-
Notifications
You must be signed in to change notification settings - Fork 14
Creating and Listening to Events
DeepLynx utilizes an event system to allow external applications (primarily data sources) to listen for and be notified of events. For example, an application may be notified that a DeepLynx data source has ingested data. This allows a workflow to be created where apps no longer need to regularly poll DeepLynx for changes, but can be notified when there is some event to act upon.
Event Actions
Events are messages that are automatically added to a queue for certain actions within DeepLynx. Each event must be associated with an existing container or data source. All events contain a source_type
(either data_source or container), source_id
, and possibly a data
value. The following table lists the types of events available for data sources and containers, along with the data that is sent with the event:
Event Source | Event Type | Data |
---|---|---|
data_source | data_imported | none |
data_source | data_ingested | import ID, status |
data_source | file_created | file ID |
data_source | file_modified | file ID |
data_source | type_mapping_created (future) | type mapping ID |
data_source | type_mapping_modified (future) | type mapping ID |
container | data_source_created | data source ID |
container | data_source_modified | data source ID |
container | data_exported | none |
Several config parameters can be set through the .env
file to determine what queue system is used (QUEUE_SYSTEM
) and the frequency with which the queue is polled for new events (QUEUE_POLL_INTERVAL
). Deep Lynx will default to using the database queue, which sends events to a table in PostgreSQL. The default poll interval is 1 second.
- An application should first query to determine the available containers or data sources and to retrieve the desired ID(s). IDs can be determined by calling the
List Containers
andList Data Sources
APIs (see Postman or Swagger docs). - Once the needed ID has been obtained, the application can register for applicable event types (see table above) on that source. This is done through the
Create Registered Event
API. A body must be included in the request that contains values for app_name, app_url (the URL to which Deep Lynx should send events), container_id or data_source_id, and type.
For example, if I wanted to listen to data_ingested
events on a datasource with the ID 123
, my application name myApp
and at the url http://myApp.com/listen
, I would supply the following body to the request:
{
"app_name": "myApp",
"app_url": "http://myApp.com/listen",
"data_source_id": "123",
"type": "data_ingested"
}
- DeepLynx will now regularly poll for new events at the specified interval and then check for any registered events that match the source ID and type of the event. A list of registered events can be retrieved through the
List Registered Events
API, and registered events may be updated through theUpdate Registered Event
API (e.g. to update the URL or set the registered event as active or inactive. Registered events are active by default).
Sections marked with ! are in progress.
- HTTP Authentication Methods
- Generating and Exchanging API Keys for Tokens
- Creating a DeepLynx Enabled OAuth2 App
- Authentication with DeepLynx Enabled OAuth2 App
- Creating an Ontology
- Creating Relationships and Relationship Pairs
- Ontology Versioning
- Ontology Inheritance
- Querying Tabular (Timeseries) Data
- Timeseries Quick Start
- Timeseries Data Source
- Timeseries Data Source via API
- Exporting Data
- Querying Data
- Querying Timeseries Data
- Querying Jazz Data
- Querying Data - Legacy
- Querying Tabular Data