Creating and Listening to Events

DeepLynx utilizes an event system to allow external applications (primarily data sources) to listen for and be notified of events. For example, an application may be notified that a DeepLynx data source has ingested data. This allows a workflow to be created where apps no longer need to regularly poll DeepLynx for changes, but can be notified when there is some event to act upon.

Available Events

Events are messages that are automatically added to a queue for certain actions within DeepLynx. Each event must be associated with an existing container or data source. All events contain a source_type (either data_source or container), source_id, and possibly a data value. The following table lists the types of events available for data sources and containers, along with the data that is sent with the event:

Event Source	Event Type	Data
data_source	data_imported	none
data_source	data_ingested	import ID, status
data_source	file_created	file ID
data_source	file_modified	file ID
data_source	type_mapping_created (future)	type mapping ID
data_source	type_mapping_modified (future)	type mapping ID
container	data_source_created	data source ID
container	data_source_modified	data source ID
container	data_exported	none

Setting up Events

Several config parameters can be set through the .env file to determine what queue system is used (QUEUE_SYSTEM) and the frequency with which the queue is polled for new events (QUEUE_POLL_INTERVAL). Deep Lynx will default to using the database queue, which sends events to a table in PostgreSQL. The default poll interval is 1 second.

Registering for Events

1. An application should first query to determine the available containers or data sources and to retrieve the desired ID(s). IDs can be determined by calling the List Containers and List Data Sources APIs (see Postman or Swagger docs).
2. Once the needed ID has been obtained, the application can register for applicable event types (see table above) on that source. This is done through the Create Registered Event API. A body must be included in the request that contains values for app_name, app_url (the URL to which Deep Lynx should send events), container_id or data_source_id, and type.

For example, if I wanted to listen to data_ingested events on a datasource with the ID 123, my application name myApp and at the url http://myApp.com/listen, I would supply the following body to the request:

{
  "app_name": "myApp",
  "app_url": "http://myApp.com/listen",
  "data_source_id": "123",
  "type": "data_ingested"
}

1. DeepLynx will now regularly poll for new events at the specified interval and then check for any registered events that match the source ID and type of the event. A list of registered events can be retrieved through the List Registered Events API, and registered events may be updated through the Update Registered Event API (e.g. to update the URL or set the registered event as active or inactive. Registered events are active by default).