technical notes

Some people consider that using a database table as a queue is an anti-pattern, while others think it's ok. As usual, it depends on the use case.

Easy Jobs is intended for small/medium apps with dozens to thousands of job requests per day (this is already a lot for a lot of apps), so I think using a table (job_execution_request) to store job execution requests is a reasonable choice.

Using a separate queue to store job requests next to the database in which jobs meta-data is stored means the transaction, which consists of:

1. pick a job execution request from the queue
2. execute the job
3. update meta-data tables in the database
4. remove the job execution request from the queue

spans two transactional resources: the queue and the database. Anyone who tried to use the XA protocol with 2PC would agree with me on how complex this approach is, in terms of operational cost as well as code and configuration.

For this reason, and since we are in the jeasy world where the goal is to KISS ;-) , Easy Jobs is implemented by using a database table as a queue. Since the job_execution_request table is only used (or intended to be only used) by Easy Jobs's server, there is no much pressure on it, and the pattern works perfectly well for small/medium apps.

---

Resources:

• The Database As Queue Anti-Pattern
• Yes, It’s OK to use a Relational Database as a Queue
• Why a database is not always the right tool for a queue based system (See Note: As long as you don't have very high-performance throughput requirements and if you do not have to communicate with any other systems then a database will probably work fine.)
• The best way to use a DB table as a job queue
• Using a database table as a queue