fixes https://github.com/TryGhost/Toolbox/issues/370
- we no longer need `bthreads` because we can use native
`worker_threads` now we don't have to support Node 10 any longer
- this allows us to clean up a dependency and stick with native
libraries
- the referenced node-sqlite3 issue should be fixed (or at least, we now
maintain it so we can fix it if not)
refs https://github.com/TryGhost/Toolbox/issues/357
- Job persisted in the database need to track job's execution status such as completion, failure, execution start and end times. This changeset allows to hook into job/bree lifecycle to track job's progress.
- NOTE: only supports "offloaded" jobs at the moment. Support for "inline" jobs will be added once there's a clear usecase for it.
- The "started" status and "started_at" timestamp are assigned to a job at the moment when the worker thread is created inside of bree
- The "finished" status and "finished_at" timestamp are assigned to a job when a "done" event is passed from the job script (NOTE: using process.exit(0) will not trigger the "finished" state")
- The "failed" status is assigned when the job execution is interrupted with an error
refs https://github.com/TryGhost/Ghost/issues/12496
- `workerMessageHandler` option allows for custom worker message handling and allows to eliminate a need for loggers of any type inside of jobs.
- removing loggers from jobs solves file hanle leak which used to cause Ghost process to crash (see referenced issue)
closes#117
- Having immediately executable offloaded jobs is necessary to be able to run usecases like: send batched emails now, or any other job that does not need to be scheduled
- Changed "simple" job timeout to make tests run faster
no issue
- When jobs are performing CPU intensive tasks they block main process'
event loop. They also can cause memory leaks or unexpected crashes
effectively crashing the parent proccess. To address these issues jobs need to be performed off of main
process. Worker Threads (https://nodejs.org/dist/latest-v12.x/docs/api/worker_threads.html)
are the best candidate for such work.
- These changes introduce an integration on top of bree
(https://github.com/breejs/bree/) which allows to run recurring
jobs in worker thereads. It falls back to child process execution for
Node v10 running without `--experimental-worker` flag.
- bree was chosen not only because it gives a polyfill for older Node
versions. It has support for some of the future use-cases Ghost is looking to
implement, like scheduled jobs.
- This changeset also includes a complete example of job running on an
interval with a possibility for graceful shutdown
