Jobs Server (resque-pool) (Obsolete)

We use resque as our background jobs systems, and resque-pool to manage multiple resque workers on a jobs server.

Starting and Stopping Workers

On a jobs server, logged in as ubuntu or a personal account.

sudo systemctl stop resque-pool.service

sudo systemctl start resque-pool.service

You might want to stop workers so they stop accepting new work, in preparation to shut down a jobs server.

We are trying to have the stop be “graceful” will it give existing workers some amount of time to finish processing their jobs, but it might end up terminating workers in the middle of a job if they were really taking a long time. Workers terminated in the middle of their task should generate an error, reported from HoneyBadger. Since all of our jobs are idempotent, it should be safe to restart them in resque admin once workers are restarted, perhaps on a new server.

However, this has all been very confusing and hard to get right with resque. It may not be working exactly how we expect, and there is also room for improvement; look for a Github issue on it.

Ansible may re-start resque-pool

You may think you’ve stopped resque-pool, but ansible may decide to start it up again. Either at the nightly 5am run, or more disturbingly if someone commits a change to repo and ansible decides it has to push it out.

Resque front end

https://digital.sciencehistory.org/admin/queues/overview


https://staging-digital.sciencehistory.org/admin/queues/overview