Page Comparison

...

1. Temporarily disable our hirefire auto-scaling manager, so it will allow our worker count to be scaled down to zero. Just toggle the “enabled” toggle for worker at https://manager.hirefire.io/

2. Scale down workers to zero: heroku ps:scale worker=0 -r production

   1. Wait until worker is actually scaled down, you can see with heroku ps -r production

3. Disable staff access, so staff can’t trigger ingest with bg jobs that might not be able to be enqueued: heroku config:set LOGINS_DISABLED=true -r production

   1. Because we use Heroku preboot, it can take 2-3 minutes to take effect, check to make sure you are really locked out of staff UI?

4. Run the maintenance now per heroku instructions, on production, eg: heroku heroku redis:maintenance --run REDIS --force -r production

   1. When it’s finished, you should get an email, you can also check on status with heroku redis:

   maintenance

   1. info -

   -run REDIS

   1. r production

5. Enable staff logins again: heroku config:set LOGINS_DISABLED=false -r production

6. Scale workers back up to their default, probably 2 (if you get it wrong hirefire will fix it): heroku ps:scale worker=2 -r production

7. Turn on hirefire manager again at https://manager.hirefire.io/

8. Because of heroku preboot, it could take 2-3 minutes for staff logins to be enabled again. Don’t leave until you confirm they are!

While this process should avoid it – if you wound up with any redis-related errors in Honeybadger despite yourself, go “resolve” them.

...