Check on status of heroku dynos
Using heroku CLI, run:
heroku ps -a scihist-digicoll-production
Is heroku itself having problems? Or are other platforms we use?
https://status.hirefire.io/ status
searchstax status: https://status.searchstax.com/
AWS status (notoriously underreports problems though): https://status.aws.amazon.com/
Restart heroku dynos
From heroku web GUI, you can restart all dynos from the “More” menu in top right navbar, choose “restart all dynos”.
Using the heroku CLI, you can restart only web or only workers, or even a specific dyno.
heroku ps:restart worker heroku ps:restart web heroku ps:restart worker.2
Note: It’s not clear to me how often this restarting heroku dynos will actually fix a problem, and in some cases it could cause a less stable state, if for instance heroku is having problems.
Restart solr on Searchstax
Login to searchstax
Use shared credentials stored in our credential spot
Click on the instance you want to restart (
scihist_digicoll(production), orscihist-digicoll-staging)At bottom of page there is a single node listed (our plan only has one node), you can click “stop solr”, and then “Start solr”
Put entire app into maintenance mode
Disable our app, it won’t be accessible to anyone, but they’ll get a nice maintainance message.
In heroku web GUI, go to “settings” tab, scroll down to “Maintenance mode” section, toggle switch.
In heroku CLI , run heroku maintenance:on and heroku maintenance:off
(Note: Right now, this is just a generic heroku maintenance message. It is possible to customize/brand this page, we may get to that eventually. https://github.com/sciencehistory/scihist_digicoll/issues/1201 )
Disable staff logins
We can effectively make the app “read-only” but still available to the public by disabling staff logins. So we don’t have a public facing outage, but if we’re dealing with some kind of data corruption issue we’re trying to diagnose, we might want to ‘freeze’ staff out.
In heroku config vars, just set LOGINS_DISABLED to true.