Ruby Version upgrade

Upgrade Steps

Switch the settings for Ruby in the grou_vars file(s) you want to use

Use the build_ami script to make a new ruby AMI.

Update the group_vars files to use the new AMI.

Switch Box steps

1. Leave current prod up
2. Build new prod boxes for app, jobs, and solr.
   1. Once boxes are done, alert staff to stop all edits and ingests
3. Make sure new prod boxes are fully connected by running update-box.yml on the new boxes.
4. Cap deploy to new prod boxes
   1. Make sure cap gives no errors.
5. Run a backup of solr
6. Backup app's postgres DB and Redis
7. On new production boxes
   1. In new-app drop the blank chf_hydra db, create a new blank chf_hydra db and then follow the postgres import instructions in Backups and Recovery
      1. Stop Redis, delete the existing redis.rdb in var/lib/redis
      2. Copy the backed up redis.rdb over to the folder and make sure it is owned by redis
      3. Restart Redis
   2. In new-solr run the solr restore commands in Application administration
8. Edit New-Prod App to allow IP access without directing to actual prod by disabling apache redirect
   1. This is not completely needed, but is a way to be sure there were no problems
   2. Check that New-Prod works properly with solr's index being loaded
   3. Return New-Prod App to normal apache configuration
9. Move Elastic IP from app-prod to the new app-prod.
10. Make sure all details in ansible's group_vars are up to date with any internal IP address changes and commit and merge with master.

Revert Box steps

Assuming the old boxes were not deleted. Follow these steps.

1. Start the old servers up.
2. Double check their internal IP addresses, they should be the same as before but always be sure.
3. Check netdata to be sure they have resque workers running
4. Tell users to stop adding new items
5. Run a backup of solr
6. Backup app's postgres DB and Redis
7. On new production boxes
   1. In new-app drop the blank chf_hydra db, create a new blank chf_hydra db and then follow the postgres import instructions in Backups and Recovery
      1. Stop Redis, delete the existing redis.rdb in var/lib/redis
      2. Copy the backed up redis.rdb over to the folder and make sure it is owned by redis
      3. Restart Redis
   2. In new-solr run the solr restore commands in Application administration
8. Edit New-Prod App to allow IP access without directing to actual prod by disabling apache redirect
   1. This is not completely needed, but is a way to be sure there were no problems
   2. Check that New-Prod works properly with solr's index being loaded
   3. Return New-Prod App to normal apache configuration
9. Move Elastic IP from app-prod to the new app-prod.
10. Make sure all details in ansible's group_vars files are up to date with any internal IP address changes and commit and merge with master.