Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Currently we are using the s3 sync tool (akin to rsync for S3) to pull over key fedora data into the chf-hydra-backup (https://s3.console.aws.amazon.com/s3/buckets/chf-hydra-backup) bucket. This is a slight misnomer as it handles backups for ArchivesSpace as well now, but Fedora data is pulled over into s3 key FedoraBackup (https://s3.console.aws.amazon.com/s3/buckets/chf-hydra-backup/FedoraBackup/?region=us-east-1&tab=overview) which contains all Fedora binary data).

PGSql (https://s3.console.aws.amazon.com/s3/buckets/chf-hydra-backup/PGSql/ ) contains the Fedora Postgres database fcrepo_backup)

Both sets are Both https://s3.console.aws.amazon.com/s3/buckets/chf-hydra-backup/FedoraBackup/ and https://s3.console.aws.amazon.com/s3/object/chf-hydra-backup/PGSql/fcrepo_backup.sql are needed to do a full restore.

Note: As a reminder, while S3's visual interface uses folders, those locations are actually just the first step in a path of individual block stored objects, folders . Folders do not exist in S3.

How to restore Fedora:

  1. Stop Tomcat
  2. Download the Postgres database fcrepo_backup.sql to an arbitrary location on the Fedora machine.
  3. Fedora might still have active connections to postgres, so run a postgres restart to kill them: sudo service postgresql restart

    will kill them

    .

  4. Import the database: psql fcrepo < fcrepo_backup.sql
    1. If the database already exists, such as when you are running a sync, you will want to drop the existing database and then run the command.
  5. Check that the user trilby has permissions to access and use the newly made fcrepo database.
  6. Delete the existing folder(s) inside /opt/fedora-data (This step is not always required but makes it simpler)
  7. Using screen or tmux start an aws s3 sync to copy all the data over in the FedoraBackup "folder" to /opt/fedora-data : aws s3 sync s3://chf-hydra-backup/FedoraBackup /opt/fedora-data/
  8. Wait a while for all the data (>800 GB) to copy over.
  9. Run chown -R tomcat7tomcat8:tomcat7 tomcat8 /opt/fedora-data to give ownership on the new files to the tomcat user so Fedora can access them.
  10. Restart Tomcat
  11. This will restore the Fedora database.  Current cost estimates (2/18) are about $.10 to do this restore.

...

How to restore users:

  1. Go to S3 and download the postgres backup files.
  2. Stop Apache
  3. Restart the postgres service, this should remove the default connection to the Hydra database that Hydra has when running so you can change it.
  4. In Postgres delete the automatically generated chf_hydra database
    1. Log in via psql -u postgres
      1. The postgres account password is in ansible-vault (groupvars/all)
    2. Run: DROP DATABASE chf_hydra;
    3. Run: CREATE DATABASE chf_hydra;
  5. Then import the downloaded database
    1. Either:
      1. pg_restore -d chf_hydra -U postgres chf_hydra.dump
      2. psql chf_hydra < chf_hydra_dump.sql
  6. Then set permissions
    1. psql -U postgres
    2. GRANT Create,Connect,Temporary ON DATABASE chf_hydra TO chf_pg_hydra;
  7. You may now restart postgres and Apache2

...