stream8_backups

This is a nomad job to backup the daily snapshots created by stream8_snapshots

The entrypoint backup.sh performs a backup of $TODAY using restic, storing the data on s3.cern.ch

A helper script interactive.sh (included in the docker image) provides the ability to query the restic repository in an interactive fashion.

To launch interactive.sh it is recommended to run a privileged docker instance as follows:

    $ export BRANCH=<branch> EMAIL=<you>@cern.ch S3REPO=s8-backups-testing PATH_SNAPSHOTS=TMP;
    docker run -it --rm --entrypoint bash \
    --privileged --cap-add SYS_ADMIN --cap-add MKNOD --device /dev/fuse
    -v /mnt/data3:/data \
    -e PATH_SNAPSHOTS="$PATH_SNAPSHOTS" \
    -e RESTIC_REPOSITORY="s3:s3.cern.ch/$S3REPO" \
    -e EMAIL_FROM="linux.support@cern.ch" \
    -e EMAIL_ADMIN="$EMAIL" \
    -e NOMAD_ADDR="https://lxsoftadm.cern.ch:4646" \
    gitlab-registry.cern.ch/linuxsupport/cronjobs/stream8_backups/stream8_backups:$BRANCH

(for the production data, use S3REPO="s8-backups-production" PATH_SNAPSHOTS="cern/centos")

interactive.sh provides the following abilities:

1. list snapshots ./interactive.sh snapshots
2. restore a snapshot ./interactive.sh restore YYYYMMDD /path/to/restore/to
3. fuse mount the restic respository ./interactive.sh mount (mounts to /root/fusemount)
4. unmount the restic repository ./interactive.sh unmount

S3 credentials are required to access the restic repo, which are embedded into the nomad job for the daily backup use case.

We are currently using credentials that map to the "Linuxsoft service" OpenStack tenant, which has a quota of 500GB

When using interactive.sh, the script will ask you to enter S3 credentials. These credentials are stored within the container in /tmp/credentials. If you wish to use restic directly (outside of the interactive.sh script), you may source /tmp/credentils after first running interactive.sh.

If you do not know where to retrieve the credentials, they can be found in the 'Variables' section of https://gitlab.cern.ch/linuxsupport/cronjobs/stream8/-/settings/ci_cd or alternatively via tbag showkeys --hg lxsoft/adm.

for i in AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY RESTIC_PASSWORD; do tbag show --hg lxsoft/adm stream8_backups_$i; done

Note: The backup repository/bucket must be initialised once, which is a manual task. You can accomplish this by:

./get_credentials.sh
source /tmp/credentials
restic init