Backups with restic

Module 8: Backups and monitoring · Reading time: 12 minutes

Learning objective: After this lesson you can use restic to make encrypted backups to storage outside your server, clean up old versions with a retention policy, check the backup and restore files, and have the whole thing run automatically.

Key concepts: restic, repository, init, password (file), backend (sftp, S3, rest-server), backup, snapshots, forget, --keep-daily/--keep-weekly/--keep-monthly, prune, check, restore, mount, deduplication, append-only, systemd timer

Why restic?

There are plenty of backup programs. restic is a good choice for your own server, because it:

• encrypts everything before it leaves the server; the storage only sees unreadable blocks;
• deduplicates: a file that has not changed is not stored again, so daily backups take up little space;
• supports many kinds of storage: a directory, another server over SFTP, S3-compatible storage, your own rest-server;
• is a single program, available in the Debian and Ubuntu package repositories (apt install restic).

An alternative with similar features is BorgBackup. The principles in this lesson apply there too.

The concepts

• Repository: the place where restic stores the backups. Always outside the server you are backing up.
• Snapshot: one backup at one point in time. (Note: this is not the same thing as your provider's snapshot from lesson 8.1.)
• Password: every repository has a password that is used to encrypt everything. The restic documentation is clear about this: if you lose the password, your data is lost for good.

Creating a repository

Say you have a storage server backup.example.net that you can reach over SFTP with a key, as the user web1. First create a password file on the server (and keep the same password in your password manager, lesson 8.1):

root@web1:~# install -m 600 /dev/null /root/.restic-wachtwoord
root@web1:~# openssl rand -base64 32 > /root/.restic-wachtwoord
root@web1:~# export RESTIC_REPOSITORY=sftp:web1@backup.example.net:/srv/restic/web1
root@web1:~# export RESTIC_PASSWORD_FILE=/root/.restic-wachtwoord
root@web1:~# restic init
created restic repository 5f1a9c2e3b at sftp:web1@backup.example.net:/srv/restic/web1

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

With the environment variables RESTIC_REPOSITORY and RESTIC_PASSWORD_FILE you don't have to type the options -r and --password-file every time.

Making a backup

First the database dump (lesson 8.1), then the backup:

root@web1:~# restic backup /etc /var/www /opt /var/backups/db \
    --exclude /var/www/*/cache --tag dagelijks
repository 5f1a9c2e opened (version 2, compression level auto)
...
Files:        8421 new,     0 changed,     0 unmodified
Dirs:         1203 new,     0 changed,     0 unmodified
Added to the repository: 612.338 MiB (402.117 MiB stored)

processed 8421 files, 1.204 GiB in 0:41
snapshot 3c7e91d4 saved

The next day it is much faster: only what changed is included.

root@web1:~# restic snapshots
ID        Time                 Host  Tags       Paths
-----------------------------------------------------------------------
3c7e91d4  2026-11-08 03:10:02  web1  dagelijks  /etc /opt /var/backups/db /var/www
a18f0b77  2026-11-09 03:10:05  web1  dagelijks  /etc /opt /var/backups/db /var/www
-----------------------------------------------------------------------
2 snapshots

Cleaning up: forget and prune

One extra snapshot every day means 365 snapshots after a year. With forget you decide which ones to keep, using a retention policy:

root@web1:~# restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune

This is how the policy works:

• --keep-daily 7: look at the last seven days that have a snapshot, and keep the newest one for each day;
• --keep-weekly 4: the same for the last four weeks (Monday to Sunday);
• --keep-monthly 6: the same for the last six months.

A snapshot is kept if it matches at least one of the rules. Anything that matches no rule at all is forgotten. Try a new policy with --dry-run first: you'll see what would be removed without anything actually changing.

An important difference: forget only removes the reference to a snapshot. The data stays in the repository until prune cleans up the blocks that are no longer used by any snapshot. The option --prune with forget does both in one go. Prune takes time and locks the repository, so don't schedule it at the same time as your backup.

Checking: check

You only notice a backup that has quietly become damaged when you try to restore it. Have restic check the repository regularly:

root@web1:~# restic check
...
no errors were found
root@web1:~# restic check --read-data-subset=5%

A plain check checks the structure. With --read-data-subset it also actually reads back part of the data and checks the contents. Do this weekly, for example, with a few per cent; that way everything gets read at some point over time.

Restoring: restore and mount

Restore to a separate directory, never straight over the original:

root@web1:~# restic restore latest --target /tmp/terug --include /etc/nginx
restoring snapshot a18f0b77 of [/etc /opt /var/backups/db /var/www] at 2026-11-09 03:10:05 ...
Summary: Restored 41 files/dirs (88.104 KiB) in 0:00
root@web1:~# diff -r /tmp/terug/etc/nginx /etc/nginx && echo gelijk
gelijk

latest is the newest snapshot; you can also use an ID (restic restore 3c7e91d4 ...). With restic mount /mnt/restic (requires FUSE) you see all snapshots as directories, so you can browse around and copy individual files. Stop with Ctrl+C.

You restore a database by retrieving the dump and importing it, for example gunzip < alles.sql.gz | mariadb.

Automatically, every night

Create a script and a systemd timer (lesson 6.3). The script:

#!/bin/sh
# /usr/local/sbin/backup.sh
set -eu
export RESTIC_REPOSITORY=sftp:web1@backup.example.net:/srv/restic/web1
export RESTIC_PASSWORD_FILE=/root/.restic-wachtwoord
mariadb-dump --single-transaction --all-databases | gzip > /var/backups/db/alles.sql.gz
restic backup /etc /var/www /opt /var/backups/db --tag dagelijks
restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune

A .service with Type=oneshot and ExecStart=/usr/local/sbin/backup.sh, and a .timer with OnCalendar=*-*-* 03:10:00 and Persistent=true. set -eu makes the script stop at the first error, so systemd sees the run as failed. Link an alert to it (lesson 8.3), otherwise you won't notice a failed backup.

Against ransomware: append-only

In the script above, the server itself can also delete the backup (forget and prune). So if an attacker takes over the server, they can do that too. Two ways to prevent this:

• rest-server in append-only mode: on the storage server you run restic's own rest-server with the option --append-only. The web server can then add backups but not delete anything. You do the cleanup (forget --prune) from another, well-secured machine with full rights.
• Storage with versioning or object locking at an S3 provider, so deleted objects can still be recovered for a while.

Watch out for a subtle pitfall that the restic documentation itself mentions: an attacker who adds fake snapshots can trick a policy such as --keep-weekly into forgetting real snapshots. With --keep-within (for example --keep-within 30d) everything from that period is kept, fake or real.

Summary

• restic encrypts, deduplicates and can write to many kinds of storage outside your server; lose the password and you lose everything.
• init creates the repository, backup makes a snapshot, snapshots lists them.
• forget with --keep-* decides what you keep; only prune (or --prune) really cleans up the data. Try it out with --dry-run.
• check (with --read-data-subset) checks, restore --target restores to a separate directory, mount lets you browse.
• Automate with a timer and an alert, and keep at least one repository out of the server's reach (append-only).

Quiz

Exercise

Exercise Z8.2: What is left after forget? The example file restic-snapshots.txt contains a list of snapshots and a retention policy. Work out which snapshots remain after restic forget with that policy. The program compares your answer with the fixed answer key.

Exercise O8.2: A restore test. On your own server, create a restic repository outside the server, make a backup of /etc and your data, and restore one directory to /tmp/terug. Compare with diff, note how long it took, and describe where your password and your storage credentials are kept outside the server.

Further reading

• Backup strategy: the 3-2-1 rule for your own business
• restic documentation, Preparing a new repository: https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html
• restic documentation, Removing backup snapshots (forget and prune): https://restic.readthedocs.io/en/stable/060_forget.html
• restic documentation, Restoring from backup: https://restic.readthedocs.io/en/stable/050_restore.html
• rest-server (append-only): https://github.com/restic/rest-server