Chapter 8 · 8.3
Monitoring from outside and inside
A server you don't monitor breaks down quietly: a full disk, an expired certificate, a backup that hasn't run for weeks. Monitor it from two sides.
From outside: does it work for your visitors?
- Is your website reachable and does it return the right page (HTTP status 200)?
- Is the certificate still valid for at least two more weeks?
- Does your server respond to ping and SSH?
You can do this with Uptime Kuma (see Uptime Kuma: monitor your sites and server). If Uptime Kuma runs on the same VPS, have the VPS monitored from somewhere else as well: a second server or an external monitoring service. Otherwise you won't hear anything if this very VPS goes down.
From inside: how is the server doing?
| What | Command | When to take action? |
|---|---|---|
| Disk | df -h | Above 80 per cent |
| Memory | free -h | The available column stays low and a lot of swap is used |
| Load | uptime | Load stays higher than the number of vCPUs for a long time |
| Services | systemctl --failed | Anything appears in the list |
| Containers | sudo docker compose ps | A container shows Restarting or Exited |
| Restart required | ls /var/run/reboot-required | The file exists |
The Charts tab in the VPS screen shows processor, memory, network and disk usage over a longer period, up to a year back (see The VPS screen: on and off, charts, system and log).
An alert when the disk fills up
A small script with a timer, as in Backups with restic to another location, can call a Push monitor in Uptime Kuma when everything is fine. That way you get an alert when the signal doesn't arrive:
#!/bin/sh
GEBRUIK=$(df --output=pcent / | tail -1 | tr -dc '0-9')
if [ "$GEBRUIK" -lt 85 ]; then
curl -fsS -m 10 "https://status.example.com/api/push/JOUWCODE?status=up&msg=schijf-$GEBRUIK" >/dev/null
fiFor example, have the timer run this every fifteen minutes with OnCalendar=*:0/15, and in Uptime Kuma set the Push monitor's wait time to a little over fifteen minutes.
A fixed routine
- Every week: check that last night's backup is there (
restic snapshots) and whether there were any alerts. - Every month:
apt list --upgradable,docker compose pulland a restart if needed. - Every quarter: restore a backup to a test location and check that everything is correct.