Chapter 9 · 9.5
Hacked? The first steps
Signs that someone else is on your server: unknown processes using a lot of CPU, a user or SSH key you don't recognise, changed files, or a notice that your server is sending spam or attacks. Stay calm and work through the steps in this order.
1. Record the current state
Take a snapshot (see Snapshots: a safety net for every major change). This preserves the evidence, so you can later work out how the attacker got in. Don't start cleaning up first.
2. Isolate the server
Turn the server off with Shut down in the VPS screen, or set the firewall so that only your IP address can still reach SSH. If you can no longer get in, use the console.
3. Reinstall
You can't clean a hacked server: you can never be sure you've found everything. Reinstall (see Reinstalling or attaching an ISO) and restore your data from a backup made before the break-in. Don't restore any programs or scripts that made the break-in possible.
4. Replace all secrets
- Create new SSH keys if you think a private key has leaked, and remove the old ones from all your servers.
- Replace the passwords for databases, apps and admin panels, and the tokens in
.env. - Replace the passwords for mailboxes and services whose details were stored on the server.
5. Close the gap
Find out how it happened: a password instead of a key, an open database or admin panel, an outdated plugin, a port that Docker published past your firewall. Go through chapter 3 and Docker and the firewall: publishing ports safely again before you open the server up again.
6. Report it
Was there personal data on the server, such as customer details or email addresses? Then you may have to report a data breach within 72 hours to the data protection authority in your country (the European Data Protection Board lists them all), and sometimes inform the people affected as well. Visit edpb.europa.eu.
