Hardening and CIS principles

Module 10: Staying secure · Reading time: 11 minutes

Learning objective: After this lesson you can explain what hardening is and which principles lie behind it, you know what the CIS Benchmarks are and how to use them wisely, and you can check your server systematically with Lynis.

Key concepts: hardening, attack surface, minimal installation, least privilege, defence in depth, CIS Benchmarks, Level 1 and Level 2, CIS Controls, Lynis, hardening index, AppArmor, SELinux, sysctl, time synchronisation

What is hardening?

A fresh installation of Debian or Ubuntu is reasonably secure, but it is built to work in as many places as possible. Hardening means adapting your server to exactly how you use it: remove everything you don't need, tighten everything that can be tightened, without breaking your own services.

You have already done this bit by bit in the previous modules: SSH with keys (module 2), a regular user with sudo (module 3), automatic updates (module 4), a firewall and fail2ban (module 5), services running under their own users (module 6). This lesson puts those separate measures into a framework, so you can see what is still missing.

Four principles

1. A small attack surface. What isn't there can't break. Every installed service, every open port, every account is a possible way in (lesson 5.1). Install as little as possible, and clean up what you no longer use:

anna@web1:~$ systemctl list-units --type=service --state=running
anna@web1:~$ sudo ss -tulpn
anna@web1:~$ apt list --installed | wc -l

If a service is running that you don't recognise, find out what it does (lesson 6.1). If you don't need it, remove the package.

2. Least privilege. Every user and every program gets only the permissions it needs (lesson 3.2). No daily work as root, services under their own users, database users with permissions on their own database only, no chmod 777.

3. Layered defence. Assume that any single measure will fail. A database that listens only on 127.0.0.1 and sits behind a firewall and has a strong password is still safe if one of those three accidentally fails. This is called defence in depth.

4. Knowing what is happening. Hardening is not just about locking things down, but also about seeing: logs that are kept (lesson 6.2), monitoring that alerts you (lesson 8.3), and backups you can fall back on (module 8). Without these, you only notice a break-in when it is too late.

The CIS Benchmarks

The Center for Internet Security (CIS) is an American non-profit organisation that produces the CIS Benchmarks: extensive lists of recommended settings per product, written by volunteers from the security community. There are more than a hundred of them, including Debian Linux and Ubuntu Linux, per version. You can download the PDFs free of charge for non-commercial use after creating an account.

A Linux benchmark contains hundreds of recommendations, each with an explanation, a way to check it and a way to set it up. They are divided into profiles:

• Level 1: basic measures you can put in place fairly quickly, with a clear security benefit, without making the server much less usable. The goal for most servers.
• Level 2: extra measures for environments where security comes first (defence in depth). They can cost functionality or speed and, according to CIS itself, can cause harm if you apply them carelessly.

For Linux there are separate profiles for servers and workstations.

Examples of topics in a Linux benchmark: separate partitions for /tmp and /var/log, disabling unused file systems and network protocols, time synchronisation, audit logging, SSH settings (you know many of these from lesson 2.2), password policy, permissions on important files, and the firewall.

Alongside the Benchmarks there are the CIS Controls (now version 8.1): not settings per product, but a list of eighteen groups of measures for a whole organisation, such as "inventory of software", "vulnerability management" and "data recovery". For a single server the Benchmarks are more useful; the Controls help when you are thinking about your whole business.

Use a benchmark wisely

A benchmark is not a button you press. A few rules:

• Understand first, then apply. Every recommendation comes with an explanation. If you don't understand why, don't apply it yet.
• One thing at a time, and test. A strict kernel or network setting can break your application. Take a snapshot, change one thing, check that everything still works.
• Not everything is for you. A recommendation for a separate /var/log partition is awkward on an existing VPS and gains you little. Choose deliberately, and write down why you are not doing something.
• Be careful with ready-made hardening scripts from the internet that apply "all of CIS" in one go. They can change SSH, your firewall or your logs in a way that locks you out, and afterwards you no longer know what has changed.

Lynis: an audit from the inside

Lynis is a free open-source program (GPL) from CISOfy that goes through your server from the inside and gives warnings and suggestions per topic. It doesn't change anything itself; it only reports.

anna@web1:~$ sudo apt install lynis
anna@web1:~$ sudo lynis audit system
...
  -[ Lynis 3.x Results ]-

  Warnings (1):
  ----------------------------
  ! Found one or more vulnerable packages. [PKGS-7392]

  Suggestions (34):
  ----------------------------
  * Consider hardening SSH configuration [SSH-7408]
      - Details  : AllowTcpForwarding (set YES to NO)
  ...
  Lynis security scan details:

  Hardening index : 66 [#############       ]
  Tests performed : 262

How to read it:

• Warnings are things you really need to look at, such as packages with known vulnerabilities (meaning you have pending updates, lesson 4.1).
• Suggestions are pieces of advice. Many are useful, some don't suit your server. A suggestion to disable AllowTcpForwarding, for example, breaks your SSH tunnels (lesson 2.3). Choose deliberately.
• Every finding has a code in brackets, such as [SSH-7408]. Use it to look up in the log /var/log/lynis.log exactly what was tested.
• The hardening index is a number up to 100. It is a pointer, not a score to chase. An increase after your changes shows you are making progress; 100 is not the goal.

The Lynis version in your distribution's package repository may lag behind a little. That is not a problem for a first audit.

Three more topics in brief

• AppArmor and SELinux are extra security layers in the kernel that define per program what it is allowed to do. Debian and Ubuntu use AppArmor, switched on by default; AlmaLinux and Rocky use SELinux. Don't switch them off "because something doesn't work": find the message in the journal and fix the specific problem.
• sysctl sets kernel settings, for example for the network. The default values of Debian and Ubuntu are reasonable. If you want to change something, put it in a file of your own in /etc/sysctl.d/ (Debian 13 no longer reads /etc/sysctl.conf).
• Time: a server with the wrong time makes logs useless and may treat certificates as invalid. Check with timedatectl that it says "System clock synchronized: yes".

In practice

At WebOké, as with other VPS providers, you manage the security inside your server yourself: the provider takes care of the hardware, the network and the virtualisation platform. This course is self-help: nobody is watching over your server. Before a round of hardening, take a snapshot in the panel, and make sure you know how to open the console in case you do lock yourself out.

Summary

• Hardening means adapting your server to your own use: remove what isn't needed, tighten what can be tightened.
• Four principles: a small attack surface, least privilege, layered defence, and seeing what is happening.
• The CIS Benchmarks give hundreds of recommendations per product in the profiles Level 1 (basic) and Level 2 (strict); use them with understanding, one step at a time.
• Lynis checks your server from the inside and gives warnings, suggestions and a hardening index; it doesn't change anything itself.
• Don't switch off AppArmor/SELinux; put your own kernel settings in /etc/sysctl.d/; check time synchronisation.

Quiz

Exercise

Exercise O10.1: An audit with Lynis. Take a snapshot, install Lynis on your server and run sudo lynis audit system. Write down the hardening index, all warnings and five suggestions. Choose three suggestions to carry out and one you deliberately won't, with your reason. Then run the audit again and write down the new index.

Further reading

• Securing your VPS: the first five minutes
• CIS Benchmarks (overview, including Debian and Ubuntu): https://www.cisecurity.org/cis-benchmarks
• CIS Controls: https://www.cisecurity.org/controls
• Lynis, documentation: https://cisofy.com/documentation/lynis/
• The Dutch NCSC, Getting started with cybersecurity (Dutch source): https://www.ncsc.nl/beginnen-met-cybersecurity