Console Login

Linux Server Hardening: SSH, iptables and sysctl Baseline (2010)

The problem

A freshly installed CentOS 5 or Ubuntu 10.04 server is reachable from the whole internet as soon as it has an address. Three defaults make that a problem:

  • The SSH daemon accepts password logins, and often root logins as well.
  • No host-level packet filter restricts which ports answer.
  • Services the machine does not need (mail transfer agent, portmap, NFS helpers) are enabled and listening.

Automated scanners try common passwords against port 22 on any address they find. The symptom is a log full of failed logins and, on a weak account, a successful one. Copy-and-paste hardening recipes also cause their own failures:

  • An administrator is locked out after moving the SSH port without opening it in the firewall.
  • SELinux denies sshd a non-standard port.
  • Setting UsePAM no breaks account and session handling.

A baseline therefore has to close the exposure and keep the administrator able to log in.

Why it happens

Distribution defaults favour compatibility and easy first logins over a minimal attack surface.

  • SSH: Password authentication lets anyone who can reach the port guess credentials indefinitely. Key authentication removes that path, because a 4096-bit RSA private key cannot be guessed over the network.
  • Keyboard-interactive logins: With UsePAM yes, PAM can still offer a password prompt through ChallengeResponseAuthentication. Setting only PasswordAuthentication no may therefore leave a password path open.
  • Network stack: The kernel keeps legacy behaviour such as accepting source-routed packets and ICMP redirects. Some distributions already set a few of the hardening values, so check what is active before assuming a gap.
  • Services: A package installed for one purpose may enable a daemon that listens on all interfaces. The result is more code exposed to the network than the server's job requires.
  • Containers: Container-style virtualization such as OpenVZ shares the host kernel, so a kernel vulnerability affects every tenant. Full virtualization such as KVM or Xen gives each guest its own kernel. This is a platform choice rather than something configured inside the guest.

Source of truth

The authoritative reference for the SSH settings is the sshd_config(5) manual page of the installed OpenSSH version. A rendered copy is at manpages.ubuntu.com, sshd_config(5). It confirms three things:

  • AllowUsers takes a list of user name patterns separated by spaces.
  • Allow and deny directives are processed in a fixed order: DenyUsers, AllowUsers, DenyGroups, then AllowGroups.
  • Debian-style packages ship with UsePAM yes as their standard.

That page describes a later OpenSSH build than the ones of 2010 (OpenSSH 4.3 on CentOS 5, 5.3 on Ubuntu 10.04), but every keyword used below exists in both. The same keywords can be checked on the target machine with man sshd_config.

The kernel settings are documented in the kernel's own ip-sysctl documentation, which describes tcp_syncookies, icmp_echo_ignore_broadcasts, rp_filter, accept_source_route and log_martians. The iptables rule syntax is documented in iptables(8) and the distribution's firewall chapter. Those two references were not link-checked for this rewrite, so verify them locally.

Fix options

SSH access

  • Keys only, no root login, user allow-list. This removes password guessing completely.
  • Keep passwords but add brute-force blocking (fail2ban, DenyHosts, or rate limiting in iptables). Blocking reduces noise but does not remove the weakness.
  • Move the port. This only shortens the logs, because a port scan finds the new port. It is cosmetic and needs a firewall rule plus an SELinux port label on CentOS.

Packet filtering

  • Raw iptables ruleset with a default-drop INPUT policy. It is available on every distribution of the era and is fully explicit.
  • Front-ends (the CentOS text-mode firewall tool, Ubuntu's ufw). They are quicker to set up but hide the rules and differ between distributions.
  • An external firewall only. This leaves the host exposed to anything on the same network segment, so it complements host filtering rather than replacing it.

Kernel and services

  • Set the relevant sysctl values in /etc/sysctl.conf.
  • Disable unused services with chkconfig (CentOS) or remove the packages entirely (yum remove, apt-get remove). Removal is stronger, because a removed package cannot be re-enabled by accident.

Why this fix

The chosen combination is key-only SSH, a raw default-drop iptables ruleset, a small sysctl block and a service audit. The reasons:

  • Key-only login removes the weakness instead of limiting its damage. Brute-force blockers can be added later as a supplement.
  • The SSH port stays at 22. Changing it adds failure modes (firewall, SELinux) without a real security gain. Anyone who still wants a different port needs both the firewall rule and, on CentOS, the SELinux port label.
  • UsePAM yes is kept, with ChallengeResponseAuthentication no. This keeps PAM account and session processing (limits, SELinux context) and still blocks password prompts.
  • A raw iptables file is the same on every system, easy to review and easy to keep in version control.
  • The ruleset is loaded before sshd is changed. Existing connections survive through the ESTABLISHED rule, so a mistake in one step does not cut off the session being used.

Step by step

The procedure assumes a regular account named sysadmin with sudo rights already exists. Keep one SSH session open for the whole procedure and test each change from a second terminal.

1. Create a key and install it

On the workstation:

ssh-keygen -t rsa -b 4096
ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]
ssh [email protected]

The last command must log in using the key (or the key passphrase) before anything else is changed. If permissions on the server are wrong, sshd ignores the key:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

2. Load the firewall first

Write the ruleset to /etc/sysconfig/iptables on CentOS:

*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set --name SSH
-A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 --name SSH -j DROP
-A INPUT -p tcp --dport 22 -m state --state NEW -j ACCEPT
-A INPUT -p tcp --dport 80 -m state --state NEW -j ACCEPT
-A INPUT -p tcp --dport 443 -m state --state NEW -j ACCEPT
-A INPUT -p icmp --icmp-type 8 -m limit --limit 1/sec -j ACCEPT
COMMIT

The two recent lines drop a source address that opens more than three new SSH connections in 60 seconds. That throttles scripted guessing and is harmless once passwords are disabled. It can also catch a legitimate user who runs many rapid scp or rsync invocations. Remove those two lines if that is a concern.

Activate and inspect the rules:

service iptables restart
chkconfig iptables on
iptables -L -n -v --line-numbers

On Ubuntu 10.04 no rule file is loaded at boot by default. Save the rules and restore them when the interface comes up:

iptables-save > /etc/iptables.rules
# in /etc/network/interfaces, under the interface stanza:
pre-up iptables-restore < /etc/iptables.rules

The ruleset above covers IPv4 only. If the server has an IPv6 address, load an equivalent default-drop policy with ip6tables (on CentOS, /etc/sysconfig/ip6tables), otherwise IPv6 remains unfiltered.

3. Harden sshd

Edit /etc/ssh/sshd_config:

PermitRootLogin no
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM yes
PubkeyAuthentication yes
AllowUsers sysadmin deploy

Check the syntax, reload, and test from a second terminal while the first stays open:

sshd -t
service sshd reload
ssh [email protected]
ssh -o PubkeyAuthentication=no [email protected]

On Ubuntu the service is named ssh, so use reload ssh. The first login must succeed with the key. The second must be refused with a public-key-only denial and no password prompt. A reload leaves established sessions running, which is why the first terminal remains usable if the test fails.

4. Apply the sysctl settings

Append to /etc/sysctl.conf:

net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.all.log_martians = 1
sysctl -p
sysctl net.ipv4.tcp_syncookies

SYN cookies are a fallback used when the SYN backlog fills up. They reduce the effect of a small SYN flood but are not a defence against a large one. Reverse-path filtering (rp_filter) drops packets whose source address could not be reached through the interface they arrived on, which blunts simple spoofing. Do not enable it on hosts with asymmetric routing.

5. Audit listening services

netstat -tulpn
chkconfig --list | grep ':on'
chkconfig sendmail off
service sendmail stop
chkconfig portmap off
service portmap stop

In the netstat listing, check the Local Address column. A daemon bound to 127.0.0.1 is only reachable from the machine itself and is a lower risk than one bound to 0.0.0.0. If a service is not needed at all, remove its package instead of just disabling it:

yum remove sendmail

6. Verify from outside

From a different host, scan the server and compare the open ports with the ruleset:

nmap -sS -p 1-1024 server.example.com

Only ports 22, 80 and 443 should be reported as open. All other ports should show as filtered, because the policy drops instead of rejecting.

Looking back

The baseline removes the biggest exposures: guessable SSH passwords, unrestricted inbound ports and unnecessary listeners. It is quick to apply and easy to audit, because everything lives in four files: sshd_config, the iptables file, sysctl.conf and the service list.

The limits are the following:

  • It does not protect the applications themselves. An outdated web application behind port 80 is still exploitable, so regular updates (yum update, apt-get upgrade) are part of the baseline.
  • Private keys must be protected with a passphrase and removed from the server when a person leaves.
  • Logs still need to be read or forwarded, or the hardening goes unnoticed when it fails.
  • Kernel vulnerabilities cannot be fixed from inside a container that shares the host kernel.
  • Where and how personal data is stored and processed is a legal and organisational question that this technical baseline does not answer.

Other methods fit particular cases:

  • Where many administrators need access, use a central user directory or a bastion host with per-user keys.
  • Where passwords cannot be removed, use fail2ban together with strong password policy.
  • Where the rules are unfamiliar, a distribution front-end is acceptable as long as the resulting rules are still read and reviewed.