SSH Security Hardening: Lock Down Your Servers in 15 Minutes

SSH is the front door to every server you manage. If it's misconfigured, you're one leaked password or brute-force script away from a breach. The default OpenSSH install on most Linux distributions is functional but not secure — it allows password authentication, listens on a well-known port, and has no rate limiting. That's fine for a throwaway test box. For production, you need to harden it.

This guide covers the changes that actually matter: key-based authentication, disabling password login, fail2ban for brute-force protection, non-standard ports, and jump host architecture. Every config snippet is tested on Ubuntu 24.04 and works on any modern Linux distribution.

1. Generate a Strong SSH Key Pair

If you're still using RSA keys, it's time to upgrade. Ed25519 keys are shorter, faster, and considered more secure against future quantum attacks:

# Generate Ed25519 key (recommended)
ssh-keygen -t ed25519 -C "your@email.com" -f ~/.ssh/id_ed25519

# If you need RSA for compatibility (older systems)
ssh-keygen -t rsa -b 4096 -C "your@email.com" -f ~/.ssh/id_rsa

Always set a passphrase. A stolen key file without a passphrase gives an attacker instant access. With a passphrase, they need both the file and the passphrase — two-factor for SSH without buying hardware tokens.

Copy your public key to the server:

# Modern way (ships with OpenSSH 8.2+)
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server

# Manual way
cat ~/.ssh/id_ed25519.pub | ssh user@server "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys"

2. Harden sshd_config

Edit /etc/ssh/sshd_config — these are the changes that move the needle:

# Disable password authentication (key-only)
PasswordAuthentication no
ChallengeResponseAuthentication no
KbdInteractiveAuthentication no

# Disable root login
PermitRootLogin no

# Disable empty passwords
PermitEmptyPasswords no

# Limit authentication attempts per connection
MaxAuthTries 3

# Idle timeout: disconnect after 5 minutes of inactivity
ClientAliveInterval 300
ClientAliveCountMax 2

# Restrict to protocol 2
Protocol 2

# Disable X11 forwarding (unless you need it)
X11Forwarding no

# Disable TCP forwarding by default (allow per-user)
AllowTcpForwarding no

# Use only strong ciphers and MACs
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org

Critical: Before disabling password authentication, make sure your key-based login works. Test it in a separate terminal session — don't lock yourself out. Keep one session open with the old settings while you verify.

# Test config syntax before restarting
sudo sshd -t

# Restart SSH (keeps current sessions alive)
sudo systemctl restart sshd

3. Fail2ban: Stop Brute-Force Attacks

Even with password auth disabled, bots will hammer your SSH port. Fail2ban monitors auth logs and bans IPs after repeated failures:

# Install
sudo apt install fail2ban -y

# Create local config (never edit jail.conf directly)
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit /etc/fail2ban/jail.local, find the [sshd] section:

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600
ignoreip = 127.0.0.1/8 YOUR_STATIC_IP

This bans an IP for 1 hour after 3 failures within 10 minutes. Add your own static IP to ignoreip so you don't accidentally lock yourself out. Start fail2ban:

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

# Check banned IPs
sudo fail2ban-client status sshd

4. Change the Default Port

Moving SSH from port 22 to a non-standard port won't stop a targeted attacker, but it eliminates 99% of automated scanners. Most bots only scan port 22:

# In /etc/ssh/sshd_config
Port 2222

Update your firewall and SSH client config:

# ~/.ssh/config — so you don't have to type -p every time
Host production-server
    HostName 203.0.113.50
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Warning: If you're behind a corporate firewall that only allows outbound port 22, changing the port will break your access. Test first.

5. Jump Host Architecture

For production environments with multiple servers, don't expose SSH on every machine. Use a bastion host (jump host) as the single entry point:

# ~/.ssh/config
Host bastion
    HostName bastion.example.com
    User admin
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Host internal-*
    User deploy
    IdentityFile ~/.ssh/id_ed25519
    ProxyJump bastion

Host internal-db
    HostName 10.0.1.50

Host internal-app
    HostName 10.0.1.51

Now ssh internal-db automatically tunnels through the bastion. Internal servers only accept connections from the bastion's IP:

# On internal servers' sshd_config
AllowUsers deploy@10.0.1.1

This means an attacker needs to compromise the bastion and the internal key to reach your database. That's two layers instead of one.

6. SSH Audit Logging

Know who's connecting and what they're doing. Add these to your sshd_config:

# Log at VERBOSE level for key fingerprints
LogLevel VERBOSE

Then check the logs:

# Who logged in recently
grep "Accepted publickey" /var/log/auth.log | tail -20

# Failed attempts (should be near zero after hardening)
grep "Failed" /var/log/auth.log | tail -20

# Key fingerprints used
grep "Accepted publickey" /var/log/auth.log | grep -oP 'SHA256:\S+'

For production, ship these logs to your centralized logging stack (ELK, Loki, etc.). A sudden spike in failed SSH attempts is an early warning sign of a targeted attack.

7. Two-Factor Authentication (Optional)

For high-security environments, add TOTP-based 2FA on top of SSH keys:

# Install Google Authenticator PAM module
sudo apt install libpam-google-authenticator -y

# Set up for your user
google-authenticator

# In /etc/pam.d/sshd (add at the end)
auth required pam_google_authenticator.so

# In /etc/ssh/sshd_config
ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

This requires both the SSH key and a TOTP code. Even a stolen key file without the TOTP seed is useless. The downside: you can't use ssh in scripts without interaction. Reserve this for human operators, not CI/CD pipelines.

Quick Hardening Checklist

✅ Ed25519 keys with passphrases
✅ Password authentication disabled
✅ Root login disabled
✅ fail2ban installed and configured
✅ Non-standard port (or at least rate-limited port 22)
✅ Jump host for multi-server environments
✅ Log level set to VERBOSE
✅ Strong ciphers and MACs only
✅ Idle timeout configured
✅ Auth logs shipped to centralized monitoring

SSH hardening isn't glamorous, but it's the foundation of server security. These changes take 15 minutes and eliminate the most common attack vectors. Do them once, document the config in your infrastructure-as-code repo, and you'll never have to think about SSH security again — until you add a new server and copy the same hardened config over.