Linux Security: How to Lock Down Your SSH Server

Table of Contents

1. Understanding SSH and Its Risks
2. Prerequisites
3. Step 1: Use SSH Keys Instead of Passwords
4. Step 2: Disable Password Authentication
5. Step 3: Change the Default SSH Port
6. Step 4: Limit User Access
7. Step 5: Disable Direct Root Login
8. Step 6: Install and Configure Fail2ban
9. Step 7: Enable Two-Factor Authentication (2FA)
10. Step 8: Harden sshd_config Further
11. Step 9: Monitor SSH Logs
12. Step 10: Keep SSH and the System Updated
13. Testing Your SSH Configuration
14. Troubleshooting Common Issues
15. Conclusion
16. References

Understanding SSH and Its Risks

SSH (Secure Shell) is a network protocol used to securely access and manage Linux/Unix servers over an unsecured network. It replaces insecure protocols like Telnet and FTP by encrypting all data (including passwords) transmitted between client and server.

Why SSH Security Matters:

• Brute-force attacks: Attackers use automated tools to guess passwords by trying thousands of combinations.
• Credential theft: Weak or reused passwords can be stolen via phishing, keyloggers, or data breaches.
• Unpatched vulnerabilities: Outdated SSH software may have known exploits (e.g., CVE-2020-14145 in OpenSSH).
• Misconfigurations: Default settings (e.g., password authentication, root login) are low-hanging fruit for attackers.

An unhardened SSH server is like leaving your house with the front door unlocked—even if you have a strong lock, improper use renders it useless.

Prerequisites

Before starting, ensure you have:

• A Linux server running OpenSSH (the most common SSH implementation).
• Sudo or root access to the server (to edit configuration files and install software).
• A local machine with an SSH client (e.g., ssh on Linux/macOS, PuTTY on Windows).
• A backup connection to the server (e.g., a physical console or out-of-band management) in case you lock yourself out during configuration.

Step 1: Use SSH Keys Instead of Passwords

Passwords are inherently weak: they can be guessed, stolen, or cracked. SSH keys provide a far more secure alternative by using cryptographic key pairs (public and private) for authentication.

How SSH Keys Work:

• Private key: Stored securely on your local machine (never shared).
• Public key: Uploaded to the server’s ~/.ssh/authorized_keys file.
• When you connect, the server uses your public key to encrypt a challenge, which your private key decrypts—proving your identity without transmitting a password.

Generate an SSH Key Pair (Local Machine)

On your local computer, generate a key pair with:

ssh-keygen -t ed25519 -C "[email protected]"

• -t ed25519: Uses the Ed25519 algorithm (more secure and faster than RSA for most use cases).
• -C: Adds a comment (e.g., your email) to identify the key.

When prompted, press Enter to save the key in the default location (~/.ssh/id_ed25519). Add a strong passphrase (even for keys—this encrypts the private key on disk).

Upload the Public Key to the Server

Copy your public key to the server using ssh-copy-id (replace user and server_ip):

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server_ip

If ssh-copy-id isn’t available (e.g., on Windows), manually copy the public key:

1. Display the public key on your local machine:

   cat ~/.ssh/id_ed25519.pub

2. SSH into the server (using a password for now) and create the .ssh directory (if missing):

   mkdir -p ~/.ssh && chmod 700 ~/.ssh

3. Append the public key to ~/.ssh/authorized_keys:

   nano ~/.ssh/authorized_keys  # Paste the public key here
   chmod 600 ~/.ssh/authorized_keys  # Critical: Restrict permissions

Test Key-Based Authentication

Verify you can log in without a password:

ssh user@server_ip

If prompted for the key passphrase, enter it. To avoid retyping the passphrase, use ssh-agent (Linux/macOS) or Pageant (Windows).

Step 2: Disable Password Authentication

Once key-based authentication works, disable password authentication to eliminate brute-force risks.

Edit the SSH Daemon Configuration

The SSH server configuration is stored in /etc/ssh/sshd_config. Open it with a text editor (e.g., nano or vim):

sudo nano /etc/ssh/sshd_config

Update Key Settings

Find and set the following options (uncomment them if they’re commented out):

PasswordAuthentication no      # Disable password login
ChallengeResponseAuthentication no  # Disable keyboard-interactive auth (e.g., 2FA may use this later)
PubkeyAuthentication yes       # Enable key-based auth (should be default)

Restart the SSH Service

Apply changes by restarting the SSH daemon. Use the command for your Linux distribution:

# Debian/Ubuntu
sudo systemctl restart sshd

# RHEL/CentOS/Fedora
sudo systemctl restart sshd

# Arch Linux
sudo systemctl restart sshd

Critical Note: Always test SSH access in a new terminal window before closing your current session. If you made a mistake, you’ll still have access to fix it!

Step 3: Change the Default SSH Port

By default, SSH listens on port 22. Attackers scan port 22 aggressively for vulnerabilities. Changing the port to a non-standard number (e.g., 2222) reduces “background noise” from automated scans.

Edit sshd_config

Open the SSH config again:

sudo nano /etc/ssh/sshd_config

Find the Port line and change it to a port between 1024–65535 (avoid well-known ports like 80, 443):

Port 2222  # Example: Use port 2222

Update Firewall Rules

Your firewall (e.g., ufw, firewalld) must allow traffic on the new port.

For ufw (Debian/Ubuntu):

sudo ufw allow 2222/tcp   # Allow the new port
sudo ufw delete allow 22/tcp  # Remove the old port (after testing!)
sudo ufw reload

For firewalld (RHEL/CentOS/Fedora):

sudo firewall-cmd --add-port=2222/tcp --permanent  # Add new port
sudo firewall-cmd --remove-port=22/tcp --permanent  # Remove old port (after testing!)
sudo firewall-cmd --reload

Restart SSH and Test

Restart SSH and test the new port (replace 2222 with your port):

sudo systemctl restart sshd
ssh -p 2222 user@server_ip  # Test connection on the new port

Only remove the old port (22) from the firewall after confirming the new port works!

Step 4: Limit User Access

Restrict which users or groups can log in via SSH to minimize exposure. For example, if only alice and bob need SSH access, explicitly allow them.

Edit sshd_config

Add one of the following options (uncomment or add a new line):

# Allow only specific users (space-separated)
AllowUsers alice [email protected]/24  # "[email protected]/24" restricts bob to the local subnet

# OR: Allow only specific groups
AllowGroups ssh-users  # Users in the "ssh-users" group can log in

Create an SSH Group (Optional)

If using AllowGroups, create a group and add users:

sudo groupadd ssh-users
sudo usermod -aG ssh-users alice  # Add user "alice" to the group

Restart SSH

sudo systemctl restart sshd

Step 5: Disable Direct Root Login

Direct root login is risky: if an attacker compromises the root account, they gain full control. Instead, log in as a regular user and use sudo for administrative tasks.

Edit sshd_config

Set:

PermitRootLogin no  # Disable direct root login

Test Root Login

Verify root cannot log in:

ssh root@server_ip -p 2222  # Should fail with "Permission denied"

Step 6: Install and Configure Fail2ban

Fail2ban is an intrusion prevention tool that blocks IP addresses with repeated failed SSH login attempts (e.g., brute-force attacks).

Install Fail2ban

# Debian/Ubuntu
sudo apt install fail2ban -y

# RHEL/CentOS/Fedora
sudo dnf install fail2ban -y  # or yum install fail2ban

# Arch Linux
sudo pacman -S fail2ban

Configure Fail2ban for SSH

Fail2ban uses “jails” (rules) to monitor services. Create a custom config file (don’t edit jail.conf directly—use jail.local):

sudo nano /etc/fail2ban/jail.local

Add the following SSH jail configuration:

[sshd]
enabled = true
port = 2222  # Use your SSH port (default 22 if unchanged)
filter = sshd
logpath = /var/log/auth.log  # Debian/Ubuntu: /var/log/auth.log; RHEL/CentOS: /var/log/secure
maxretry = 3  # Ban after 3 failed attempts
bantime = 3600  # Ban for 1 hour (3600 seconds)
findtime = 600  # Count attempts in the last 10 minutes (600 seconds)
ignoreip = 192.168.1.0/24  # Whitelist your local subnet (optional but recommended)

Start and Enable Fail2ban

sudo systemctl enable --now fail2ban  # Start on boot and now
sudo systemctl status fail2ban  # Verify it's running

Test Fail2ban

Simulate failed login attempts (use a different IP or a VM):

for i in {1..5}; do ssh user@server_ip -p 2222; done  # Enter wrong passwords

Check if the IP is banned:

sudo fail2ban-client status sshd

Step 7: Enable Two-Factor Authentication (2FA)

Two-factor authentication (2FA) adds a second layer of security: something you know (SSH key passphrase) and something you have (a mobile app like Google Authenticator).

Install Google Authenticator PAM Module

We’ll use libpam-google-authenticator, which integrates with PAM (Pluggable Authentication Modules) for SSH.

# Debian/Ubuntu
sudo apt install libpam-google-authenticator -y

# RHEL/CentOS/Fedora
sudo dnf install google-authenticator -y  # EPEL repo may be required: sudo dnf install epel-release

# Arch Linux
sudo pacman -S libpam-google-authenticator

Configure 2FA for a User

Run the Google Authenticator setup as the user who will log in (e.g., alice):

google-authenticator

Answer the prompts:

• “Do you want authentication tokens to be time-based? (y/n)”: y
• Scan the QR code with the Google Authenticator app on your phone.
• Save the emergency scratch codes (store them securely—they’re for account recovery).
• “Do you want me to update your “/home/alice/.google_authenticator” file? (y/n)”: y
• “Do you want to disallow multiple uses of the same authentication token? (y/n)”: y
• “Do you want to enable rate-limiting? (y/n)”: y

Update PAM and SSH Config

1. Edit the PAM SSH config to require 2FA:

   sudo nano /etc/pam.d/sshd

   Add this line at the top (before other auth rules):

   auth required pam_google_authenticator.so

2. Edit sshd_config to enable challenge-response authentication (required for 2FA):

   sudo nano /etc/ssh/sshd_config

   Set:

   ChallengeResponseAuthentication yes  # Override earlier "no" setting
   UsePAM yes  # Enable PAM (default)

Restart SSH

sudo systemctl restart sshd

Test 2FA Login

Connect via SSH—you’ll now be prompted for a verification code from Google Authenticator:

ssh user@server_ip -p 2222
# Output:
# Verification code: 123456  # Enter code from app

Step 8: Harden sshd_config Further

Tweak additional sshd_config settings to reduce attack surface:

# Limit failed attempts per connection
MaxAuthTries 3  # Default is 6; reduce to 3

# Reduce login window (time to enter credentials)
LoginGraceTime 20s  # Default is 2m; auto-logout after 20 seconds

# Disable X11 forwarding (if not needed)
X11Forwarding no

# Disable TCP forwarding (if not needed for tunnels)
AllowTcpForwarding no

# Use only SSH Protocol 2 (Protocol 1 is insecure)
Protocol 2

# Disable unused authentication methods
KerberosAuthentication no
GSSAPIAuthentication no

# Add a legal banner (optional but recommended for compliance)
Banner /etc/ssh/banner  # Create /etc/ssh/banner with a warning (e.g., "Unauthorized access prohibited")

Create a banner file (optional):

sudo nano /etc/ssh/banner

Add text like:

WARNING: Unauthorized access to this system is prohibited. All activities are logged.

Restart SSH after changes.

Step 9: Monitor SSH Logs

SSH logs track login attempts, successes, and failures—critical for detecting breaches.

Where Are SSH Logs Stored?

• Debian/Ubuntu: /var/log/auth.log
• RHEL/CentOS/Fedora: /var/log/secure

Monitor Logs in Real Time

Use tail to watch logs as they update:

sudo tail -f /var/log/auth.log  # Debian/Ubuntu
# or
sudo tail -f /var/log/secure   # RHEL/CentOS

Search for Failed Attempts

sudo grep "Failed password" /var/log/auth.log  # Debian/Ubuntu
sudo grep "Failed password" /var/log/secure    # RHEL/CentOS

Automated Log Monitoring

For ongoing monitoring:

• Logwatch: Sends daily email summaries of logs.

  sudo apt install logwatch -y  # Debian/Ubuntu

• Centralized logging: Tools like ELK Stack (Elasticsearch, Logstash, Kibana) or Graylog aggregate logs from multiple servers.

Step 10: Keep SSH and the System Updated

Outdated software is a major security risk. Regularly update OpenSSH and your OS to patch vulnerabilities.

Update OpenSSH

# Debian/Ubuntu
sudo apt update && sudo apt upgrade openssh-server -y

# RHEL/CentOS/Fedora
sudo dnf update openssh-server -y

# Arch Linux
sudo pacman -Syu openssh

Enable Automatic Updates

For Debian/Ubuntu:

sudo apt install unattended-upgrades -y
sudo dpkg-reconfigure -plow unattended-upgrades  # Enable automatic updates

Testing Your SSH Configuration

After making changes, validate the SSH config for errors:

sudo sshd -t  # Checks for syntax errors in sshd_config

If you see Syntax OK, restart SSH. Always test login in a new terminal!

Troubleshooting Common Issues

• Locked out of the server: Use a console/IPMI connection to revert sshd_config changes.
• Permission errors: Ensure .ssh (700), authorized_keys (600), and ~ (700–755) have correct permissions.
• Fail2ban banned your IP: Unban with sudo fail2ban-client set sshd unbanip YOUR_IP.
• 2FA not working: Verify ChallengeResponseAuthentication yes in sshd_config and PAM config.

Conclusion

Securing your SSH server is a critical step in protecting your Linux infrastructure. By combining key-based authentication, 2FA, fail2ban, and strict configuration hardening, you create a “defense in depth” strategy that thwarts most attacks. Remember: security is ongoing—regularly update your setup, monitor logs, and stay informed about new threats.

References

• OpenSSH Official Documentation
• Fail2ban Documentation
• Google Authenticator PAM Module
• SSH Key Best Practices
• NIST Guidelines for SSH (Section IA-2)
• Linux Server Hardening Guide