Configure SSH for Secure Remote Access on Linux

SSH (Secure Shell) is the standard way to remotely access Linux servers. A default SSH installation works, but it is not hardened. This guide walks through securing SSH properly — from key-based authentication to sshd_config tuning — so your server is not a sitting target.

How SSH Works

SSH uses asymmetric cryptography. The server has a host key pair. When you connect, the client verifies the server identity, then authenticates itself via password or key. All traffic is encrypted. The SSH daemon (sshd) runs on the server and listens for connections.

Installing SSH

# Ubuntu/Debian
sudo apt update && sudo apt install openssh-server -y

# CentOS/Rocky/RHEL
sudo dnf install openssh-server -y

sudo systemctl enable --now sshd
sudo systemctl status sshd

Connecting to a Server

ssh username@192.168.1.10
ssh -p 2222 username@192.168.1.10
ssh -i ~/.ssh/my_key username@host

Generating SSH Key Pairs

Generate a key pair on your client machine:

ssh-keygen -t ed25519 -C "alok@workstation"

This creates ~/.ssh/id_ed25519 (private — never share) and ~/.ssh/id_ed25519.pub (public — copy to server).

Copying the Public Key to the Server

ssh-copy-id -i ~/.ssh/id_ed25519.pub username@192.168.1.10

# Manual method
cat ~/.ssh/id_ed25519.pub | ssh username@host "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

# Fix permissions on server
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Hardening sshd_config

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak
sudo nano /etc/ssh/sshd_config

Port 2222
PermitRootLogin no
PasswordAuthentication no
AllowUsers alok devuser
PermitEmptyPasswords no
LoginGraceTime 30
MaxAuthTries 3
X11Forwarding no

sudo sshd -t           # Test config for syntax errors
sudo systemctl restart sshd

Keep your current session open and test a new connection before closing the existing one.

SSH Config File on the Client

Host myserver
    HostName 192.168.1.10
    Port 2222
    User alok
    IdentityFile ~/.ssh/id_ed25519

Connect with: ssh myserver

Using SSH Tunnels

# Local tunnel: access remote MySQL on local port 3307
ssh -L 3307:localhost:3306 alok@192.168.1.10

# Dynamic SOCKS proxy
ssh -D 8080 alok@192.168.1.10

Blocking Brute Force with fail2ban

sudo apt install fail2ban -y
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local

[sshd]
enabled = true
port = 2222
maxretry = 3
bantime = 3600
findtime = 600

sudo systemctl enable --now fail2ban
sudo fail2ban-client status sshd

Checking Who Is Logged In

who
w
last | head -20
lastb | head -10

Summary

A properly configured SSH setup uses key-based authentication, disables root login, restricts allowed users, changes the default port, and runs fail2ban. These steps eliminate the vast majority of automated attacks. Treat SSH configuration as non-negotiable on any internet-facing server — it is the front door to your system.