Secure SSH and SFTP behind a bastion with 2FA

To protect access to a system or network from the outside world, SSH authentication can be accompanied by 2FA by using google-authenticator alongside enforcing use of an SSH bastion.

Firstly, instead of accessing one or more servers by port forwarding SSH connectivity directly to the server, you should set up an extra server to handle SSH connectivity; port forward only to this bastion server; install and configure google-authenticator.

We will set up an SSH Bastion to accept either public key authentication or password authentication + TOTP.
Publickey auth happens outside of PAM, so it's not possible to have the option of TOTP + publickey/password. The only options are:

- publickey auth + TOTP
- password + TOTP
- publickey OR {password + TOTP}

Since transmitting the private SSH key, or carrying it around with you, isn't desirable we will opt for authentication with publickey-only (if presented) OR password+TOTP.
This way we can SSH by password (enforcing TOTP as 2FA) whenever we do not have access to the private key, or simply log straight in using private key if we have it (no 2FA required).

SSH Bastion

Deploy a new server on your network with only openssh-server installed.

Configure your new SSH Server

/etc/ssh/sshd_config

Port 22
Protocol 2
PermitRootLogin no
MaxAuthTries 6 # (password and TOTP identify as 2 attempts, one each - this will provide 3 login attempts with 2FA)
MaxSessions 10
PubkeyAuthentication yes # (allow publickey authentication if presented)
HostbasedAuthentication no
IgnoreUserKnownHosts yes
IgnoreRhosts yes
PasswordAuthentication no # (non-intuitively, this is required along with 'UsePAM yes' to force password+TOTP authentication using PAM instead of SSH built-in)
PermitEmptyPasswords no
ChallengeResponseAuthentication yes # (this is required for prompting TOTP verification codes)
AuthenticationMethods publickey keyboard-interactive:pam # (pubkey auth OR password+TOTP via PAM - publickey,keyboard-interactive means key FOLLOWED BY password+TOTP)
UsePAM yes

Install google-authenticator

• Use time-based tokens
• Save / update your ~/.google_authenticator file (protect it with chmod 400)
• Disallow multiple uses of auth token
• 90 seconds should be time enough to enter a valid auth token
• Enable rate-limiting

You can either use the generated QR code to set up an authentication app, or use the 2FA code store in ~/.google_authenticator (the very first line is your 2FA token). I can recommend:

• Mobile : FreeOTP
• Linux desktop app : authenticator

Enable google-authenticator TOTP in the sshd PAM module

/etc/pam.d/sshd

auth      required  pam_google_authenticator.so
auth      include   system-remote-login
account   include   system-remote-login
password  include   system-remote-login
session   include   system-remote-login

(if there are users on the SSH Bastion that may need to login without TOTP, i.e. service accounts, use auth required pam_google_authenticator.so nullok instead - this means service accounts that haven't set up google-authenticator won't be re-prompted after providing the password.)

Enable time synchronisation

As the google-authenticator module is configured to use Time-based One Time Passcodes, it should be evident that we need to ensure system time is accurate.

/etc/systemd/timesyncd.conf

[Time]
NTP=0.uk.pool.ntp.org 1.uk.pool.ntp.org 2.uk.pool.ntp.org 3.uk.pool.ntp.org
FallbackNTP=0.arch.pool.ntp.org 1.arch.pool.ntp.org 2.arch.pool.ntp.org 3.arch.pool.ntp.org

Start the time timesyncd service

sudo timedatectl set-ntp true

Enable the timesyncd service.

sudo systemctl enable timesyncd

Install fail2ban

We're going to block any malicious attempts at logging into our SSH Bastion. We will do this by using fail2ban, and banning any IP addresses that fail 3 login attempts.

Create a jail configuration for the sshd service

/etc/fail2ban/jail.d/sshd.local

[DEFAULT]
bantime = 1d
destemail = <YOUR_EMAIL>
sender = <SMTP_SENDER_EMAIL>

# to ban & send an e-mail with whois report to the destemail.
action = %(action_mw)s

# same as action_mw but also send relevant log lines
#action = %(action_mwl)s

[sshd]
enabled   = true
filter    = sshd
banaction = iptables
backend   = systemd
maxretry  = 3
findtime  = 1d
bantime   = 2w # adjust this to your liking
ignoreip  = 192.168.0.0/24 # adjust this to your private network, or any other IP addresses to be ignored

(you may want to install and configure a Mail Transport Agent to send email notifications about blocked IPs).

Finish

Now start or restart the sshd service and confirm you are able to login.

• Attempt login with your SSH private key - you should log in immediately with no TOTP prompt
• Attempt login without SSH private key (ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no) - you should be prompted to provide a TOTP verification code from your 2FA app, followed by the account password.

SFTP Bastion

Now that we have a functioning SSH Bastion that has access to our private network; is secured by 2FA; and actively blocks malicious login attempts, we can use the following SSH Port Forwarding command to gain FTP access to files on any of the systems on our private network.

This will:

• Authenticate against our SSH Bastion and open an SSH encrypted channel - using 2FA token if no privatekey is present
• Create a port forward from our local machine (listening on LOCAL_PORT)
• Pass any network traffic through the SSH tunnel to our private PROTECTED_SERVER at port FORWARDED_PORT on the private server.

(you will need to make sure the SSH Bastion has network access to the PROTECTED_SERVER on port FORWARDED_PORT).

Set up the SSH Tunnel and Port Forward

Use the following command to authenticate against the SSH Bastion and create a local listening port that forwards traffic to our private server.

ssh -L<LOCAL_PORT>:<PROTECTED_SERVER>:<FORWARDED_PORT> -p <BASTION_PORT> <BASTION_SERVER>

Now that we have an encrypted network tunnel, with the local machine listening for traffic on port LOCAL_PORT and forwarding everything from that port through the SSH Bastion and onto the PROTECTED_SERVER, we can initiate and SFTP connection to the private server.

The USER in this instance is the user account that exists on the private server (PROTECTED_SERVER).
The LOCAL_PORT is the currently listening port that is forwarding connections through the SSH Bastion.

File manager access

sftp://<USER>@localhost:<LOCAL_PORT>/home

Manual mountpoint access

sshfs <USER>@localhost:/remote/path /local/path -C -p <LOCAL_PORT> -o idmap=user

fstab on-demand

# You may need to mount at least once manually AS ROOT so the host's signature is added to the /root/.ssh/known_hosts file.

<USER>@localhost:/remote/folder /mount/point  -p <LOCAL_PORT> fuse.sshfs noauto,x-systemd.automount,_netdev,users,idmap=user,IdentityFile=/home/<USER>/.ssh/id_rsa,allow_other,reconnect 0 0