External command-line access of DoC machines via Secure Shell (SSH)

Secure shell ('ssh') is a network protocol allowing secure, remote machine access. 'ssh' also refers to applications that allow you to connect using this network protocol. In order to connect, the remote machine (the server or DoC machine) must be running a secure shell daemon/server; all Linux machines in the Department are already running ssh daemons.

The local machine (the client or your home machine) must be running a secure shell client; ssh client implementations are freely available for all major platforms (please see below).

From outside the College network, you can ONLY directly ssh to the following five DoC Linux servers:

  • shell1.doc.ic.ac.uk

  • shell2.doc.ic.ac.uk

  • shell3.doc.ic.ac.uk

  • shell4.doc.ic.ac.uk

  • shell5.doc.ic.ac.uk 

(and use sftp to sftp.doc.ic.ac.uk)

You may ssh to any other DoC machine either:

  • in two hops (first ssh into one of the above shell servers, then ssh from that shell server to whichever DoC linux machine you wanted to connect to). See the later section on Two Hop SSH Access for details.
  • or by first connecting to the College VPN (or the newer Zscaler) and then ssh'ing directly to your chosen DoC machine.

Note that you should not run any CPU-intensive or memory-intensive code on the shell servers: they exist only as gateway machines.  Do the second hop as we describe below!  Long running jobs (that is, over a day, and with very large RAM requirements) should run on batch1.doc.ic.ac.uk or batch2.doc.ic.ac.uk.

If your home machine runs Windows 10/11, or MacOS X, or practically all versions of Linux, ssh should come with it, or be trivially installable.  The rest of this guide concentrates on using your built-in ssh client.  Please note that whenever you enter a hostname remember that you need to use the fully qualified hostnames as above, as in shell1.doc.ic.ac.uk not just shell1.

From any ssh window you can run text-mode programs (eg. editors such as Vim, Pico or Emacs) but you won't be able to run X clients (graphical programs). To do that, you need an X server installed on the machine you are sitting at, and to tell ssh to forward the X protocol data over the connection, or some similar graphical network transport (such as X2go). We'll talk about graphical remote applications later.

Under Linux, Mac OS X, or Windows 10/11, you can run the command in your shell/command line interpreter (eg. Terminal, cmd or Powershell on Windows):

ssh USERNAME@shell3.doc.ic.ac.uk

(replacing USERNAME with your DoC user-name) and you will then be prompted for your college password.

You can replace shell3 with shell1, shell2, shell4 or shell5 in the above example to use one of the other shell servers.

For now, we recommend that you do not attempt to run graphical programs on lab machines remotely for now.  Learn to use the power of the command line.  If you want to edit files and you can't learn a Linux editor, you could always use scp to grab a copy of the file, edit it locally, and scp it back again. Scp should be part of OpenSSH on Linux, Mac and windows.

Cygwin is also another Windows possibility as it provides Linux API support in Windows as well as X/SSH capabilities. WSL2 is a newer option. VcXsrv may also be worth considering.

 

Two Hop SSH Access via a Shell server

Suppose you want to ssh to ray17 from offsite.  Of course you can't just ssh USERNAME@ray17.doc.ic.ac.uk from offsite because it's firewalled off.  You have to go via a shell server.  You could do it in two explicit steps:

ssh USERNAME@shell3.doc.ic.ac.uk

and then

ssh ray17

which should work, but there's a neater way:

ssh -J USERNAME@shell3.doc.ic.ac.uk USERNAME@ray17.doc.ic.ac.uk

should "jump" or "proxy" through shell3 to ray17.   It might ask for your password twice, of course.  Read on, we'll show you ways of improving this below..

Simplifying your ssh access

It's a pain to have to type:

ssh USERNAME@shell3.doc.ic.ac.uk

every time. That’s a mouthful, wouldn’t it be nicer to have to type only:

ssh shell5

and have it not ask who you are, or require you to fully qualify the hostname?

You can achieve this as follows: On your home PC, edit ~/.ssh/config (that's the path on Linux or Mac OS X, for Windows your path is usually c:\users\your-username\.ssh\config - you should create the .ssh folder if it does not exist) using your favourite editor (vim in my case, whatever you like). Note that your .ssh/config file may or may not already exist. Add the following to it:

 

Host shell1
        User USERNAME
        HostName shell1.doc.ic.ac.uk

Host shell2
        User USERNAME
        HostName shell2.doc.ic.ac.uk

Host shell3
        User USERNAME
        HostName shell3.doc.ic.ac.uk

Host shell4
        User USERNAME
        HostName shell4.doc.ic.ac.uk

Host shell5
        User USERNAME
        HostName shell5.doc.ic.ac.uk

(replacing, as ever, USERNAME with your College username). You may wish to add additional entries, for particular DoC linux machines you regularly ssh to, such as batch1 and batch2, or specific lab machines that you like to use.

You can now ssh shell3 and have it know that you meant ssh USERNAME@shell3.doc.ic.ac.uk.

But it will still ask you for a password or passphrase, each time you ssh in.  That's still quite a pain.

Note that you can put an explicit "fake two hop host" entry into your .ssh/config.  Suppose you often login to ray17 via shell3.  Edit your .ssh/config and add:

 

Host ray17
        User USERNAME
        HostName ray17.doc.ic.ac.uk

Host ray17viashell3
        User USERNAME
        HostName ray17
        ProxyJump shell3

Now, the equivalent of the enormous ssh -J command from the two hop section is simply:

ssh ray17viashell3

which is a lot simpler.  Note: the name "ray17viashell3" can be anything you want.

Avoiding ssh asking for a password

(Please note there are several methods of doing this, this guide describes one - albeit the way we recommend. As the Perl slogan has it: There’s More Than One Way to Do It.  In particular, another way is to set up a non-passphrase protected ssh key, and trust that, but we regard this as very insecure and request that you DON'T do that).

First, do the following - unless you have already done it before, of course:

Setting up a passphrase protected SSH RSA key at work, and trusting it

Ok, so ssh into a DoC shell server, giving the password as usual:

ssh shell3

Now, create an ssh key here at DoC by:

ssh-keygen -t rsa

or, for a newer even more secure type of key: 

ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519 (if you do this, mentally replace all references to id_rsa below with id_ed25519).

Choose a decently complex passphrase for your new SSH key, different from (often longer than!) your ordinary College password - perhaps more of a phrase than a password. Accept most of the ssh-keygen defaults and enter your fresh passphrase (twice) when asked. This creates ~/.ssh/id_rsa (the private key) and ~/.ssh/id_rsa.pub (the public key).

Never give the private key to anyone [not even a member of CSG], never email it etc; treat it like a password.

Next, still on the shell server, append the contents of your ~/.ssh/id_rsa.pub public key onto the ~/.ssh/authorized_keys file.
Be careful now: If no such file exists:

cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys

or if it already exists,

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

This tells the ssh daemon on all DoC linux machines (because they all access your DoC home directory) to accept ssh connections from any machine in the world without asking for a password or phrase – if that remote end can cryptographically demonstrate prior knowledge of the ssh passphrase associated with that key.

Using your DoC passphrase-protected SSH key at home, to avoid ssh asking for a password

Now, you can use your (new) SSH host key from home as follows.  These instructions assume that your home PC runs Linux or Mac OS X too, for Windows 10/11 ssh and ssh-add and ssh-agent are usually installed, but you may have to install  ssh-agent or start an ssh-agent service, see note at the end of this section:

Back on your home Linux PC/Mac, copy the private key from DoC to your home PC:

scp USERNAME@shell3.doc.ic.ac.uk:.ssh/id_rsa ~/.ssh

(Be careful again: make absolutely sure that you’re not overwriting an existing, different and vitally important, id_rsa private key at home).

To use this key automatically, each time you reboot your home PC, start an ssh-agent:

eval `ssh-agent` (for bash users, csh users should add “-c" after "ssh-agent”).

Then add your DoC key into the agent via:

ssh-add ~/.ssh/id_rsa

It will prompt you for your passphrase at this time: enter it. Check that it worked via:

ssh-add -l

Now that you’ve done all this, as long as the ssh-agent is running and has your key in it, you should be able to:

ssh USERNAME@shell3.doc.ic.ac.uk echo hi

and have it say hi without prompting you for a password or passphrase.  If you've done all the simplifications mentioned above you should even be able to say:

ssh shell3 echo hi

Note that this ssh-agent, loaded with your ssh key, lasts forever, and lets anyone with physical access to your home machine to ssh, as you, to all DoC linux machines.  This is convenient - but dangerous.  For security, at the end of your working day (or when leaving a secure environment with this setup on your laptop), we recommend that you destroy your keys via:

ssh-add -D

This leaves your ssh-agent running, but loaded with no keys.  The next day, simply redo

ssh-add ~/.ssh/id_rsa

and enter your passphrase again.

Windows addendum: If ssh-add.exe does not exist, or fails to connect to ssh-agent, try running this command in Powershell (Run as adminstrator):

Get-Service ssh-agent | Set-Service -StartupType Automatic -PassThru | Start-Service

And then try the ssh-add and subsequent commands from earlier in this section.

We've also been told that ssh-agent has been setup as a Windows service in recent versions of Windows.  If you can find the "OpenSSH Authentication Agent" service in Settings -> Services, you can mark that as Automatic start - and then start that now via a "Start" button - instead.

We've also been told that RSA keys don't work with the version of OpenSSH in Windows, create an ED25519 key instead.

Using Graphical Remote Applications via ssh

Wherever possible, we strongly recommend that you use ssh in text mode.  The Linux command line is incredibly powerful, learn to use it.

But, once in a while, you may have no choice but to run a graphical (X Windows) application on a DoC linux machine, and then you will want to have it display on your home machine's screen.  Please note that this will use a lot more resources than the command line, so don't even try to do it "just because you want to run some GUI app like Eclipse".  In such cases: install the GUI app on your home machine and use that instead.

 

But, ok, if you are running some form of X Window server on your home machine (Linux machines do this by default, on Mac OS you can install XQuartz, and it's much more complicated on Windows 10), and enable both 'X11 forwarding' and 'Agent Forwarding' in your ssh client configuration, then you may be able to use the command: 

ssh -Y USERNAME@shell3.doc.ic.ac.uk

and then, on shell3, remotely run a graphical program too, though they may be slow due to unoptimised graphics updates.  In addition, remember that very little GUI software is installed on the shell servers, and that you're not supposed to run any intensive programs on a shell server anyway.  Getting X-forwarding to work over two ssh hops is quite tricky.

We will discuss a much more efficient alternative that works over two ssh hops - called X2go - later, in a separate document.

Advanced - Secure Shell Tunneling

Secure shell not only provides the means for securely interacting with a remote machine, it can also be used to enable remote operation of otherwise independent applications. This concept is called ssh tunnelling, since the ssh connection tunnels the network traffic between a remote machine and a local application.

Under Linux and Mac OS X, the syntax for setting up an ssh tunnel is as follows:

 

ssh username@remotehost -L localport:remotehost2:remoteport

Under Windows, use Plink: 

plink username@remotehost -L localport:remotehost2:remoteport

Subject to a successful authentication on remotehost, the above command specifies that connections on the ssh client machine (one's 'home' machine) to port localport should be tunnelled (through the SSH connection to remotehost) to port remoteport on remotehost2.