External command-line access of DoC machines via Secure Shell (SSH)

Secure shell ('ssh') is a network protocol allowing secure, remote machine access. 'ssh' also refers to applications that allow you to connect using this network protocol. In order to connect, the remote machine (the server or DoC machine) must be running a secure shell daemon/server; all Linux machines in the Department are already running ssh daemons.

The local machine (the client or your home machine) must be running a secure shell client; ssh client implementations are freely available for all major platforms (please see below).

From outside the College network, you can ONLY directly ssh to the following five DoC Linux servers:

  • shell1.doc.ic.ac.uk

  • shell2.doc.ic.ac.uk

  • shell3.doc.ic.ac.uk

  • shell4.doc.ic.ac.uk

  • shell5.doc.ic.ac.uk 

You may ssh to any other DoC machine either:

  • in two hops (first ssh into one of the above shell servers, then ssh from that shell server to whichever DoC linux machine you wanted to connect to)
  • or by first connecting to the College VPN and then ssh'ing directly to your chosen DoC machine.

Note that you should not run any CPU-intensive or memory-intensive code on the shell servers: they exist only as gateway machines.  Do the second hop!  Long running jobs (that is, over a day, and with very large ram requirements) should run on batch1.doc.ic.ac.uk or batch2.doc.ic.ac.uk.

If your home machine runs Windows 10, or MacOS X, or practically all versions of Linux, ssh should come with it, or be trivially installable.  If you still use Windows 7 (why would you?) then you can download and use a freeware application called PuTTY.  The rest of this guide concentrates on using your built-in ssh client.  Please note that whenever you enter a hostname remember that you need to use the fully qualified hostnames as above, as in shell1.doc.ic.ac.uk not just shell1.

From any ssh/putty window you can run text-mode programs (eg. editors such as Vim, Pico or Emacs) but you won't be able to run X clients (graphical programs). To do that, you need an X server installed on the machine you are sitting at, and to tell ssh to forward the X protocol data over the connection, or some similar graphical network transport (such as X2go). We'll talk about graphical remote applications later.

Under Linux, Mac OS X, or Windows 10, you can run the command in your shell/command line interpreter:

ssh USERNAME@shell3.doc.ic.ac.uk

(replacing USERNAME with your DoC user-name) and you will then be prompted for your college password.

You can replace shell3 with shell1, shell2, shell4 or shell5 in the above example to use one of the other shell servers.

For now, we recommend that you do not attempt to run graphical programs on lab machines remotely for now.  Learn to use the power of the command line.  If you want to edit files and you can't learn a Linux editor, you could always use scp to grab a copy of the file, edit it locally, and scp it back again. Scp should be part of OpenSSH on Linux; you can get it for Windows as part of the PuTTY suite but there's no pretty GUI to help you use it.

Cygwin is also another Windows possibility as it provides Linux API support in Windows as well as X/SSH capabilities. VcXsrv may also be worth considering.

Simplifying your ssh access

It's a pain to have to type:

ssh USERNAME@shell3.doc.ic.ac.uk

and enter your College password every time.  That’s a mouthful, wouldn’t it be nicer to have to type only:

ssh shell5

and have it not ask who you are?

You can achieve this as follows: On your home PC, edit ~/.ssh/config (that's the path on Linux or Mac OS X, not sure where Windows 10 puts it's per-user ssh client configuration, that's left up to you to find out) using your favourite editor (vim in my case, whatever you like). Note that your .ssh/config file may or may not already exist. Add the following to it:

Host shell1
        User USERNAME
        HostName shell1.doc.ic.ac.uk

Host shell2
        User USERNAME
        HostName shell2.doc.ic.ac.uk

Host shell3
        User USERNAME
        HostName shell3.doc.ic.ac.uk

Host shell4
        User USERNAME
        HostName shell4.doc.ic.ac.uk

Host shell5
        User USERNAME
        HostName shell5.doc.ic.ac.uk

(replacing, as ever, USERNAME with your College username). You may wish to add additional entries, for particular DoC linux machines you regularly ssh to, such as batch1 and batch2, or specific lab machines that you like to use.

You can now ssh shell3 and have it know that you meant ssh USERNAME@shell3.doc.ic.ac.uk.

But it will still ask you for a password or passphrase, each time you ssh in.  That's still quite a pain.

Avoiding ssh asking for a password

(Please note there are several methods of doing this, this guide describes one. As the Perl slogan has it: There’s More Than One Way to Do It.  In particular, another way is to set up a non-passphrase protected ssh key, and trust that, but we regard this as very insecure and cannot recommend it).

First, do the following - unless you have already done it before, of course:

Setting up a passphrase protected SSH RSA key at work, and trusting it

Ok, so ssh into a DoC shell server, giving the password as usual:

ssh shell3

Now, create an ssh key here at DoC by:

ssh-keygen -t rsa

Choose a decently complex passphrase for your new SSH key, different from (often longer than!) your ordinary College password - perhaps more of a phrase than a password. Accept most of the ssh-keygen defaults and enter your fresh passphrase when asked. This creates ~/.ssh/id_rsa (the private key) and ~/.ssh/id_rsa.pub (the public key). Never give the private key to anyone [not even a member of CSG], never email it etc; treat it like a password.

Next, append the contents of your ~/.ssh/id_rsa.pub public key onto the ~/.ssh/authorized_keys2 file.
Be careful now: If no such file exists:

cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys2

or if it already exists,

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys2

This tells the ssh daemon on all DoC linux machines (because they all access your DoC home directory) to accept ssh connections from any machine in the world without asking for a password or phrase – if that remote end can cryptographically demonstrate prior knowledge of the ssh passphrase associated with that key.

Using your DoC passphrase-protected SSH key at home, to avoid ssh asking for a password

Now, you can use your (new) SSH host key frpm home as follows.  These instructions assume that your home PC runs Linux or Mac OS X too, if not, you'll have to work out how to make the equivalent changes yourself:

Back on your home Linux PC, copy the private key from DoC to your home PC:

scp shell3:.ssh/id_rsa ~/.ssh

(Be careful again: make absolutely sure that you’re not overwriting an existing, different and vitally important, id_rsa private key at home).

To use this key automatically, each time you reboot your home PC, start an ssh-agent:

eval `ssh-agent` (for bash users, csh users should add “-c" after "ssh-agent”).

Then add your DoC key into the agent via:

ssh-add ~/.ssh/id_rsa

It will prompt you for your passphrase at this time: enter it. Check that it worked via:

ssh-add -l

Now that you’ve done all this, as long as the ssh-agent is running and has your key in it, you should be able to:

ssh shell3 echo hi

and have it say hi without prompting you for a password or passphrase.

Note that this ssh-agent, loaded with your ssh key, lasts forever, and lets anyone with physical access to your home machine to ssh, as you, to all DoC linux machines.  This is convenient - but dangerous.  For security, at the end of your working day (or when leaving a secure environment with this setup on your laptop), we recommend that you destroy your keys via:

ssh-add -D

This leaves your ssh-agent running, but loaded with no keys.  The next day, simply redo

ssh-add ~/.ssh/id_rsa

and enter your passphrase again.

Using Graphical Remote Applications via ssh

Wherever possible, we strongly recommend that you use ssh in text mode.  The Linux command line is incredibly powerful, learn to use it.

But, once in a while, you may have no choice but to run a graphical (X Windows) application on a DoC linux machine, and then you will want to have it display on your home machine's screen.  Please note that this will use a lot more resources than the command line, so don't even try to do it "just because you want to run some GUI app like Eclipse".  In such cases: install the GUI app on your home machine and use that instead.

But, ok, if you are running some form of X Window server on your home machine (Linux machines do this by default, on Mac OS you can install XQuartz, and it's much more complicated on Windows 10), and enable both 'X11 forwarding' and 'Agent Forwarding' in your ssh client configuration, then you may be able to use the command: 

ssh -Y shell3.doc.ic.ac.uk

and then, on shell3, remotely run a graphical program too, though they may be slow due to unoptimised graphics updates.  In addition, remember that very little GUI software is installed on the shell servers, and that you're not supposed to run any intensive programs on a shell server anyway.  Getting X-forwarding to work over two ssh hops is quite tricky.

We will discuss a much more efficient alternative that works over two ssh hops - called X2go - later, in a separate document.

Advanced - Secure Shell Tunneling

Secure shell not only provides the means for securely interacting with a remote machine, it can also be used to enable remote operation of otherwise independent applications. This concept is called ssh tunnelling, since the ssh connection tunnels the network traffic between a remote machine and a local application.

Under Linux and Mac OS X, the syntax for setting up an ssh tunnel is as follows: 

ssh username@remotehost -L localport:remotehost2:remoteport

Under Windows, use Plink: 

plink username@remotehost -L localport:remotehost2:remoteport

Subject to a successful authentication on remotehost, the above command specifies that connections on the ssh client machine (one's 'home' machine) to port localport should be tunnelled (through the SSH connection to remotehost) to port remoteport on remotehost2.