SSH: RSA keys, and ssh-agent for SSH keys and their passwords management

By | 12/01/2019
 

During keyring configuration for the Nextcloud client (see the Linux: the Nextcloud client, qtkeychain and the “The name org.freedesktop.secrets was not provided by any .service files” error post) – I decided to clean up the mess in my SSH keys, as I have a lot of them and sometimes authentication became just pain.

In general to make this simpler one can use system-wide storage like gnome-keyring or KeeyPassXC, but we will speak about them in the next post.

Today, let’s discuss ssh-agent and how to use it to manage password-protected RSA keys for SSH authentication without such a backends.

Examples below performed on an Arch Linux installation with some additional tests Manjaro Linux with Budgie DE.

ssh-agent

ssh-agent is intended to manage a user’s SSH keys and their passwords to avoid the necessity to enter a key’s password each time you need to log in a remote host using such a key for your authentication.

Running the agent

Just perform:

[simterm]

$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-dMDE5mED77tM/agent.436347; export SSH_AUTH_SOCK;
SSH_AGENT_PID=436348; export SSH_AGENT_PID;
echo Agent pid 436348;

[/simterm]

For clients, such as ssh-client or git, they need to know the following variables:

  • SSH_AGENT_PID: a started ssh-agent PID, that will be sued for example to kill it with ssh-agent -k
  • SSH_AUTH_SOCK: a path to a UNIX socket file which will be used to communicate to the ssh-agent from clients (ssh, git, etc)

To run an agent without displaying these variables and to apply them – do the next:

[simterm]

$ eval $(ssh-agent) > /dev/null

[/simterm]

There are few various ways to run the agent, we will take a closer look at the Running ssh-agent with multitype terminals part.

Examples

Let’s take an overview of ssh-agent basic usage.

A key generation

Create a new key:

[simterm]

$ ssh-keygen -t rsa -b 2048 -f /home/setevoy/.ssh/test-key -C "Testing key" -P pass 
Generating public/private rsa key pair. 
Your identification has been saved in /home/setevoy/.ssh/test-key. 
Your public key has been saved in /home/setevoy/.ssh/test-key.pub. 
The key fingerprint is: 
SHA256:pTyrGtk1hnNHB6b8ilp5jRe1+K4KrLHg50yUGilApLY Testing key 
The key's randomart image is: 
+---[RSA 2048]----+ 
|.o        o      | 
|o      . o .     | 
|o.      o o o    | 
|o .. . o = + .   | 
|.Eo o o S = .    | 
| . + + B O o     | 
|  o = B = o .    | 
| . +.B + . .     | 
|  .oB.. .....    | 
+----[SHA256]-----+

[/simterm]

Options here:

  • -t: type, RSA
  • -b: a key’s length in bits (by default 3072 for RSA)
  • -f: a path to the key’s file (by default ~/.ssh/id_rsa)
  • -C: a comment for the key (by default username@hostname)
  • -P: a key’s password
Checking SSH-key’s password

To check a password for a key you can use  thessh-keygen with -y to display information about this key, and this will ask you to enter this key’s password:

[simterm]

$ ssh-keygen -y -f /home/setevoy/.ssh/test-key
Enter passphrase:
ssh-rsa AAAAB***gud2vedL/V Testing key

[/simterm]

ssh-copy-id – copy a key to a remote host

You can copy a key manually, by getting its public part from the test-key.pub file:

[simterm]

$ cat .ssh/test-key.pub
ssh-rsa AAAAB***gud2vedL/V Testing key

[/simterm]

And by adding it to the ~/.ssh/authorized_keys on a target host.

Another way, the recommended one, is to use an ssh-copy-id utility which will do the same but also will keep an eye on folders/files permissions – the most frequent problem during SSH RSA-based authentification:

[simterm]

$ ssh-copy-id -i /home/setevoy/.ssh/test-key [email protected]
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/setevoy/.ssh/test-key.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

[/simterm]

Now you can log in using this key:

[simterm]

$ ssh [email protected] -i .ssh/test-key
Enter passphrase for key '.ssh/test-key':
Linux rtfm-do-production 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19) x86_64
...
setevoy@rtfm-do-production:~$

[/simterm]

ssh-add

Okay, so now we do have a password-protected RSA key for SSH authentication.

But during each SSH-login, you’ll have to enter its password again and again and this will be a real pain when using a lot of connections and keys.

To avoid this issue – add a key to the ssh-agent using ssh-add.

Check if it is running:

[simterm]

$ ps aux | grep ssh-agent
setevoy     1322  0.0  0.0   5796   456 ?        Ss   Nov30   0:00 ssh-agent -s
setevoy     1324  0.0  0.0   5796  2160 ?        Ss   Nov30   0:00 ssh-agent -s
...

[/simterm]

Could not open a connection to your authentication agent

The most common problem is when ssh-add is not able to connect to an agent:

[simterm]

$ ssh-add
Could not open a connection to your authentication agent.

[/simterm]

At first – check if it’s PID is set from the SSH_AGENT_PID, or by checking the $SSH_AUTH_SOCK variable as all communication is gone via the socket-file specified by this variable:

[simterm]

$ test -z $SSH_AGENT_PID; echo $?
0

[/simterm]

Here is it empty, because thessh-agent was started in another terminal instance (we will speak soon how to handle it).

For now – kill all already running instances:

[simterm]

$ killall ssh-agent

[/simterm]

And run an agent’s instance anew:

[simterm]

$ eval $(ssh-agent -s)
Agent pid 452333

[/simterm]

We are using the-s option as not everybody will do the steps above from the exactly bash shell and eval to apply the strings from the agent’s output (export SSH_AUTH_SOCK).

Check again:

[simterm]

$ test -z $SSH_AGENT_PID; echo $?
1

[/simterm]

And ssh-add:

[simterm]

$ ssh-add -l
The agent has no identities.

[/simterm]

All done here.

Adding a key

Run:

[simterm]

$ ssh-add /home/setevoy/.ssh/test-key
Enter passphrase for /home/setevoy/.ssh/test-key:
Identity added: /home/setevoy/.ssh/test-key (Testing key)

[/simterm]

Checking keys

Use the -l option to check which keys are already loaded to an agent’s instance:

[simterm]

$ ssh-add -l
2048 SHA256:pTyrGtk1hnNHB6b8ilp5jRe1+K4KrLHg50yUGilApLY Testing key (RSA)

[/simterm]

Deleting key(s)

Use -d to delete one key:

[simterm]

$ ssh-add -d .ssh/test-key
Identity removed: .ssh/test-key (Testing key)

[/simterm]

And -D to delete all keys at once:

[simterm]

$ ssh-add -D
All identities removed.

[/simterm]

Automatically adding keys to ssh-agent

To make ssh (and git for example) adding used keys to an ssh-agent without the necessity to run ssh-add manually you can add the AddKeysToAgent parameter to theв ~/.ssh/config and specify one of the following options – yes, confirm or ask (см. SSH_ASKPASS):

[simterm]

$ head -1 .ssh/config
AddKeysToAgent yes

[/simterm]

Let’s check – there is nothing added at this moment:

[simterm]

$ ssh-add -l
The agent has no identities.

[/simterm]

Make a connection, enter a key’s password:

[simterm]

$ ssh -i .ssh/test-key [email protected]
Enter passphrase for key '.ssh/test-key':
...
setevoy@rtfm-do-production:~$

[/simterm]

Disconnect, and check keys in the agent now:

[simterm]

setevoy@rtfm-do-production:~$ logout
Connection to rtfm.co.ua closed.
$ ssh-add -l
2048 SHA256:pTyrGtk1hnNHB6b8ilp5jRe1+K4KrLHg50yUGilApLY Testing key (RSA)

[/simterm]

On the next connection – the ssh client will use the key from the agent and will not ask you for the key’s password again:

[simterm]

$ ssh -i .ssh/test-key [email protected]
...
setevoy@rtfm-do-production:~$

[/simterm]

Running ssh-agent with multitype terminals

Another big question is what to do when you have few bash-sessions, for example in various terminals’ tabs, as it will not has the $SSH_AUTH_SOCK variable set and an ssh client will not be able to communicate with an already running ssh-agent instance.

I.e. when you’ll run ssh-add in a new terminal – you’ll see the already mentioned “Could not open a connection to your authentication agent” error:

[simterm]

$ ssh-add -l
Could not open a connection to your authentication agent.

[/simterm]

~/.bashrc

There is a few ways to make the initialization of the variables during new bash session initialization, for example, you can add the following to your  ~/.bashrc:

if [ -z "$SSH_AUTH_SOCK" ] ; then
  eval `ssh-agent -s`
  ssh-add /home/setevoy/.ssh/test-key
fi

But in this case, each bash-sessions will has its own ssh-agent running, which is not a problem but maybe not what you’d like to have.

Another way could be the following code added to the ~/.bashrc:

ssh-add -l &>/dev/null
if [ "$?" == 2 ]; then
  test -r ~/.ssh-agent-env && \
    eval "$(<~/.ssh-agent-env)" >/dev/null

  ssh-add -l &>/dev/null
  if [ "$?" == 2 ]; then
    (umask 066; ssh-agent > ~/.ssh-agent-env)
    eval "$(<~/.ssh-agent-env)" >/dev/null
    ssh-add /home/setevoy/.ssh/test-key
  fi
fi

Here (see response codes in the ssh-agent documentation):

  1. try to execute ssh-add -l, and redirect output to the /dev/null
  2. check returned code of the previous command:
    1. if it is == 2 (error connect to an agent):
      1. check if ~/.ssh-agent-env is present and available for reading,  read it and pass its output to the bash
      2. retry ssh-add -l
      3. if code still 2:
        1. create the ~/.ssh-agent-env file with the 660 permissions (read-write for an owner only)
        2. start ssh-agent and redirects its output into the .ssh-agent-env file
        3. read the .ssh-agent-env content and pass it via a pipe to the bash
        4. run ssh-add /home/setevoy/.ssh/test-key

Not a bad solution, and in this way all our sessions will use the same agent, although some guides suggesting to have different agents for personal and work usage

systemd

Another solution could be to create a dedicated systemd service by adding a unit  file and by running ssh-agent as a systemd service, see the Arch Wiki for the details.

Create a directory if not added yet:

[simterm]

$ mkdir -p .config/systemd/user/

[/simterm]

And create a ~/.config/systemd/user/ssh-agent.service file there:

[Unit]
Description=SSH key agent

[Service]
Type=simple
Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket
ExecStart=/usr/bin/ssh-agent -D -a $SSH_AUTH_SOCK

[Install]
WantedBy=default.target

Next, Wiki told about the ~/.pam_environment file for variables, but in my current case I have Openbox and usually set variables via .config/openbox/autostart file:

[simterm]

$ head -2 .config/openbox/autostart
# ssh-agent.service
SSH_AUTH_SOCK="${XDG_RUNTIME_DIR}/ssh-agent.socket"

[/simterm]

By the way, recalled about about such a thing as setting default values in BASH – BASH: переменные — передача значений по-умолчанию ${var:-defaultvalue}, замена значений — ${var:+alternatevalue} и сообщений — ${var:?message} (Rus)

Now, stop all agents running:

[simterm]

$ killall ssh-agent

[/simterm]

Check the $XDG_RUNTIME_DIR variable value:

[simterm]

$ echo $XDG_RUNTIME_DIR
/run/user/1000

[/simterm]

For now, set the $SSH_AUTH_SOCK variable manually:

[simterm]

$ SSH_AUTH_SOCK="${XDG_RUNTIME_DIR}/ssh-agent.socket"

[/simterm]

And run an agent via systemctl --user:

[simterm]

$ systemctl --user start ssh-agent

[/simterm]

Check it:

[simterm]

$ systemctl --user status ssh-agent
● ssh-agent.service - SSH key agent
  Loaded: loaded (/home/setevoy/.config/systemd/user/ssh-agent.service; disabled; vendor preset: enabled)
  Active: active (running) since Sun 2019-12-01 09:15:18 EET; 2s ago 
Main PID: 497687 (ssh-agent) 
  CGroup: /user.slice/user-1000.slice/[email protected]/ssh-agent.service 
          └─497687 /usr/bin/ssh-agent -D -a /run/user/1000/ssh-agent.socket 

Dec 01 09:15:18 setevoy-arch-pc systemd[670]: Started SSH key agent. 
Dec 01 09:15:19 setevoy-arch-pc ssh-agent[497687]: SSH_AUTH_SOCK=/run/user/1000/ssh-agent.socket; export SSH_AUTH_SOCK; 
Dec 01 09:15:19 setevoy-arch-pc ssh-agent[497687]: echo Agent pid 497687;

[/simterm]

A socket’s variable:

[simterm]

$ echo $SSH_AUTH_SOCK
/run/user/1000/ssh-agent.socket

[/simterm]

And try ssh-add:

[simterm]

$ ssh-add -l
The agent has no identities.

[/simterm]

“It works!” (c)

You can add to autostart now:

[simterm]

$ systemctl --user enable ssh-agent
Created symlink /home/setevoy/.config/systemd/user/default.target.wants/ssh-agent.service → /home/setevoy/.config/systemd/user/ssh-agent.service.

[/simterm]

~/.xinitrc

One more way you can use  is by adding the agent’s start to the~/.xinitrc.

In this case, when you’ll execute the startx (for example, as in my case, when I have no any login manager, and X.Org is started manually by entering the startx in the console) – at first, an agent will be started and the – an Openbox session, see the documentation:

[simterm]

$ cat ~/.xinitrc
eval $(ssh-agent) &
exec openbox-session

[/simterm]

Also, as already mentioned at the very beginning of this post, there other implementations for the key’s backends that can be used alongside or instead of the ssh-agent – kind of “wrappers” that will be or “proxy” requests from an ssh client to an ssh-agent‘s instance, or will fully replace the ssh-agent itself and will store keys and passwords themselves, but we will speak about them in a following post(s?)..