Every developer needs access to some servers for example to check the application logs.

Usually, this is done using public-private key encryption where each developer generates their own public-private key pair. The public keys of each developer are added to the authorized_keys file on each server they should have access to.

Painful manual changes

So far so good. However, what happens when one developer leaves the company?

In that case, the public keys of that developer should be removed from all servers. This could be quite a bit of work depending on how many servers they had access to. And even worse, if it’s done manually, there is quite some risk that the key is still forgotten on some server, so the access remains open.

Alternative solutions

There are some commercial and open source solutions out there that want to help out with this problem. The basic idea is that you add and maintain the keys and the access lists on that service and when you remove a key, they will remove it from all your servers.

Sounds good, but it has one very big disadvantage: it’s a potential single source of failure. If someone captures access to that service, they can gain access to all your servers. And if you lose access to that service, you also lose access to all your servers in the worst case.

The solution: signing keys

When I was facing this problem, I asked on HackerNews how others are tackling this problem.

There were some great suggestions and insights from the community and the best solution to the problem seems to be the signing of keys which I will present to you here in detail.