Selfhosted

I would switch to certificate based SSH authentication.

All the server keys gets signed by your CA, all clients also gets signed by your CA. Everyone implicitly trust eachother though the CA and it's as safe as regular SSH keys.

You can also sign short lived client keys if you want to make revocations easier, the servers don't care because now all it cares is that it's a valid cert issues by the CA, which can be done entirely offline!

HashiCorp Vault can also help managing the above, but it's also pretty easy to do manually.

I've been using https://github.com/warp-tech/warpgate for essentially this purpose. It does kind of put all of your eggs in one basket so don't expose this to the Internet and probably keep at least one other machine that has all the keys. I haven't had any catastrophic issues so far other than my host going down (unrelated to this tool).

You could use an LDAP and/or Kerberos solution to centralize user management. Alternatively you could use ansible

Terrible idea of the day: You could use something like NFS and map the drive on all clients. On that drive you can have the latest keys then use symlinking to update, etc.

Something like puppet, chef, ansible are likely better choices.

You could try SSH certificates using something like https://smallstep.com/sso-ssh/ - essentially you delegate validation of your public key to a IDP, which your servers are configured to trust.

The other approach would be something like ansible or puppet to deploy trusted keys to all servers

I quite like Tailscale SSH for this, but I don't have as many machines, so not sure how it will scale. You can definitely assign roles here to allow/deny SSH between hosts in your fleet though.

Are you initiating SSH connections from all these hosts?

If you just need to SSH to these hosts, use a single key and copy the public key only to the hosts you need to connect to. If you don't want to copy the pubkeys to target hosts, use LDAP + SSSD or certificates.

Then, if you do need to initiate connections from these hosts and use an SSH agent you can forward your agent and SSH to another host

client> ssh -A host1
host1> ssh host2
host2>

client> ssh -A host1
host1> ssh -A host2
host2> ssh -A host3
host3> 

This is one of the jobs of OpenLDAP.

You could use ldap with OpenLdap, Keycloak, freeipa, etc to set ssh keys for users.

If you want something simpler, you could use Ansible (or another cm) or just have a startup script that downloads the authorized keys file from GitHub or wherever you can store it.

And if you want something less simple, hashicorp Vault supports dynamic ssh keys using certificates.

Sometimes the obvious solution is the way to go.

Your idea sounds good to go ahead and publish your pubkey(s) to fully public URL you control and can memorize.

Then you can stash or memorize the curl command needed to grab it (them) and authorize something to it (them).

A lot of more complicated solutions are just fancy ways to safely nice private keys around.

For my private keys, I prefer to generate a new one for each use case, and throw them out when I'm done with them. That way I don't need a solution to move, share or store them.

Ansible

I use ansible for that: https://docs.ansible.com/ansible/latest/collections/ansible/posix/authorized_key_module.html

Keys stored alongside my playbook in a git repository.

all I know is that on NixOS you can declare the authorized keys for each user in the config