- Only public keys should be on the various servers (required for external facing systems; ex: app systems, proxy servers, sc9-admin1), and the critical private keys should be on an internal 'admin' type box (such as a grid server, puppet master, or other trusted host system).
- Our laptops must not be configured with a key such that it can automatically connect to an externally available server. This is required, due to the security around our products, since we're allowed to take our laptops outside of the office, and they occasionally get stolen.
- Keys should not be automatically invoked. when generating keys, do not take the default filename (which will automatically get used for all ssh related connections), but use a different filename which includes the name of the system it was generated on, and a name for the function the key will be used for (ex; id_dsa-app1_named for syncing DNS/named configs to the various nameservers, 'named' is the user that the process is running under).
- When keys are used, they should be locked down to access from specific systems (authorized_keys file), and disallow any port forwarding that can be initiated by a user or hacker, AND if possible, locked down to a specific command, ex;
# cat authorized_keys # remote commands, no ssh (no pty) but works with scp/sftp from="126.96.36.199,188.8.131.52",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa AAAAB3NzaC1... OR # remote commands & ssh (w/pty) from="184.108.40.206,220.127.116.11",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa AAAAB3NzaC1... OR # a single remote command (you would need an entry for each command allowed, or have one key per command) command="/bin/myscript",from="18.104.22.168,22.214.171.124",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa AAAAB3NzaC1... OR # a single remote scp copy (you would need an entry for each directory allowed, or have one key per directory) command="/usr/bin/scp -p -t /home/appuser/data",from="126.96.36.199,188.8.131.52",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa AAAAB3NzaC1... OR # a single remote rsync copy (you would need an entry for each directory allowed, or have one key per directory) command="/usr/bin/rsync --server -logDtpre.iLs --partial . /home/appuser/data",from="184.108.40.206,220.127.116.11",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa AAAAB3NzaC1...
- Keys should not be under a normal application user account, if possible..
The appuser contains the perms for the app (for the app running as appuser), plus this acct is used generally for deployments and making any changes to the app. Anyone using this account, or breaching this acct, would have access to all the other envs/servers, if such a key existed.
The applog user is used to pull logs out of an appuser log dir and the transfer keys are owned by applog user, appuser does not have access/perms to the key to get to all the other envs. In this particular case, the group for the appuser home dir was set to applog, and the group perms were set to read-only (r-x for dir, r-- for files). The applog user is not an actual account (cannot login directly, like the appuser acct cannot login directly), but is used by an automated script that is pulling the logs. To further define 'pulling', only public keys exist on the individual servers (must be this way (w/a public key), if system is external facing), and the logs are copied from a central/internal system (appadmin1).
Key Setup Example / Trusted Host Setup Example
example for hostname gwikip-nhpm1
1. generate key on an internal/admin system/trusted host system (not allowed on systems with public IP's), naming convention is id_rsa-`hostname -s`_`whoami` OR (optional) id_rsa-`hostname -s`_`whoami`-keyfunctionORscriptname
Note: as mentioned above; default path/default keys MUST NOT BE USED (keys must not be automatically picked up/used by ssh), using a particular key must be specified with ssh/scp/sftp/rsync options, hostname field in the naming convention can be `hostname -s` or another abbreviated representation of the host (enough to distinguish that particular host)
Enter file in which to save the key (/root/.ssh/id_rsa): /root/.ssh/id_rsa-nhpm1_root
2. remove domain info (leave `hostname -s` at end of key but remove the domain, .prd.example.com, .stg.example.net, etc.) and lock down the pub key (the key that gets pushed out to the various systems)
Note: check the list of possible lockdown strings above, use of the non-shell (non-tty) lockdown is preferred, but may not work for your use case (scenario)
# vi /root/.ssh/id_rsa-nhpm1_root.pub CHANGE (example) ...BtJ9bww== email@example.com TO ...BtJ9bww== ice@gwikip-nhpm1 AND add from="18.104.22.168",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa AAAAB3NzaC1... OR from="22.214.171.124",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa AAAAB3NzaC1...
3. setup the .pub key on the other systems which this internal box needs to access 3a. copy the .pub key to another system
# scp -pP 222 /root/.ssh/id_rsa-nhpm1_root.pub yourlogin@glis1p-nhpm1r1:/tmp/
3b. connect to that system to setup the key
# ssh -p 222 yourlogin@gwikip-nhpm1r1
3c. become the user you are setting up the key for (usually an app user), in this case; root
# su - OR # sudo su - webapp
3d. make sure the .ssh dir exists, if not, you can have it automatically generated with the proper permissions for SSH
Note: answer 'yes' but don't login (simply answering 'yes' creates the dir and sets the perms)
# ssh -p 222 localhost
3e. copy the .pub key into the .ssh dir and copy it into/append it to the authorized_keys file
# cat /tmp/id_rsa-nhpm1_root.pub >~/.ssh/id_rsa-nhpm1_root.pub # cp ~/.ssh/id_rsa-nhpm1_root.pub ~/.ssh/authorized_keys
4. On Trusted Host systems for an application, we normally add all application users (wikiapp, webapp, hornetq, oracle, etc.) to the trusted host, and setup keys for each user to be able to connect to the nodes/systems which have one of those accounts. For general accounts like root and monitoring users (monupdate), the keys would be distributed to all systems in that application env (Beagle Production, Squawk QA, etc.). !th.jpg|thumbnail!
Locking Down Specific Users
Note: this is just some additional info on how to do something like this, this is not currently used on any of our systems
User Fred cannot access anything outside of his home directory (cannot see /tmp/, /etc/, etc..)
Match User fred.jones ChrootDirectory %h
Note: %h refers to the home directory of the user
Trusted Host Reverse Copy (forall suite of scripts)
# forall p-all 'cp -p /etc/postfix/main.cf /tmp/main.cf-$(hostname -s)' # forcpr p-all /tmp/main.cf-* /tmp/
If you are using a windows system, you should be able to run putty (Google for putty)
If you are using a Mac (or Linux) system, you can use command line ssh if you open a Terminal window. Syntax would be something like:
# ssh -p 222 user@hostname # ssh -p 222 firstname.lastname@example.org