The Secure Shell (SSH) architecture is a set of protocols and tools based on the ability to enable encrypted remote system login. SSH has largely replaced tools such as TELNET and rsh for most system administrative needs. Most SSH implementations have had good track records, having been responsible for relatively few bugs that lead to widespread anonymous attacks against the protocol or implementation. However, like many systems, SSH is only as safe as its weakest link. Often the weakest link in SSH configurations is the reliance on username and password authentication. When passwords are weak or easily guessed, other underlying SSH benefits are rendered worthless. Unfortunately, many SSH systems are susceptible to brute force password guessing and dictionary attacks. Recent report data from the SANS Internet Storm Center has shown that TCP port 22, most commonly associated with SSH, is one of the most prevalent and widely reported threat vectors seen in the wild today. The Dragon Research Group (DRG), through the use of tools such as the DRG Distro are helping to monitor and combat this threat. This whitepaper is another addition in a series to help educate and foster security best practices within the Internet community. Below we discuss some common SSH password authentication threats and countermeasures.
It might be argued that any reliance on static usernames and passwords for critical system authentication is prone to failure, particularly at scale. The threat of weak passwords and brute force password attacks (aka key search and dictionary attacks) have been discussed in literature for many years. Improving the strength of user selected passwords without a loss in the memorability has also been studied. With so many services and systems making use of usernames and passwords, it is wise to publish a sound password selection strategy like the one outlined by J. Yan, et al. Gene Spafford, has discussed password change policies and other password best practice issues on his blog. More recently, Joe St. Sauver gave a good summary talk about password authentication issues. With the history and threat of password security in mind, it may be tempting to hand-wave the problem away and dictate a different, or at least enhanced, authentication system. We will cover some SSH-specific strategies to consider below, but for many, password authentication is not likely to be completely eliminated for years to come, if ever. Here are some things you can consider if password authentication is required on your systems:
In this section we will use OpenSSH in our examples and to describe a set of useful configuration and policy options you may wish to examine further. Consult the specific documentation for other implementations about similar features and usage instructions. The following is a list of OpenSSH server daemon configuration options that may be worth adjusting from their default settings. Each is briefly described. Options not listed are generally presumed to be set to reasonable defaults as listed in the OpenSSH sshd_config manual page.
In addition to the server configuration options listed above, some sites limit accessibility to the server daemon from a range of source addresses such as a VPN. Doing this may limit the exposure, but it alone does not completely eliminate the threat of password attacks. We have seen some VPN credentials compromised. If an organization is not addressing issues throughout the infrastructure and is not carefully monitoring access and usage except at a few well known "choke" points, more targeted attacks can be devastating in those environments. Other options might be to limit accessibility to SSH servers during specific times of the day or to require a predetermined series of events before the SSH process will accept a connection, such as through the use of port knocking.
Most SSH servers support the use of public key authentication as it is typically enabled by default. However, users and often even many administrators choose to use password authentication if available. We encourage the use of public key authentication. Jeremy Mates and David McNett have each written walk-through instructions for those new to the process.
In this section we briefly discuss four general areas that offer additional insight and ability to alert an administrator about SSH password authentication attacks. Each offers their own unique perspective on attacks. Not all sites need to make full use of each, but having multiple views of your landscape improves your ability to detect and respond. Each category consists of an approach that has been widely implemented in other networks. Other systems and processes are possible, including many packages available from commercial vendors. However, in each section below, use of these subsystems can generally be done with little to no additional cost except for the time required for setup and management.
Netflow was a concept originally developed by Cisco to summarize network traffic traversing routers. Today it is widely implemented in many vendor hardware platforms as well as a general tool to summarize network traffic by other means. There are a number of tools available for Netflow collected data. Using them for SSH attack monitoring is relatively straightforward since generally SSH attacks occur over TCP port 22. Netflow tools are good at monitoring and alerting on well known traffic characteristics such as this. Furthermore, you can use Netflow to monitor for TCP port 22 traffic from your hosts since a successful brute force attack may result in the compromised system becoming part of a larger brute force attack network.
By default, many SSH server daemons will perform a series of DNS-based checks on a connecting IP address. This consists first of a DNS PTR type query for the connecting IP address (e.g. 22.214.171.124.in-addr.arpa.). The name associated with the answer is then used in an A or AAAA query to see if that answer matches the IP address of the connecting system. If not, typically a warning message is logged. If your SSH server performs this action, there are two places to watch for anomalies. First, you can watch your SSH server logs for warning messages when this DNS query check fails. Consult your SSH implementation documentation for details. In addition, you could monitor DNS queries at the recursive name server used by the SSH server. If you see an SSH server making unusual PTR queries for an IP address you do not recognize, or perhaps making an unusual number of PTR queries, this may be a telltale sign that the SSH server is undergoing an SSH attack.
Most SSH implementations will log some minimal information about login attempts. Typically systems will log to the local disk, but to ensure survivability of those logs, you should have SSH server logs sent to a collector that can withstand attack even when other systems may be penetrated. If an SSH attack is successful, local logs may be one of the first bits of information to be wiped out by an attacker. Since syslog operates in real-time, an offline collector will help ensure you have some sort of audit trail of events leading up to an attack. Be aware that some login monitoring tools will not display some active SSH sessions. For instance, most login tools such as w, who, and last will not show SFTP connections. Be sure to examine the SSH logs and not just the lastlog entries. Ideally if you are able to understand login patterns for all users on the systems, you may be able to alert on anomalous patterns. For example, if a user normally logs in from a source address of 192.0.2.1, but suddenly is logging in from an IPv6 address 2001:DB8::1 then you may wish to investigate.
There are a number of freely available sources of alert data related to SSH authentication attacks. Typically these are, minimally, IP addresses that have been identified as being associated with malicious SSH activity in the recent past. As with any third party provided data, you must use the data wisely and be wary of relying on it if you are going to use it to actively affect traffic. The Dragon Research Group provides its own sshpwauth report. The DenyHosts project is a cooperative among anonymous parties who share IP addresses that each identifies as malicious. The SSH blacklist (sshbl.org) is another source of IP addresses they identify as malicious.
There are a number of tools available that aim to help administrators both detect and respond to SSH password authentication attacks. Most fall into one of two types; alert-only or alert-and-respond. We list a number of the tools we are aware of below. Note, we avoid including generic log parsing tools, but many generic log parsers also support SSH-related events including password authentication issues.
SSH password authentication attacks exist and are a popular form of malicious activity because they yield results for an attacker. Take away password authentication mechanisms and you only eliminate one vector in an architecture where multiple avenues of attack are possible. The aim for any administrator should be to improve their monitoring, prevention, detection and response capabilities. The aim of this whitepaper was to help shed some light on SSH password authentication threats and countermeasures as a means to help improve your security posture in one or more of those four areas. If we can be of anymore help with actionable intelligence, please contact us at email@example.com.