Fuzzing with American Fuzzy Lop (AFL)

In a previous entry we gave a brief introduction to the concept of fuzzing and why we use it. In this entry we’ll guide you through using a fuzzer on Linux to help identify bugs and vulnerabilities in Linux’s main archiving application “tar”.

A little about the fuzzer

”American Fuzzy Lop” as well as being a variety of rabbit, is a well-designed and versatile file fuzzer. The software is built and maintained by Michal Zalewski and offered freely under an Apache 2.0 licence. As a file fuzzer “American Fuzzy Lop” is targeting issues in data parsing, particularly in binary file formats which are prone to problematic issues and strong candidates for fuzzing.

AFL has two main components, an instrumentation suite that can be used to get our target application ready for fuzzing, and the fuzzer itself which controls mutation of the input files, execution and monitoring of the target.

The instrumentation suite allows for binaries to be prepared for fuzzing in one of two ways. The preferred manner is to rebuild the target using the AFL compiler which is stated to be “a drop-in replacement for gcc or clang”. This allows for accurate injection of code that will allow the fuzzer to trace the path of execution that each file takes when run through the target. Where the source code isn’t available, however, AFL also has experimental support for instrumentation of the native binaries though QEMU. This is a more complex topic and during this tutorial we’ll focus on instrumentation via source code.

The fuzzer itself is the real “brains” of the utility. It takes an initial set of samples to benchmark what is considered “normal” behaviour, reduce the input complexity and create the “input queue”. It then begins fuzzing by taking an item from the input queue (1), using targeted algorithms to mutate the input (2), and then uses these derived children as input to the target application (3). A monitor observes each execution (4) and where new behaviours are observed the derived input which caused them is added back into the “input queue” so it too can be further mutated (5). Note that different behaviour is not necessarily “bad” just different. This cumulative mutation helps the fuzzer fully exercise by getting better “coverage”.

Figure 1 - The Fuzzer

Figure 1 – The Fuzzer

Downloading and building AFL

We’ll be working with version AFL 1.56b (the latest version available at this time) on a vanilla Ubuntu 14.10 virtual machine. The commands might differ slightly if you are using a different version or operating system but will hopefully be similar. The fuzzer is built from source code. Our first task is to get and build this code. This can done in the following manner:

What the script actually does:

1. Gets the “American Fuzzy Lop” source code and extract it on your local machine.

[cpp]
user@ubuntu:~$ wget -q <a href="http://lcamtuf.coredump.cx/afl/releases/afl-1.56b.tgz">http://lcamtuf.coredump.cx/afl/releases/afl-1.56b.tgz</a>

user@ubuntu:~$ tar zxf afl-1.56b.tgz

[/cpp]

2. Builds the fuzzer from the source code.

[cpp]
user@ubuntu:~$ cd afl-1.56b

user@ubuntu:~/afl-1.56b$ make -s

[*] Checking for the ability to compile x86 code…

[+] Everything seems to be working, ready to compile.

[*] Testing the CC wrapper and instrumentation output…

[+] All right, the instrumentation seems to be working!

[+] All done! Be sure to review README – it’s pretty short and useful.

NOTE: If you can read this, your terminal probably uses white background.

This will make the UI hard to read. See docs/status_screen.txt for advice.

[/cpp]

Assuming everything was successful, you should now have all the tools built in the directory. These can be used to fuzz our target application.

 

Instrumenting a binary from source

We are going to test the linux “tar” archive utility. To do this we’ll start by building a version of the target from source using the AFL compiler so that it can be instrumented. The instrumentation process injects code that can communicate with American Fuzzy Lops monitoring system allowing the fuzzer to accurately observe the path of execution through the target.

1. Start by getting the tar source code

[cpp]
user@ubuntu:~/afl-1.56b$ cd

user@ubuntu:~$ wget -q http://ftp.gnu.org/gnu/tar/tar-latest.tar.gz

user@ubuntu:~$ tar zxf tar-latest.tar.gz

[/cpp]

2. Next we need to build our target using AFL’s compiler. The correct way to do this can be complex and AFL’s README covers this topic in greater detail. For tar we can do the following:

[cpp]
user@ubuntu:~$ cd tar-1.28

user@ubuntu:~/tar-1.28$ CC=~/afl-1.56b/afl-gcc ./configure

[Output truncated for this blog article]

user@ubuntu:~/tar-1.28$ make

[Output truncated for this blog article]

[/cpp]

3. Assuming the build is successful this will result in an instrumented version of tar being built in ‘src’ as shown below:

[cpp]
user@ubuntu:~/tar-1.28$ file src/tar

src/tar: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=28d3095ca5633069721e79cb09bba7ff6d6110b7, not stripped

[/cpp]

Fuzzing commences

Finally, to the actual fuzzing. First just a little more preparation. American fuzzy lop requires two directories one for the inputs, and a working directory that’ll contain current state and output information, the exact structure of which we’ll cover later. Let’s start by creating an input and output directory and the initial sample file for the fuzzer to work with (if you want to make more, that’s great). We’ll also create a third directory to collect any output from tar.

[cpp]
user@ubuntu:~/tar-1.28$ cd

user@ubuntu:~$ mkdir fuzz-state fuzz-input fuzz-garbage

user@ubuntu:~$ cd fuzz-input/

user@ubuntu:~/fuzz-input$ echo "Hello World" > file1

user@ubuntu:~/fuzz-input$ echo "Hello Fuzzer" > file2

user@ubuntu:~/fuzz-input$ tar cfJ sample1.tar.xz *

user@ubuntu:~/fuzz-input$ rm file*

[/cpp]

The above commands create three directories, one for input, one for output and one for any garbage thrown out by our instrumented version of tar. The input directory will contain a single xz archive containing two simple files. The final piece of preparation involves preventing core dump notifications being passed to apport, as this can hinder the fuzzer if the application actually crashes.

[cpp]
user@ubuntu:~$ echo core | sudo tee /proc/sys/kernel/core_pattern

[/cpp]

To start fuzzing:

[cpp]
user@ubuntu:~$ ./afl-1.56b/afl-fuzz -i fuzz-input/ -o fuzz-state/ -t 10000 ~/tar-1.28/src/tar xfJ @@ -C fuzz-garbage/ –force-local

[/cpp]

That’s a bit of a mouthful, lets break it down a little.

1.  Run the fuzzing tool: ./afl-1.56b/afl-fuzz

a. Source input samples from ‘fuzz-input’:  -i fuzz-input/
b. Use the following directory for state and output:  -o fuzz-state/
c.
Timeout of 10 seconds (usually this isn’t needed):  -t 10000
    d. Everything else is the command line to fuzz, in our case:

2.  Run our instrumented “tar” binary: – ~/tar-1.28/src/tar

a. Extract the contents of the following XZ archive file:  xfJ
b. @@ is replaced by the fuzzer with a fuzzed file name:  @@
c. Write output to fuzz-garbage:  -C fuzz-garbage/
d. Required due to the naming convention used by AFL: –force-local

Interpreting the output

The following is a brief overview of the UI. This will be displayed whilst the fuzzer is running.

Figure 2 - American Fuzzy Lop

Figure 2 – American Fuzzy Lop

 

  1. Process timing: How long things have been running, and how long since we last saw results.
  2. Cycle progress: How far through the input queue we are.
  3. Stage progress: How far through mutating the current file we are.
  4. Details about which methods of mutation are yielding the most new behaviours and results.
  5. General overview of the fuzzer’s current state.
  6. Coverage metrics (how much of the target we have found paths through).
  7. Information about the number of execution paths, exceptions and hangs we have found.
  8. Information about the execution paths we have found.

For each unique hang and error, the fuzzer will create a symbolic link to the file that caused the behaviour in the output directory at “fuzz-state/hangs” and “fuzz-state/crashes” respectively. The symbolic link will point to the file in the input queue so beware modifying or moving these directly when the fuzzer is running. Another interesting metric is “variable” (in “path geometry”, number 8 above) which indicates paths that are giving variable or unpredictable behaviour. This could be by design, via a call to rand() for example, for an indicator of uninitialised memory. Files with variable behaviour are symbolic linked from “fuzz-state/queue/.state/variable_behavior”

Fuzzing in less than 3 minutes?

Feeling lazy? From a clean install of Ubuntu 14.10 you could be fuzzing in under 3 minutes.

[cpp]
user@ubuntu:~$ echo core | sudo tee /proc/sys/kernel/core_pattern

core

user@ubuntu:~$ wget -q -O – http://nettitude.com/scripts/fuzz-xz.sh | bash

[/cpp]

 

 

To contact Nettitude’s editor, please email media@nettitude.com.

VMware Multiple Products – Privilege Escalation

CVE-2015-3650

Introduction

This article summarises the findings and the impact of a vulnerability that we recently discovered in three major VMware Windows products. The affected products are ‘VMware Workstation’, ‘Horizon Client’ (with Local Mode Option), and ‘Player’.

Successful exploitation of this vulnerability allows a local attacker to execute code in the context of other logged-on user accounts in the same host.

The resulting impact can be either horizontal or vertical privilege escalation depending on the account from which the attack is launched and the targeted user account that uses vulnerable VMware software for Windows.

Securable Objects

As their name dictates, these are Windows objects that live in the kernel address space and have security descriptors so that the operating system can control who can access what and at what level.

These descriptors are structures that contain security information associated with those objects. Normally, a security descriptor should have a valid pointer to discretionary access control list (DACL) which can be comprised by one or more access control entries (ACEs) as shown in Figure 1.

Figure 1. Discretionary Access Control List

Figure 1. Discretionary Access Control List

An ACE describes which groups and individual users can or cannot access a securable object, and the type of access such as Read, Write, Execute, Read|Write etc…

For most of these securable objects, we can set our own security descriptors when we create them in order to customise access to them.

As shown in Figure 1, we want to allow full access to the users that belong to the ‘A’ group (Write + Read|Execute for Everyone group), and Read|Execute to everyone else, but we want to exclude Andrew from being able to access that object.

Unless for some reason we need to grant or deny access to a specific user or group of users, it is generally not necessary to specify our own security descriptors for securable objects that we create. These objects will only allow access by default to users or groups that have the necessary privileges to do so. Indeed, the system will provide a default DACL for a securable object when we don’t specify a security descriptor for it.

If for example we create a process that runs in the security context of an account that belongs in the Administrators group, then another process that runs in the context of a standard or guest user account should not be able to access it, or at least it should have very limited access to that process.

Indeed, customising access to securable objects that we create can be risky and lead to critical vulnerabilities. For this reason developers must be very careful when they do so.

As already mentioned, security descriptors should have a valid pointer to a DACL that contains one or more ACEs. However, a DACL can also be empty which means that access to this object is denied to everyone. A design error of this type would probably cause some sort of denial of service to any process that would need access to that particular object.

On the other hand, a more severe design error that can be created in the attempt of customising access to a securable object is to provide a NULL DACL pointer to the security descriptor of that object. This would allow any user to fully access that object, even if that object has been created in the security context of a higher privileged user, in which case the system should normally deny full access (Read|Write|Execute) to lower privileged users.

The Ugly DACLing

Unfortunately, not every fairy tale has a happy ending, especially when it is pretty much associated with a 0-day vulnerability in a real life scenario. Yes, NULL DACLs can be really ugly when it comes to security, and they don’t get any prettier.

Recently, VMware patched a critical security issue that we reported to them. This issue was mainly affecting all VMware Workstation, Horizon Client (with Local Mode Option), and Player versions since the virtual printer device was introduced.

This device allows a virtual machine with VMware tools installed to access a printer configured in the host. Adding this device to the virtual machine device settings was originally optional. However, in the more recent versions of these products the virtual printer device is added by default whenever a user creates a new virtual machine.

The discovered vulnerability is based on a NULL DACL assignment to the security descriptor of the vprintproxy.exe process which handles the virtual printer device in the host.
Every time a user launches a virtual machine, an associated vmware-vmx.exe process will run in the security context of the user that launched the virtual machine.

However, if the virtual machine has the virtual printer device enabled, or if the user connects that device while the virtual machine is active, then vmware-vmx.exe process will spawn vprintproxy.exe process. In the steps taken to configure a custom security descriptor for this child process, the privilege escalation vulnerability is introduced.

Figure 2. Assigning the NULL DACL

Figure 2. Assigning the NULL DACL

As shown in the above figure, vmware-vmx.exe makes a call to the SetSecurityDescriptorDacl function, but fails to provide a valid pointer (R8 Register) to an ACL structure that specifies the DACL for the created security descriptor.
Once the new security descriptor is created, it will be used in a subsequent call to CreateProcess function (Figure 3) to launch vprintproxy.exe which will create a process object with a NULL DACL assigned to it.

Figure 3. Launching vprintproxy.exe with a NULL DACL

Figure 3. Launching vprintproxy.exe with a NULL DACL

The Impact

The newly created vprintproxy.exe process runs in the security context of its parent process, which in turn runs in the security context of the user that launched the virtual machine instance.

However, because the object associated with vprintproxy.exe process has a NULL DACL in its security descriptor, it allows any user to fully access it and execute code in the context of this process by injecting a remote thread to it.

In a few words, if a virtual machine has been launched from an Administrator account, then a guest or a standard user that is currently logged-on in the same host can execute code in the security context of the Administrator account, thus obtaining full access to files and other resources that normally the system wouldn’t grant access to.

The Vendor Fix

VMware has provided a fix for the two most recent major versions of VMware Workstation, which are v10.x (v10.0.7) and v11.x (v11.1.1), as well as for the associated VMware Horizon Client v5.4.2, and Player v7.x (v7.1.1) and v6.x (v6.0.7) respectively.

Earlier versions of these products will not be patched, which means that using these products in a Windows host where multiple users are logged on at the same time, might pose a critical security risk.

The following two figures demonstrate the results of the applied fix.

Figure 4. Assigning a valid pointer to an ACL structure

Figure 4. Assigning a valid pointer to an ACL structure

Figure 5. Launching vprintproxy.exe with a valid DACL

Figure 5. Launching vprintproxy.exe with a valid DACL

VMware has created the following security advisory: https://www.vmware.com/security/advisories/VMSA-2015-0005

A Workaround Fix

If for any reason terminating one of the aforementioned VMware products in order to apply the updates is not an immediate option, the account owner can choose to permanently disable the virtual printer device from all the virtual machines, and ensure that there are no vprintproxy.exe processes still active. However, this should only be a temporary solution since the Vendor’s supplied update can contain fixes for other security related issues as well.

Conclusion

Creating custom security descriptors might be quite risky, and most of the time this is really not necessary, since the system always applies a default DACL to protect securable objects from being fully accessed by a lower privileged user.

However, it is necessary to do so if we want to create more strict rules for protecting a securable object from specific users or groups.

In this case, I had initially patched, for testing, the vulnerability in vmware-vmx.exe by simply passing a NULL pointer to the SECURITY_DESCRIPTOR parameter of the CreateProcess function. This forced the system to assign a default DACL to the vprintproxy.exe process which didn’t allow any more lower-privileged users to access this process. Even though this was just a POC fix, it didn’t cause any issues while using the software.

The demo video below demonstrates how a local attacker elevates his privileges from a Guest to SYSTEM account by exploiting CVE-2015-3650 in VMware Workstation v11.1.0.

 

References

  1. https://msdn.microsoft.com/en-us/library/windows/desktop/aa379557%28v=vs.85%29.aspx
  2. https://msdn.microsoft.com/en-us/library/windows/desktop/aa379563%28v=vs.85%29.aspx
  3. https://msdn.microsoft.com/en-us/library/windows/desktop/aa446597%28v=vs.85%29.aspx
  4. https://msdn.microsoft.com/en-us/library/windows/desktop/aa379583%28v=vs.85%29.aspx
  5. https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425%28v=vs.85%29.aspx

To contact the Nettitude editor, please email media@nettitude.com.

The Problem of Data Loss Intelligence

Data Loss Intelligence (DLI) concerns the information that is available to you when your data has been compromised. It’s distinct from Data Loss Protection (DLP) technologies, which are more concerned with preventing your data being compromised in the first place. Think of DLI as your last line; it tries to let you know when DLP has failed, and what is happening now that your data is out in the wild.

Tracking data is hard. It’s even harder when you lose control of its distribution. Whether it’s through an insider leak, external attack or innocent mistake, once data leaves the confines of your regular security perimeter, it is often entirely lost. Knowing who has your data, where they are keeping it and what they are doing with it is an increasingly important part of managing your sensitive business assets and intellectual property.

 Data is Inert

The heart of the problem lies with the fact that data is inherently inert. Typically it cannot itself execute or exhibit behaviours. It is entirely dependent on a counterpart piece of software, a reader, to parse its content and interpret any actions that file suggest be performed. Any attempt to contact the outside world or “call home”, for example, may be honoured by the reader. Often this is not the case, however, as this directly conflicts with an end user’s desire for privacy by allowing their actions to be tracked. Where an external call is permitted, usually each data type has a plethora of available readers and the behaviour may not be consistent across them all.

 The Problem of Consistency

While on the topic of consistency, even if you could guarantee a call home with one data type, your data is often not in a single discrete format. With each separate data type and reader, a different track technique will be needed, resulting in confusing and tedious operating procedures. One of the approaches taken to address this is to abstract the concept of protection away from being embedded in the data itself. This is done by wrapping, or “enveloping”, the data in a protective layer of encryption. The problem here is that at some point, in order to allow legitimate access, you must provide the key to the end user. Once decrypted, the content is unsecured again, and is subject to traditional threats. Even if the software which applies the envelope does not explicitly allow access to the decrypted material, a skilled attacker may be able to capture the key and use this to decrypt the original content.

Ultimately, to have a robust DLI solution, we need to be able to guarantee a behaviour on a computer we don’t control. This is a challenge as it’s near identical to the goals of malware and, as such, is rightly proactively hindered by various security mechanisms and product updates. Somehow we need to balance the need for user privacy and data control. It’s a problem that’s been going for years, with strong similarity to DRM (digital rights management). This is a problem we are actively investigating at Nettitude.

To contact Nettitude’s editor, please email media@nettitude.com.

 

Context Triggered Piecewise Hashing To Detect Malware Similarity

At Nettitude we collect a large amount of malware binary samples, both from our Honeypot network, from our customers and from incident response. One of the first steps we take is to calculate the MD5 hash of the malware and compare this hash to known samples, while unknown samples can be examined further by an analyst.

Traditional hashes like MD5 take an input and create a fixed length output hash that represents the data. A one bit change to the data will produce an entirely different MD5 hash, so quite often during this examination process we find that the malware is essentially identical to another known piece of malware and only slight variations in the file cause the MD5 hash to differ. Eliminating these duplicates is something I’ve recently been looking into.

SSDeep 

Initially I took a look at the well-known SSDeep tool, which can be used to determine document similarity. This uses a technique known as context triggered piecewise hashing (CTPH) to compare files for similarities.

SSDeep can be used to hash and compare any two files; it doesn’t, however, take into account the internal structure of those files when hashing. Therefore in order to get more reliable results I decided to implement a version of CTPH myself, which takes into account the format of a windows executable image (also known as the portable executable file format PE).

What Is Context Triggered Piecewise Hashing? 

CTPH – also known as fuzzy hashing – is based on using a rolling hash, where the hash has a siding window and a ‘state’. The state maintains the hash of the last few bytes of the data that are in the current window and is constructed in such a way that allows the removal of influence of previous bytes of data, and the addition of new data, as the sliding window position moves.

The Rabin-Karp string search algorithm uses this rolling hash technique to locate substrings that only require one comparison per character:

Figure 1 - Rabin-Karp String Search

Figure 1 – Rabin-Karp String Search

 

When examining documents for similarity using CTPH, a rolling hash of the data is performed and, at an arbitrarily chosen trigger point. For example, when the modulus of the hash matches a certain value, a traditional hash such as MD5 is calculated on the data processed since the previous trigger point. Part of the traditional hash is saved, for instance, the last two bytes and the rolling hash continues. The final CTP hash consists of the saved parts of the traditional hash.

Comparing two CTP hashes

CTP hashes can be compared by using a Bloom filter and a Hamming distance calculation. A bloom filter is a data structure that can be used to test if an element of data is a member of a set and the Hamming distance gives a weighting value based on the difference of two strings.

An example of this is Sdhash, which uses a bloom filter combined with a calculation of the Hamming distance.

The parts of the two CTP hashes to be compared can be added to two separate bloom filters, and then the Hamming distance between the bloom filters is calculated to determine the weight of differences between the two hashes.

Results

After some initial research, I have implemented a prototype tool.

The prototype uses pebliss to read the PE file, SQLite as a backend to store the results, the Rabin-Fingerprint rolling hash algorithm and MD5 hashing. For hash comparison, a Bloom filter is also used and the percentage matching bits is used to gauge similarity. To begin with, only the code section of the malware is hashed and compared.

The CTPH algorithm allows an arbitrary trigger point to be chosen. For the triggering I experimented with various values and finally settled on a system which looks at the next byte to be processed and compares it to an assembler “jmp”, “call” or “ret” instruction. Although the byte being examined may actually not be a jmp and might simply be part of another instruction, that can’t be any worse than choosing an arbitrary modulus value of the rolling hash. At the trigger point I generate an MD5 sum.

Hashing 200 malware samples with the tool shows that the code section for a large number of our unique samples is in fact the same. This may of course be because a packer has been used to compress or obfuscate the malware.

Figure 2 - SQLite Similarity Results

Figure 2 – SQLite Similarity Results

Conclusion 

The initial prototype has proved that the concept of fuzzy hashing the individual malware sections can be useful.

This still, however, needs to be developed into a complete tool. I’ll be moving towards implementing the following features over the next few weeks:

  • Hash and compare the resource and data sections of the PE files
  • Identify which malware has been packed
  • Move away from SQLite and place the data into a NoSql database (mongo or elasticsearch)
  • Calculating the Hamming distance of two hashes

Some interesting research has already been done on hashing disassembled functions using CTPH. This is also something that I will be investigating in the coming weeks:

http://www.shadowserver.org/wiki/uploads/Information/FuzzyHashing.pdf

http://www.blackhat.com/presentations/bh-usa-09/RAYGOZA/BHUSA09-Raygoza-MalwareSimAnalysis-PAPER.pdf

 

 

To contact Nettitude’s editor, please email media@nettitude.com.