In this talk we will disclose our recent findings about never before seen PE or Portable executable format malformations. These findings have serious consequences on security and reverse engineering tools and lead to multiple exploit vectors.

PE is the main executable image file format on Windows operating system since its introduction in Windows NT 18 years ago. PE file format itself can be found on numerous Windows-based devices including PCs, mobile and gaming devices, BIOS environments and others. Its proper understanding is the key for securing these platforms. The talk will focus on all aspects of PE file format parsing that leads to undesired behavior or prevents security and reverse engineering tools from inspecting malformated files due to incorrect parsing. Special attention will be given to differences between PECOFF documentation and the actual implementation done by the operating system loader. With respect to these differences we will demonstrate existence of files that can't possibly be considered valid from a documentation standpoint but which are still correctly processed and loaded by the operating system. These differences and numerous design logic flaws can lead to PE processing errors that have serious and hardly detectable security implications. Effects of these PE file format malformations will be compared against several reverse engineering tools, security applications and unpacking systems. Special attention will be given to following PE file format aspects and their malformation consequences:

• General PE header layout in respect to data positioning and consequences of different memory model implementations as specified by PECOFF documentation. Use of multiple PE headers in a single file along with self-destructing headers.
• Alignment fields with their impact on disk and memory layout with the section layout issues that can occur due to disk or memory data overlapping or splicing. In addition to this, section table content will be inspected for issues of data hiding and its limits will be tested for upper and lower content boundaries. We will demonstrate how such issues affect existing static and dynamic PE unpacking systems.
• Data tables, including imports and exports, will be discussed in detail to show how their malformated content can break analysis tools but is still considered valid from the operating system loader standpoint. We will demonstrate existence of files that can miss use existing PE features in order to cloak important file information and omit reverse engineering process. Furthermore based upon these methods a unique undetectable method of API hooking that requires no code for hooks insertion will be presented.
• PE file format will be inspected for integer overflows and we will show how their presence can lead to arbitrary code execution in otherwise safe analysis environments. We will show how PE fields themselves could be used to deliver code payload resulting in a completely new field of programming; via the file format itself.
• In addition to single field and table malformations more complex ones involving multiple fields and tables will also be discussed. As a demonstration of such use case scenario a unique malformation requiring multiple fields working together to establish custom file encryption will be presented. This simple, yet effective, encryption that is reversed during runtime by the operating system loader itself requires no code in the malformated binary itself to be executed. Its effectiveness is in a unique approach to encryption trough file format features themselves in order to prevent static and dynamic file analysis tools from processing such files.

This talk will be a Black Hat exclusive; Whitepaper accompanying the presentation materials will contain detailed description of all malformations discussed during the talk. This whitepaper aims to be a mandatory reading material for security analysts. It will continue to be maintained as new information on PE format malformations are discovered.

More information here.

Instructors: Tomislav Pericin and Nicolas Brulez
Dates: 6-7 July 2011
Availability: 20 Seats

Day 1: Inside the PECOFF file format

During the first day of the training we will focus on reviewing the PECOFF file format and examining its aspects to determine the structures and tables most commonly compressed and protected by PE modifiers. General memory models used by all known PE format modifiers will be described based upon which software compressions will be classified into groups. Key features of crypters, packers and protectors will be analyzed on real world samples and the most representative formats will be manually unpacked.

PE file format properties obscured by the format modifier will be discussed. These properties include import, export, resource, relocation and tls tables and the ways that PE modifiers transform them from standard PECOFF to packer specific formats. By applying reverse engineering techniques we will decipher these internal packer specific formats and restore them to their original state. In addition to this attendees will learn how to reverse engineer custom compression and encryption algorithms and implement them in their code in order to produce fully functional format unpackers. Special attention will be given to static unpacker coding layout and the benefits of using TitanEngine to minimize the time it takes to create an unpacker.

Attendees will learn how to identify and reverse engineer key PE file format modifier sections. Single PE packer format that supports x86/x64/.net packing will be inspected in detail for which static unpacker will be coded.

Day 2: Inside the nightmares of file analysis

During the second day of the training we will focus on analyzing and unpacking complex software protections. Special attention will be given to methods used to harden against format reverse engineering and prevent unpacking. We will describe common protection techniques utilized by both legitimate software protectors and those specifically designed for use in malware. We will then use information to show coding techniques needed for such complex static unpackers and ways to counter all the tricks used to harden detection, analysis and unpacking.

Single PE protection format will be inspected in detail for which dynamic and/or static unpackers will be coded.

Class Requirements

Very basic knowledge of C/C++ or any other programming language.
Very basic understanding of assembly, debugging and Windows internals.
OllyDBG 1.10 and IDA Pro 5 (free version will be sufficient).
Microsoft Visual Studio 2008 (express will be sufficient).
Additional tools and scripts will be provided by the instrutor.

More information here.

Reverse engineering is the only weapon of choice when it comes to malware unpacking and analysis. It gives us an inside look into the malware creations and enables us to understand their ins and outs. One such malicious sample was sent to us today for analysis. The file in question is an update for a rogue anti-virus solution and it uses an interesting encryption and packing options to hide its presence from legitimate security software solutions. For our today's blog we demonstrate the actions needed to remove the protections utilized by malicious software in order to get to the core malware functionality. Until next week...

Looks cool? Want one? All you have to do is solve this challenge and tell us what is the password we seek. Sounds easy? Its not... Mail us with your solution at: blog(at)reversinglabs(dot)com; Challenge is now closed! Thanks to everyone who participated. Click read more for the solution...

We didn't even dream about getting so many people to participate in our little challenge. The sheer number of emails simply flooded our mailbox with possible solutions and compliments about  our challenge! One of those compliments expresses just what we want to do in the not-so-distant future, we quote: "Fun challenge, do more of these!" We definitely will!

Now to the solution, and discussion of the parts that proved troublesome for many...

We start by downloading the file and doing our initial analysis. Since the file is a ZIP archive we open it with any program that works with this archive format to find a folder named "r" with the file named "r.zip" in it. This part of the challenge is just a decoy to keep you busy and distracted from the real content which is appended to the archive as an archive comment. That real content is another archive in 7Zip file format, which once extracted produces a single CAB file, and that is where the things get interesting. The CAB file has a single PNG file in it, but to solve this challenge we must observe the image and the archive as two separate objects.

First the image part of the puzzle. The image, once opened, shows a normal picture with the logo of our company. However the picture itself has embedded steganography data. Since we didn't want to do any hard stego which can be solved by inspecting image pixels we embedded our hidden information between valid records inside the PNG file header. Something very similar to what we demonstrated on BlackHat Barcelona earlier this year. With the obvious difference that the file format is an image not an archive. Nonetheless the principle is the same. So, what's hidden? If you open the image file with any hex editor you will see a string "pSWD" near the start of the file. That string is followed by a 16 number sequence: 538B327278BBAB654747288999FBCDA1 which isn't the password we need. Nope, its not - even though many of you thought that that was the end solution. Why isn't it?

Because of the fact that the CAB file that compressed that PNG image holds the last piece of the puzzle. If we scan that CAB file with our NyxEngine we get the following output:

Steganography ID: 0x00000b
Possible steganography due to suspicious CAB extra data present between entries!
Data start: 0x5a; Data size: 0x0000f6

And in that data there is the following text block:

UmFyIRoHAM6Zc4AADQAAAAAAAAA8MSAOyRZcWCVhcEFcUfp
P4JdbtU2derwgjSYp+BpxVYkWJPDtQ/TITifo4qO7qyYz+yLpd9+6
nkwwxmomWHbHK0Bt6UPHOwL/pEKm6IGXo/5dioeP66Fq5brTldgi
Z7do5bbFjykQIsx6PMCBre4iUJ7jcwrwD2MDs69XwuuHL+fMKy9hD
UJQPDEgDskWXFjp6jPWFXoWVSNb4H1zjQpW

Which is, in fact, a base64 encoded password protected RAR file. But, what's the password? The password is the PNG image number sequence converted to lower case text. So, its: 538b327278bbab654747288999fbcda1 which isn't an MD5 and needs not to be bruteforced. Once its entered and the RAR file is decrypted we can see the file named "file" containing the following text: "Password is: 9ec4c12949a4f31474f299058ce2b22a". And that's it, the challenge is successfully completed at that point. No more stego or hidden files.

There are six accepted solutions to this challenge, but the one that simply astonished us is the following python script which solves our challenge:

#! /usr/bin/env python
URL="http://blog.reversinglabs.com/wp-content/uploads/2010/07/r.zip"

import os
import urllib2
import struct

os.chdir("/tmp")
rzip=urllib2.urlopen("http://blog.reversinglabs.com/wp-content/uploads/2010/07/r.zip").read()
r7z = rzip[rzip.find("7z"):]
open("r.7z","w").write(r7z)
os.system("7z e r.7z")
cab = open("puzzle.cab").read()
os.system("cabextract puzzle.cab")
open("r.rar","w").write(cab[0x5a:0x5a+250].decode("base64"))
png = open("ReversingLabs.png").read()
ppos = png.find("pSWD")
sz, = struct.unpack(">I", png[ppos-4:ppos])
pwd = png[ppos+4:ppos+4+sz]
os.system("unrar e -P%s r.rar" % pwd.encode("hex"))
print open("file").read()

Thanks to everyone who participated in our little competition. Winners, your T-shirts are in the mail. Until our next challenge...

NyxEngine ReversingLabs Corporation

Our challenge got beaten by our own NyxEngine! Oh, Nyx...

When talking about new concepts, its always best to demonstrate them on something everyone is familiar with. In our case that's of-course UPX with which we are fairly familiar. It almost feels like we write one UPX unpacker each week, doesn't it?

Today we are presenting an optimization concept that enables us to unpack everything in a single go. Now, when talking about file unpacking we always unpack everything in one go, but we never unpack both the main executable module and all of its packed dependencies in a single run. Normally, you wold do this by batching through individual files.  But from a speed perspective, the best optimization imaginable comes from unpacking the main module and all of its dependencies at once. Since TitanEngine wasn't really designed to do that out-of-the-box, it needs just a little bit of help to pull it off.

The problem is the existence of multiple relocation tables, and more importantly multiple import tables. Since TitanEngine was designed to unpack files one at the time, we must do some additional coding around these boundaries to achieve our goal. Compared to a traditional TitanEngine dynamic unpacker, the only difference is the need to collect import table data for modules in one place, and use that data for any module that has reached its entry point jump. The UPX is a special case because it always imports packed file dependencies through the import table. This is, of course, a static way of importing libraries but our approach must be flexible enough to cover both dynamic and static importing.

To achieve our goal we have to scan the main module and all loaded libraries and try to find  the appropriate patterns. Once the patterns are found, we set breakpoints and store info about them so we know which module triggered which callback event. Normally we have three callbacks for UPX unpackers (LoadLibrary, GetProcAddress and EP jump) but since we are doing transverse unpacking we need one more: the load library event custom handler, which determines whether the loaded dependencies are packed with UPX by trying to find the neccessary breakpoint patterns. Even though it is impossible to have more than one module loading at a time, we still need to store the import data because the import tables for the main executable and dependencies might overlap if the modules are loaded dynamically. Once stored, the import info for each module is retrieved when it hits its entry point callback. Relocations aren't really a problem since there is just one module loading at a time, so we can use our "snapshot and compare" model, provided that modules load on non-default image bases. This can be done in numerous ways - one of the easiest is to compile the sample files so that they do that by default (which is considered cheating in the unpacking game), alternatively, we can pre-allocate the memory so that the modules have no choice but to pick another base address. For the purpose of this blog we cheated, in a real world application of this approach you mustn't.

In the real world you will hardly ever see this kind of case but if you do, you now know how to get everything in one go. Until next week...

TitanEngine ReversingLabs Corporation

RL!deUPX (package contains the unpacker with source and the samples used)

If you remember not so long ago we wrote about minor inconveniences we encountered while working with OllyDBG. Today we return to that subject with challenges we face when reversing modern software protectors. One such protection is SafeEngine or NoobyProtect, which uses a simple portable executable malformation in order to crash OllyDBG. Even though this crash is limited to old OllyDBG versions, it is interesting for us nevertheless. The crash itself wouldn't be interesting if, in fact, the application didn't then operate normally. But it does, which makes it even more sinister. This leads us to conclude that this portable executable malformation is either ignored by the system or irrelevant to the system loader ... but not to our debugger. So, where's the trick? What does it apply to?

If you look at the import table you will see that the first entry is rather unusual. Its name in LordPE is "kernel32...". Which could be fine since the trailing ".dll" is optional. If only it wasn't for those three little dots. Those dots represent multiple new lines after the name. In fact if we open the file with a hex editor we see that there are exactly 0x100 bytes following the first import library name. And in addition, there are 0x3000 bytes that follow the name of the first imported function in the first library trunk. These extra bytes seem to be too large for Olly to handle, overwriting some memory and ultimately making it crash. Since EIP is rewritten to 0x0d0a0d0a this could be an exploitable attack vector. Now we knew about this kind of attack for a while, having first seen it as an attack via too long names in the export table. But this is a bigger issue, since the import table is affected.

If we investigate further we find an even more interesting issue that arises because of the way Windows processes the import table. If the import address table pointer inside the image import descriptor isn't nil and if it points to a sequence of eight zero bytes, the import entry will be ignored completely. Also, if the import lookup table pointer is valid and it points to at least one valid import address pointer, tools such as LordPE will read that data and display a list of functions that never get imported. This means that Windows will ignore the image import descriptor entry completely, which makes the dynamic link library name irrelevant as it will never reach the loading stage. This enables anyone to craft a file that only looks like its importing a specified library which may or may not be present on the system. And that can create confusion about which library is actually needed and which isn't.

This isn't something that NoobyProtect does - but it could if it wanted to. Working around our crash problem is as easy as updating to OllyDBG 2.0 or patching the broken import table. Until next week...

sampleFile
(package contains a harmless sample file that imports a non existent DLL)

We can, with a little help from TitanEngine's hooking library. The idea is to use our unpacker as a library which will be injected into the packed file during its execution. Such a library would place hooks inside the packer code, redirecting the control flow to our unpacker wherever data collection or execution handling is needed. Those places are usually spots where the packer processes the import table or relocations, jumps to the original entry point, or just switches execution from one layer to another.

What are the benefits of such an approach? Even though its slightly harder to create and test such unpackers, the most notable benefit of unpacking by hooking is total immunity to various anti-debugging tricks used to detect the unpacking process. The only detection applicable to this unpacking scenario is anti-hooking and memory checksumming. The first is hardly ever used in modern protections due to the large number of false positives it gives, which are triggered by the operating system itself, security software and various window skinning applications. The second one is rarely present, and when it is it only covers specific memory regions that correspond to a single protection layer. In conclusion this method of implementing the unpacking process should result in fewer things to worry about.

Implementing this kind of hooking requires building custom functions to process the hook events. This is necessary to maintain the packed program work flow, and is exactly why we preserve the register state with PUSHAD, and if there is a jump affected by our hook, even EFLAGS with PUSHFD. These ASM instructions are embedded in our C code and with the help of naked pre-processor instruction they become the prologue and epilogue of the function. To apply the hooks we use the DLL_PROCESS_ATTACH event. For example if we were to hook the UPX code which loads libraries the hook code flow would look like this:

Since our hooks are 5 bytes we need to "borrow" as many instructions as we need to insert the hook. In this case we are "borrowing" three instructions. These instructions will be executed right after our inserted function is called. This is done to preserve the packer work flow. As you can see from this diagram we are using hooks instead of breakpoints. Therefore these hooks will be placed on at least three places: when UPX calls LoadLibraryA, GetProcAddress and finally once it jumps to the entry point. The most basic sample UPX unpacker is limited to working on executables that don't import functions by ordinals and use the old jump to entry point method. It's quite limited, but it's enough for a proof-of-concept of our technique.

Debugging this kind of unpacker can be rather tricky. This video shows a quick and easy way to do it:

Since we are creating a hook library unpacker, we also need a loader which will execute the unpacking target and inject the unpacker library in it. This can be done in number of ways but we decided to do it via the debug - detach method. Once both the unpacker hook library and the loader are made, our unpacker is complete. We hope you got the idea on how to use this technique to build your own hooking unpackers from our short blog. Until next week...

TitanEngine ReversingLabs Corporation

upxHooks (package contains the unpacker with source and the samples used)

%~~~~~~~~~~~~~~~~~~~~~~~~%
|H4sIAAAAAAACA3P3dLOwTOxh|
|YGF4zsBg7tHJMApGwYgE////|
|V/zJwsjF8I9BB8QH5QkGjhYG|
|xj/MD'              gULH|
|JrY'                BbVi|
|Tlx|   Y4NgmoOxWoxH4yL5d|
|VDR|   oTseHh8f6WK359lQU|
|qJy\              \YJOGt|
|xhN5I\              \dlr|
|qoJvnIznRDXvHjPWZ   |SY7|
|Lz31nKtYPklkV0F6w   |AKr|
|1E17                ,Vk5|
|afng              ,hp63R|
|VsvNzy8u9qpU670lon11hvnS|
|KNWuSS+vrvNf3HV05beU0NXB|
|p71kJQQYrAFt8kQCpwMAAA==|
%~~~~~~~~~~~~~~~~~~~~~~~~%
  D  E  C  O  D  E  M  E

We are pretty sure that "S" stands for Sophos not Superman. Now, the first thing that comes to mind when you look at the "picture" is that the data around the "S" is important. And if we look at the last two letters we see the base64 trademark signature. Which means that all that data is an encoded message or a file. To decode it, we must strip that "S" to form a proper base64 data chain. Once done, the data looks like this:

H4sIAAAAAAACA3P3dLOwTOxhYGF4zsBg7tHJMApGwYgE////V/zJwsj
F8I9BB8QH5QkGjhYGxj/MDgULHJrYBbViTlxY4NgmoOxWoxH4yL5dVD
RoTseHh8f6WK359lQUqJyYJOGtxhN5IdlrqoJvnIznRDXvHjPWZSY7L
z31nKtYPklkV0F6wAKr1E17Vk5afnghp63RVsvNzy8u9qpU670lon11
hvnSKNWuSS+vrvNf3HV05beU0NXBp71kJQQYrAFt8kQCpwMAAA==

That data must be reverted to either text or binary to continue. First, we tried  an online base64 decoder but it returns a very strange string. So then, we decoded the data to a binary file and opened that with a hex editor, where we see the well known 0x1F 0x8B signature, which indicates that the decoded data is in fact a GZIP file. Now, we know GZip files may or may not store a file name, so when we decompress the packed data we do another hex data inspection to discover that the decompressed file is a GIF file. It's an image showing us this: Not quite readable, but once you zoom in on it, and lower-case it, it points to: http://www.sophos.com/anz/sofarsogood.html which holds the last piece of the puzzle.

Sadly last piece of the puzzle has nothing to do with file analysis whatsoever. Its a crypto challenge requiring you to play with letter substitution crypto algorithms. And this isn't something we are really interested in. You are however more than welcome to fiddle with it if you like. For some help on solving it check this out. Until next week...

We have already talked about fixing the damaged, broken or missing files in several occasions. Based on what we know we created the Nexus TitanEngine plugin to deal with cases of missing dependencies and damaged files. Implementing the basic TitanEngine features to correct file abnormalities does however change the file checksum since modifications  needed to correct detected problems modify file and memory content. And that doesn't go well with software protections that check the file integrity during execution. One of those software protectors is tELock, and that is the starting point for today's blog. That and a question "How can we work around checksums when file repairing is necessary?".

Luckily for us most software protections only check the file integrity on disk while the memory integrity checks are only limited to protected data and the protection itself. Therefore we only need to worry about the integrity of the file on disk. To be able to fool any software protection integrity check in a generic way we need to know how these checks are performed. Usually is as simple as opening a file, reading its content in a buffer, hashing it with a custom hashing algorithm and checking if the hash is different then the one stored during file protection. So the logical place to catch the integrity checks is by hooking functions used open the file. Most commonly that involves hooking CreateFile API since all protections use it to gain access to protected file.

Hooking an API in a remote process is easy but not very practical since it involves injecting a DLL into the unpacking process and that isn't something we want to do. Other option is to set a breakpoint at the selected API and filter the information returned to the protection. In order to fool the checksum checks we do the following:

• Detect if the file is broken (Nexus already did this)
• Correct the damaged file and produce a backup file (Nexus already did this)
• Catch all calls to CreateFileW API to determine when the integrity check is performed
• Open a handle to backup file (which is valid for execution since its checksum is unaltered)
• Pass the open handle back to protector so that backup file is hashed and its checksum is confirmed

Since we only place a breakpoint on CreateFileW API we need to filter the information somehow to make the program open the backup file which is unaltered and therefore has the correct checksum. We can alter the parameter string and possibly corrupt the memory or we can pass the correct handle back to the protection. To do that we open a handle to backup file inside the context of the debugger and duplicate it inside the context of the unpacking process. That new handle is then used by the software protection to read the data from the backup file which successfully fools any integrity check regardless of the checksum algorithm used. We do this handle switch only if the file which the protected file is trying to open is the file we are currently unpacking. Since this method is generic we can use it for any software protection, not just tELock.

To test out theory we intentionally damage the sample file by modifying a single non relevant byte. This damaged file is now named damaged.exe and the backup file which is the original one is named damaged.exe.bak. If we try to unpack damaged.exe file the unpacker will unpack the file correctly regardless of the damage done to the file. This process effectively simulates the scenario in which the Nexus plugin automatically corrects the damaged file. Until next week...

TitanEngine ReversingLabs Corporation

NexusCheckSum (package contains the plugin with source and the samples used)