Instructors: Tomislav Pericin and Nicolas Brulez
Dates: 6-7 July 2011
Availability: 20 Seats

Day 1: Inside the PECOFF file format

During the first day of the training we will focus on reviewing the PECOFF file format and examining its aspects to determine the structures and tables most commonly compressed and protected by PE modifiers. General memory models used by all known PE format modifiers will be described based upon which software compressions will be classified into groups. Key features of crypters, packers and protectors will be analyzed on real world samples and the most representative formats will be manually unpacked.

PE file format properties obscured by the format modifier will be discussed. These properties include import, export, resource, relocation and tls tables and the ways that PE modifiers transform them from standard PECOFF to packer specific formats. By applying reverse engineering techniques we will decipher these internal packer specific formats and restore them to their original state. In addition to this attendees will learn how to reverse engineer custom compression and encryption algorithms and implement them in their code in order to produce fully functional format unpackers. Special attention will be given to static unpacker coding layout and the benefits of using TitanEngine to minimize the time it takes to create an unpacker.

Attendees will learn how to identify and reverse engineer key PE file format modifier sections. Single PE packer format that supports x86/x64/.net packing will be inspected in detail for which static unpacker will be coded.

Day 2: Inside the nightmares of file analysis

During the second day of the training we will focus on analyzing and unpacking complex software protections. Special attention will be given to methods used to harden against format reverse engineering and prevent unpacking. We will describe common protection techniques utilized by both legitimate software protectors and those specifically designed for use in malware. We will then use information to show coding techniques needed for such complex static unpackers and ways to counter all the tricks used to harden detection, analysis and unpacking.

Single PE protection format will be inspected in detail for which dynamic and/or static unpackers will be coded.

Class Requirements

Very basic knowledge of C/C++ or any other programming language.
Very basic understanding of assembly, debugging and Windows internals.
OllyDBG 1.10 and IDA Pro 5 (free version will be sufficient).
Microsoft Visual Studio 2008 (express will be sufficient).
Additional tools and scripts will be provided by the instrutor.

More information here.

BlackHat, one of the world's biggest security conferences, was held in Las Vegas two weeks ago. Among the BlackHat conferences this year, Las Vegas was by far the biggest event  - bringing thousands of security researchers to the heart of the Sin City. Bigger then ever before, BlackHat featured eleven tracks with an impressive number of high quality talks and trainings. We were there, and we were more than proud to present our newest file analysis tool, TitanMist, to the World.

In a fairly full room, for a reversing track that is, we presented the TitanMist project we have been working on for the past couple of months. Best described as  an automated PE32 file format identification and unpacking tool, TitanMist aims to improve collaboration among reverse engineers across the globe. That is an ambitious goal, but we have high hopes for this project and believe that we can grow it into something of great value in the coming months. Our detailed project roadmap will be unveiled on our blog next week.

The TitanMist presentation was accompanied by the Arsenal presentation that featured all of our tools: TitanEngine, NyxEngine and TitanMist. This kind of tool demo was introduced at this year's BlackHat USA and we must admit that we like the idea of tool presentations, during which authors receive direct feedback from the community. This feedback enables us to add new features and improve our tools even further. We got  many great ideas from talking to attendees with an interest in our tools and what we do. Ones attendees with particularly intriguing questions or ideas questions were rewarded with one of our TitanEngine T-Shirts. But t-shirt winner or not, we thank you all for your continuing support of ReversingLabs and the TitanEngine project!

In the picture on the left you can see our Arsenal booth and one of our senior software engineers, Mario Suvajac. Mario is one of the guys behind the TitanMist project, in charge of the byte pattern matching and overall tool design.

That is it for this report, until next week...

When talking about new concepts, its always best to demonstrate them on something everyone is familiar with. In our case that's of-course UPX with which we are fairly familiar. It almost feels like we write one UPX unpacker each week, doesn't it?

Today we are presenting an optimization concept that enables us to unpack everything in a single go. Now, when talking about file unpacking we always unpack everything in one go, but we never unpack both the main executable module and all of its packed dependencies in a single run. Normally, you wold do this by batching through individual files.  But from a speed perspective, the best optimization imaginable comes from unpacking the main module and all of its dependencies at once. Since TitanEngine wasn't really designed to do that out-of-the-box, it needs just a little bit of help to pull it off.

The problem is the existence of multiple relocation tables, and more importantly multiple import tables. Since TitanEngine was designed to unpack files one at the time, we must do some additional coding around these boundaries to achieve our goal. Compared to a traditional TitanEngine dynamic unpacker, the only difference is the need to collect import table data for modules in one place, and use that data for any module that has reached its entry point jump. The UPX is a special case because it always imports packed file dependencies through the import table. This is, of course, a static way of importing libraries but our approach must be flexible enough to cover both dynamic and static importing.

To achieve our goal we have to scan the main module and all loaded libraries and try to find  the appropriate patterns. Once the patterns are found, we set breakpoints and store info about them so we know which module triggered which callback event. Normally we have three callbacks for UPX unpackers (LoadLibrary, GetProcAddress and EP jump) but since we are doing transverse unpacking we need one more: the load library event custom handler, which determines whether the loaded dependencies are packed with UPX by trying to find the neccessary breakpoint patterns. Even though it is impossible to have more than one module loading at a time, we still need to store the import data because the import tables for the main executable and dependencies might overlap if the modules are loaded dynamically. Once stored, the import info for each module is retrieved when it hits its entry point callback. Relocations aren't really a problem since there is just one module loading at a time, so we can use our "snapshot and compare" model, provided that modules load on non-default image bases. This can be done in numerous ways - one of the easiest is to compile the sample files so that they do that by default (which is considered cheating in the unpacking game), alternatively, we can pre-allocate the memory so that the modules have no choice but to pick another base address. For the purpose of this blog we cheated, in a real world application of this approach you mustn't.

In the real world you will hardly ever see this kind of case but if you do, you now know how to get everything in one go. Until next week...

TitanEngine ReversingLabs Corporation

RL!deUPX (package contains the unpacker with source and the samples used)

With the increase in packed and protected malicious payloads, collaboration and quick response among researchers has become critical. As new sample numbers are quickly approaching 40M samples per year, the solution to this problem has to come from reverse-engineers themselves, integrating their prior and current work. Huge databases of format identification data and unpacking scripts can be reused to maximize automation. Yet, where do we find a definite collection of functional tools, identification signatures and unpacking tools? And how do we integrate them in a meaningful and accurate way?

Come to this talk to hear how we plan to raise reversing collaboration to a whole new level with TitanMist. We will address today's and future challenges, source code, packaging and distribution, and define your role in making TitanMist the most powerful community tool for years to come.

This talk will be a BlackHat exclusive; a launch and demonstration of TitanMist, a new open-source project based on TitanEngine. All components will be available for distribution with the conference materials.

See you in Vegas...

We can, with a little help from TitanEngine's hooking library. The idea is to use our unpacker as a library which will be injected into the packed file during its execution. Such a library would place hooks inside the packer code, redirecting the control flow to our unpacker wherever data collection or execution handling is needed. Those places are usually spots where the packer processes the import table or relocations, jumps to the original entry point, or just switches execution from one layer to another.

What are the benefits of such an approach? Even though its slightly harder to create and test such unpackers, the most notable benefit of unpacking by hooking is total immunity to various anti-debugging tricks used to detect the unpacking process. The only detection applicable to this unpacking scenario is anti-hooking and memory checksumming. The first is hardly ever used in modern protections due to the large number of false positives it gives, which are triggered by the operating system itself, security software and various window skinning applications. The second one is rarely present, and when it is it only covers specific memory regions that correspond to a single protection layer. In conclusion this method of implementing the unpacking process should result in fewer things to worry about.

Implementing this kind of hooking requires building custom functions to process the hook events. This is necessary to maintain the packed program work flow, and is exactly why we preserve the register state with PUSHAD, and if there is a jump affected by our hook, even EFLAGS with PUSHFD. These ASM instructions are embedded in our C code and with the help of naked pre-processor instruction they become the prologue and epilogue of the function. To apply the hooks we use the DLL_PROCESS_ATTACH event. For example if we were to hook the UPX code which loads libraries the hook code flow would look like this:

Since our hooks are 5 bytes we need to "borrow" as many instructions as we need to insert the hook. In this case we are "borrowing" three instructions. These instructions will be executed right after our inserted function is called. This is done to preserve the packer work flow. As you can see from this diagram we are using hooks instead of breakpoints. Therefore these hooks will be placed on at least three places: when UPX calls LoadLibraryA, GetProcAddress and finally once it jumps to the entry point. The most basic sample UPX unpacker is limited to working on executables that don't import functions by ordinals and use the old jump to entry point method. It's quite limited, but it's enough for a proof-of-concept of our technique.

Debugging this kind of unpacker can be rather tricky. This video shows a quick and easy way to do it:

Since we are creating a hook library unpacker, we also need a loader which will execute the unpacking target and inject the unpacker library in it. This can be done in number of ways but we decided to do it via the debug - detach method. Once both the unpacker hook library and the loader are made, our unpacker is complete. We hope you got the idea on how to use this technique to build your own hooking unpackers from our short blog. Until next week...

TitanEngine ReversingLabs Corporation

upxHooks (package contains the unpacker with source and the samples used)

We had a great time during this year's CARO Workshop Conference held in Helsinki last week.  Now it is the time to sort out our impressions.  First of all, thanks to all that have made it to our talk and asked us many intriguing questions. Slides for our talk are available here. The picture you see above is from the brilliant keynote held by Dr. Alan Solomon. We absolutely enjoyed the keynote and Dr. Solomon's remark regarding the perfect antivirus represented by his three batch files.

Our talk was focused on improving the file analysis metrics and on unpacking technology performance testing,  comparing different solutions. During the talk we have presented a new idea for unpacking optimization.  We proposed unpacking through "binary layering" which enables the reuse of unpacking technology as much as possible. Put simply binary layering enables scanning various parts of the binary object and attributing them to known packing formats. Since multiple segments of the same file can have different formats attached to them we recognize that files commonly don't have simple identities but instead their complex layout is viewed as file's complex identity. These complex identities give much more detailed picture about the file itself and enable easy file categorization and further analysis.

We also talked about optimization that can improve file analysis system's metrics.   In this regard, we have shown that binary layering can improve the unpacking speed when identified segments are processed in parallel. Most objects can take advantage of this kind of optimization, but with some exceptions.  Specifically, this applies to cases where binary object requires other objects to be present in predefined way prevents unpacking one file at the time.   Similarly, it also applies to cases where there are multiple one way unpacking layers with output of the previous layer serving as input for the next one.

To test our hypothesis we did a comparative test using our lab tools and Kaspersky Anti-virus, which incorporates both file unpacking and malicious payload detection.   For the test to be relevant enough and to avoid inclusion of  malware scanning into unpacking metrics we have performed the following:

• Inspecting metrics for our internal lab unpacking tools
• Inspecting metrics for KAV on the predefined set of packed files
• Inspecting metrics for KAV on the set of unpacked files produced by our internal lab tools

It is necessary to perform these three steps together in order to obtain relevant results.  Third step excludes unpacking from scanning results and therefore gets a relatively good comparison for unpacking metrics.  For the purpose of our presentation we performed two distinct tests, one on packed portable executable files and one on installer packages.  The first test has employed one way unpacking while the second test has used non-parallel "binary layering" to detect and unpack files. Here are the results for the first test:

This first test was performed on 1627 portable executable files packed with 140 different packer families. It  demonstrated that our internal tool (referred here as the "BlackBox") has successfully unpacked 95% of the files in 530 seconds. Remaining 74 files we declared as invalid either for static or dynamic analysis, indicating that file recovery can not be applied to salvage corrupt data. This means that reported 1568 objects is the number of output files that were processed by this unpacking library. KAV processed the same amount of files in 534 seconds reporting 4533 objects and 249 events. To clarify, KAV counts all files it finds inside the packed content (every packing level is counted) and then reports the actual number of files detected by its signatures. Number of events refers to all additional operations KAV performs on scanned files such as malware detection, quarantine or deletion action. Finally, in the last step KAV scanned 1568 unpacked files that were produced by BlackBox. Third step eliminates the need for unpacking since all files are already unpacked. This part completed in 300 seconds and KAV reported2042 objects and 35 events. To take into the account the unpacking that was initially performed with BlackBox we have added its execution time to the scan time. Results: KAV performs its scan faster with fewer objects that need scanning. Additionally, there are less events indicating false positive detection on the packer formats themselves .  Granted a small amount of packers used in our test base should be blacklisted as their main use historically has been to hide malicious payload.

Its important to note that the unpacking methods used by BlackBox and KAV are completely different. While KAV mostly uses static unpacking to decompress data to memory, our BlackBox uses both dynamic and static unpacking  methods to decompress data to disk with multiple drive accesses. It is slowed down even further when unpacking dynamic link libraries due to snapshot comparison to repair relocation table. Optimizations can be performed to improve these unpacking results, but none were used. Hence we feel confidant that if all of these unpackers were done using TitanEngine, a significant unpacking speed increase would be gained.

Now, lets move to our second, more interesting test.  Here are the results:

Our second test was performed on 20 selected non-malicious installer packages. We used another internally developed tool, here referred to as "Core", to produce 4275 files in 95 seconds.  In comparison, KAV scanned these same input packages in 300 seconds, reporting 9174 found files. In our last step, we have performed the scan on unpacked files produced by Core.  In that case KAV reported 12175 files with the unpacking finishing in 360 seconds (this is with the added time for file unpacking done by Core). Number of events reported is two and they refer to scan start and scan finish. No malicious objects were detected. Results: This test shows that when performing unpacking on files  that have been already unpacked by Core, KAV is able to scan 3000 more files in the time that is very close to the time needed to scan the packed content.  Further optimizations could certainly apply that would reduce this number even further.

In conclusion, our initial "binary layering" experiment has performed great in comparison to existing solutions., while our first test has demonstrated the value of diligent support for various packing formats.  As these were only lab experiments, much space is left for further optimization and implementation improvements. Until next week...

We have already talked about fixing the damaged, broken or missing files in several occasions. Based on what we know we created the Nexus TitanEngine plugin to deal with cases of missing dependencies and damaged files. Implementing the basic TitanEngine features to correct file abnormalities does however change the file checksum since modifications  needed to correct detected problems modify file and memory content. And that doesn't go well with software protections that check the file integrity during execution. One of those software protectors is tELock, and that is the starting point for today's blog. That and a question "How can we work around checksums when file repairing is necessary?".

Luckily for us most software protections only check the file integrity on disk while the memory integrity checks are only limited to protected data and the protection itself. Therefore we only need to worry about the integrity of the file on disk. To be able to fool any software protection integrity check in a generic way we need to know how these checks are performed. Usually is as simple as opening a file, reading its content in a buffer, hashing it with a custom hashing algorithm and checking if the hash is different then the one stored during file protection. So the logical place to catch the integrity checks is by hooking functions used open the file. Most commonly that involves hooking CreateFile API since all protections use it to gain access to protected file.

Hooking an API in a remote process is easy but not very practical since it involves injecting a DLL into the unpacking process and that isn't something we want to do. Other option is to set a breakpoint at the selected API and filter the information returned to the protection. In order to fool the checksum checks we do the following:

• Detect if the file is broken (Nexus already did this)
• Correct the damaged file and produce a backup file (Nexus already did this)
• Catch all calls to CreateFileW API to determine when the integrity check is performed
• Open a handle to backup file (which is valid for execution since its checksum is unaltered)
• Pass the open handle back to protector so that backup file is hashed and its checksum is confirmed

Since we only place a breakpoint on CreateFileW API we need to filter the information somehow to make the program open the backup file which is unaltered and therefore has the correct checksum. We can alter the parameter string and possibly corrupt the memory or we can pass the correct handle back to the protection. To do that we open a handle to backup file inside the context of the debugger and duplicate it inside the context of the unpacking process. That new handle is then used by the software protection to read the data from the backup file which successfully fools any integrity check regardless of the checksum algorithm used. We do this handle switch only if the file which the protected file is trying to open is the file we are currently unpacking. Since this method is generic we can use it for any software protection, not just tELock.

To test out theory we intentionally damage the sample file by modifying a single non relevant byte. This damaged file is now named damaged.exe and the backup file which is the original one is named damaged.exe.bak. If we try to unpack damaged.exe file the unpacker will unpack the file correctly regardless of the damage done to the file. This process effectively simulates the scenario in which the Nexus plugin automatically corrects the damaged file. Until next week...

TitanEngine ReversingLabs Corporation

NexusCheckSum (package contains the plugin with source and the samples used)

There are a lot of optimizations one can do with the TitanEngine to make it work even faster then lightning. During the related unpacker execution timing research for our upcoming CARO Workshop talk we measured the impact that certain operations inside the engine itself have on the total unpacking time. We realized that there is significant space for performance improvement in certain unpacking areas which is especially important when we are processing large file volumes. Now, when unpacking files with unpackers built around the TitanEngine you get unpacker execution times quite similar to the sample execution time, except for cases where dynamic link library unpacking requires snapshots to correct the relocation table. in those cases we see a significant unpacking execution time increase. To counter this we can either do memory snapshots to memory or optimize relocation processing and avoid using snapshots at all.

Generally when talking about fixing relocation table we refer to the easy snap-and-compare method. However there is another way of making the unpacked dynamic link library valid for loading on non default base. We can use RelocaterGrabRelocationTableEx function for cases when the packer uses non modified relocation table, defined as it is in the PECOFF document. Relocation data is still compressed and can only be accessed just before the file is relocated, which is why we need a function to inspect the memory and determine the relocation table size. And that is exactly what RelocaterGrabRelocationTableEx does. It determines the size of the relocation table at the provided address and copies it to the engine for later exporting. If we look at the following PackMan code snippet which does the image relocation:

  OR ECX,ECX
  JE L018
  MOV EDI,DWORD PTR DS:[EBX+24]
  JMP L013
L004:
  XOR EAX,EAX
  LODS WORD PTR DS:[ESI]
  OR EAX,EAX
  JE L011
  AND AH,0F
  ADD EAX,DWORD PTR DS:[EBX]
  ADD DWORD PTR DS:[EDX+EAX],ECX
L011:
  CMP ESI,EDI
  JNZ L004
L013:
  MOV EDX,DWORD PTR DS:[EDI]
  LEA ESI,DWORD PTR DS:[EDI+8]
  ADD EDI,DWORD PTR DS:[EDI+4]
  TEST EDX,EDX
  JNZ L011
L018:
  POPAD
 

We can see that the relocation table is stored at EBX+0x24 address. Therefore by reading that memory pointer before the actual relocation occurs we have all the parameters we need to fix the relocation table. Passing that parameter to the RelocaterGrabRelocationTableEx will result in the engine reading the relocation table and estimating its size. Therefore we can just use the pointer we read at the EBX+0x24 address and the return from RelocaterEstimatedSize to correct the PE header for the unpacked file. However RelocaterEstimatedSize doesn't return the accurate size due to the system design. It must be reduced by 8 to be correct for all cases.

Since we are only updating the PE header data we can free the relocation table stored inside the engine with RelocaterCleanup. Once we dump the process relocation table fixing is as easy as updating the PE header fields. By doing the relocation table fixing this way we optimize the speed of execution by a significant percent. No actual data needs to be written to the file on the disk since it is already there and in the correct format. Furthermore you can start the debugging without the previously necessary DLL loading on the address other then default. If you choose to use that optimization as well packer execution time will be shorter since the file might not be relocated at all thus saving CPU cycles. Until next week...

TitanEngine ReversingLabs Corporation

RL!dePackMan (package contains the unpacker with source and the samples used)