ReversingLabs | Blog

In the last couple of years we have seen a drastic increase in numbers of malicious sample we see a day. These numbers are quickly closing to 40M samples a year mark that we expect to see hit this year. That is why the sheer volume of data we are bombarded with each day raises an important question, where is the relevant data in this sea of information?  And even is all data we have relevant?

Prioritization is the main way of extracting relevant data with the techniques and methods used to highlight interesting information varying from one antivirus company to another. However we can think differently in order to sort this information. We can think in reverse asking ourselves which of this data isn’t interesting. With that question in mind we developed a system to exclude damaged, invalid and broken files from our sample bases. In depth file analysis tell us exactly which files have zero chance of execution on any system flagging them as crapware. But is everything broken to that extent?

If you remember recently we gave you a good idea what to do with broken files and how to implement TitanEngine statical validity analysis to identify and fix broken files. For this purpose we will update the TitanEngine Nexus plugin to automatically identify and fix broken files. This will extend this plugin functionality from creating missing dynamic link library dependencies to fixing every aspect of the broken inputted file. And since the plugin will work automatically it needs to be compatible with all existing unpackers. To achieve this we must recognize the basic dynamic unpacker model which looks like this:

As we can see from this flow chart all dynamic unpackers share a certain logic model. Perfect place for Nexus to handle the inputted file is at the start of unpacking process which is achieved by hooking TitanEngine's function IsPE32FileValidExW. This function is called before the unpacking process starts by all unpackers and if it estimates the file as invalid or broken unpacking is aborted. So what does our hook need to do? List of steps to do would be:

• Perform statical validity analysis by calling IsPE32FileValid
• Determine if file is valid or not and if it isn't do the following
  • Create a backup for inputted file
  • Perform statical file fixing by calling FixBrokenPE32FileEx
  • Validate the file fixing success
  • Return TRUE

But this is just the first step because in order to fix the file the TitanEngine can temporarily disable certain fields by removing them from PE header. To revert these changes we must add another hook to revert these changes. Since we are improving Nexus to automatically correct broken files for dynamic unpackers the function to hook is easily recognized as DumpProcessW. This function is called at the start of the unpacking process finalization, just before the necessary data is exported to file on the disk. That makes this function a perfect place to revert the changes to temporarily disabled PE fields. To do this we just need to call FixBrokenPE32FileEx again with the saved FILE_FIX_INFO structure.

By implementing these changes to TitanEngine's Nexus plugin we convert it to all purpose dynamic unpacker helper module because with its help we can unpack broken files and files that are missing their dependencies. And all this done with no modification to the source code of any unpacker we made in the past. As a demonstration of the plugin capabilities we have attached it and a broken UPX sample file with this blog. Until next week...

TitanEngine ReversingLabs Corporation

Nexus plugin (package contains Nexus plugin, UPX unpacker and a broken sample file)

In addition to TitanEngine course in Montreal on Recon there is another course that will be teaching you how to use the TitanEngine. So, if you are in Vegas for BlackHat you might want to check out Advanced Malware Deobfuscation training by Jason Geffner & Scott Lambert. Here is the course description:

Advanced Malware Deobfuscationby Jason Geffner & Scott Lambert

Overview:

Security researchers are facing a growing problem in the complexity of malicious executables. With an ever-increasing number of tools that malware authors use to compress and obfuscate executables, and the pressing urgency that analysts often face, it is vital for analysts to know the best methods to remove protections that they have never seen before.

Unpacking is the process of removing the compression and obfuscation applied by a “packer” (or “protector”) to a compiled and linked binary. This class will focus on teaching attendees the steps required to effectively deal with both known and previously unknown packing techniques.

This is a hands-on course. Attendees will work on real-world malware through a series of lab exercises designed to build their expertise in thwarting anti-debugging and anti-disassembling techniques.

Day One:

The first day will focus on understanding the problems presented by obfuscated malware and the steps required to effectively return the malware to an analyzable state. You will begin the day by learning the fundamentals of the Portable Executable (PE) file format. Then, through a series of lab exercises you will learn reliable methods for finding the Original Entry Point. With this knowledge in-hand, you will write software to construct a valid PE file on disk from the memory of a running process. You will complete this exercise by reconstructing the Import Table, effectively returning the executable to its pre-obfuscated state. With this virgin executable, you will apply static analysis techniques to determine the malware’s malicious capabilities.

The day will include a series of lab exercises focused on defeating anti-debugging tricks such as hardware/software breakpoint detection, generic/specific debugger detection, unpacker stub detection, Thread Local Storage callback functions, and more.

• PE File Format Essentials
• Fundamentals of Win32 Debugging
• Methods for Finding the Original Entry Point
• Manual and Assisted Import Table Reconstruction
• Overcoming Anti-Debugging Tricks
• User-Mode and Kernel-Mode Hooking and Code-Splicing

Day Two:

The second day will focus on how to unpack a heavily armored malware sample. You will learn about the concept of protected processes and how to decouple parent/child processes. Next, you will learn how API redirection utilizes stolen bytes. Then, you will master everything there is to know about Structured Exception Handling injection and redirection. Lastly, you will learn how chunked packing works, how to recognize it, and how to defeat it.

The day will end in a contest in which attendees will pit their wits against one another to analyze a heavily armored executable.

• Protected Processes
• Exception Injection and Redirection
• API Redirection
• Chunked Packing
• Utilizing TitanEngine from ReversingLabs as an Unpacking Framework

Who Should Attend:

This class is for skilled security analysts who wish to learn how to remove binary obfuscation from malware for analysis purposes. It is expected that attendees have a firm understanding of x86 assembly language and the Microsoft Windows API. Reverse engineering experience is desired, though not required.

What do i get:

• Hard copies of lecture slides and lab exercises.
• A CD containing links to all tools and reference materials used throughout the course.
• Solutions and written walkthroughs for all lab exercises.

Course Length:

Two days. All course materials, lunch and two coffee breaks will be provided. A Certificate of Completion will be offered. You must provide your own laptop.

Software Requirements:

Attendees must bring their own laptop with a 32-bit version of Windows XP, Windows Server 2003, Windows Vista, Windows Server 2008 or Windows 7 installed inside of a virtual machine (such as Microsoft® Virtual PC 2007 or VMware Workstation). Prior to the first day of the course, attendees are expected to have the following software installed in a virtual machine:

• API Imports/Exports Viewer - Dependency Walker
• API Logger - Auto Debug Professional
• C++ Compiler - Microsoft® Visual C++ 2008 Express Edition
• Debugger - OllyDbg
• Disassembler - IDA Pro Demo
• Hex Editor - Hex Workshop
• IT Reconstructor / Memory Dumper - Import REConstructor
• Microsoft® Windows® SDK - Windows SDK for Windows Server 2008
• Packer Detector - PEiD
• Packer Detector - ExeInfo PE
• Packer Detector - AT4RE FastScanner
• PE Editor - LordPE
• Strings Dumper - BinText
• Unpacking Framework - TitanEngine

Trainers:

Jason Geffner joined Next Generation Security Software Ltd. in June of 2007 as a Principal Security Consultant. Jason focuses on performing security reviews of source code and designs, reverse engineering software protection methods and DRM protection methods, penetration testing web applications and network infrastructures, and developing automated security analysis tools.

Prior to joining NGS, Jason spent three years as a Reverse Engineer on Microsoft Corporation's Anti-Malware Team, where his work involved analyzing malware samples, deobfuscating binaries, and writing tools for analysis and automation. Jason was the Security Research & Response owner of the Windows Malicious Software Removal Tool (MSRT). He chose which new malware families for the MSRT to detect and clean each month based on his analysis of the telemetry and trends of the underground malware community. Jason authored tens of thousands of malware signatures and dozens of malware analyses based on static and dynamic analyses of obfuscated binaries. His work on the MSRT helped hundreds of millions of Windows users each month keep their computers safe and secure. While at Microsoft, Jason was recognized for his reverse engineering skills and for his efforts to drive awareness of reverse engineering practices throughout the company by being given the formal job title "Reverse Engineer"; Jason was the only Microsoft employee with this title.

Jason holds several patents in the fields of reverse engineering and network security. He is a Program Committee member of the Reverse Engineering Conference (REcon) and of the International Conference on Malicious and Unwanted Software, is a regular trainer at Black Hat and other industry conferences, is often credited in industry talks and publications, and has been actively reverse engineering and analyzing software protection methods since 1995.

Scott Lambert is a senior Security Researcher on the Microsoft Malware Protection Center (MMPC) team. Much of Scott's current research centers around binary reverse engineering frameworks that leverage a combination of both static and dynamic binary instrumentation, taint analysis and SMT solvers to aid in vulnerability analysis and signature development. In his spare time he supports the Microsoft Vulnerability Research (MSVR) program by developing proof of concept code execution exploits and serving as a technical expert on 3rd party vendor engagements.

Prior to joining Microsoft, Lambert developed, maintained and supported numerous computer security applications ranging from Vulnerability Assessment and Risk Management software to Network and Host-Based Intrusion Detection/Prevention Systems for companies such as L-3 Network Security, Veridian Information Solutions, Symantec Corporation and TippingPoint, a division of 3Com.

Coding Unpackers for Fun and Profit: TitanEngine Training by
Tomislav Pericin and Nicolas Brulez

Learn how to analyze, unpack and code unpackers for software packers and protectors. Attendees will receive hands-on experience working with the ReversingLabs TitanEngine framework, designed for unpacker creation.

Instructors: Tomislav Pericin and Nicolas Brulez
Dates: 6-8 July 2010
Availability: 10 Seats

Day 1: Static file analysis and static unpacker coding

The focus of the first day is manual file unpacking and static file analysis. We go into deep format analysis to create both simple and more complex static unpackers.

We will focus on real-world protections you are likely to encounter on a day-to-day basis.

Day 2: Dynamic file analysis and dynamic unpacker coding

The second day will cover manual file unpacking and dynamic file analysis. We go into deep format analysis for creating simple and more complex dynamic unpackers. Special attention will be given to dynamic unpacker coding layout and the benefits of using TitanEngine to minimize the time it takes to create an unpacker.

Our focus will be on real world packers you are likely to encounter on a day-to-day basis. These packers top the charts in legitimate software compression, but are often used as malware envelopes.

Day 3: Advanced file analysis and coding complex unpackers

On day 3, we will cover the manual unpacking of complex file packing and protection systems. Special attention will be given to methods used to harden against format reverse engineering and prevent unpacking. We will describe common protection techniques utilized by both legitimate software protectors and those specifically designed for use in malware. We will then use information to show coding techniques needed for such complex dynamic unpackers and ways to counter all the tricks used to harden detection, analysis and unpacking.

Our focus will be on the real-world protections you are likely to encounter on a day-to-day basis.

More info here...

This is the second "Ask a developer Monday," in which we answer the most common question we've received recently. The current No. 1 question is: "Why is the entry point after unpacking located in the section named UPX0?"

This is a more complex question than you might think, because it requires understanding the memory models used by software packers, which makes it a perfect question for "Ask a developer Monday"! To answer it we must explore the memory models used, and the possible results that an unpacker could produce.

The first memory model is the one  typically used by software packers. Its main characteristic is a greatly reduced number of portable executable sections - lowered to only two or three, depending on the packer solution itself. With this model, all sections in the file before packing are merged into a single section that is always the first section in the packed file. Most commonly this first section in the packed file holds no data and it is used just as a slot to reserve memory which will be filled after the packer stub finishes the decompression. The virtual size of this first packed file section is equal to the SizeOfImage of the file before it was packed. This gives the packers a powerful option for compressing all code and data in one pass as a single compressed data stream. It also speeds up the time needed to decompress the entire packed content and lowers the compressed file's size because it needs only a single decompression dictionary. The compressed data is commonly stored in the packer section which is either in the second or the third section of the packed file. Since resources must be aligned to SectionAlignment, they commonly get their own section - usually the last section of the file packed  using this memory model.

Although the first model brings faster decompression and smaller files to the table, it has the disadvantage of slightly increasing the memory usage for the packed file. Since memory usage is only increased by the size of the compressed content if that content is displaced from its original location and moved to the packer section, the problem can be avoided by  using a packer that uses a memory model in which the compressed data is stored at its original location. This kind of packer individually compresses the portable executable sections and stores the compressed data inside the same section. With this model, the packer preserves the  section layout the file had prior to packing. Commonly, only one section is added to the original file layout, and that section only contains the packer stub. Compression here is achieved by reducing the physical size of the individual sections. There is a hybrid approach which combines these two memory models, but there isn't a software compression solution that uses it.

Now how does this apply to dynamic unpackers? Since a dynamic unpacker executes the file until it reaches its original entry point and performs a memory dump once that point is reached, it has no impact whatsoever on the memory model used by the packer. That means that the file section layout before and after unpacking will remain the same, with exception of the sections added to the file by the unpacker. These new sections contain the import and relocation data, while the old sections hold the decompressed code, resources and data. There is no way to restore the memory model to its original layout if that kind of data isn't preserved in the packer stub. Since that data isn't commonly preserved by any software packers, a dynamic unpacker can't restore the original memory model layout. And since UPX uses the first memory model as its entry point after the file is unpacked, it will be moved from the section UPX1 to UPX0.  The thing to remember is that section names are not important, what is important is that the data and the code itself are decompressed and this is achieved by the dynamic unpackers.

• Examples for the fist memory model are: UPX, FSG, RLPack, etc.
• Examples for the second memory model are: PackMan, ASPack, AlexProtector, etc.

That is it for this weeks Q&A, until next time...