SymHash: An ImpHash for Mach-O

In the past the Windows Portable Executable (PE) format has been analyzed far and wide due to the historical large scale adoption of the platform. In contrast the Mach-O binary format (executable file format used by MacOS X, IOS, and other Mach based systems) has received much less attention. This is partially due to the comparative lack of malware and forensics engagements focusing on the Mach-O platform. As the Apple operating systems gain adoption this is changing. A few weeks ago Kaspersky’s GREAT released a <a href="https://securelist.com/blog/research/75990/the-missing-piece-sophisticated-os-x-backdoor-discovered/" target="_blank">report</a> on a Mach-O variant of the ekoms spyware.There are many methods to find similar PE files as described below. This post describes a new form of finding similar Mach-O files.The Mach-O executable format is similar but significantly different than the Windows PE format. For starters Mach-O FAT files can include binaries for multiple architectures in a single package as seen in the simplistic overview of the format in Figure 1. In the Windows PE format, there are multiple PE sections, which can be named by the compiler.<img src="https://cdn.filestackcontent.com/eBXC3a1QOGMtSolHa8By"/>Figure 1: Simplified Mach-O FAT file formatIn 2014 FireEye released <a href="https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html" target="_blank">Import Hashing</a> as a tool for analyzing the Windows Application Program Interface (API) functions used by Windows PE files. The Imphash was integrated within the <a href="http://blog.virustotal.com/2014/02/virustotal-imphash.html" target="_blank">VirusTotal platform</a> shortly afterward, and has been a favorite pivoting tool of analysts ever since.While Anomali Labs was experimenting with many different methods of hashing to aid in analyzing Mach-O binaries,most of the forms of hashing were found to be extremely brittle. As we gained an understanding of the Mach-O format, we realized that none of the current segments were a 1:1 match to the Windows PE ImpHash, nor were any of them as powerful as that method.Anomali Labs used open source libraries such as ‘macholib’ and ‘machoinfo’ from CRITS. Macholib provides an API call to retrieve the hash of different MachO Segments. Due to the large amount of Sections within each segment. The machoinfo library from CRITS provided an easy to use interface to retrieve Mach-O section hashes. This technique worked better than Segment Hashing, but due to the wide variety and placement of Mach-O sections comparing sections well was complicated. Additionally sections that were semantically the same, could result in different hash values because of additional data included with the section.<img src="https://cdn.filestackcontent.com/BFQqevVqTLeb18ZXGi9c"/>Figure 2: Symhash CodeSymHash follows the same methodology as ImpHash but for Mach-O executables. Using the CRITS machoinfo library, we retrieve the Symbol Table from each Mach-O entity (that is, each Architecture’s binary within the Mach-O FAT file). Then for each symbol we check to see if it is an ‘external’ symbol, that is, a symbol that is resolved in an external library, and of n_type 0x00. We then append that string to a list, and finally create the md5 hash value of that list of externally referenced symbols.The symhash allows the analyst to find additional executables that share the exact same set of externally referenced symbols. The symhash fails in the same situations as the imphash. This includes:<ul><li>Programs with simple functionality with very little dynamic / external libraries.</li><li>Packed programs. While packing is much less prevalent in Mach-O executables due to the executable architecture and restrictions on Read/Write/Execute sections, X-code offers a built in form of copyright protection which is used. (insert link)</li><li>There are cases where programs reference the same dynamic / external libraries.</li></ul>In our expirement, a corpus of 1140 files produced multiple different segment, section, and symbol table hashes. In comparison tests, the symbol table hashes proved to be much more useful in connecting files. Within the 1140 Mach-O files, there were 1450 entities which resulted in 586 unique symbol table hashes. Many of these Symbol tables are found within Potentially Unwanted Applications and Adware, which make up the vast majority of Mach-O AV triggering software. One cluster of malware identified in table 1 below, has the symhash of 9190c790e6404b0eacfd05402ad85e0e which corresponds to the NetWire RAT.<table><tbody><tr><td>Malware SHA1 Hash</td><td>Symbol Table MD5 Hash</td></tr><tr><td>28d834252374a3c605e139790a43487153c2804e</td><td>9190c790e6404b0eacfd05402ad85e0e</td></tr><tr><td>61c2c1e8d4ba4c39d0fc77ba6cf707bdbaa0e6e7</td><td>9190c790e6404b0eacfd05402ad85e0e</td></tr><tr><td>3ae364f9621e490f8ad5c7d5e85bb913e55d0985</td><td>9190c790e6404b0eacfd05402ad85e0e</td></tr><tr><td>652a6adfa999ad89042d049f33a769f1716650e4</td><td>9190c790e6404b0eacfd05402ad85e0e</td></tr><tr><td>c74192554222d915329be67b4630c7110199226e</td><td>9190c790e6404b0eacfd05402ad85e0e</td></tr></tbody></table>Table 1: Samples of the NetWire RAT which share the symhash of 9190c790e6404b0eacfd05402ad85e0eIf you would like to use symhash in your analysis, checkout our newly open sourced (MIT license) library and command line tool found at <a href="https://github.com/threatstream/symhash/" target="_blank">https://github.com/threatstream/symhash/</a>. Feedback and pull requests are welcome.