Reverse Engineering - Getting Started Guide

Reverse Engineering - Getting Started Guide

Can't get SoftICE to work with Windows 2000, XP?, read here first!



What is Reverse Engineering?



What do I Need?



Getting Started



CompuWare Products Overview



Cracking Etiquette



Other Resources



Protection Types



Patchers



Jump Right In!

If you are considering studying the art of software reverse engineering, then this guide below is for you. I'll try to outline here everything you need to know and do (of course this is by no means an exhaustive list or guarantee that you'll become a reversing god overnight but it might just get you started in a whole new world). If you are at all serious then you should take heed and the time to download all of my recommended materials, all the time you invest learning now will serve you well in the future. It will also be worth your while to visit some of the other sites I've linked too on the web. After reading this document and attempting the 2 small sample programs I've made available you'll know whether or not this really is the art for you.

What is Reverse Engineering (precisely)?

Software reverse engineering is the art and process of understanding the intricacies of your own and commercial software at a lower level than the compiler, a fuller definition can be found here. Many reversers focus initially on the various protection schemes used by software writers to disable or otherwise prohibit the full use of their software since this is a convenient (if somewhat legally dubious) starting point with a definite challenge and end point. I personally however have used the knowledge I have gained through 'reversing' to :

i). Produce my own custom tools for circumventing / identifying protections.

ii). Recover usable source code to lost projects.

iii). Identify and understand how specific functionality is implemented.

iv). Debug hard to find errors.

v). Perform analysis of potentially hostile programs.

Sometimes reverse engineering can be the only way out of a development tight spot, however it is not a decision to be taken lightly.

Reverse Engineering is NOT cracking per se, although it is sometimes difficult to draw the fine line between them in the early stages. Most reversers deplore the tens of thousands of warez sites that waste good server space on the web (you probably know them already). If you are looking for easy cracks, key generators or just serial numbers lists then this site and reverse engineering will NOT be for you, even though this information can be obtained with fairly minimal effort I expect most warez aficionados will not find themselves reading this in the first place and certainly won't have a clue how to code, assemble and link a key generator, let alone spend hours upon end studying assembly routines.

By learning to reverse engineer yourself, you are gaining a set of valuable and marketable skills (malware analysis, intellectual property rights management and anti-virus / vulnerability research are booming industries), thus distinguishing yourself from the many losers who would rather waste their time searching through pages of bloated graphics and commercial porn sponsors than learning anything themselves. You'll also find (over a period of time) that your reversing efforts will become less focused on protection schemes and that your interest will move away from simple protection cracking, who knows, perhaps a job in hostile code analysis beckons.....

What do I need to know / learn ?

To learn reverse engineering from scratch you will probably need to spend a significant amount of time enhancing your low level knowledge, don't think you can crack any target you fancy by just learning ad nauseam simple techniques. A familiarity with the x86 architecture and instruction set is essential, an awareness of the 6 basic digital logic circuits (binary) will also be useful (AND/OR (inclusive), NOT, NAND, NOR & exclusive OR (XOR)).

I recommend the following reading resources :-

Art of Assembly Language :- A 25 chapter PDF guide to virtually everything you might ever want to know about x86 processors. These documents are very complete yet reading them all will probably take you in excess of a few years so read just the first few chapters and keep the rest like Chapter 14 on the FPU for reference purposes as you improve / require.

HelpPC :- A 220k quick and convenient DOS instruction viewing program from 1991. If you've forgotten a particular assembler command or need to quickly look up how many clocks a particular instruction takes, then this is the guide for you (it is somewhat dated though).

Iczelion's Win32 ASM Resources :- A great site with literally tons of useful resources. Download everything there :-). If you want to really 'get into' windows assembly language programming there isn't much better for free than Iczelion's tutorials.

Intel Developer Manuals :- Anything you ever wanted to know about the nitty-gritty internals of your x86. I recommend Volume 3 (System Programming). I have been told recently that the previous link does not lead to all 3 manuals, you might like to try this link instead. You could also search for 386intel.txt for a good overview. Update 2004 : I believe now the Developer manuals now stretch to 4 guides, either way you shouldn't have much problem finding them.

Mammon_'s Tales to his Grandson & Mammon_'s coming to the Iceage :- 2 definitive guides to configuring your SoftICE and synopses of the main 3 disassemblers by one of the very best reverse engineers out there (25k). Mammon_ abandoned the Windows scene a considerable amount of years ago, an eccentric and enigmatic character, his website still makes for fascinating reading.

Nolan Blender's "Making Tools Work Together" :- How you can use IDA & SoftICE to maximum effect (related to FLEXlm but applicable elsewhere).

PC Assembly Tutorial :- Dr Paul Carter's free introduction to assembly language (32-bit) using NASM (since its free), taught previously as a university course. Recommended.

Ralph Browns Interrupt List :- A maintained list of all DOS BIOS/Interrupt Services, most of the time you'll be looking for subfunctions of INT 10/13/21. Invaluable for older 16-bit programs or coding your own graphics demos / key generators (even understanding old virii). Somewhat dated now thus I've changed my recommendation from learning this to keeping it just for reference.

Getting and Setting up your Tools

*Updated 2007* : CompuWare have now officially ceased all development upon SoftICE as a product, those of us who watch the scene closely could see this coming for sometime, the text below I leave now as a dedication to the past. Farewell.

Any reverser will tell you that you will only ever be as good as the tools you use and the competency with which you use and customise them. Your best weapons are your tools, invest the time learning how to use them. I suggest you obtain at the minimum the following (either download them from my tools page (if the links are even working) or locate them around the web using various searching techniques).

- A Windows (preferably protected-mode) Debugger - The standard tool in this category is NuMega's SoftICE which can trace just about anything, you will not break some protections without it. Download the versions relevant to the platform you plan to investigate, better still download every version you can. Pre-2000 most of my guides use v3.2x/v4.0x for Windows 98. Pay a regular visit also to CompuWare's (formerly NuMega's) web site to keep informed of any new developments, these guys really know how to produce useful tools (need I also mention BoundsChecker & SmartCheck). Its also worth hunting down the various homepages and articles by (ex & current) NuMega developers, need I mention Matt Pietrek & John Robbins ;-).

* The advent of more recent Microsoft OS's (Windows 2000, XP) & CompuWare's acquisition of NuMega requires that you now source SoftICE as part of a CompuWare package; in fact I've heard that CompuWare won't even sell legitimate developers SoftICE standalone any longer.

DriverStudio (approx. size 184Mb's)

* Requires Installation Serial Number + FLEXlm license

DriverWorks

DriverNetworks

VtoolsD

SoftICE / Visual SoftICE

Boundschecker / TrueTime / TrueCoverage

The sale of NuMega to CompuWare also seems to have contributed to a major decline in quality control, many users have reported significant problems with SoftICE under the newer OS's, most of these relate to breakpoints not behaving as they should. There are some workarounds and custom patches, which you might find on the RCE MessageBoard (use the search facility), a lot of reversers however have given up trying to get SoftICE to behave reliably and have resorted instead to using the capable ring 3 debugger OllyDbg. This has also the added capacity to work under VMWare which seems to be all the rage right now.

SoftICE symbols

Getting debug symbols loaded into SoftICE can be a challenge to say the least, before attempting to do so, make sure that you download and install the latest 'Debugging Tools for Windows' from Microsoft. Next replace all copies of symsrv.dll & dbghelp.dll installed by DriverStudio with those from the Debugging Tools folder, if I remember rightly the DriverStudio root directory, the SoftICE root directory and the SymbolRetriever subdirectory all have copies of those files that need to be replaced. Also be sure that your 'Path to NMS' is set to a directory that exists.

SoftICE under VMWare

This advice from my good friend nc. If you browse to your VM directory on the hard disk and open the config file in a text editor (.vmx file), add the following lines to the config file :

vmmouse.present = FALSE

svga.maxFullscreenRefreshTick = 5

If you want to verify that SoftICE is working correctly, try the following advice that I shamelessly borrowed from Kayaker.

"If you break at the start of a program with the SoftICE loader (assuming you can), and set a breakpoint say a few lines down, either on an address or an API call - does SoftICE break? It should. Make sure you set your bp *while in the context* of the application you want to break into. This is irrespective of the ADDR command, which you shouldn't have to use since you're already in the correct context. In other words, don't expect to be able to just change the context with ADDR from the desktop and have a reliable bp set. If you do, you also need to specify the CS: portion of the address else you'll set up a bp with the wrong code segment. If all else fails, you could try BPM x breakpoints, they can be more reliable than BPX bp's for "sticking". However, they especially should be set while *in* the context of the app.

This small table should provide you with a means to identify which version of SoftICE you have installed on your system.

DriverStudio v2.7

SoftICE, DriverWorkbench, BoundsChecker, TrueTime, TrueCoverage, DriverWorks, DriverNetworks, VtoolsD (requires installation serial number only).

NTICE.sys file version 4.0.1381, product version 4.2.7 (Build 562). osinfo.dat (191,340 bytes).

DriverStudio v3.1

As v2.7, also Visual SoftICE (requires installation serial number + FLEXlm license).

NTICE.sys file version 5.1.2601.0, product version 4.3.1 (Build 1722). osinfo.dat (304,588 bytes), osinfob.dat (200,027) bytes.

DriverStudio v3.2

As v3.1.

NTICE.sys file version 4.3.2.2485, product version 4.3.2 (Build 2485). osinfo.dat (350,737 bytes), osinfob.dat (375,319 bytes).

Latest from Compuware FTP

N/A.

osinfo.dat (474,346 bytes). osinfob.dat (356,884 bytes - most likely out of date).

DriverStudio v3.2.1 Update

As v3.1. *Update Only* here (1.65Mb).

NTICE.sys file version 3.2.1 (Build 2560), product version 3.2.1 (Build 2560). osinfo.dat (474,346 bytes). osinfob.dat (356,884 bytes).

As SoftICE is virtually every reversers choice of debugger, some of the more intelligent protections will use various techniques to detect its presence. More likely than not you can find a way around most of these yet in certain cases e.g. Hardlock's wrapper and VBox, you'll need to identify precisely the trick before you can work around it, Hardlock is particularly nasty because after disabling the CreateFileA detection you'll wind up with a frozen computer. In said circumstances an alternative debugger can be very useful, such possibilities include Borland's Turbo Debugger (included with TASM & BC++), Microsoft's WinDbg and LiuTaoTao's superb TRW, you know where to look for these :-).

OllyDbg is now highly recommended as the best alternative if your system simply won't take to SoftICE.

- A Disassembler - There are probably 2 main choices for this category, the quicker but less technical W32Dasm v8.9x from URSoftware and the slower more advanced Intelligent Disassembler Pro from Data Rescue. The differences between these 2 are immense, however for instances where you need a quick 'dumb deadlisting' W32Dasm may suffice, serious analysis and analysts however choose IDA. If you have a few spare moments you might also care to investigate some of the older disassemblers such as Sourcer (more for DOS) and WCB for Windows 3.1 although these are largely obsolete. The choice between the main 2 here is really a question of personal preference. Visual Basic v3 and v4 decompilers are also available, although I've never had a great deal of luck with the VB4 edition. For VB5 & VB6 there exists now a p-code debugger courtesy of the WKT team.

If you are really interested in disassemblers then you should check out dsassm02e, a Win32 disassembler written by a South Korean professor, visit his homepage here and download the program with full C source code. Web searchers might like to try looking for material written by Australian Christina Ciffuentes, especially her thesis on decompiling to recover source code.

- A HEX Editor - In this category there at least a dozen choices, most reversers will however develop their favourite, mine being DOS Hiew. Conventional search engines (e.g. the Simtel archive) will find at least 30 HEX editors (some better than others), of the many out there in the woods the following seem to be popular with reversers. Hex Workshop, UltraEdit, HEdit (* note HEdit appears now to be unsupported) you should of course learn how to reverse your tools first)).

- Our Tools - Progress is constantly being made in this area (although it is sporadic), this section is probably out of date several weeks after I write it. Retrospectively, arguably the 2 best developments have been IceDump by The Owl et al & ProcDump courtesy of G-RoM & Stone (now integrated into IceDump). Many other tools have also made an appearance, for example r!sc has done some very good work in the unpacking and CD protection fields, others have contributed with unpackers for specific packers (check out the Unpacking Gods webpage if you can) & Tsehp has contributed Revirgin.

The games scene has also pushed forward the boundaries of our tools, an entire scene is now built around in-memory patching (or 'training') courtesy of Stone and others delving inside the Win32 debug API. In late 1999 Stone's Webnote (a very interesting collection of his own exploits) disappeared from the web, for personal reasons he is reluctant to ever re-upload it, a decision you might not agree with but should respect, a final archive of some of the very interesting material on his site can be found here (1.08Mb's, 1,141,940 bytes).

- Support Tools, room must also be found in any reversers toolbox for the following tools :-

i) File Monitoring (FileMon) & Registry Monitoring (RegMon) from the wizards at SysInternals.

ii) InstallShield script decompiling (isDCC, Wisdec).

iii) Installation Monitoring (CleanSweep from Quarterdeck).

iv) Resource Editor (BRW 4.5, eXeScope, Symantec Resource Studio, Resource Hacker, Restorator).

Cracking Etiquette

Indeed, there is such a thing as the above. When starting out you should probably adhere closely to these pieces of advice else you might make some very nasty enemies (this applies mainly to IRC and message boards).

i) DON'T the first time you join one of these forums issue long lists of requests for tools, specifically SoftICE and IDA. At best you'll be politely told to "learn how to search" and at worst you'll be flamed out of existence, not a great way to make friends in this world. However, there are ways and means of obtaining said tools, public forums being not the place. I know that many reversers in private will help you obtain what you need, yet you'll need to develop some skills identifying those that might help and those that will never.

ii) When you've actually cracked a few programs it is very easy to become aloof and maybe somewhat egotistical, I know this to my cost because I've been there and done it too. As a general rule, its best never to boast or be cocky, trust me someone out there knows more than you & will eventually shoot you down in flames no matter how clever you think you are ;-), you aren't compelled to reply to 'lamer requests' so maintaining a respectful silence is often 10x more effective. No-one on a message board appreciates a reply to a request for help along the lines of "man, you must be stupid, I cracked that in 5 minutes", real help rather than ridicule is the order of the day.

iii) Joining warez groups is a matter for your own consciences, I would guess 50% of the community deplores such groups and 50% tolerates them, I'm one of the tolerant group because you may be able to obtain some very interesting specific targets from these sources, naturally I wouldn't dream of cracking these targets or making them available for the losers to download for free of course. If you are offered hardware incentives to crack for any group you should turn it down immediately (unless of course you have a very secure place to send it).

iv) If you should encounter me on IRC not following my own rules be sure to tell me I'm a hypocrite ;-). The reversing community is much like any other, "do unto others as you would have them do unto you", apply basic common sense and you won't go far wrong.

Other Resources

Download the documentation for SoftICE and please do read it, else read this (something I shamelessly borrowed from a Programming FAQ) :-

One day a Novice came to the Master.

"Master," he said, "How is it that I may become a Writer of Programs?".

The Master looked solemnly at the Novice.

"Have you in your possession a Compiler of Source Code?" the Master asked.

"No," replied the Novice. The Master sent the Novice on a quest to the Store of Software.

Many hours later the Novice returned.

"Master," he said, "How is it that I may become a Writer of Programs?".

The Master looked solemnly at the Novice.

"Have you in your possession a Compiler of Source Code?" the Master asked.

"Yes," replied the Novice.

The Master frowned at the Novice.

"You have a Compiler of Source. What now can prevent you from becoming a Writer of Programs?".

The Novice fidgeted nervously and presented his Compiler of Source to the Master.

"How is this used?" asked the Novice.

"Have you in your possession a Manual of Operation?" the Master asked.

"No," replied the Novice.

The Master instructed the Novice as to where he could find the Manual of Operation.

Many days later the Novice returned.

"Master," he said, "How is it that I may become a Writer of Programs?".

The Master looked solemnly at the Novice.

"Have you in your possession a Compiler of Source Code?" the Master asked.

"Yes," replied the Novice.

"Have you in your possession a Manual of Operation?" the Master asked.

"Yes," replied the Novice.

The Master frowned at the Novice.

"You have a Compiler of Source, and a Manual of Operation. What now can prevent you from becoming a Writer of Programs?".

At this the Novice fidgeted nervously and presented his Manual of Operations to the Master.

"How is this used?" asked the Novice.

The Master closed his eyes, and heaved a great sigh.

The Master sent the Novice on a quest to the School of Elementary.

Many years later the Novice returned.

"Master," he said, "How is it that I may become a Writer of Programs?".

The Master looked solemnly at the Novice.

"Have you in your possession a Compiler of Source Code, a Manual of Operation and an Education of Elementary?" the Master asked.

"Yes," replied the Novice.

The Master frowned at the Novice.

"What then can prevent you from becoming a Writer of Programs?".

The Novice fidgeted nervously. He looked around but could find nothing to present to the Master.

The Master smiled at the Novice.

"I see what problem plagues you." said the Master.

"Oh great master, please tell me." asked the Novice.

The Master turned the Novice toward the door, and with a supportive hand on his shoulder said, "Go young Novice, and Read The Fucking Manual." And so the Novice became enlightened.

Both the Command Reference and Users Manual used to be available at NuMega's public ftp but now ship by default with the installations. There are many tutorials on how to use and customise SoftICE, including mine which forms part of the 1st tutorial. The most common problems with SoftICE relate to the configuration file winice.dat, download Mammon_'s superb guide on all aspects of SoftICE configuration (linked above).

Whilst at Greythorne's site (check out his new Security Nexus too), download all of the +ORC teachings which have the added advantage of including the relevant files, you will also find some useful ASM and other snippets (e.g. gij's tutorials). Even though the +ORC programs are fairly old, read the texts very carefully indeed, I have found them useful on many occasions. If you already have some Windows programming knowledge then you will most likely already possess a Windows 32 API guide, otherwise locate the pertinent help file and download it (all C compilers that I know of carry the guide).

Protections

As a reverse engineer you will encounter several protectionist strategies, a brief appraisal of the most common schemes are listed below.

1. Serial Number/Password protections - These type of schemes are ubiquitous, just look around the web at the serial number lists and key generators available for losers. Usually the protection of choice for cheaper software, you'll usually find only variations upon very simple schemes, maybe some interesting mathematical manipulations, however you should not dismiss programs using these schemes, some such as ACDSee or WinRAR will prove more than enough challenge to the casual reverser.

Recently several serial number schemes have been based on RSA public/private key encryption (ADC v1.2+, IDA v4.x, Hiew, The Bat! to name but a few), so beware of the target requesting just a serial number. I've heard only of several examples of RSA factoring, the maximum key length being 512-bit, I recommend Ghiri's RSA tutorial on Hiew and also the MIRACL maths libraries for factoring sometime this year. RSA of course is crackable, you could for example simply replace the key with a known quantity :-), often using 1 as the decryption exponent will be satisfactory. An increasing number of serial number schemes are using good off the shelf encryption algorithms, ask around for known targets or check out how the algorithms look when compiled.

2. Time trials - With the explosion in magazine cover CD-ROM's, 30 day trials or 'cinderellas' are also common, although Microsoft prefers to allow you 60 or even 90 days to try their software. Time trials are also fairly easy because the amount of tricks a programmer can use is so limited, remember also that in most cases you will have the opportunity to study such a scheme before your time has elapsed. On smaller software be aware that the authors often change the version fairly regularly so reversing a 30-day trial may not even be necessary.

3. Function Disabled - These are becoming less common now, a program author will lock out certain operations (most commonly Save and Print) allowing you to trial his crippled software. In marketing terms disabled software is less likely to encourage potential buyers to try the software, who will spend 2hrs constructing a work of art which cannot be saved.

Disabled software can be either easily reversed or virtually unreversable depending on whether the program author just locked out the functionality or removed the code altogether, more recently I've seen instances of where reversers have actually added back in the relevant saving code as required although this will depend on how much you know about the missing functionality and whether you have the technical information / skills to add it back in. In most cases (like the Adobe trials) your going to need advanced knowledge of the file format and I have my doubts as to whether its practical to invest the time without referring to the widely available full version.

4. Commercial protection schemes - Now becoming more common as the capitalists seek to market web-ready software, you'll almost certainly run into SalesAgent from Release Software & VBox v4.x from Preview Systems, the latter is a pathetic protection which will require no more than 30 seconds SoftICE work, the former is somewhat trickier. Packers such as ASPack, Petite, Shrinker are also becoming more common, but you'll need to read more about these elsewhere (see the newly added 6. section).

5. Hardware/Dongle Protections - Termed as hardware protection, a dongle is a small device that is usually connected to the parallel port of the computer (serial & USB devices also exist). The strength of any dongle protection will be influenced a lot by the quality of the implementation, a lot of them are fairly weak. If you have the actual dongle then reversing it will obviously be a lot easier as you can just examine the relevant INs and OUTs.

As with any protection, information is power so always identify what flavour of dongle you are dealing with and visit the relevant manufacturers web sites, often you'll be able to download full API sources. In some instances, if you do not have the dongle you may have to pray, although no dongle is unreversable, some of the wrappers incorporate sophisticated and unreversable encryption, anti-SoftICE tricks and self-modifying code, so in some cases you would be well-advised to leave the dongle code as it is and patch the application side. The 2 most common dongles are HASP & Sentinel, if you are serious about the dongle game, visit my dedicated LPT parasite page.

6. Packers - Packing software is now very common and is typically marketed more as a code obfuscation tool rather than a protection in its own right (although some do incorporate their own license schemes). The classic symptoms of encountering a packer, either the packers debugger detection lets you know ("Debugger detected" "Please unload your debugger" etc, etc) or you load the file into a disassembler and see nothing but junk and a program entry point somewhere other than the first code section. A packer generally works by compressing the programs main code and attaching a loading stub, at runtime the stub decompresses the program and runs it, this is a complete oversimplification of an entire protection field, however it will suffice for now. I suggest you download PEId v0.92 (2004 version) if you want to check quickly a target for a known packer and then search for various unpacking programs for a quick fix; of course you could do it manually but will need to improve your skills considerably before doing so.

Patchers / How to Patch

Although I disagree with the concept of making ready-made patches for software I recognise that in certain circumstances it can be beneficial for reversers to publish examples of their work. Pages which just distribute lamer cracks are wasted space, hence why I mostly avoid including a patch file leaving you to probe on your own. Anyhow, this inevitably raises the question of whether you should use 1 of the existing patching engines or code your own.

For ease of use I recommend Jes's GPatch (tutorial included in the ready to start tutorial), which generates 4-5k COM files. You may like to examine the source code to a very quick C patcher which I wrote fairly hastily, cranking up the compiler options may well reduce the file size, or you may like to calibrate pitty's very good C++ patcher. Pascal guru's (I am not one) may like to use/modify MisterE's Pascal patcher, you can download all the pertinent source codes here (31k). There are many other patchers available, those written in ASM usually produce the smallest file size, although with the size of modern day HD clusters I doubt this is a real consideration.

Windows patchers and patch generators are also available, RTPatch and WinPatch are 2 that I know of and which are used by some fairly high profile software companies, many of the scene groups have their own sophisticated patchers these days. For those of you who insist on complete optimisation I recommend PCOM (Private COMpiler), although you might have to invest a little time getting to know it.

Ready to Start?

Well, if you've downloaded all of the tools and documentation I recommended and perhaps invested a few days internally digesting all that information, then you might be ready to attempt your first and second projects, see the link below, after attempting these examples you might find some more in the newbies section.

Start Menu Cleaner v1.2 (1) & Teleport Pro v1.29 (2)

Still stuck for a target or 2 to test your skills on?, well what about WinZip or mIRC ;-).

Additionally, why not enhance your OllyDbg skills with this SWF movie tutorial courtesy of lena151 (1.45Mb's).



Return to Main Index

© 1997-2007 CrackZ. Updated 15th July 2007.