How to Start Reverse Engineering in 2021 / Habr - The first knowledge sharing application in Vietnam

Reverse engineering might seem so complex, that not everyone has the bravery required to tackle it. But is it really that hard? Today we are gonna dive into the process of learning how to reverse engineer.

First of all, try to answer yourself, what are you hoping to achieve with reverse engineering? Because reverse engineering is a tool. And you should choose the right tool for your task. So when reverse engineering might be useful:

If you want to know, how some piece of closed source software works. For example, there are a number of communities dedicated to reverse engineering old videogames.
Malware analysis. Malware often comes in a form of one or several binary files, usually heavy obfuscated. And all of the major Antivirus companies have departments dedicated to analyzing malware. They use a number of methods including reverse engineering. And their efforts aimed to detect malware faster and more efficiently. Sometimes they can even take down C&C centers of malware.
Security audit of closed source SW. Find vulnerabilities and stuff.

And if you have a clear goal in your mind, you can always keep your eyes on the prize while you are learning. It’ll help you to focus on the most important information and skills. But if you haven’t decided yet, it’s completely OK. You can start now, get a taste of reverse engineering and then decide later what are you liking the most. Or maybe you even want to do all of that thing simultaneously.

Now to the fun stuff! I’ve divided beginner reverse engineer skills into 3 tiers. As always, links to all the resources I talk about are in the description.

Mục lục bài viết

Tier 1

It’s when you have nothing but an eager desire to learn reverse engineering. So that should you learn on this tier?

Assembly language. It doesn’t really matter what assembly language to learn first. Because most assembly languages are very similar. But I suggest starting with x86, because there are far more resources about it, and far more tools that support it, than any other language. My top resources for learning x86 assembly are: https://github.com/0xZ0F/Z0FCourse_ReverseEngineering and https://beginners.re
Also, a bit of knowledge of executable file formats will be useful for you. Each operating system uses its own executable: Windows uses PE, Linux and other Unix-like systems use ELF.

Tier 2

Don’t spend too much time on tier 1. In fact, after just about 5 hours spent on tier 1, you can start learning something from tier 2. It will give you a much greater sense of progress, and learning tier 2 will help you with tier 1.

Start using reverse engineering tools like Ghidra (free), Radare 2 (free), Binary Ninja (paid), IDA Home/Pro (paid). IDA is the oldest of them and it’s still considered the industry standard for static reverse engineering. But also it’s the most expensive option. Ghidra might be a great tool for beginners: it’s free, it offers lots of features you can’t find in other free tools, and it has a lot of extensibility. There are plenty of online tutorials on Ghidra, plus there’s a solid book called ‘The Ghidra Book: The Definitive Guide’ on the topic. You can find the link to this book in the description.
Maybe even write some simple programs in C and then load them in the tool of your choice to see how they work. Also, study executable file formats.
Then, Learn how debugging works. It’s a process of dynamically examining and modifying the workflow of a program. Your go-to debuggers are GDB on Linux and x64dbg on Windows.

Tier 3

This is where it gets real. It’s time for you to get your feet wet and check out some close to real-life scenarios. Also, it’s where you can get a taste of different aspects of reverse engineering and pick some areas you want to focus on.

Find some challenges online. Those might be simple crackme challenges from resources like https://crackmes.one. It’s where you can put your reverse engineering skills to good use.
Then, you can learn about common security vulnerabilities in binary software and try to find and exploit them. You can start your journey into vulnerability exploitation from https://www.vulnhub.com, http://root-me.org, http://pwnable.kr, and https://www.hackthebox.eu.
Get some info on packing and obfuscation. Though those are not super-common in legitimate software, they’re often used in malware and stuff like that. So those skills might be handy.
Play some CTFs. There are CTFs and CTF challenges for every taste. REing and exploitation challenges are probably included in every CTF nowadays. Flareon CTF is all about reversing malware-like challenges, and RealworldCTF is about exploiting vulnerabilities in real-life software. All CTFs are time-bound but don’t be ashamed if you can’t solve a challenge right away. Take your time, there is no rush. You can try to solve a challenge even if a CTF has ended. Also, if you’re stuck, try to google write-ups for your challenge. http://ctftime.org is a great CTFs and writeups database.
And finally — try to reverse engineer real stuff. Maybe there’s a piece of software you’ve always been curious about. Or you always wanted to hack and get a root shell on your home router. It’s time to give it a try.

Although I talked about 3 tiers, I highly encourage you to do all of them almost simultaneously. Learning is an iterative process. Those tiers can work off one another and help you get as much knowledge and practice as possible.

Final thoughts

And never stop learning: find online tutorials, follow people on Twitter, join communities Reddit, be curious, and don’t be afraid to fail!