Malware Analysis: A Beginners Guide
An introductory guide to malware analysis, covering static and dynamic analysis, and types of malware.
In this post, we’ll explore the basics of malware analysis, from understanding the common techniques used to the different strains of malware that create the digital ecosystem of today. The goal of this post is to introduce these concepts to you at a high level. We’ll be diving into greater detail on each section in the later part of this series.
What is Malware Analysis?
Malware analysis is a critical skill in the cybersecurity world. It allows researchers to dissect malware and study its inner workings. Once the malware has been analyzed, the artifacts and behaviors extracted can help us understand and defend against it in the future.
With over 1 billion variants of malware and millions of cyberattacks occurring annually, understanding the malware ecosystem and how to defend against it is more crucial than ever.
Before diving into the details of malware analysis, it’s important to understand its purpose, common terminology, types of malware, and the approach we’ll take. Malware can be considered anything that causes harm to a user, computer, or network. This includes software like RATs (Remote Access Trojans), worms, spyware, ransomware, rootkits, credential stealers, and more.
To see where these steps fit into the bigger picture, take a look at the malware attack lifecycle graphic below.
Malware and its variants are constantly evolving, but many common techniques are used to analyze it. The specific technique used will depend on the malware being analyzed and the goals of the analyst.
The Purpose of Malware Analysis
The goal of a malware analyst is to gather the information needed to respond to a network intrusion or create rules to defend the network in the future. After analyzing malware, the analyst should be able to determine what the binary is capable of, how it can be detected on the network, and how its damage can be contained.
Imagine being tasked with analyzing malware that wreaked havoc on your network. You’re sent the sample, equipped with the tools to analyze it, and now everyone is looking to you for answers. What would you do next?
If you’re unsure, the goal of this blog series is to get you up to speed so you can take immediate action when faced with such a task.
One of the most effective ways to defend against malware is by developing host-based and network-based signatures.
Host-Based Signatures: Also known as Indicators of Compromise (IOCs), are used to detect the presence of malware on a victim’s computer. Indicators include the presence of unknown files, execution of unknown processes, unusual outbound connections, and more. These signatures are particularly effective against malware that modifies the host system, as such changes leave detectable artifacts. Once identified, rules such as YARA rules can be created to consistently detect the same behavior or artifacts across multiple systems.
Network Signatures: Network signatures are observable indicators or patterns in network traffic generated by the malware, which are often captured during its execution. While network IOCs can be identified without performing malware analysis, analyzing the malware sample itself can reveal additional indicators. Combining indicators and cross referencing malware analysis with network traffic monitoring during execution provides the most comprehensive understanding of the malware’s behavior. This is essential for understanding the extent of the infection.
For example, here’s a sample YARA rule that illustrates how such host-based indicators can be translated into actionable detection logic:
Pre-requisites
Basic malware analysis techniques can be performed by those with little to no experience in security or programming. Thanks to advanced tools, you can go from knowing nothing about malware to extracting Indicators of Compromise (IOCs) rather quickly. However, having programming experience is recommended before diving into malware analysis. The necessary security-related knowledge will be taught as we progress through this series.
Types of Malware
- Ransomware: Malware that encrypts files or systems, demanding payment (often in cryptocurrency) for decryption keys. It frequently targets large businesses, critical infrastructure, and even hospitals.
- Example strain: WannaCry, which surfaced in 2017 and caused a global crisis by exploiting a Microsoft Windows vulnerability. Within hours, WannaCry infected hundreds of thousands of systems, affecting healthcare institutions, government agencies, and businesses worldwide. We’ll conduct a detailed analysis of WannaCry in a future post—stay tuned.
- Trojans: Malware disguised as legitimate software to trick users into installing it, often used to deliver additional payloads. Once inside the system, trojans provide attackers with a foothold, enabling further malicious activities.
- Example strain: Emotet, known for distributing other malware and evolving its techniques over time.
- Worms: Self-replicating malware that spreads autonomously across networks without user interaction, often leading to rapid, widespread infections.
- Example strain: Conficker, which famously compromised millions of systems worldwide.
- Rootkits: Malware designed to hide itself and other malicious software at a deep, often kernel-level layer. This can evade traditional detection methods and make removal particularly challenging.
- Example strain: ZeroAccess, known for its stealth and resilience.
- Spyware: Malware that collects user data without consent, often used for espionage or financial gain. Some spyware strains are highly advanced, exploiting zero-day vulnerabilities and bypassing security measures.
Example strain: FinFisher, a surveillance toolkit known for its sophisticated anti-analysis measures.
Another notable example is Pegasus, which gained notoriety for its ability to infiltrate mobile devices—both iPhones and Androids—through zero-day exploits. Discovered in 2016, Pegasus could access messages, calls, and even the device’s microphone and camera, targeting journalists, activists, and government officials worldwide. If interested, read this Citizen Lab article.
- Keyloggers & Credential Stealers: Malware that records keystrokes or extracts stored credentials from browsers and applications. These threats are often combined with other malware types to maximize damage, such as granting attackers access to sensitive accounts or corporate systems.
- Example Keylogger strain: Agent Tesla
- Example Credential Stealer strain: RedLine Stealer
- Botnets: Malware that turns infected machines into a “bot” network for orchestrating large-scale attacks, spam campaigns, or DDoS assaults.
Example strain: Mirai Botnet, which exploited IoT devices for record-breaking distributed attacks.
The image below shows a dashboard from a DDoS-as-a-service platform built around a variant of the Bushido botnet. It displays nearly 17,000 compromised bots collectively capable of generating over 400 GB/s of traffic—more than enough firepower to overwhelm many online targets. Fortinet
- Backdoors: Malware that provides unauthorized, persistent access to a system. Backdoors allow attackers to maintain control and continue stealing data or deploying additional payloads.
Example strain: PlugX, commonly used by advanced threat actors for long-term espionage campaigns.
A real-world incident involved a threat group known as Salt Typhoon, believed to be operating from China, which exploited vulnerabilities in U.S. telecommunications companies. By leveraging backdoors created for U.S Law Enforcement/Intelligence, they gained access to sensitive call records and communications, including systems involved in wiretapping requests. This case underscores the severe national security implications that backdoors can pose when abused at scale. Read more in this Trend Micro article.
- Fileless Malware: Malware that resides in memory and avoids leaving traditional file-based artifacts. This makes detection more difficult, as there are fewer conventional signs to analyze.
- Example strain: Cobalt Strike, a legitimate penetration testing framework frequently abused by attackers to deploy malicious payloads directly into memory and maintain control of compromised systems.
- Cryptominers: Malware that hijacks a system’s resources—CPU, GPU, or both—to mine cryptocurrency. This can degrade system performance, increase power usage, and lead to higher operating costs for businesses.
- Example strain: CoinMiner, often targeting personal and enterprise devices to generate illicit profit for attackers.
Malware often shares multiple attributes from the categories mentioned above. For example, a sample could be a trojan containing a keylogger and backdoor, which enables initial access for deploying ransomware. These classifications help categorize malware at a high level.
Techniques for Malware Analysis
If you thought you’d be given the source code of the malware sample to analyze you’d be sorely mistaken. In most cases, you’re given the compiled version which isn’t human readable and you’re goal is to try and understand it. This is where the tools and techniques mentioned come into play. Each tool has a purpose in unveiling what the sample is doing bit by bit. There are two approaches for analysis, one being static analysis and the other dynamic analysis. These can later be broken into basic static analysis, advanced static analysis, basic dynamic analysis, and advanced dynamic analysis. As the name suggest, static analysis is analyzing the malware without running it and dynamic analysis involves analyzing the sample while running it. Hybrid analysis is another popular technique which combines both static and dynamic analysis.
Basic Static Analysis
Basic static analysis involves extracting information from a binary without examining its instructions. The goal is to quickly assess whether a file is malicious or not by gathering data such as file metadata, strings, and hashes. Common tools for this phase include strings, FLOSS, and PEStudio. Indicators gathered during this phase can be used to create network signatures. Basic static analysis is not comprehensive and may fail to detect advanced malware, so this limitation should always be kept in mind when analyzing a sample.
Advanced Static Analysis
Advanced static analysis consist of reverse engineering the malware’s internal workings by loading the binary into a disassembler to examine its low-level instructions. A disassembler is a tool that converts machine code from a binary into assembly language, some of the most popular disassemblers include Ghidra, IDA Pro, and Binary Ninja. Mastering advanced static analysis can be very challenging, as it demands a strong understanding of operating system fundamentals, assembly language, and programming constructs.
Basic Dynamic Analysis
Basic dynamic analysis is one of the most engaging parts of malware analysis. This is where the sample is detonated to observe its behaviors on the system. By monitoring the network and host before, during, and after detonation, you can identify the Indicators of Compromise (IOCs) mentioned earlier. These indicators can then be turned into signatures to help detect or protect against the malware in the future.
However, similar to basic static analysis, dynamic analysis may not always provide a comprehensive picture. Malware often employs techniques like virtual machine (VM) detection and other safeguards to evade analysis. Detonating a sample with these safeguards in place may lead you to believe the file is not malicious, when in reality, it recognized it was being analyzed and either withheld certain functions or activated decoy behaviors to mask its true malicious nature.
An isolated and secure environment is required for dynamic analysis
Advanced Dynamic Analysis
Advanced dynamic analysis often involves using a debugger. A debugger is a tool that allows analysts to run a program step by step, inspect its execution flow, and analyze its behavior in real time. It’s essential for identifying runtime actions such as function calls, memory modifications, and network activity. Popular debuggers include x64dbg, GDB, and WinDbg.
One of the key advantages of using a debugger is the ability to walk through the code step by step. For example, in basic dynamic analysis, I mentioned how malware can detect that it is running in a virtual environment and withhold its malicious behavior. How can you uncover this behavior? By stepping through the code, you can identify checks for the presence of a hypervisor and modify the relevant flags, forcing the malware to execute its hidden functionality. This technique enables you to extract more information from the sample than would otherwise be possible.
Conclusion
In this post, we introduced the core concepts of malware analysis, including its purpose, techniques, and types of malware. As we progress through this series, we’ll dive deeper into the tools and methodologies needed to perform both static and dynamic analysis. Stay tuned for the next post, where we’ll explore the foundational steps for setting up a secure malware analysis lab.
Ready to learn more? Stay tuned for the next post in our malware analysis series!