Malware Analysis: Types, Behavior, Detection & Reverse Engineering

Malware: Definition and Classification

Malware (malicious software) is any program or code designed to harm, exploit, or infiltrate computer systems without user consent. It exploits vulnerabilities in software, operating systems, or human behavior to gain unauthorized access or damage data.

Types of Malware

Malware can be classified based on its behavior, propagation method, and intent. Below is a structured classification with examples:

Key Characteristics of Malware

---

In the Real World

1. LockBit Ransomware (Global)

   • How it uses malware analysis concepts: LockBit is a ransomware-as-a-service (RaaS) that encrypts victim data and demands Bitcoin payments. Security researchers analyze its behavioral patterns (e.g., lateral movement in networks) and static code (e.g., strings like LockBit2.0) to develop decryption tools. Companies like Ncell or NTC in Nepal must monitor for such attacks via intrusion detection systems (IDS).
   • Real-world impact: In 2023, LockBit targeted hospitals in Europe, disrupting patient care. Analyzing its command-and-control (C2) communication helps block attacks.

2. Emotet Trojan (Nepalese Banks)

   • How it uses malware analysis: Emotet spreads via phishing emails with malicious Word/Excel macros. Banks like Nabil Bank or Global IME analyze its packed executable (using tools like PEiD or Ghidra) to reverse-engineer its C2 server communication. Dynamic analysis (e.g., Wireshark) reveals its keylogging and credential-stealing behavior.
   • Worked example: If a bank employee opens an infected Excel file, Emotet drops a Trojan dropper, which then downloads the full payload. Analyzing the network traffic during infection helps detect similar attacks.

3. Android Spyware in Mobile Banking Apps (eSewa, Khalti)

   • How it uses malware analysis: Fake banking apps (e.g., Fake Khalti) contain spyware that logs keystrokes and steals OTPs. Security firms use static analysis (e.g., AndroGuard) to decompile the APK and identify hardcoded C2 servers. Dynamic analysis on a sandboxed device captures the malware’s data exfiltration to malicious servers.
   • Real-world impact: In 2022, a spyware campaign targeted Nepali users via fake eSewa update links, stealing ₹100M+ in transactions.

---

Malware Analysis Techniques

Malware analysis is divided into two primary methods: static and dynamic.

1. Static Analysis

Static analysis involves examining malware without executing it. This is safer and often used for initial triage.

Key Static Analysis Methods

• File Header Analysis: Inspecting the Magic Numbers (e.g., MZ for PE files, PDF for PDFs) to identify file types.

• String Analysis: Extracting readable text (e.g., URLs, registry keys) using tools like strings or PEiD.
• Packer Detection: Identifying if malware is packed (compressed) using tools like Detect It Easy (DIE).
  • Why it matters: Packed malware evades signature-based antivirus (AV) scans.
• Yara Rules: Writing custom rules to detect malware families.

  rule LockBitFamily {
      meta:
          description = "Detects LockBit ransomware"
          author = "Nepal Cyber Security Team"
      strings:
          $s1 = "LockBit2.0"
          $s2 = "!@#$%^&*()"
      condition:
          all of them
  }
  

Tools for Static Analysis

---

2. Dynamic Analysis

Dynamic analysis involves running malware in a controlled environment (e.g., sandbox) to observe its behavior.

Key Dynamic Analysis Methods

• Sandboxing: Running malware in an isolated VM (e.g., Cuckoo Sandbox, Joe Sandbox).
• Behavioral Monitoring:
  • Process Injection: Malware injects code into legitimate processes (e.g., svchost.exe).
  • Network Traffic Analysis: Capturing C2 communication (e.g., DNS tunneling, HTTP POST requests).
  • File System Changes: Detecting data encryption (ransomware) or file deletion (wipers).
• Debugging: Using x64dbg or OllyDbg to step through malicious code.

Tools for Dynamic Analysis

---

Reverse Engineering Malware

Reverse engineering involves disassembling and analyzing malware code to understand its functionality.

Steps in Reverse Engineering

1. Disassembly: Convert binary to assembly (e.g., using Ghidra or IDA Pro).

2. Decompilation: Convert assembly to high-level pseudocode (e.g., Ghidra’s decompiler).
3. Function Analysis:
   • Identify cryptographic functions (e.g., AES encryption in ransomware).
   • Find hardcoded keys or C2 server IPs.
4. Patch or Decrypt:
   • Modify malware to prevent execution (e.g., patching a jmp instruction).
   • Extract decryption keys to recover encrypted files.

Real-World Example: Analyzing a Ransomware Sample

Scenario: A Nepali university’s server is infected with WannaCry ransomware. Security analysts need to reverse-engineer it to develop a decryption tool.

1. Static Analysis:
   • Use PEiD to confirm it’s a Windows PE executable.
   • Extract strings to find the encryption key (0x1234ABCD).
2. Dynamic Analysis:
   • Run in Cuckoo Sandbox to observe:
     • File encryption (.txt → .txt.wcry).
     • Network call to 8.8.8.8 (WannaCry’s C2).
3. Reverse Engineering:
   • Use Ghidra to disassemble the encryption function.
   • Find the XOR key used for encryption.
4. Decryption:
   • Write a Python script to reverse the XOR operation and recover files.

---

Malware Detection and Mitigation

Detection Methods

Mitigation Strategies

---

Exam Tip

What to Expect in the Exam

1. Theory Questions (30-40%):

   • Define malware, static vs. dynamic analysis, and reverse engineering.
   • Explain how ransomware encrypts files (e.g., AES, RSA).
   • Compare Trojan vs. Worm propagation methods.

2. Scenario-Based Questions (40-50%):

   • Case Study: Given a malware sample (e.g., a packed executable), describe how you would analyze it.
     • Example: "A bank’s ATM system is infected with a malware that logs keystrokes. How would you analyze it?"
     • Expected Answer:
       • Use static analysis (PEiD, strings) to check for packers.
       • Run in Cuckoo Sandbox to capture keystroke logging.
       • Reverse-engineer the keylogger DLL using Ghidra.
   • Mitigation: "How would you protect a university network from Emotet?"
     • Expected Answer:
       • Deploy email filtering to block phishing.
       • Use EDR tools to detect suspicious processes.
       • Train staff on not opening unknown attachments.

3. Tool-Based Questions (20-30%):

   • "Show how you would use Wireshark to detect C2 traffic."
     • Expected Answer:
       • Filter for unusual DNS requests (dns.query).
       • Look for HTTP POST to suspicious IPs.
   • "Explain how YARA rules work with an example."
     • Expected Answer: Provide a YARA rule for LockBit (as shown earlier).

Key Formulas/Concepts to Remember

• Polymorphic Malware: Changes its signature on each infection.
• Metamorphic Malware: Rewrites its entire code while keeping functionality.
• C2 Communication: Malware phones home to a command-and-control server (often via DNS tunneling or HTTP).
• Encryption Algorithms: Ransomware often uses AES (symmetric) + RSA (asymmetric).

Common Pitfalls

• Assuming all malware is a virus: Some malware (e.g., spyware) doesn’t replicate.
• Ignoring static analysis: Always check strings and headers before dynamic analysis.
• Running malware on a live system: Always use a sandbox or VM.

---