IP Security: Protocols, Attacks, and Protection Mechanisms

Core Concepts of IP Security

IP Security (IPsec) is a suite of protocols designed to secure IPv4 and IPv6 communications by authenticating and encrypting packets at the network layer (Layer 3). Unlike TLS (Transport Layer Security), which operates at Layer 4 (Transport), IPsec secures all IP traffic, making it ideal for VPNs, remote access, and sensitive data transfers.

Why IPsec?

• Transparency: Works at the network layer, so any application (email, VoIP, file transfer) benefits without modification.
• Performance: Encryption/decryption happens once per session, not per packet (unlike TLS).
• Flexibility: Supports site-to-site VPNs (e.g., Ncell’s corporate network) and remote access VPNs (e.g., eSewa’s secure login).

1. IPsec Architecture: AH, ESP, and IKE

IPsec consists of three main protocols:

1. Authentication Header (AH): Ensures data integrity and authentication but does not encrypt data.
2. Encapsulating Security Payload (ESP): Provides confidentiality (encryption), integrity, and authentication.
3. Internet Key Exchange (IKE): A protocol to negotiate and establish shared secrets (keys) for AH/ESP.

How AH and ESP Work

IPsec Modes

IPsec operates in two modes:

1. Transport Mode: Encrypts only the payload (used for end-to-end security, e.g., eSewa’s login traffic).
2. Tunnel Mode: Encrypts the entire IP packet (used for VPNs, e.g., Ncell’s remote office connections).

flowchart TD
    A["IPsec Modes"] --> B["Transport Mode\n(End-to-End)"]
    A --> C["Tunnel Mode\n(VPN/Gateway)"]
    B --> D["Encrypts payload only\n(e.g., HTTP, DNS)"]
    C --> E["Encrypts entire packet\n(e.g., VPN between routers)"]

2. Internet Key Exchange (IKE)

IKE is a hybrid protocol combining:

• Oakley (key exchange)
• ISAKMP (key management)
• SKIP (session management)

IKE Phases

1. Phase 1 (IKE SA Establishment)

   • Negotiates a secure channel using Diffie-Hellman (DH) for key exchange.
   • Supports authentication (pre-shared keys, digital certificates, or RSA).
   • Creates an IKE Security Association (SA).

2. Phase 2 (IPsec SA Establishment)

   • Uses the IKE SA to negotiate AH/ESP SAs for actual data transfer.
   • Supports perfect forward secrecy (PFS) (new keys per session).

sequenceDiagram
    participant Initiator as IKE Initiator
    participant Responder as IKE Responder
    Initiator->>Responder: Phase 1: DH + Auth (Pre-shared Key)
    Responder->>Initiator: Respond with DH params
    Initiator->>Responder: Phase 2: AH/ESP SA Negotiation
    Responder->>Initiator: Confirm SA
    Initiator->>Responder: Encrypted Data (ESP)

3. IPsec Attacks and Countermeasures

Real-World Example: Ncell’s VPN Security

Ncell uses IPsec in tunnel mode to secure:

• Remote office connections (AH for integrity, ESP for encryption).
• IKEv2 for mobile users (faster than IKEv1).
• PFS to prevent key compromise if one session is hacked.

In the Real World

1. eSewa’s Secure Transactions

   • Uses IPsec ESP in tunnel mode between eSewa’s servers and banks (e.g., NMB, Global IME) to encrypt all financial data in transit.
   • IKEv2 ensures fast, secure key exchange for mobile payments.

2. Ncell’s Corporate VPN

   • Employees connect to Ncell’s HQ via IPsec VPN (ESP + AES-256).
   • AH ensures no one tampers with routing updates between offices.

3. Nepal Stock Exchange (NEPSE) Data Security

   • NEPSE’s trading servers use IPsec AH to verify integrity of stock price updates and ESP to encrypt sensitive trades.

4. Virtual Private Networks (VPNs) and IPsec

VPNs use IPsec to create secure tunnels over untrusted networks (e.g., the internet).

• Site-to-Site VPN: Connects two offices (e.g., Daraz’s warehouse to HQ).
• Remote Access VPN: Allows employees to connect securely (e.g., Pathao drivers accessing dispatch systems).

graph TD
    A["User\n(Home)"]
    B["ISP\n(Untrusted)"]
    C["VPN Gateway\n(IPsec Tunnel)"]
    D["Corporate Network\n(Secure)"]
    A -->|"IPsec Tunnel"| B
    B -->|"ESP Encrypted"| C
    C -->|"Decrypted"| D

5. IPsec vs. TLS/SSL

When to Use Which?

• Use IPsec for entire network security (e.g., NTC’s backbone).
• Use TLS for application-level security (e.g., WhatsApp’s end-to-end encryption).

Exam Tip

1. Memorize AH vs. ESP: Know which provides encryption and which does not.
2. IKE Phases: Phase 1 = key exchange, Phase 2 = IPsec SA setup.
3. VPN Modes: Transport = end-to-end, Tunnel = gateway-to-gateway.
4. Attacks: Spoofing → AH, Replay → Sequence numbers, DoS → Rate limiting.
5. Real-World Links:
   • eSewa → IPsec ESP for payments.
   • Ncell VPN → Tunnel mode for offices.
   • NEPSE → AH for stock data integrity.

Common Exam Questions:

• Draw an IPsec ESP packet format.
• Explain IKEv2 vs. IKEv1 (IKEv2 supports mobility).
• Compare IPsec and TLS in a table.
• Describe how PFS prevents key compromise.

How IKE establishes shared secrets (Image: de:Benutzer:DaMutz, CC BY-SA 4.0, via Wikimedia Commons)