OS Protection, Security & Linux Case Study

TAKEAWAYS:

• Protection uses access control matrices and capabilities to restrict processes from accessing unauthorized resources (e.g., a bank app blocking a user from modifying loan data).
• Security threats include malware (viruses, worms), DoS attacks, and privilege escalation—Linux mitigates these via mandatory access control (MAC) and discretionary access control (DAC).
• Linux security relies on SELinux/AppArmor for fine-grained policy enforcement (e.g., WhatsApp’s end-to-end encryption uses similar principles to Linux’s seccomp).
• Case study: Linux’s kernel hardening (ASLR, stack canaries) protects against exploits like buffer overflows (used in real attacks on Ncell’s billing system).
• Exam focus: Compare DAC vs. MAC, trace a security breach scenario, and explain Linux’s security layers (userspace → kernel → hardware).

1. Protection in Operating Systems

Protection ensures that processes, users, or system components cannot access or modify resources they are not authorized to use. It is implemented via:

• Access Control Mechanisms
• Capabilities
• Domain and Type Enforcement

1.1 Access Control Mechanisms

Access control defines who (or what) can access a resource and what operations they can perform. Two primary models:

1. Discretionary Access Control (DAC)

   • Owners of resources (files, devices) decide who gets access.
   • Example: In Linux, file permissions (chmod 755 file.txt) use DAC.
   • Weakness: If an attacker gains root access, they can modify permissions.

2. Mandatory Access Control (MAC)

   • Access is controlled by a central authority (e.g., system administrator).
   • Used in military/enterprise systems (e.g., SELinux in Linux).
   • Example: A hospital’s patient records system might use MAC to ensure only doctors can view certain files.

stateDiagram-v2
    [*] --> AccessRequest
    AccessRequest --> CheckPolicy: "Is request allowed?"
    CheckPolicy --> GrantAccess: "Yes (DAC/MAC)"
    CheckPolicy --> DenyAccess: "No"
    GrantAccess --> [*]
    DenyAccess --> [*]

1.2 Capabilities

Instead of relying solely on user IDs, capabilities are unforgeable tokens that grant specific rights to processes.

• Example: A web browser (like Chrome) runs with limited capabilities—it cannot modify system files unless explicitly granted (e.g., via sudo).
• Advantage: More granular than DAC (e.g., a process can have "read file" but not "delete file").

1.3 Domain and Type Enforcement

• Domain: A set of rights/privileges assigned to a process (e.g., "web server domain" can only serve HTTP requests).
• Type Enforcement (TE): Classifies objects (files, processes) into types and enforces rules between them.
  • Used in SELinux (Linux) to prevent unauthorized interactions.

Real-World Example:

• Khalti’s Payment System
  • Uses MAC-like policies to ensure that only verified merchants can process transactions.
  • If a hacker gains access to a merchant’s system, MAC prevents them from modifying Khalti’s core transaction database.

2. Security Threats and Countermeasures

Security threats exploit vulnerabilities in protection mechanisms. Common threats:

2.1 Case Study: Buffer Overflow Attack on Ncell’s Billing System

• Attack: A hacker sends malformed input to Ncell’s billing software, causing a buffer overflow that executes arbitrary code.
• Linux Defense:
  • ASLR (Address Space Layout Randomization): Randomizes memory addresses to make exploits harder.
  • Stack Canaries: Detects buffer overflows by placing a "canary" value on the stack.
  • Seccomp: Restricts system calls (e.g., prevents execve() unless explicitly allowed).

sequenceDiagram
    participant Hacker
    participant NcellServer
    participant LinuxKernel
    Hacker->>NcellServer: Malicious Input (Buffer Overflow)
    NcellServer->>LinuxKernel: System Call (e.g., execve())
    LinuxKernel-->>NcellServer: Blocked (Seccomp)
    LinuxKernel-->>NcellServer: Crash (Stack Canary)

3. Linux Security Features

Linux is widely used in servers (e.g., Google’s data centers, NTC’s network infrastructure) due to its robust security model.

3.1 Mandatory Access Control (MAC) in Linux

• SELinux (Security-Enhanced Linux)

  • Enforces MAC policies (e.g., httpd_t domain can only serve web pages).
  • Example: If a hacker compromises a web server, SELinux prevents them from accessing /etc/passwd.

• AppArmor

  • Simpler than SELinux, profiles applications (e.g., restricts nginx to only handle HTTP requests).

classDiagram
    class User {
        +UID
        +Groups
    }
    class Process {
        +PID
        +Domain (SELinux)
    }
    class File {
        +Type (SELinux)
        +Permissions
    }
    User --> Process : "Runs"
    Process --> File : "Accesses (MAC/DAC)"
    Process --> Kernel : "Enforced by SELinux/AppArmor"

3.2 Kernel Hardening

• ASLR (Address Space Layout Randomization)
  • Randomizes memory addresses to prevent exploits from predicting locations (e.g., stack, heap).
• Stack Canaries
  • Detects buffer overflows by checking a "canary" value before returning from a function.
• Seccomp (Secure Computing Mode)
  • Restricts system calls (e.g., a containerized app can only call read() and write()).

Real-World Example:

• Pathao’s Ride-Hailing App
  • Uses Linux containers with seccomp to restrict driver apps from accessing the system’s GPS data unless explicitly allowed.
  • If a driver app is hacked, seccomp prevents it from modifying Pathao’s payment records.

3.3 Userspace Security

• Sandboxing (Firejail, Docker)
  • Runs untrusted apps in isolated environments (e.g., WhatsApp Web runs in a sandboxed Chrome tab).
• Firewalls (iptables/nftables)
  • Blocks unauthorized network access (e.g., NTC’s routers use iptables to filter traffic).

4. Case Study: Linux Security Architecture

Linux security is layered:

1. Hardware: TPM (Trusted Platform Module) for secure boot.
2. Kernel:
   • LSM (Linux Security Modules): Framework for SELinux/AppArmor.
   • Integrity Measurement Architecture (IMA): Ensures files haven’t been tampered with.
3. Userspace:
   • Sudo: Restricts root access.
   • AppArmor/SELinux: Enforces policies.

Example Trace: Secure File Transfer

1. User uploads a file via SFTP (encrypted).
2. SELinux checks if the user’s domain (user_t) can write to the target file (public_content_t).
3. AppArmor ensures the sftp-server process cannot access /etc/shadow.
4. ASLR randomizes memory to prevent exploits.

5. Protection vs. Security: Key Differences

## In the real world

1. eSewa’s Payment Security

   • Uses TLS/SSL (encryption) and Linux’s SELinux to protect transaction data.
   • If a hacker breaches a merchant’s system, SELinux prevents them from accessing eSewa’s database.

2. Ncell’s Billing System

   • ASLR and stack canaries protect against buffer overflows in legacy billing software.
   • Firewalls (iptables) block DDoS attacks on Ncell’s SMS gateway.

3. Daraz’s Order Processing

   • Docker containers (Linux-based) isolate order-processing microservices.
   • If one service is compromised, seccomp prevents it from accessing inventory databases.

## Exam Tip

1. Compare DAC vs. MAC:

   • DAC = "Owner decides" (e.g., chmod).
   • MAC = "System enforces" (e.g., SELinux).
   • Exam Question: "Why is MAC more secure than DAC in a hospital’s patient records system?" Answer: MAC prevents even administrators from bypassing access rules, whereas DAC allows privilege escalation if an attacker gains root access.

2. Trace a Security Breach:

   • Scenario: A hacker exploits a buffer overflow in a Linux web server.
   • Steps:
     1. Hacker sends malformed input → buffer overflow.
     2. Without ASLR, the exploit predicts stack address.
     3. Defense: ASLR randomizes stack → exploit fails.
     4. Final Step: SELinux blocks the compromised process from accessing /etc/passwd.

3. Linux Security Layers:

   • Hardware → Kernel (LSM) → Userspace (SELinux/AppArmor).
   • Exam Question: "How does Linux prevent a rootkit from hiding in memory?"
     • Answer: IMA (Integrity Measurement) checks file hashes; ASLR randomizes memory layout.

4. Real-World Applications:

   • Always relate to Nepali tech (e.g., Khalti’s MAC policies, Ncell’s ASLR).
   • Example: "Explain how Pathao uses Linux security to prevent driver fraud."
     • Answer: Containers with seccomp restrict driver apps to only GPS/payment APIs; SELinux ensures they cannot modify ride data.