File Systems, Storage Types, RAID, LVM, Partitioning & Backup

What is a File System?

A file system is a method and data structure that an operating system uses to control how data is stored and retrieved on storage devices (HDDs, SSDs, USB drives). It defines:

• How files and folders are named and organized.
• How data is stored on disk blocks.
• How permissions and metadata (size, date, owner) are managed.

Common File Systems

How ext4 Organizes Data

classDiagram
    class BlockGroup {
        +Superblock (metadata)
        +Group Descriptor Table
        +Inode Table (file metadata)
        +Data Blocks (actual file content)
    }
    class Superblock {
        +Filesystem UUID
        +Block size
        +Total blocks
        +Free blocks count
    }
    class Inode {
        +File size
        +Permissions (rwx)
        +Owner:Group
        +Timestamps (atime, mtime, ctime)
        +Pointers to data blocks
    }
    BlockGroup "1" --> "many" Superblock : contains
    BlockGroup "1" --> "many" Inode : contains
    BlockD["Data Block"] "1" --> "many" Inode : referenced by

Storage Devices: HDD vs. SSD vs. NAS

1. HDD (Hard Disk Drive)

• Mechanism: Magnetic platters + read/write heads on an actuator arm.
• Speed: 50–150 MB/s (slower due to mechanical movement).
• Capacity: 500GB–20TB (cheaper per GB).
• Use Case: Bulk storage (e.g., NEPSE’s historical stock data archives).

2. SSD (Solid State Drive)

• Mechanism: NAND flash memory (no moving parts).
• Speed: 300–3500 MB/s (faster random access).
• Capacity: 120GB–15TB (expensive per GB).
• Use Case: OS boot drives, databases (e.g., eSewa transaction logs).

3. NAS (Network-Attached Storage)

• Definition: A dedicated file-level storage device connected to a network (e.g., Synology, QNAP).
• Use Case: Centralized storage for multiple users (e.g., Daraz’s order processing servers).
• Protocols: NFS (Linux), SMB/CIFS (Windows), AFP (macOS).

RAID (Redundant Array of Independent Disks)

RAID combines multiple physical disks into a logical unit for performance, redundancy, or both. Common levels:

graph LR
    subgraph RAID 0
        A["Disk 1"] -->|"Striping"| B["Disk 2"]
        B -->|"No redundancy"| A
    end
    subgraph RAID 1
        C["Disk 1"] -->|"Mirroring"| D["Disk 2"]
        D -->|"Exact copy"| C
    end
    subgraph RAID 5
        E["Disk 1"] -->|"Striping + Parity"| F["Disk 2"]
        F -->|"Parity block"| G["Disk 3"]
        G -->|"Tolerates 1 failure"| E
    end
    subgraph RAID 10
        H["Disk 1"] -->|"Mirrored"| I["Disk 2"]
        J["Disk 3"] -->|"Mirrored"| K["Disk 4"]
        I -->|"Striped"| J
        K -->|"Striped"| H
    end

Worked Example: Daraz’s Order Queue Storage

Daraz processes 10,000 orders/hour. If they use:

• RAID 0 (2x 1TB HDDs): 2TB total, but 1 disk failure = all data lost.
• RAID 1 (2x 1TB HDDs): 1TB usable, but faster reads (mirrored).
• RAID 5 (3x 1TB HDDs): 2TB usable, tolerates 1 failure, good balance.

Recommendation: RAID 10 for high availability (e.g., order processing DB) + RAID 5 for backups.

Logical Volume Management (LVM)

LVM allows dynamic resizing of storage volumes without downtime. Key components:

1. Physical Volumes (PVs): Physical disks or partitions.
2. Volume Groups (VGs): Pool of PVs.
3. Logical Volumes (LVs): Filesystems or swap space created from VGs.

Steps to Create an LVM Setup

sequenceDiagram
    participant Admin
    participant PV as Physical Volume
    participant VG as Volume Group
    participant LV as Logical Volume
    participant FS as Filesystem

    Admin->>PV: pvcreate /dev/sdb
    Admin->>VG: vgcreate myvg /dev/sdb
    Admin->>LV: lvcreate -n data -L 50G myvg
    Admin->>FS: mkfs.ext4 /dev/myvg/data
    Admin->>FS: mount /dev/myvg/data /mnt/data

Advantages of LVM

• Extend storage: Add a new disk to a VG and expand an LV.
• Snapshots: Create point-in-time copies (e.g., before a software update).
• Flexibility: Move data between LVs without reformatting.

Disk Partitioning

Partitioning divides a disk into independent sections (e.g., /boot, /home, /var). Common schemes:

Worked Example: Ncell’s Server Partitioning

Ncell’s billing server requires:

• / (root): 30GB (OS + MySQL, Apache).
• /var: 50GB (logs, call records).
• /home: 200GB (customer data).
• swap: 8GB (RAM size).

Command to create partitions:

fdisk /dev/sda
n (new partition)
p (primary)
1 (partition number)
+30G (size)
t (type)
83 (Linux filesystem)
w (write)
mkfs.ext4 /dev/sda1
mount /dev/sda1 /

Backup Strategies

3-2-1 Backup Rule

1. 3 copies of data (e.g., primary + 2 backups).
2. 2 different media (e.g., HDD + cloud).
3. 1 offsite (e.g., encrypted backup at a remote data center).

In the Real World

1. eSewa’s Transaction Logs

   • Uses RAID 10 for high availability (mirrored + striped) to handle 10,000+ transactions/minute.
   • LVM allows dynamic scaling during Diwali (peak season).
   • ext4 filesystem for journaling (prevents corruption during power failures).

2. Daraz’s Order Processing

   • RAID 5 for order databases (tolerates 1 disk failure).
   • Partitioning: Separate /var for logs (analyzed for fraud detection).
   • Incremental backups nightly to reduce storage costs.

3. Ncell’s Network Equipment Logs

   • ZFS for checksums (detects corrupted logs from hardware failures).
   • NAS centralizes logs from 50+ cell towers for analysis.

Exam Tip

1. Diagrams are worth marks:

   • Draw RAID configurations (show parity blocks for RAID 5).
   • Sketch LVM components (PV → VG → LV).
   • Label partition tables (MBR vs. GPT).

2. Compare file systems:

   • ext4 vs. NTFS: journaling, permissions, max file size.
   • FAT32 vs. exFAT: file size limit, compatibility.

3. Worked examples:

   • Calculate RAID 5 usable space: (n-1)*disk_size (e.g., 3x1TB → 2TB usable).
   • Design a partition scheme for a given scenario (e.g., web server vs. database server).

4. Backup questions:

   • Given a scenario, choose the best backup type (full vs. incremental).
   • Explain the 3-2-1 rule with real-world constraints (e.g., "NTC has limited cloud storage").

5. Commands:

   • fdisk, mkfs, mount, pvcreate, vgcreate, lvcreate.
   • rsync for backups (e.g., rsync -avz /data/ backup@server:/backups/).

Summary Table