Pipelining & Performance: Speeding Up Computers

TAKEAWAYS:

• Pipelining splits instruction execution into 5 stages (IF, ID, EX, MEM, WB) to overlap operations and increase instructions per cycle (IPC).
• Hazards (structural, data, control) disrupt pipelines, requiring stalls or forwarding to maintain efficiency.
• Performance metrics (CPI, clock cycles, MIPS) quantify speedups, with pipelining reducing average CPI below 1.
• Superpipelining and superscalar architectures extend pipelining to deeper stages or wider execution units.
• Real-world apps like eSewa’s payment processing and Pathao’s route optimization rely on pipelined CPUs for low-latency responses.
• Exam focus: Define pipelining, draw the 5-stage pipeline, explain hazards, and calculate speedup for given CPI values.

1. Why Pipelining? The Bottleneck Problem

Computers execute instructions sequentially: fetch → decode → execute → memory access → writeback. Without pipelining, the CPU idles while waiting for each stage to finish. For example, if a CPU takes 4 clock cycles to complete one instruction, it processes 1 instruction per 4 cycles—wasting 75% of its potential.

Real-world analogy:

• Pathao’s driver matching system must process thousands of ride requests per second. Without pipelining, each request would block the next, causing delays. Pipelining lets the system fetch a new request while the previous one is being executed, reducing average wait time.

2. The 5-Stage Pipeline: How It Works

Pipelining divides instruction execution into 5 parallel stages, each taking 1 clock cycle. While one instruction is in the Execute (EX) stage, the next can be Decoded (ID), and another Fetched (IF).

Visual:

graph LR
    subgraph Pipeline Stages
        IF["IF: Fetch"] -->|"1 cycle"| ID["ID: Decode"]
        ID -->|"1 cycle"| EX["EX: Execute"]
        EX -->|"1 cycle"| MEM["MEM: Memory"]
        MEM -->|"1 cycle"| WB["WB: Writeback"]
    end
        **Throughput**:
        1 instruction per cycle
        (vs. 1 per 5 cycles without pipelining)
    end note

Worked Example: Speedup Calculation Assume a CPU without pipelining takes 5 cycles per instruction (CPI = 5). With pipelining, CPI = 1 (ideal case).

• Speedup = Old CPI / New CPI = 5 / 1 = 5× faster.
• Real-world tie-in: Nepal Rastra Bank’s transaction processing uses pipelined servers to handle thousands of online payments (via eSewa/Khalti) per second without delays.

3. Hazards: When Pipelining Breaks Down

Even with pipelining, 3 types of hazards can stall the pipeline:

Visual: Data Hazard Example

Real-world example:

• NTC’s ticket booking system uses pipelining but must handle data hazards when checking seat availability. If Instruction 1 updates seat A1 and Instruction 2 checks A1, a stall ensures the correct data is read.

4. Performance Metrics: Measuring Speedups

Key metrics to evaluate pipelining:

Worked Example: Calculating MIPS

• A pipelined CPU runs at 2 GHz (2 × 10⁹ cycles/sec) with CPI = 1.2 (due to hazards).
• MIPS = (2 × 10⁹) / (1.2 × 10⁶) = 1.67 × 10³ MIPS ≈ 1670 MIPS.

Real-world tie-in:

• Google’s data centers use pipelined CPUs to achieve >10,000 MIPS per server, enabling real-time searches and ad targeting.

5. Advanced Pipelining Techniques

To further improve performance, architects use:

Visual: Superscalar Pipeline

Real-world example:

• WhatsApp’s servers use superscalar CPUs to handle millions of messages/sec, processing encryption and routing in parallel.

6. Trade-offs: Pros and Cons of Pipelining

Real-world trade-off:

• Nepal’s NEPSE stock trading system uses pipelined servers for low-latency order matching, but branch mispredictions (e.g., in volatile markets) can cause temporary slowdowns.

7. Real-World Applications in Nepal

1. eSewa/Khalti Payments

   • Idea Used: Pipelined transaction processing to handle 10,000+ payments/sec during Dashain.
   • How: Each payment goes through stages:
     • IF: Fetch user details
     • ID: Validate amount
     • EX: Deduct from account
     • MEM: Update bank DB
     • WB: Send confirmation SMS.

2. Pathao’s Driver Matching

   • Idea Used: Superpipelining to match riders to drivers in <500ms.
   • How: Parallel stages for:
     • Location fetch (IF)
     • Driver availability check (ID)
     • Route calculation (EX)
     • Notification send (MEM/WB).

3. NTC’s Ticket Booking

   • Idea Used: Data hazard handling to prevent double-booking.
   • How: If Instruction 1 books seat A1 and Instruction 2 checks A1, the pipeline stalls until A1 is marked as booked.

Exam Tip

What examiners want to see:

1. Definition: Pipelining is "dividing instruction execution into parallel stages to overlap operations and increase throughput."
2. 5-Stage Pipeline Diagram: Draw and label IF, ID, EX, MEM, WB with arrows showing overlap.
3. Hazard Examples: For each hazard type (structural, data, control), give one example and its solution.
4. Speedup Calculation: If given CPI before/after, calculate speedup as:

   Speedup = Old CPI / New CPI
   

   Example: If CPI drops from 4 to 1.5, speedup = 4 / 1.5 ≈ 2.67×.
5. Real-World Link: Relate to eSewa, Pathao, or NTC in your answer (e.g., "Like Pathao’s driver matching, pipelining reduces average wait time by overlapping stages.").

Common Mistakes to Avoid:

• ❌ Forgetting to mention hazards (examiners love this!).
• ❌ Drawing a pipeline with non-overlapping stages (show overlap with arrows).
• ❌ Ignoring CPI > 1 in real systems (always mention stalls/hazards reduce ideal speedup).

Final Visual Summary: