Message Passing: IPC, Protocols, Formats & Reliability

TAKEAWAYS:

• Message passing enables processes to exchange data across distributed systems using synchronous (blocking) or asynchronous (non-blocking) communication.
• Message formats define fields like headers (source/destination, sequence numbers) and payloads, critical for parsing and routing (e.g., TCP segments vs. UDP datagrams).
• Reliability mechanisms (acknowledgments, timeouts, retransmissions) ensure data integrity in unreliable networks (e.g., Ncell’s SMS delivery).
• Protocols (TCP, UDP, HTTP) dictate rules for handshakes, error recovery, and flow control—visible in every web request or WhatsApp message.
• Real-world systems (eSewa, Daraz, banks) use message passing for secure transactions, order processing, and multi-server coordination.
• Exam focus: Trace message flows, compare protocols (TCP vs. UDP), and explain reliability techniques with worked examples (e.g., a failed Daraz order retry).

1. Interprocess Communication (IPC) via Message Passing

Message passing is the core mechanism for distributed systems to exchange data between processes running on different machines (or even the same machine). Unlike shared-memory IPC, it avoids race conditions by enforcing explicit communication.

Key Concepts

• Process: An instance of a program with its own address space (e.g., a Daraz order-processing service vs. a payment gateway).
• Message: A structured packet containing:
  • Header: Metadata (source/destination IDs, sequence numbers, checksums).
  • Payload: The actual data (e.g., JSON for an eSewa transaction).
  • Trailer: Optional error-checking fields (e.g., CRC).
• Channels: Logical or physical links (e.g., TCP sockets, UDP ports) over which messages travel.

classDiagram
    class Process {
        +send(message)
        +receive(message)
    }
    class Message {
        -header: Metadata
        -payload: Data
        -trailer: Checksum
    }
    Process "1" --> "0..*" Message : sends/receives
    Message --> Process : delivered to

Synchronous vs. Asynchronous Communication

Worked Example: eSewa Transaction

1. User Process → Sends PaymentRequest (synchronous) to eSewa Server.
2. Server → Validates, deducts amount, sends PaymentConfirmation back.
3. Failure Handling: If no reply in 5 seconds, user retries (timeout + retransmission).

2. Message Formats and Protocols

Messages must be structured for correct parsing and routing. Protocols define these formats and rules.

Message Fields (Generic)

+---------------------+---------------------+---------------------+
|       Header        |      Payload        |      Trailer       |
+---------------------+---------------------+---------------------+
| Source ID (4B)      | Transaction Data   | Checksum (2B)       |
| Destination ID (4B) | (JSON/XML)         |                     |
| Sequence # (4B)     |                     |                     |
| Timestamp (8B)      |                     |                     |
+---------------------+---------------------+---------------------+

Protocol-Specific Formats

3. Reliability Mechanisms

Networks are unreliable (packets lost, corrupted, or delayed). Message passing uses:

1. Acknowledgments (ACKs): Receiver sends ACK to confirm delivery.
2. Timeouts: Sender retries if no ACK after a set time (e.g., 3 seconds).
3. Sequence Numbers: Detect lost/reordered messages (e.g., TCP’s cumulative ACKs).
4. Checksums/CRCs: Detect corrupted messages (e.g., UDP’s checksum).

How TCP Ensures Reliability

sequenceDiagram
    participant Sender
    participant Network
    participant Receiver
    Sender->>Network: Segment 1 (Seq=100)
    Network-->>Receiver: Segment 1 (corrupted)
    Receiver->>Network: ACK 100 (discarded due to corruption)
    Sender->>Network: Segment 1 (retransmitted)
    Network-->>Receiver: Segment 1 (delivered)
    Receiver->>Network: ACK 101
    Sender->>Network: Segment 2 (Seq=101)

Worked Example: Ncell SMS Delivery

• Problem: SMS may be lost in transit.
• Solution:
  1. Sender (Ncell server) assigns Seq=X to SMS.
  2. Receiver (user phone) sends ACK if delivered.
  3. If no ACK in 30 seconds, Ncell retransmits (seen as "SMS failed to send" retries).

4. Real-World Applications

Example 1: eSewa (Financial Transactions)

• IPC Model: Synchronous (user waits for payment confirmation).
• Protocols: HTTPS (TCP) for secure data transfer.
• Reliability: Retries on failed transactions (e.g., insufficient balance).
• Message Flow:

  sequenceDiagram
      participant User
      participant eSewaServer
      participant Bank
      User->>eSewaServer: PaymentRequest (Rs. 500)
      eSewaServer->>Bank: DebitRequest
      Bank-->>eSewaServer: DebitConfirmation
      eSewaServer-->>User: PaymentConfirmation

Example 2: Pathao (Ride Matching)

• IPC Model: Asynchronous (driver/rider processes run independently).
• Protocols: WebSockets (TCP) for real-time updates.
• Message Types:
  • RideRequest (rider → Pathao server).
  • DriverAssigned (server → rider/driver).
  • TripUpdate (driver → server → rider).
• Reliability: If a driver’s TripUpdate fails, Pathao queues it and retries.

Example 3: Daraz Order Processing

• Problem: High traffic causes message loss.
• Solution: Message Queues (e.g., RabbitMQ) buffer orders.
• Flow:
  1. User clicks "Buy" → Daraz frontend sends OrderCreated to queue.
  2. Order service picks it up, processes payment, updates inventory.
  3. If payment fails, queue retains the message for retry.

5. Advantages and Disadvantages

6. Common Pitfalls and Best Practices

• Pitfall: Assuming UDP is "faster" without considering retransmissions (e.g., VoIP drops calls if packets are lost).
• Best Practice: Use TCP for critical data (e.g., bank transfers) and UDP for real-time but loss-tolerant apps (e.g., online gaming).
• Pitfall: Ignoring timeout values (e.g., Ncell’s 30-second SMS retry vs. eSewa’s 5-second payment timeout).
• Best Practice: Log sequence numbers and timestamps for debugging (e.g., Daraz’s order tracking).

## In the Real World

1. eSewa (Nepal)

   • Idea Used: Synchronous message passing with TCP for secure transactions.
   • How: When you pay Rs. 1000 for a bill, your phone sends a PaymentRequest to eSewa’s server, which waits for an ACK before confirming. If the network drops the message, eSewa retransmits it (you see "Processing..." for up to 30 seconds).

2. Pathao (Ride-Hailing)

   • Idea Used: Asynchronous pub/sub model with WebSockets.
   • How: When you request a ride, Pathao’s server publishes a RideRequest event. Drivers subscribe to this topic; the first to accept gets an AssignmentConfirmation message. If your phone loses the assignment message, Pathao queues it and redelivers it when the connection recovers.

3. Ncell SMS Service

   • Idea Used: Reliable UDP with retransmissions.
   • How: SMS uses UDP (fast) but adds its own ACK/retry logic. If your phone doesn’t ACK a message, Ncell’s server retransmits it every 30 seconds until delivered or discarded after 72 hours (hence delayed SMS).

4. Daraz Order Fulfillment

   • Idea Used: Message queues for decoupling.
   • How: When you order a product, Daraz’s frontend sends an OrderCreated message to a queue. The payment service and inventory service compete to consume this message. If payment fails, the message stays in the queue for manual review (you see "Order processing..." for hours).

5. NEPSE Stock Trading

   • Idea Used: Synchronous RPC (Remote Procedure Call).
   • How: When you buy shares, your broker’s app sends a BuyOrder RPC to NEPSE’s server. The server blocks until it confirms the trade or rejects it (e.g., insufficient funds). This ensures no partial trades.

## Exam Tip

1. Trace Message Flows: Exams often ask to draw a sequence diagram for a scenario (e.g., "A user logs into eSewa. Show the message exchange between client, server, and bank."). Always include:

   • Protocols (HTTP/TCP for web, UDP for SMS).
   • Reliability steps (ACKs, timeouts).
   • Failure cases (e.g., "What if the ACK is lost?").

2. Compare TCP vs. UDP:

   • TCP: Reliable, ordered, connection-oriented (use for data integrity like bank transfers).
   • UDP: Fast, no guarantees (use for real-time like VoIP or gaming).
   • Example Question: "Why does Pathao use WebSockets (TCP) for ride updates instead of UDP?" Answer: Because ride updates must arrive in order (e.g., "Driver arrived" → "Starting trip"), and WebSockets provide retransmission if packets are lost.

3. Worked Examples Are Key:

   • Bank Loan Interest Calculation: Not directly IPC, but message passing is how loan applications are routed between bank servers. If the system uses asynchronous processing, the bank’s server might send a LoanApproved message to a queue, and your app consumes it later.
   • Daraz Order Queue: If an order fails, Daraz’s message queue (RabbitMQ) ensures it’s not lost. Draw this as a queue with producers/consumers.

4. Common Exam Questions:

   • "Explain how eSewa ensures a payment is not lost if the network drops the confirmation message." Answer: Uses TCP (reliable transport) + application-level retries. If the ACK is lost, TCP retransmits the segment; if the app doesn’t get a reply, it resends the payment request.
   • "Why might UDP be preferred over TCP for a live cricket score app?" Answer: UDP has lower latency (no ACKs/retries), and occasional missing scores are tolerable (unlike missing a bank transaction).

5. Visuals to Master:

   • Sequence diagrams for handshakes (e.g., TCP 3-way handshake).
   • Message format tables (compare TCP/UDP headers).
   • Queue diagrams for async systems (e.g., Daraz’s order processing).

SYN → SYN-ACK → ACK exchange between client (e.g., your browser) and server (e.g., Daraz). (Image: CC BY-SA 3.0, via Wikimedia Commons)