Sync & Replication: Offline-First, Conflict Handling, Data Consistency

TAKEAWAYS:

• Offline-first design lets apps work without internet, syncing later (e.g., WhatsApp saves messages offline).
• Conflict resolution merges changes from multiple devices (e.g., Google Docs’ last-writer-wins or manual merge).
• Replication strategies (strong vs. eventual consistency) balance speed and accuracy (e.g., banks use strong consistency; social media uses eventual).
• Optimistic vs. pessimistic locking controls concurrent edits (e.g., Google Sheets uses optimistic; banking apps use pessimistic).
• Delta sync reduces data transfer by sending only changes (e.g., email apps syncing new emails only).
• Mobile-specific challenges like battery, bandwidth, and intermittent connectivity require adaptive sync strategies.

Core Concepts: Why Sync and Replication Matter

Mobile apps often operate in unreliable networks (e.g., rural Nepal, public transport). Users expect seamless access to data even when offline. Synchronization ensures data consistency across devices and servers, while replication distributes data to improve availability and performance.

1. Offline-First Design

Apps store data locally (SQLite, Realm) and sync later. Key components:

• Local database: SQLite, Room (Android), Core Data (iOS).
• Sync manager: Detects network changes and triggers sync (e.g., Firebase, CouchDB).
• Queue system: Stores pending changes (e.g., WhatsApp’s unsent messages).

How It Works (Step-by-Step)

flowchart LR
    A["User edits data offline"] --> B["Local DB updates"]
    B --> C["Sync queue adds change"]
    C --> D["Network available?"]
    D -->|"Yes"| E["Sync to server"]
    D -->|"No"| F["Retry later"]
    E --> G["Server updates"]
    G --> H["Conflict check"]
    H -->|"Resolved"| I["Data merged"]
    H -->|"Conflict"| J["Manual resolution"]

Example: eSewa Wallet

• Problem: Users pay bills or transfer money in areas with poor connectivity.
• Solution:
  1. Transaction is stored locally in SQLite.
  2. When online, syncs with eSewa’s server.
  3. If conflict (e.g., double-spent money), server validates and rejects duplicates.

2. Conflict Resolution Strategies

When two devices edit the same data offline, conflicts arise. Common strategies:

Worked Example: Pathao Rider Earnings

• Scenario: A Pathao rider earns ₹500 offline. Another device (e.g., admin dashboard) deducts ₹100 for a penalty while the rider is offline.
• Conflict:
  • Rider’s device: ₹500 → ₹600 (new ride).
  • Server: ₹500 → ₹400 (penalty applied).
• Resolution:
  • Last-Write-Wins: Server wins → rider sees ₹400 (losing ₹100).
  • Manual Merge: Server shows both changes; admin approves the penalty.
  • Timestamp-Based: If rider’s edit has a later timestamp, server applies ₹600.

3. Replication Models

Data replication distributes copies to improve reliability and performance. Two extremes:

A. Strong Consistency (Centralized)

• Definition: All replicas are identical at all times (e.g., banking transactions).
• Mechanism: Locks prevent concurrent writes (pessimistic concurrency).
• Example: Ncell’s account balance syncs instantly across all systems.

graph TD
    A["User requests balance"] --> B["Server locks account"]
    B --> C["Reads latest balance"]
    C --> D["User updates balance"]
    D --> E["Server unlocks"]

B. Eventual Consistency (Decentralized)

• Definition: Replicas will converge eventually (e.g., social media posts).
• Mechanism: Asynchronous updates (optimistic concurrency).
• Example: Facebook posts appear on your feed after a delay.

Visual: Conflict-Free Replicated Data Types (CRDTs) CRDTs are data structures that guarantee convergence without conflicts. Example: A G-Counter (grow-only counter) for likes:

graph LR
    A["User 1 likes (P1)"] --> B["P1: {A:1}"]
    C["User 2 likes (P2)"] --> D["P2: {B:1}"]
    B & D --> E["Merged: {A:1, B:1}"]
    E --> F["Total: 2"]

4. Synchronization Techniques

A. Full Sync vs. Delta Sync

Example: Daraz Order Queue

• Full Sync: Downloads all 10,000 pending orders every time → slow.
• Delta Sync: Only syncs orders updated in the last hour → efficient.

B. Sync Triggers

• Manual: User clicks "Sync" (e.g., old email clients).
• Automatic: On Wi-Fi, low battery, or idle (e.g., WhatsApp).
• Event-Based: Sync after a transaction (e.g., eSewa after payment).

5. Mobile-Specific Challenges

In the Real World

1. WhatsApp (Optimistic Locking + Delta Sync)

   • How it works: Messages are stored locally in SQLite. When online, WhatsApp syncs only new/updated messages (delta sync). If two devices edit the same chat (e.g., typing indicators), WhatsApp uses last-write-wins for timestamps.
   • Conflict example: If you and a friend edit the same group chat name offline, the change with the later timestamp wins.

2. eSewa (Strong Consistency for Transactions)

   • How it works: Every transaction (e.g., bill payment) is locked on the server until confirmed. If you pay a bill offline, eSewa queues the transaction and syncs only after network confirmation.
   • Conflict example: If you pay ₹1000 twice offline, the second payment fails on sync (duplicate detection).

3. Google Docs (Operational Transformation)

   • How it works: When two users edit a document simultaneously, Google Docs transforms their operations to maintain a consistent order. For example, if User A deletes "hello" and User B inserts "world" at the same time, the system ensures the final text is "world" (not "worl" or "hellow").
   • Visual:

     sequenceDiagram
         UserA->>Doc: Delete "hello" (pos 0-5)
         UserB->>Doc: Insert "world" (pos 0)
         Doc-->>UserA: Transform: Delete "hello" → "world" + delete "hello" → "world"
         Doc-->>UserB: Transform: Insert "world" → "world" (no change)

Exam Tip

1. Define Key Terms Clearly:

   • Differentiate strong vs. eventual consistency with examples (banks vs. social media).
   • Explain optimistic vs. pessimistic locking (e.g., Google Sheets vs. bank transfers).

2. Compare Sync Strategies:

   • Draw a table (like above) comparing full vs. delta sync, and full vs. incremental replication.

3. Conflict Resolution Scenarios:

   • Expect questions like: "A user edits a Daraz order offline, and another user cancels it online. How would you resolve this conflict using last-write-wins? What’s the drawback?"
   • Answer:
     • Last-write-wins picks the newer timestamp.
     • Drawback: The offline edit is lost if the online change is newer.

4. Diagrams Are Your Friend:

   • For sync processes, draw sequence diagrams (like WhatsApp sync).
   • For conflict resolution, show state transitions (e.g., before/after merge).

5. Real-World Mapping:

   • Relate concepts to apps you know:
     • Offline-first: WhatsApp, eSewa.
     • Strong consistency: Ncell recharge, NEPSE trading.
     • Eventual consistency: Facebook posts, YouTube comments.

Practice Question: "Design a synchronization strategy for a mobile app that tracks Kathmandu traffic routes. Users can report jams offline. How would you handle conflicts if two users report the same route as jammed at different times?" Answer Outline:

1. Use delta sync to send only new jam reports.
2. Implement timestamp-based conflict resolution (latest report wins).
3. For critical routes (e.g., Ring Road), use pessimistic locking to prevent duplicates.
4. Visualize with a mermaid sequence diagram showing offline → online sync.