Cloud Databases: Models, Services, Security & Migration

Key Concepts and Cloud Database Models

What is a Cloud Database?

A cloud database is a database service hosted on remote servers (cloud infrastructure) instead of on-premises hardware. It provides on-demand scalability, automated backups, and pay-as-you-go pricing. Unlike traditional databases, cloud databases abstract hardware management, allowing DBAs to focus on schema design, queries, and performance tuning.

classDiagram
    class CloudDatabase {
        +Scalability: Auto-scaling
        +High Availability: Multi-zone replication
        +Pay-as-you-go: Billing per usage
        +Managed Services: Backups, patches, upgrades
    }
    class OnPremDatabase {
        +Fixed Hardware: Manual scaling
        +Self-Managed: Admin controls everything
        +High Upfront Cost: Servers, licenses, maintenance
    }
    CloudDatabase --> "Replaces" OnPremDatabase

Why Cloud?

• Cost Efficiency: No need to invest in physical servers or data centers.
• Scalability: Scale up/down based on demand (e.g., Black Friday sales for Daraz).
• Disaster Recovery: Built-in redundancy across regions.
• Global Access: Deploy databases closer to users (e.g., Ncell’s customer data in Kathmandu vs. Pokhara).

Deployment Models: IaaS vs. PaaS vs. SaaS

Cloud databases are deployed in three primary models, each offering different levels of control and management:

Worked Example: eSewa’s Cloud Database eSewa, Nepal’s leading digital payment platform, uses a PaaS model (likely AWS RDS or Google Cloud SQL) for:

• High Availability: Replicates transactions across multiple zones to prevent downtime during peak hours (e.g., Dashain festivals).
• Auto-Scaling: Handles 10x traffic during events like Dashain without manual intervention.
• Security: Encrypts sensitive data (e.g., bank details) at rest and in transit.

Cloud Database Services: AWS vs. Google vs. Azure

1. Amazon RDS (Relational Database Service)

• Supported DBs: MySQL, PostgreSQL, Oracle, SQL Server.
• Features:
  • Multi-AZ deployments for failover.
  • Automated backups and patching.
  • Read replicas for read-heavy workloads.
• Pricing: Pay for allocated storage, compute, and I/O operations.
• Use Case: NEPSE (Nepal Stock Exchange) could use RDS for its transaction logs to ensure 99.99% uptime.

sequenceDiagram
    participant User as eSewa User
    participant App as eSewa App (EC2)
    participant DB as RDS (Multi-AZ)
    User->>App: Initiate Payment
    App->>DB: INSERT INTO transactions(...)
    DB-->>App: Transaction ID
    App->>User: Payment Successful
    Note right of DB: Automated backup to S3\nMulti-AZ failover

2. Google Cloud Spanner

• Features:
  • Globally distributed SQL database with strong consistency.
  • Horizontal scaling across regions (e.g., Kathmandu + Singapore).
  • Built-in encryption and IAM integration.
• Use Case: Pathao’s ride-hailing app uses Spanner to sync driver locations and passenger requests in real-time across Nepal.

3. Azure SQL Database

• Features:
  • Hyperscale tier for petabyte-scale databases.
  • Threat detection via Microsoft Defender for SQL.
  • Hybrid cloud support (e.g., NTC’s legacy systems + cloud).
• Pricing: DTU (Database Transaction Units) or vCore model.

Comparison Table:

Security Challenges and Best Practices

Threats in Cloud Databases

1. Data Breaches: Unauthorized access due to weak IAM policies.
   • Example: In 2021, a misconfigured S3 bucket exposed Khalti’s customer data.
2. DDoS Attacks: Overwhelming query traffic to degrade performance.
   • Example: A DDoS attack on Daraz’s database during the COVID-19 lockdown caused outages.
3. Compliance Risks: Violating GDPR, PCI-DSS, or Nepal’s Data Privacy Act (2018).
   • Example: Ncell must ensure customer call logs (stored in cloud) comply with local laws.

Security Measures

Worked Example: Bank of Kathmandu’s Cloud Security

• Problem: Migrating loan processing to AWS RDS while ensuring PCI-DSS compliance.
• Solution:
  1. Encryption: Enable AWS KMS for encrypting loan application data.
  2. Access Control: Restrict DB access to only the loan processing microservice using IAM roles.
  3. Audit Trails: Enable AWS CloudTrail to log all SELECT, INSERT, and UPDATE operations on sensitive tables.
  4. Backup: Automate daily backups with point-in-time recovery for 30 days.

Database Migration to the Cloud

Migration Strategies

1. Lift-and-Shift (Rehosting)

   • Move on-prem DB to cloud with minimal changes.
   • Example: NTC migrated its billing system to AWS RDS with minimal downtime.
   • Tools: AWS Database Migration Service (DMS), Azure Data Factory.

2. Refactoring (Replatforming)

   • Optimize schema/query for cloud (e.g., add sharding for scalability).
   • Example: Daraz refactored its inventory DB to use Google Spanner for global low-latency access.

3. Re-architecting

   • Redesign for cloud-native features (e.g., serverless, NoSQL).
   • Example: Pathao replaced its monolithic MySQL DB with a microservices architecture using DynamoDB.

Step-by-Step Migration Plan

flowchart TD
    A["Assess Current DB"] --> B["Choose Cloud Model<br/>(IaaS/PaaS/SaaS)"]
    B --> C["Design Schema for Cloud<br/>(Sharding, Denormalization)"]
    C --> D["Set Up Cloud DB<br/>(Provision, Configure)"]
    D --> E["Test Migration<br/>(Staging Environment)"]
    E --> F["Cutover<br/>(Minimal Downtime)"]
    F --> G["Monitor & Optimize<br/>(Query Tuning, Scaling)"]

Worked Example: Kathmandu Traffic Management System

• Challenge: Kathmandu’s traffic police needed a real-time DB to track violations (e.g., red-light jumps) across 50+ cameras.
• Solution:
  1. Assessment: Current MySQL DB on-prem was slow during peak hours (6–9 PM).
  2. Migration: Moved to Azure SQL Database with:
     • Read Replicas: Offload analytics queries.
     • Elastic Pools: Share resources across multiple DBs (e.g., fines, citations).
  3. Result: Reduced query latency from 2s to 50ms; added real-time dashboards for police.

Performance Optimization in Cloud Databases

Key Metrics to Monitor

Optimization Techniques

1. Indexing:

   • Add indexes on frequently queried columns (e.g., customer_id in eSewa’s transactions table).
   • Example: Daraz added a composite index on (product_id, warehouse_id) to speed up inventory checks.

2. Caching:

   • Use Redis or Memcached for session data (e.g., WhatsApp’s message status cache).
   • Example: Ncell caches customer login tokens in Redis to reduce DB load.

3. Query Optimization:

   • Avoid SELECT *; fetch only required columns.
   • Use EXPLAIN to analyze slow queries (e.g., nested loops in NEPSE’s portfolio queries).

stateDiagram-v2
    [*] --> Idle
    Idle --> Active : User Query
    Active --> Optimized : Indexing/Caching
    Active --> Slow : Poor Query
    Optimized --> [*]
    Slow --> Timeout : >5s
    Slow --> Retry : Add Index

Cost Management in Cloud Databases

Cost Drivers

1. Compute: vCPUs, memory (e.g., db.m5.large vs. db.m5.xlarge).
2. Storage: GB-months, I/O operations (e.g., EBS volumes in AWS).
3. Data Transfer: Outbound traffic (e.g., syncing Daraz’s global inventory).
4. Backups: Automated snapshots (e.g., daily backups for Ncell’s call logs).

Cost-Saving Strategies

Worked Example: Khalti’s Cost Optimization

• Problem: Khalti’s PostgreSQL DB costs were rising due to unpredictable traffic (e.g., Dashain spikes).
• Solution:
  1. Auto-Scaling: Configured AWS RDS to scale between 2–10 vCPUs based on CPU load.
  2. Reserved Instances: Purchased 3-year RI for the base 2 vCPUs.
  3. Cold Storage: Archived transactions older than 1 year to S3 Glacier.
• Result: Reduced costs by 60% while maintaining 99.9% uptime.

Compliance and Legal Considerations

Key Regulations

Data Residency Requirements

• Nepal: Sensitive data (e.g., bank transactions, Aadhaar-linked records) must reside in Nepalese data centers (e.g., NTC’s cloud).
• Workaround: Use multi-region deployments with strict access controls (e.g., Ncell’s data in Kathmandu + backup in Singapore).

Example: NEPSE’s Compliance

• Challenge: Store stock trade data in the cloud while complying with Nepal’s Securities Board Act.
• Solution:
  1. Deployed Azure SQL Database in Nepal’s Nepal Data Center (NDC).
  2. Enabled Transparent Data Encryption (TDE).
  3. Implemented audit logs for all INSERT/UPDATE operations on trade tables.

In the Real World

1. eSewa: Pay-as-you-go Scalability

   • Idea Used: Auto-scaling in AWS RDS.
   • How: During Dashain, eSewa’s DB scales from 4 vCPUs to 20 automatically to handle 5x transaction volume. Without cloud, they’d need to over-provision hardware year-round, increasing costs by 300%.

2. Pathao: Global Low-Latency with Google Spanner

   • Idea Used: Strong consistency across regions.
   • How: Pathao’s driver-passenger matching system uses Spanner to ensure real-time updates in Kathmandu and Pokhara. Without it, a driver in Lalitpur might see outdated passenger locations, increasing ride cancellation rates by 20%.

3. Ncell: Hybrid Cloud for Legacy Systems

   • Idea Used: Azure Arc for on-prem integration.
   • How: Ncell’s legacy billing system (COBOL) runs on-prem, but customer data syncs to Azure SQL Database via Arc. This allows modern analytics (e.g., churn prediction) without rewriting the entire system.

Exam Tip

This unit is heavily tested on:

1. Deployment Models: Expect questions comparing IaaS/PaaS/SaaS with real-world examples (e.g., "Which model would eSewa use?").
2. Security: Always relate to Nepal’s Data Privacy Act or PCI-DSS. Example question:

   "Design a security plan for Khalti’s cloud database to comply with Nepal’s laws. Include encryption, access control, and audit measures."

3. Migration: Trace a step-by-step migration plan (e.g., "How would you move NTC’s billing system to AWS?").
4. Cost Optimization: Calculate savings using reserved instances vs. on-demand (e.g., "Khalti’s DB costs $500/month on-demand. How much would they save with a 3-year RI?").
5. Performance: Explain how to reduce query latency in a cloud DB (e.g., "Optimize Daraz’s product search queries").

Common Pitfalls:

• Forgetting data residency laws (e.g., assuming all data can be stored in AWS US-East).
• Ignoring multi-region failover in high-availability questions.
• Overlooking cost in comparisons (e.g., Spanner is more expensive than RDS but offers global consistency).