Linux Server Upgrades & Maintenance: Cron, CUPS, Patches, Backups & Monitoring

TAKEAWAYS:

• Cron/Crontab automates repetitive tasks (e.g., log rotation, backups) using time-based scheduling syntax like * * * * *.
• CUPS (Common Unix Printing System) manages print queues, drivers, and jobs via lpadmin, lpstat, and cancel commands.
• Server upgrades require pre-upgrade checks (disk space, dependencies), staged rollouts, and rollback plans to avoid downtime.
• Backups use tar, rsync, and cron to create incremental snapshots; test restores to verify integrity.
• Monitoring tools (top, htop, netstat, sar) track CPU, memory, disk I/O, and network traffic in real time.
• Security patches must be tested in staging before production deployment to avoid breaking services.

1. Linux Server Maintenance: Core Concepts

Linux servers require regular maintenance to ensure stability, security, and performance. Key tasks include:

• Updating software (OS, applications, security patches).
• Monitoring system health (CPU, memory, disk, network).
• Backing up critical data (configurations, databases, logs).
• Managing logs (rotation, archiving, analysis).
• Optimizing performance (cron jobs, service tuning).

Why Maintenance Matters

• Prevents downtime: Unpatched servers are vulnerable to exploits (e.g., Heartbleed, Log4j).
• Improves efficiency: Optimized cron jobs reduce unnecessary resource usage.
• Ensures compliance: Many industries (banks, healthcare) require regular audits and backups.

2. Job Scheduling with Cron and Crontab

Cron is a time-based job scheduler in Linux that runs commands or scripts at fixed intervals. It is widely used for:

• Automated backups (e.g., daily at 2 AM).
• Log rotation (clearing old logs to free disk space).
• Database maintenance (index optimization, cleanup).
• System monitoring (checking disk space, restarting failed services).

How Cron Works

Cron reads a crontab file (/etc/crontab or user-specific ~/.crontab) containing entries like:

* * * * *  command_to_execute

Where the 5 fields represent:

Worked Example: Daily Backup Script

Suppose you need to back up /var/www/html to /backup/www every night at 1 AM.

1. Create a script backup.sh:

   #!/bin/bash
   tar -czf /backup/www/$(date +\%Y-\%m-\%d).tar.gz /var/www/html
   

2. Make it executable:

   chmod +x /usr/local/bin/backup.sh
   

3. Add to root’s crontab:

   crontab -e
   

   Insert:

   0 1 * * * /usr/local/bin/backup.sh
   

Real-World Application: Ncell’s Billing System

Ncell uses cron jobs to:

• Generate daily billing reports at 3 AM (run generate_bill.sh).
• Archive old call logs weekly (delete logs older than 30 days).
• Restart failed services if they crash overnight.

3. Print Server Configuration with CUPS

CUPS (Common Unix Printing System) is the standard printing system in Linux. It manages:

• Print queues (jobs waiting to print).
• Printer drivers (PPD files for different models).
• Access control (who can print to which printer).

Key CUPS Commands

Worked Example: Setting Up a Printer for an Office

1. Install CUPS (if not installed):

   sudo apt install cups  # Debian/Ubuntu
   sudo yum install cups  # RHEL/CentOS
   

2. Add a printer (e.g., HP LaserJet MFP M426):

   lpadmin -p office_printer -v usb://HP/LaserJet -E -m hpcups.ppd
   

3. Share the printer (allow network access):

   lpadmin -p office_printer -o auth-info-required=none
   

4. Test printing:

   echo "Test page" | lp -d office_printer
   

Real-World Application: Daraz’s Warehouse Printing

Daraz uses CUPS to:

• Print shipping labels automatically when an order is processed.
• Manage multiple printers (thermal for labels, laser for invoices).
• Log print jobs for auditing (who printed what and when).

4. Server Upgrades: Process and Best Practices

Upgrading a Linux server involves planning, testing, and execution to minimize downtime. Steps include:

Pre-Upgrade Checklist

1. Check disk space:

   df -h
   

2. Verify dependencies:

   apt-cache depends package_name  # Debian/Ubuntu
   yum deplist package_name        # RHEL/CentOS
   

3. Backup critical data:

   tar -czf backup.tar.gz /etc /var/www
   

4. Test in a staging environment (clone the server in a VM).

Upgrade Workflow

1. Update package lists:

   apt update && apt upgrade -y  # Debian/Ubuntu
   yum update -y                  # RHEL/CentOS
   

2. Upgrade the OS:

   do-release-upgrade  # Ubuntu
   

3. Reboot and verify:

   uname -a  # Check new kernel version
   

4. Rollback plan: If the upgrade fails, restore from backup or revert to a snapshot.

Real-World Application: Bank of Kathmandu’s Server Upgrade

Bank of Kathmandu upgraded its core banking system from Ubuntu 18.04 to 20.04 in stages:

1. Tested in a VM with a replica of production data.
2. Upgraded non-critical servers first (e.g., web servers).
3. Scheduled the upgrade during low-traffic hours (3 AM).
4. Used rsync to sync configurations before rebooting.
5. Monitored logs (/var/log/syslog) for errors post-upgrade.

5. Backup Strategies for Linux Servers

Backups are critical for disaster recovery. Common methods:

Backup Tools and Commands

Backup Types

Worked Example: Automated Backup Script

Create /usr/local/bin/backup.sh:

#!/bin/bash
DATE=$(date +\%Y-\%m-\%d)
BACKUP_DIR="/backup/server_$DATE"

# Create backup directory
mkdir -p $BACKUP_DIR

# Backup /etc and /var/www
tar -czf $BACKUP_DIR/etc.tar.gz /etc
tar -czf $BACKUP_DIR/www.tar.gz /var/www

# Log the backup
echo "Backup completed at $(date)" >> /var/log/backup.log

Schedule with cron:

0 2 * * * /usr/local/bin/backup.sh

Real-World Application: NEPSE’s Trading System Backups

NEPSE (Nepal Stock Exchange) uses:

• Daily full backups of trading databases at midnight.
• Hourly incremental backups during market hours.
• Offsite storage (encrypted backups sent to a secure cloud provider).
• Automated testing of restore procedures weekly.

6. Server Monitoring and Logging

Monitoring ensures servers run efficiently and securely. Key tools:

Monitoring Tools

Log Files to Monitor

Worked Example: Monitoring CPU Usage

1. Check current CPU usage:

   top
   

2. Log CPU usage every 5 minutes:

   watch -n 5 "sar -u 1"
   

3. Set up alerts for high CPU (>90%):

   # Add to crontab
   */5 * * * * [ $(grep '^%user ' /proc/loadavg | cut -d. -f1) -gt 90 ] && echo "High CPU!" | mail admin@example.com
   

Real-World Application: Pathao’s Driver App Servers

Pathao monitors its Linux servers for:

• High memory usage (alerts if >80% for 5 minutes).
• Failed driver app connections (logs in /var/log/nginx/error.log).
• Disk space (alerts if /var/lib/docker fills up).
• Network latency (ping checks to AWS regions).

7. Security Patches and Updates

Security patches fix vulnerabilities (e.g., buffer overflows, privilege escalations). Steps:

1. Check for updates:

   apt list --upgradable  # Debian/Ubuntu
   yum check-update      # RHEL/CentOS
   

2. Test patches in staging.
3. Apply updates:

   apt upgrade -y
   

4. Reboot if required:

   reboot
   

5. Verify patch status:

   uname -r  # Check kernel version
   

Real-World Application: Khalti’s Payment Gateway

Khalti’s Linux servers:

• Patch critical vulnerabilities within 24 hours (e.g., Log4j in 2021).
• Use automated tools (e.g., apt-get update && apt-get upgrade -y) for minor updates.
• Schedule major updates during low-traffic hours (e.g., 4 AM).
• Maintain a patch history log for audits.

In the Real World

1. eSewa’s Payment Processing

   • Cron jobs run daily to reconcile transactions and generate reports.
   • CUPS prints receipts for cash deposits at collection centers.
   • Monitoring tools (netstat, sar) track high traffic during festivals (Dashain, Tihar).

2. Daraz’s Order Fulfillment

   • Backup scripts (rsync) sync inventory data between warehouses and the main server every hour.
   • Server upgrades are tested in a staging environment before deploying to production during off-peak hours (2–5 AM).
   • Logging (/var/log/nginx/access.log) helps analyze traffic spikes during sales events.

3. NTC’s Network Monitoring

   • Cron automates daily bandwidth usage reports sent to NTC engineers.
   • CUPS manages print queues for technical reports in NTC’s offices.
   • Patch management ensures routers and switches run the latest firmware to prevent outages.

Exam Tip

This unit is theoretical but practical—expect:

1. Short-answer questions on:
   • Cron syntax (e.g., "Write a crontab entry to run a script every Monday at 3 PM").
   • CUPS commands (e.g., "How to enable a printer?").
   • Backup strategies (e.g., "Difference between full and incremental backups").
2. Scenario-based questions (e.g.):
   • "A bank’s server crashed after an upgrade. Explain the steps to recover it."
   • "How would you monitor CPU usage on a server handling 10,000 concurrent WhatsApp-like messages?"
3. Command-based questions:
   • "Write commands to backup /home to /backup using tar."
   • "How to check pending print jobs in CUPS?"
4. Real-world applications:
   • "How does Daraz use cron jobs in its order processing system?"
   • "Why is patch management critical for eSewa’s payment servers?"

Focus on:

• Cron syntax (memorize the 5 fields).
• CUPS workflow (adding, enabling, sharing printers).
• Backup methods (tar, rsync, LVM snapshots).
• Upgrade steps (check, test, execute, rollback).
• Monitoring tools (top, sar, netstat).

stateDiagram-v2
    [*] --> Idle
    Idle --> Running: cron job triggered
    Running --> Success: command completes
    Running --> Failure: error occurs
    Success --> [*]
    Failure --> [*]

sequenceDiagram
    Client->>CUPS: lp -d printer_name file.pdf
    CUPS->>Printer: Send job to queue
    Printer->>CUPS: Acknowledge receipt
    CUPS->>Client: Job ID assigned
    CUPS->>Printer: Process and print
    Printer-->>Client: Print complete