Serialization, I/O Streams, File Handling & Advanced Data Persistence

TAKEAWAYS:

• Understand serialization (converting objects to byte streams) and its role in saving/loading objects to/from files or networks.
• Master I/O streams (byte vs. character, buffered vs. unbuffered) and their hierarchy in Java’s java.io package.
• Learn file handling techniques (reading/writing text/binary files, random access) with practical examples like logs or databases.
• Compare serialization vs. JSON/XML for data persistence, including security and compatibility trade-offs.
• Optimize I/O performance using buffering, compression (GZIP), and object streams.
• Apply exception handling for file operations (e.g., FileNotFoundException, IOException).

1. Serialization: Saving and Loading Objects

Serialization is the process of converting an object’s state (data) into a byte stream to store it or transmit it over a network. Java provides built-in support via java.io.Serializable interface.

How Serialization Works

1. An object implements Serializable.
2. ObjectOutputStream converts the object to bytes.
3. Bytes are written to a file/network.
4. ObjectInputStream reconstructs the object from bytes.

import java.io.*;

class Student implements Serializable {
    private static final long serialVersionUID = 1L;
    String name;
    int rollNo;
    // Constructor, getters, setters
}

public class SerializeDemo {
    public static void main(String[] args) {
        Student s = new Student("Rohan", 101);
        try {
            // Serialize to file
            FileOutputStream fos = new FileOutputStream("student.ser");
            ObjectOutputStream oos = new ObjectOutputStream(fos);
            oos.writeObject(s);
            oos.close();
            fos.close();
            System.out.println("Serialized!");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

State After Serialization

graph LR
    A["Student Object (s)"]
    B["ObjectOutputStream"]
    C["File: student.ser (byte stream)"]
    A -->|"writeObject()"| B
    B -->|"write()"| C

Deserialization (Loading Object)

try {
    FileInputStream fis = new FileInputStream("student.ser");
    ObjectInputStream ois = new ObjectInputStream(fis);
    Student loadedStudent = (Student) ois.readObject();
    System.out.println("Name: " + loadedStudent.name);
    ois.close();
    fis.close();
} catch (IOException | ClassNotFoundException e) {
    e.printStackTrace();
}

Key Points

• serialVersionUID: Ensures version compatibility. Without it, Java generates one automatically.
• Transient Fields: Marked with transient keyword, these fields are ignored during serialization.
• Security: Serialization can be a security risk (e.g., malicious object reconstruction). Use ObjectInputFilter to mitigate.

Real-World Example: eSewa Transaction Logs

eSewa stores transaction records (e.g., user IDs, amounts) as serialized objects in a database. This allows:

• Fast retrieval of past transactions.
• Efficient storage (binary format is compact).
• Cross-platform compatibility (Java objects can be read anywhere).

2. I/O Streams: Byte vs. Character Streams

Java’s java.io package provides two types of streams:

1. Byte Streams (InputStream, OutputStream): Handle raw bytes (e.g., images, binary files).
2. Character Streams (Reader, Writer): Handle text (Unicode characters).

Stream Hierarchy

classDiagram
    class InputStream
    class OutputStream
    class Reader
    class Writer
    class FileInputStream <|-- InputStream
    class FileOutputStream <|-- OutputStream
    class FileReader <|-- Reader
    class FileWriter <|-- Writer
    class BufferedInputStream <|-- InputStream
    class BufferedOutputStream <|-- OutputStream
    class BufferedReader <|-- Reader
    class BufferedWriter <|-- Writer
    class ObjectOutputStream <|-- OutputStream
    class ObjectInputStream <|-- InputStream

Example: Reading a Text File

try (FileReader fr = new FileReader("data.txt");
     BufferedReader br = new BufferedReader(fr)) {
    String line;
    while ((line = br.readLine()) != null) {
        System.out.println(line);
    }
} catch (IOException e) {
    e.printStackTrace();
}

State After Reading a File

graph LR
    A["File: data.txt"] -->|"FileReader"| B["BufferedReader"]
    B -->|"readLine()"| C["Line 1: Hello"]
    B -->|"readLine()"| D["Line 2: World"]

Buffered Streams for Performance

• BufferedInputStream/BufferedOutputStream: Reduce I/O operations by buffering data in memory.
• BufferedReader/BufferedWriter: Improve text reading/writing efficiency.

Example: Writing to a File with Buffering

try (FileWriter fw = new FileWriter("output.txt");
     BufferedWriter bw = new BufferedWriter(fw)) {
    bw.write("Buffered I/O is efficient!");
    bw.newLine();
    bw.write("Java makes it easy.");
} catch (IOException e) {
    e.printStackTrace();
}

3. File Handling: Reading and Writing Files

Common File Operations

Example: Copying a File

public static void copyFile(String src, String dest) throws IOException {
    try (FileInputStream fis = new FileInputStream(src);
         FileOutputStream fos = new FileOutputStream(dest)) {
        byte[] buffer = new byte[1024];
        int bytesRead;
        while ((bytesRead = fis.read(buffer)) != -1) {
            fos.write(buffer, 0, bytesRead);
        }
    }
}

State After Copying a File

graph LR
    A["Source File (src.txt)"] -->|"FileInputStream"| B["Buffer (1KB chunks)"]
    B -->|"FileOutputStream"| C["Destination File (dest.txt)"]

Real-World Example: Daraz Order Processing

Daraz uses file I/O to:

• Log orders in orders.log (appended daily).
• Store product images in binary format (product_images/).
• Read/write inventory files (inventory.dat) for stock updates.

Example: Logging an Order

try (FileWriter fw = new FileWriter("orders.log", true);
     BufferedWriter bw = new BufferedWriter(fw)) {
    bw.write("Order#123: User=John, Product=Laptop, Amount=50000");
    bw.newLine();
} catch (IOException e) {
    e.printStackTrace();
}

4. Random Access Files

RandomAccessFile allows reading/writing at any position in a file (useful for databases or large logs).

Example: Updating a Record in a File

try (RandomAccessFile raf = new RandomAccessFile("data.dat", "rw")) {
    // Seek to position 100 (bytes)
    raf.seek(100);
    raf.writeUTF("Updated Value");
} catch (IOException e) {
    e.printStackTrace();
}

State After Random Access Update

graph LR
    A["File: data.dat"] -->|"seek(100)"| B["Position 100"]
    B -->|"writeUTF()"| C["Updated Value"]

Real-World Example: NTC Electricity Bill Updates

NTC stores customer bills in a binary file (bills.dat). When a user pays, the system:

1. Opens bills.dat with RandomAccessFile.
2. Seeks to the customer’s record (e.g., position 5000).
3. Updates the paid flag to true.

5. Serialization vs. JSON/XML

Example: JSON Alternative (Using org.json)

import org.json.*;

JSONObject student = new JSONObject();
student.put("name", "Rohan");
student.put("rollNo", 101);
String jsonString = student.toString(); // '{"name":"Rohan","rollNo":101}'

6. Advanced I/O: Compression and Performance

Compressing Files with GZIP

try (FileOutputStream fos = new FileOutputStream("data.gz");
     GZIPOutputStream gzos = new GZIPOutputStream(fos);
     FileInputStream fis = new FileInputStream("data.txt")) {
    byte[] buffer = new byte[1024];
    int bytesRead;
    while ((bytesRead = fis.read(buffer)) != -1) {
        gzos.write(buffer, 0, bytesRead);
    }
} catch (IOException e) {
    e.printStackTrace();
}

Real-World Example: WhatsApp Media Storage

WhatsApp compresses images/videos using algorithms like GZIP or ZIP before sending them to save bandwidth. This reduces file size by ~70% for photos.

7. Exception Handling in I/O

Common exceptions:

• FileNotFoundException: File does not exist.
• IOException: General I/O errors (e.g., disk full).
• ClassNotFoundException: During deserialization (invalid class).

Example: Handling File Errors

try {
    FileInputStream fis = new FileInputStream("nonexistent.txt");
} catch (FileNotFoundException e) {
    System.err.println("File not found: " + e.getMessage());
} catch (IOException e) {
    System.err.println("I/O Error: " + e.getMessage());
}

In the Real World

1. eSewa Transaction Records

   • Idea Used: Serialization + File I/O
   • How: Transactions are serialized into transactions.ser for fast retrieval. During audits, eSewa deserializes these objects to verify amounts.

2. Khalti Payment Logs

   • Idea Used: Buffered Character Streams
   • How: Khalti appends payment logs (payments.log) using BufferedWriter to minimize disk writes, improving speed.

3. Daraz Inventory Management

   • Idea Used: Random Access Files
   • How: Daraz uses RandomAccessFile to update stock levels in inventory.dat without rewriting the entire file, saving time.

4. Ncell Customer Data Backup

   • Idea Used: Compression (GZIP)
   • How: Nightly backups of customer data are compressed with GZIP to reduce storage costs and backup time.

Exam Tip

1. Serialization Questions:

   • Always include serialVersionUID in your code.
   • Explain why transient fields are excluded.
   • Compare serialization with JSON/XML in terms of speed and portability.

2. I/O Streams:

   • Draw the stream hierarchy (byte vs. character, buffered vs. unbuffered).
   • Show the state of buffers after operations (e.g., reading a file line by line).

3. File Handling:

   • For file copying, explain the role of buffers (e.g., 1KB chunks).
   • In random access examples, show the seek() position and updated data.

4. Real-World Scenarios:

   • Relate file operations to apps like eSewa (logs), Daraz (inventory), or Ncell (backups).
   • For compression, mention bandwidth savings (e.g., WhatsApp, YouTube).

5. Common Pitfalls:

   • Forgetting to close streams (use try-with-resources).
   • Not handling IOException or ClassNotFoundException.
   • Serializing non-serializable objects (e.g., ArrayList without implementing Serializable).

Practice: Implement a program that:

1. Serializes a BankAccount object to a file.
2. Uses RandomAccessFile to update the balance.
3. Compresses the file with GZIP for backup.