Network ProgrammingUnit 710 min read

URL, HTTP, Cookies & Web Data Handling

Unit 7 of Network Programming: Explores Java’s URL/URI classes, HTTP methods, headers, cookies, and web data retrieval with hands-on code examples, real-world apps (e.g., Daraz, eSewa), and step-by-step traces of HTTP requests/responses.

Padhai · Based on the official syllabus and past papers
Updated 29 Sept 2026

TAKEAWAYS:

Java’s URL/URI classes parse web addresses into components (scheme, host, path) for programmatic use.
HTTP methods (GET, POST, etc.) define client-server interactions, while status codes (200, 404) indicate success/failure.
Cookies store session data (e.g., user login tokens) via CookieManager and HttpCookie in Java.
URL connections (URLConnection) fetch headers/data from servers, enabling dynamic web scraping or API calls.
HTTP Keep-Alive optimizes performance by reusing TCP connections for multiple requests.
Real-world use: Daraz uses HTTP GET/POST for product listings; eSewa relies on cookies for session persistence.

1. URL vs. URI: Definitions and Java Classes

Key Definitions

Uniform Resource Locator (URL): A specific subset of URI that includes scheme (e.g., https), host (e.g., www.daraz.com), and path (e.g., /products). Example:
```
https://www.daraz.com/products/laptop?brand=Dell
```
Uniform Resource Identifier (URI): Broader than URL; includes URNs (e.g., urn:isbn:9780132350884). URLs are URIs, but not all URIs are URLs.

Java Classes

Class	Purpose
`java.net.URL`	Represents a URL; parses components (scheme, host, port, path, query).
`java.net.URI`	More general; handles URIs/URLs; supports encoding/decoding.

Example: Parsing a URL

URL url = new URL("https://www.daraz.com/products/laptop?brand=Dell");
System.out.println("Scheme: " + url.getProtocol()); // https
System.out.println("Host: " + url.getHost());       // www.daraz.com
System.out.println("Path: " + url.getPath());       // /products/laptop
System.out.println("Query: " + url.getQuery());     // brand=Dell

Trace Table:

Step	Code Snippet	Output
1	`url.getProtocol()`	`https`
2	`url.getHost()`	`www.daraz.com`
3	`url.getPath()`	`/products/laptop`
4	`url.getQuery()`	`brand=Dell`

Visual: URL Structure

Scheme: https Host: www.daraz.com Port: 443 (default for HTTPS) Path: /products/laptop Query: brand=Dell Fragment: (none)

2. HTTP Methods and Status Codes

HTTP Methods (Request Types)

Method	Purpose	Example Use Case
GET	Retrieve data from server.	Fetching Daraz product details.
POST	Send data to server (e.g., form submission).	Logging into eSewa.
PUT	Update existing resource.	Updating user profile on Pathao.
DELETE	Remove a resource.	Deleting a saved order on Daraz.
HEAD	Fetch headers only (no body).	Checking if a file exists.
OPTIONS	Describe allowed methods for a resource.	API documentation.

HTTP Status Codes

Code	Category	Example Meaning	Real-World Impact
200	Success	OK – Request succeeded.	Daraz shows product page.
404	Client Error	Not Found – Resource missing.	NTC website returns "Page Not Found".
500	Server Error	Internal Server Error.	Ncell app crashes during payment.

Example: HTTP GET Request Trace

Scenario: Fetching Daraz’s homepage headers.

URL url = new URL("https://www.daraz.com");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
int responseCode = conn.getResponseCode(); // 200
System.out.println("Response Code: " + responseCode);

Trace Table:

Step	Action	Response Code	Data Fetched
1	Open connection to `daraz.com`	N/A	Connection established.
2	Set `GET` method	N/A	Headers sent to server.
3	`conn.getResponseCode()`	200	Server returns success + headers.

Visual: HTTP Request/Response Flow

sequenceDiagram
    participant Client
    participant Server
    Client->>Server: GET / HTTP/1.1\nHost: www.daraz.com
    Server-->>Client: HTTP/1.1 200 OK\nHeaders...\n<body>

3. Cookies: Session Management

What Are Cookies?

Small data packets sent by servers to clients (browsers) to track sessions.
Example: eSewa stores your login token in a cookie to avoid re-entering credentials.

Class	Purpose
`java.net.CookieManager`	Manages cookie storage/retrieval.
`java.net.HttpCookie`	Represents a single cookie (name-value pairs).

Example: Blocking `.gov` Cookies

CookieManager manager = new CookieManager();
manager.setCookiePolicy(CookiePolicy.ACCEPT_ALL); // Default
// Block .gov cookies
manager.setCookiePolicy(new CookiePolicy() {
    @Override
    public boolean shouldAccept(HttpCookie cookie) {
        return !cookie.getDomain().endsWith(".gov");
    }
});

Trace:

If a server sends Set-Cookie: id=123; Domain=.gov, the manager rejects it.
All other cookies (e.g., Set-Cookie: session=abc; Domain=.esewa) are accepted.

How cookie blocking policies prevent third-party access to government session data

Client (Browser) <-->[Set-Cookie: session=abc]--> Server (eSewa) Client <-->[Cookie: session=abc]--> Server (for subsequent requests)

4. URLConnection: Fetching Data

Steps to Read Headers/Data

Open a URLConnection.
Set request method (GET/POST).
Read headers (getHeaderField()).
Read response body (getInputStream()).

Example: Reading Headers from NTC

URLConnection conn = new URL("https://www.ntc.gov.np").openConnection();
conn.setRequestMethod("GET");
for (int i = 1; ; i++) {
    String header = conn.getHeaderField(i);
    if (header == null) break;
    System.out.println(header);
}

Output Snippet:

Date: Mon, 01 Jan 2024 00:00:00 GMT
Server: Apache/2.4.41
Content-Type: text/html

Trace Table:

Header Field	Value
`Date`	`Mon, 01 Jan 2024 00:00:00 GMT`
`Server`	`Apache/2.4.41`
`Content-Type`	`text/html`

Visual: URLConnection Workflow

flowchart TD
    A["Open URLConnection"] --> B["Set Request Method"]
    B --> C["Read Headers (getHeaderField())"]
    C --> D["Read Body (getInputStream())"]
    D --> E["Process Data"]

5. HTTP Keep-Alive

What It Does

Reuses a single TCP connection for multiple HTTP requests, reducing latency.
Example: Daraz loads product images, descriptions, and reviews in one session.

Java Implementation

System.setProperty("http.keepAlive", "true"); // Enable Keep-Alive
URLConnection conn = new URL("https://www.daraz.com").openConnection();
conn.setRequestProperty("Connection", "keep-alive");

Impact:

Fewer connection handshakes → faster page loads.
Saves bandwidth (e.g., Ncell users save data).

6. Real-World Applications

In the Real World

Daraz:
- Uses HTTP GET to fetch product listings and POST for order submissions.
- Relies on cookies to remember user carts across sessions.
eSewa:
- Stores session cookies (eSewaSessionID) to validate payments without re-login.
- Uses HTTP 302 Redirect after successful transactions.
NTC Website:
- Returns HTTP 404 if a page is deleted (e.g., old tariff schedules).
- Uses URLConnection to scrape dynamic content (e.g., service status).

Worked Example: Simulating a Daraz Order

Scenario: A user adds a laptop to their cart (POST) and checks the cart (GET).

sequenceDiagram
  participant User
  participant DarazServer
  User->>DarazServer: POST /cart/add
  {productId: 123, quantity: 1}
  DarazServer-->>User: HTTP/1.1 201 Created
  Set-Cookie: session=abc123; Path=/; Secure
  User->>DarazServer: GET /cart
  Cookie: session=abc123
  DarazServer-->>User: HTTP/1.1 200 OK
  <html>...</html>

Key Takeaways:

POST creates/modifies data (adds item to cart).
Cookie persists cart state for subsequent GET requests.

7. Exam Tips

URL/URI:
- Always distinguish between URL (specific) and URI (general).
- Memorize getProtocol(), getHost(), getQuery() methods.
HTTP Methods:
- GET = safe (idempotent), POST = state-changing.
- Status codes: 2xx = success, 4xx = client error, 5xx = server error.
Cookies:
- Know CookieManager and HttpCookie for session handling.
- Example: Block .gov cookies as shown in the note.
URLConnection:
- Focus on header reading (getHeaderField) and body parsing.
- Trace steps: open → set method → read headers → read body.
Keep-Alive:
- Highlight its role in performance optimization (e.g., Daraz/Ncell).
Real-World Mapping:
- Link Daraz/eSewa to HTTP methods/cookies.
- NTC/Ncell to status codes/URLConnection.

Final Advice:

For programming questions, always include:
- import java.net.*; and import java.io.*;.
- Step-by-step trace tables (like the ones above).
For theory, use flowcharts (e.g., HTTP request flow) to visualize processes.

Based on the TU BCA syllabus for Network Programming (CACS355), unit 7.

Discussion

Loading…

URL, HTTP, Cookies & Web Data Handling

1. URL vs. URI: Definitions and Java Classes

Key Definitions

Java Classes

Example: Parsing a URL

Visual: URL Structure

Scheme: https Host: www.daraz.com Port: 443 (default for HTTPS) Path: /products/laptop Query: brand=Dell Fragment: (none)

2. HTTP Methods and Status Codes

HTTP Methods (Request Types)

HTTP Status Codes

Example: HTTP GET Request Trace

Visual: HTTP Request/Response Flow

3. Cookies: Session Management

What Are Cookies?

Java Cookie Handling

Example: Blocking .gov Cookies

Visual: Cookie Flow in eSewa

4. URLConnection: Fetching Data

Steps to Read Headers/Data

Example: Reading Headers from NTC

Visual: URLConnection Workflow

5. HTTP Keep-Alive

What It Does

Java Implementation

6. Real-World Applications

In the Real World

Worked Example: Simulating a Daraz Order

7. Exam Tips

Discussion

Example: Blocking `.gov` Cookies