Neural Networks: Layers, Learning & Applications

Neural Networks: The Brain of Modern AI

What is a Neural Network?

A neural network (NN) is a computational model inspired by the human brain’s neural structure. It consists of interconnected nodes (neurons) organized in layers that process input data, learn patterns, and make predictions or decisions. Unlike traditional programming, NNs adapt through training on data rather than following rigid rules.

Biological vs. Artificial Neurons

Structure of a Neural Network

A NN is built in layers:

1. Input Layer: Receives raw data (e.g., pixel values in an image).
2. Hidden Layers: Perform computations (can be 1–100+ layers in deep learning).
3. Output Layer: Produces the final result (e.g., a class label or probability).

Example: Handwritten Digit Recognition (MNIST Dataset)

• Input: 28×28 pixel image (flattened into 784 values).
• Hidden Layers: 2 layers with 128 and 64 neurons (ReLU activation).
• Output: 10 neurons (one for each digit 0–9, softmax activation).

graph LR
    A["Input Layer: 784 neurons"] --> B["Hidden Layer 1: 128 neurons<br/>ReLU"]
    B --> C["Hidden Layer 2: 64 neurons<br/>ReLU"]
    C --> D["Output Layer: 10 neurons<br/>Softmax"]

Key Components

1. Activation Functions

Convert weighted sums into outputs. Common types:

2. Loss Functions

Measure error between predicted and actual outputs. Examples:

• Mean Squared Error (MSE): For regression. $\text{MSE}=\frac{1}{n}{\sum }_{i=1}^n(y_i-{\hat{y}}_i{)}^2$
• Cross-Entropy: For classification. $\text{CE}=-{\sum }_i{y}_i\log ({\hat{y}}_i)$

Worked Example: Binary Classification Loss Suppose:

• True label $y=1$, predicted probability $\hat{y}=0.7$.
• Loss (binary cross-entropy): $-\left(1\cdot \log (0.7)+0\cdot \log (0.3)\right)=0.3567$.

How Neural Networks Learn: Backpropagation

Neural networks learn by adjusting weights to minimize loss. The process:

1. Forward Pass: Compute predictions and loss.
2. Backward Pass: Calculate gradients of loss w.r.t. weights using the chain rule.
3. Update Weights: Use an optimizer (e.g., Stochastic Gradient Descent, SGD) to adjust weights.

Example: Training a NN to Predict House Prices

Data: 3 features (size, bedrooms, location score) → predict price. Steps:

1. Initialize random weights $w_1,w_2,w_3$ and bias $b$.
2. Forward pass: $\text{output}=\sigma (w_1\cdot \text{size}+w_2\cdot \text{bedrooms}+w_3\cdot \text{location}+b)$.
3. Compute loss (MSE) vs. true price.
4. Backpropagate to find $\frac{\partial \text{Loss}}{\partial w_i}$.
5. Update weights: $w_i=w_i-\alpha \cdot \frac{\partial \text{Loss}}{\partial w_i}$ (where $\alpha$ = learning rate).

Types of Neural Networks

Real-World Applications in Nepal and Globally

1. eSewa Fraud Detection

• Idea Used: Feedforward NN with binary classification.
• How: Trained on transaction data (amount, time, location) to flag fraudulent payments (e.g., unusual high-value transfers at odd hours).
• Output: Probability score (e.g., 0.92 = "high risk").

2. Ncell Chatbot (Nepali Language Processing)

• Idea Used: RNN/LSTM for sequence processing.
• How: Processes customer queries (e.g., "My bill is pending") as sequences of words, predicts intent, and generates responses.
• Example Trace: Input: ["My", "bill", "is", "pending"] → Encoded as vectors → LSTM hidden states → Output: "Please check your payment method."

3. Daraz Recommendation System

• Idea Used: Collaborative Filtering + NN Embeddings.
• How: NN learns user-item interaction patterns (e.g., users who bought X also buy Y) to recommend products.
• Example: If you view shoes, the NN predicts "Buy these sandals" based on similar users' behavior.

4. NTC Traffic Route Optimization

• Idea Used: Reinforcement Learning (RL).
• How: An RL agent (NN) learns to adjust traffic light timings in Kathmandu to minimize congestion by simulating thousands of scenarios.
• Worked Example:
  • State: Traffic density at 3 intersections.
  • Action: Green light duration for each road.
  • Reward: Negative delay time for vehicles.
  • Policy: NN outputs optimal timing (e.g., "Road A: 45s green, Road B: 30s green").

Advantages and Limitations

Worked Example: Predicting Loan Defaults (Nepali Banks)

Problem: A bank wants to predict if a customer will default on a loan using NN. Data: 5 features (income, credit score, loan amount, employment status, age). Steps:

1. Preprocess:
   • Normalize income (scale to 0–1).
   • Encode employment status (e.g., "employed" → 1, "unemployed" → 0).
2. Model Architecture:

   Model:
     Input Layer: 5 neurons
     Hidden Layer 1: 16 neurons (ReLU)
     Hidden Layer 2: 8 neurons (ReLU)
     Output Layer: 1 neuron (Sigmoid for probability)
   

3. Training:
   • Loss: Binary cross-entropy.
   • Optimizer: Adam (adaptive learning rate).
   • Batch size: 32, epochs: 50.
4. Prediction: Input: [income=0.8, credit_score=0.6, loan_amount=0.5, employed=1, age=0.4] Output: 0.2 → "Low risk of default (78% confidence)."

Exam Tip

1. Diagrams Are Key: Always draw the NN architecture (layers, activations) for questions on design. Label inputs, hidden layers, and outputs clearly.
2. Math Shortcuts: For backpropagation, focus on the chain rule and gradient flow. Memorize:
   • $\frac{\partial \text{ReLU}}{\partial x}=1$ if $x>0$, else 0.
   • $\frac{\partial \text{Sigmoid}}{\partial x}=\sigma (x)(1-\sigma (x))$.
3. Real-World Links: Examiners love connections to local tech. For example:
   • "How would you design a NN for NEPSE stock prediction?" → Use LSTM for time-series data.
   • "Explain how Pathao uses CNNs." → Object detection for rider locations in satellite images.
4. Common Pitfalls:
   • Forgetting to normalize input data (e.g., pixel values 0–255 → 0–1).
   • Confusing activation functions (e.g., using ReLU in the output layer for classification).
5. Practical Question Tricks:
   • If asked to "explain backpropagation," trace one weight’s gradient through the network.
   • For "advantages of CNNs," mention parameter sharing (same filter applied to all regions of an image).

Final Note: Neural networks are the backbone of modern AI. Master the architecture, math of learning, and real-world mapping to ace this unit. Practice coding a simple NN (even in Python’s numpy) to solidify concepts!