Probabilistic Reasoning: Uncertainty, Bayes, and Real-World Applications

TAKEAWAYS:

• Uncertainty ≠ ignorance: Probabilistic reasoning quantifies doubt (e.g., "80% chance of rain") using probability distributions, not just binary logic.
• Bayes’ Theorem flips conditional probabilities: $P(H|E)=\frac{P(E|H)\cdot P(H)}{P(E)}$ lets you update beliefs as evidence arrives (e.g., spam filters learning from misclassified emails).
• Graphical models (Bayesian networks) visualize dependencies between uncertain events (e.g., a patient’s symptoms → disease likelihood).
• Case-based reasoning (CBR) solves new problems by reusing past cases with probabilistic similarity matching (e.g., Daraz’s customer support matching similar order issues).
• Real-world trade-offs: Probabilistic systems balance accuracy vs. computational cost (e.g., Ncell’s network congestion predictions use simplified models for speed).
• Exam focus: Define terms precisely, show calculations step-by-step, and link theory to applications (e.g., medical diagnosis, fraud detection).

Core Concepts: Why Probability?

Knowledge engineering often deals with incomplete or noisy data. Probabilistic reasoning lets systems:

1. Handle uncertainty (e.g., "Is this email spam?" → 72% confidence).
2. Update beliefs dynamically (e.g., weather forecasts refining as new data arrives).
3. Make decisions under risk (e.g., banks approving loans based on credit-score probabilities).

1. Probability Basics: From Dice to AI

Probability is the measure of belief in an event’s occurrence, ranging from 0 (impossible) to 1 (certain). Key terms:

• Prior probability $P(H)$: Initial belief (e.g., "20% of emails are spam").
• Likelihood $P(E|H)$: How evidence supports a hypothesis (e.g., "Spam emails often contain ‘FREE’").
• Posterior probability $P(H|E)$: Updated belief after seeing evidence (e.g., "Given ‘FREE’, spam probability jumps to 85%").

Visual: Probability Space

Example: Rolling a die. $P(\text{odd})=\frac{3}{6}=0.5$.

2. Bayesian Reasoning: Updating Beliefs

Bayes’ Theorem formalizes how to revise probabilities when new evidence arrives. Formula: $$P(H|E)=\frac{P(E|H)\cdot P(H)}{P(E)}$$ Where:

• $P(H|E)$: Posterior (belief after evidence).
• $P(E|H)$: Likelihood (how evidence fits the hypothesis).
• $P(H)$: Prior (initial belief).
• $P(E)$: Marginal probability (total probability of evidence).

Worked Example: Medical Diagnosis

Scenario: A test for a rare disease (prevalence = 1%) has 99% accuracy (true positive rate). You test positive. What’s the probability you have the disease?

• Prior $P(D)$: 0.01 (1% prevalence).
• Likelihood $P(+|D)$: 0.99 (true positive).
• False positive rate $P(+|\neg D)$: 0.01 (1% of healthy people test positive).
• Marginal $P(+)$: $P(+|D)P(D)+P(+|\neg D)P(\neg D)=0.99\times 0.01+0.01\times 0.99=0.0198$.

Calculation: P(D|+) = \frac{0.99 \times 0.01}{0.0198} \approx 0.50 \text{ (50%)} Insight: Even with a positive test, the disease probability is only 50% due to its rarity. This is why doctors use multiple tests or Bayesian networks to combine evidence.

3. Bayesian Networks: Modeling Uncertain Dependencies

Bayesian networks (or belief networks) are directed acyclic graphs (DAGs) where:

• Nodes = random variables (e.g., "Rain," "Traffic Jam").
• Edges = conditional dependencies (e.g., "Rain → Traffic Jam").
• Conditional Probability Tables (CPTs) define $P(\text{Child}|\text{Parents})$.

Example: Traffic Prediction for Kathmandu

graph TD
    A["Rain"] --> B["Road Wet"]
    A --> C["Traffic Jam"]
    B --> C

CPT for "Traffic Jam":

Question: If it rained today ($P(\text{Rain})=0.3$) and roads are wet ($P(\text{Wet}|\text{Rain})=0.8$), what’s $P(\text{Traffic Jam})$? Solution:

1. $P(\text{Wet})=P(\text{Wet}|\text{Rain})P(\text{Rain})+P(\text{Wet}|\neg \text{Rain})P(\neg \text{Rain})=0.8\times 0.3+0.1\times 0.7=0.29$.
2. Use the CPT row where Rain=Yes and Wet=Yes: $P(\text{Jam}|\text{Rain, Wet})=0.9$.
3. Joint probability: $P(\text{Jam})=P(\text{Jam}|\text{Rain, Wet})\cdot P(\text{Wet}|\text{Rain})\cdot P(\text{Rain})=0.9\times 0.8\times 0.3=0.216$.

Real-world tie-in: NTC uses similar models to predict network congestion during festivals (e.g., Dashain), adjusting bandwidth allocation probabilistically.

4. Handling Uncertainty in Knowledge Systems

Probabilistic reasoning is used in:

5. Case-Based Reasoning (CBR): Reusing Probabilistic Cases

CBR solves new problems by matching them to past cases with similar features. Steps:

1. Retrieve: Find the most similar past case(s) (using probabilistic similarity metrics).
2. Reuse: Adapt the solution to the new problem.
3. Revise: Test the solution and adjust if needed.
4. Retain: Store the new case for future use.

Example: Daraz Customer Support

• Case: A user reports a delayed order.
• Retrieval: System finds 10 similar cases (same product, same delivery zone, same delay reason).
• Reuse: Applies the most common solution (e.g., "Reschedule delivery").
• Revise: If the user rejects the solution, the system escalates to a human agent.
• Retain: Adds the new case to the database with updated probabilities (e.g., "This carrier has a 60% delay rate in this area").

Probabilistic Similarity: Use Euclidean distance or cosine similarity on case features (weighted by importance). For example:

• Case 1: {Delay: 3 days, Product: Electronics, Carrier: Ncell} → Vector [3, 1, 1].
• Case 2: {Delay: 2 days, Product: Electronics, Carrier: Ncell} → Vector [2, 1, 1].
• Similarity: $1-\frac{\sqrt{(3-2{)}^2+(1-1{)}^2+(1-1{)}^2}}{\sqrt{3^2+1^2+1^2}}=1-\frac{1}{3.74}\approx 0.73$ (73% similar).

6. Probabilistic Logic vs. Classical Logic

When to Use Which?

• Use classical logic for definite rules (e.g., "If temperature > 100°C, then water boils").
• Use probabilistic logic for uncertain or noisy data (e.g., "Given symptoms X, Y, and Z, 80% chance of disease D").

In the Real World

1. Khalti’s Fraud Detection

   • Idea: Naive Bayes classifiers analyze transaction patterns (e.g., sudden large payments, unusual locations).
   • How: Each transaction feature (amount, time, device) contributes probabilistically to a "fraud score." If $P(\text{Fraud}|\text{Features})>0.9$, the transaction is blocked.
   • Example: A user in Pokhara suddenly sends ₹50,000 to an unknown account in India. The system flags it because:
     • $P(\text{Fraud}|\text{High Amount})=0.7$
     • $P(\text{Fraud}|\text{International})=0.6$
     • Combined posterior: $P(\text{Fraud})\approx 0.85$.

2. Pathao’s Dynamic Pricing

   • Idea: Bayesian networks model rider demand, driver availability, and traffic to adjust fares in real time.
   • How: Nodes include:
     • Demand (time of day, events like Dashain).
     • Supply (number of available drivers).
     • Traffic (probability of delays from NTC data).
   • Example: During a sudden rain in Lalitpur, the system predicts:
     • $P(\text{Traffic Jam})=0.8$ → $P(\text{High Demand})=0.9$ → Fare surcharge of 30%.

3. NEPSE Stock Predictions

   • Idea: Time-series probabilistic models (e.g., Hidden Markov Models) forecast stock prices based on historical trends and news sentiment.
   • How: Analysts input:
     • Technical indicators (e.g., moving averages).
     • Fundamental data (e.g., company earnings reports).
     • Sentiment analysis (e.g., Twitter mentions of "NEPSE crash").
   • Example: If $P(\text{Price Drop}|\text{Negative News})=0.7$ and $P(\text{Negative News})=0.6$, the model predicts a 42% chance of a drop tomorrow.

7. Common Pitfalls and Exam Traps

1. Ignoring Base Rates: Always consider the prior probability (e.g., rare diseases have low $P(D)$, so even high $P(+|D)$ may not make $P(D|+)$ high).
2. Assuming Independence: Naive Bayes assumes features are independent (e.g., "sunny" and "hot" are independent in weather prediction). This simplifies calculations but may reduce accuracy.
3. Overfitting: A Bayesian network with too many parameters (e.g., too many CPT entries) may fit training data but fail on real-world cases.
4. Misapplying Bayes’ Theorem: Forgetting to normalize by $P(E)$ (the denominator) leads to incorrect posteriors.

Exam Tip: If asked to "explain probabilistic reasoning," structure your answer as:

1. Definition: "A framework to handle uncertainty using probability theory."
2. Key Components: Priors, likelihoods, posteriors, Bayes’ Theorem.
3. Example: Medical diagnosis or spam filtering (show calculations).
4. Applications: Bayesian networks, CBR, or NLP.
5. Advantages: Handles incomplete data; updates beliefs dynamically.
6. Limitations: Computational cost; requires accurate probability estimates.

Practice Questions (Self-Check)

1. Bayes’ Theorem: If 5% of people have a disease and a test is 95% accurate, what’s $P(\text{Disease}|\text{Positive Test})$?
2. Bayesian Network: Draw a network for "Student Passes Exam" with nodes: {Study Hours, Attendance, Stress Level, Passes Exam}. Add one edge.
3. CBR: How would you retrieve cases for a "battery drain" issue in a smartphone support system?
4. Comparison: Differentiate between probabilistic clustering (e.g., Gaussian Mixture Models) and deterministic clustering (e.g., k-means).

Exam Tip

• For numerical questions: Show every step of Bayes’ Theorem or CPT calculation. Use fractions or decimals consistently.
• For conceptual questions: Link theory to real systems (e.g., "Like Khalti’s fraud detection, Bayesian filters...").
• For diagrams: Always label nodes and edges clearly. Use directed edges for Bayesian networks (→) and undirected for similarity graphs (—).
• Avoid: Vague statements like "probability is used everywhere." Instead, name the specific technique (e.g., "Naive Bayes for spam classification").

Final Visual Summary