Probability Basics, Conditional Probability & Applications

TAKEAWAYS:

• Understand sample space (S), events (E), and probability axioms (P(E) ∈ [0,1], P(S)=1) as the foundation for all calculations.
• Master conditional probability $P(A|B)=\frac{P(A\cap B)}{P(B)}$ and its independence condition ($P(A\cap B)=P(A)P(B)$).
• Apply Bayes’ Theorem $P(A|B)=\frac{P(B|A)P(A)}{P(B)}$ to reverse probabilities in medical testing or spam detection.
• Solve combinatorial probability problems using $P(E)=\frac{\text{favorable outcomes}}{\text{total outcomes}}$ with permutations/combinations.
• Recognize applications in risk assessment (e.g., "How many ways can you invite 6 out of 9 families?" uses combinations).
• Memorize common pitfalls: ignoring sample space constraints, misapplying independence, or confusing $P(A|B)$ vs. $P(B|A)$.

---

1. Probability Basics

1.1 Definitions

• Experiment: A procedure with well-defined outcomes (e.g., rolling a die, flipping a coin).
• Sample Space (S): Set of all possible outcomes. Example: For a die, $S=\{1,2,3,4,5,6\}$.
• Event (E): A subset of $S$. Can be simple (single outcome) or compound (multiple outcomes). Example: $E=\{2,4,6\}$ (even numbers).
• Probability Axioms (Kolmogorov):
  1. $0\le P(E)\le 1$ for any event $E$.
  2. $P(S)=1$.
  3. For mutually exclusive events $E_1,E_2,\dots$, $P(E_1\cup E_2\cup \dots )=\sum P(E_i)$.

1.2 Probability of an Event

For finite sample spaces, probability is calculated as: $$P(E)=\frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}}$$ Worked Example: Problem: What is the probability of drawing a king from a standard deck of 52 cards? Solution:

• Favorable outcomes = 4 (kings of hearts, diamonds, clubs, spades).
• Total outcomes = 52.
• $P(\text{King})=\frac{4}{52}=\frac{1}{13}$.

1.3 Key Properties

---

2. Conditional Probability

2.1 Definition

Conditional probability measures the likelihood of an event $A$ given that another event $B$ has occurred. $$P(A|B)=\frac{P(A\cap B)}{P(B)},\text{where }P(B)>0$$ Example:

• Event A: Drawing a red card from a deck.
• Event B: Drawing a king.
• $P(A|B)=\frac{P(\text{red king})}{P(\text{king})}=\frac{2/52}{4/52}=\frac{1}{2}$.

2.2 Multiplication Rule

The probability of both $A$ and $B$ occurring: $$P(A\cap B)=P(A|B)\cdot P(B)=P(B|A)\cdot P(A)$$ Worked Example: Problem: A bag has 3 red and 2 blue balls. Two balls are drawn without replacement. What is the probability both are red? Solution:

1. $P(\text{First red})=\frac{3}{5}$.
2. $P(\text{Second red}|\text{First red})=\frac{2}{4}$.
3. $P(\text{Both red})=\frac{3}{5}\times \frac{2}{4}=\frac{6}{20}=\frac{3}{10}$.

2.3 Independent Events

Two events $A$ and $B$ are independent if: $$P(A\cap B)=P(A)\cdot P(B)$$ Implication: $P(A|B)=P(A)$ and $P(B|A)=P(B)$. Example:

• Flipping a coin twice: $P(\text{Heads first})=\frac{1}{2}$, $P(\text{Heads second})=\frac{1}{2}$.
• $P(\text{Heads first and second})=\frac{1}{2}\times \frac{1}{2}=\frac{1}{4}$. Independent.

Non-Example:

• Drawing two cards without replacement from a deck: $P(\text{King second}|\text{King first})\ne P(\text{King second})$. Dependent.

---

3. Bayes’ Theorem

3.1 Statement

Bayes’ Theorem relates conditional probabilities and is used to "reverse" probabilities: $$P(A|B)=\frac{P(B|A)\cdot P(A)}{P(B)}$$ Where:

• $P(B|A)$: Likelihood of $B$ given $A$.
• $P(A)$: Prior probability of $A$.
• $P(B)$: Total probability of $B$ (can be expanded using the Law of Total Probability).

3.2 Law of Total Probability

For mutually exclusive and exhaustive events $A_1,A_2,\dots ,A_n$: $$P(B)=\sum _{i=1}^{n}P(B|A_i)\cdot P(A_i)$$ Worked Example: Problem: A disease affects 1% of a population. A test is 95% accurate (95% true positive, 95% true negative). If a randomly selected person tests positive, what is the probability they have the disease? Solution:

1. Define:
   • $D$: Disease present ($P(D)=0.01$).
   • $\neg D$: Disease absent ($P(\neg D)=0.99$).
   • $+$: Test positive.
2. Given:
   • $P(+|D)=0.95$ (true positive rate).
   • $P(-|\neg D)=0.95$ ⇒ $P(+|\neg D)=0.05$ (false positive rate).
3. Apply Bayes’ Theorem: $$P(D|+)=\frac{P(+|D)\cdot P(D)}{P(+)}=\frac{0.95\times 0.01}{P(+)}$$
4. Calculate $P(+)$ using the Law of Total Probability: $$P(+)=P(+|D)\cdot P(D)+P(+|\neg D)\cdot P(\neg D)=0.95\times 0.01+0.05\times 0.99=0.0095+0.0495=0.059$$
5. Final probability: $$P(D|+)=\frac{0.0095}{0.059}\approx 0.161\text{ or }16.1\%$$ Interpretation: Even with a positive test, there’s only a 16.1% chance the person has the disease due to the low prior probability.

---

4. Applications of Conditional Probability

4.1 Real-World Scenarios

4.2 Combinatorial Problems

Worked Example (from past exams): Problem: You have 9 families to invite to a wedding but can only invite 6. How many different sets of invitations can you write? Solution:

• This is a combinatorial probability problem if we consider probabilities (e.g., "What’s the chance a specific family is invited?").
• Total ways to choose 6 out of 9: $\left(\genfrac{}{}{0ex}{}{9}{6}\right)=\frac{9!}{6!3!}=84$.
• If probabilities are involved (e.g., "What’s the probability Family A is invited?"), use: $$P(\text{Family A invited})=\frac{\left(\genfrac{}{}{0ex}{}{8}{5}\right)}{\left(\genfrac{}{}{0ex}{}{9}{6}\right)}=\frac{56}{84}=\frac{2}{3}$$ (Since if Family A is invited, we choose 5 more from the remaining 8.)

---

5. Common Mistakes and Pitfalls

1. Ignoring Sample Space:

   • Mistake: Calculating $P(E)$ without defining $S$.
   • Fix: Always state the total number of possible outcomes.

2. Misapplying Independence:

   • Mistake: Assuming events are independent when they are not (e.g., drawing cards without replacement).
   • Fix: Use the multiplication rule $P(A\cap B)=P(A|B)P(B)$.

3. Confusing $P(A|B)$ and $P(B|A)$:

   • Mistake: Reversing the condition (e.g., using $P(B|A)$ when $P(A|B)$ is needed).
   • Fix: Draw a probability tree or use Bayes’ Theorem.

4. Forgetting Mutually Exclusivity:

   • Mistake: Adding probabilities of non-mutually exclusive events without subtracting $P(A\cap B)$.
   • Fix: Use $P(A\cup B)=P(A)+P(B)-P(A\cap B)$.

5. Incorrect Use of Combinations:

   • Mistake: Using permutations when order doesn’t matter (or vice versa).
   • Fix: Recall $\left(\genfrac{}{}{0ex}{}{n}{k}\right)=\frac{n!}{k!(n-k)!}$ for combinations.

---

6. Exam Tip: How to Score Full Marks

6.1 Understanding the Question

• Key Phrases:
  • "Given that..." → Conditional probability $P(A|B)$.
  • "Probability of both..." → Multiplication rule $P(A\cap B)$.
  • "At least one..." → Use complement rule $1-P(\text{none})$.

6.2 Step-by-Step Solutions

1. Define Events Clearly:
   • Label all events (e.g., $A,B,C$) and their complements.
2. Draw a Diagram:
   • Use Venn diagrams for overlapping events or probability trees for conditional problems.
3. Apply the Correct Formula:
   • For Bayes’ Theorem, always write: $$P(A|B)=\frac{P(B|A)P(A)}{P(B)}$$ and expand $P(B)$ using the Law of Total Probability.
4. Show All Calculations:
   • Partial credit is given for correct intermediate steps. For example:
     • Calculate $P(A\cap B)$ separately.
     • Expand $P(B)$ explicitly.
5. Check Units and Logic:
   • Ensure probabilities are between 0 and 1.
   • Verify if the answer makes intuitive sense (e.g., a 90% chance of a rare disease testing positive should seem low).

6.3 Common Exam Questions and Strategies

6.4 Practice Problems for TU Exams

1. Problem: A box contains 5 red, 3 blue, and 2 green balls. Two balls are drawn with replacement. What is the probability both are blue? Solution: $$P(\text{Blue first})=\frac{3}{10},P(\text{Blue second})=\frac{3}{10}$$ Since independent: $$P(\text{Both blue})={\left(\frac{3}{10}\right)}^2=\frac{9}{100}$$

2. Problem: In a class, 60% pass Math, 70% pass Science, and 50% pass both. What is the probability a student passes at least one subject? Solution: Use the inclusion-exclusion principle: $$P(M\cup S)=P(M)+P(S)-P(M\cap S)=0.6+0.7-0.5=0.8$$

3. Problem: A factory produces 10% defective items. A test detects 90% of defects but has a 5% false positive rate. What is the probability an item is defective given it tested positive? Solution: Apply Bayes’ Theorem:

   • $P(D)=0.1$, $P(\neg D)=0.9$.
   • $P(+|D)=0.9$, $P(+|\neg D)=0.05$.
   • $P(D|+)=\frac{0.9\times 0.1}{0.9\times 0.1+0.05\times 0.9}=\frac{0.09}{0.09+0.045}=\frac{0.09}{0.135}\approx 0.667$.

---

7. Summary Table: Key Formulas

---

8. Final Notes

• Memorize: The definitions of sample space, events, and the three probability axioms.
• Practice: Work through problems involving with/without replacement, with/without order, and dependent/independent events.
• Bayes’ Theorem: This is a high-weight topic in TU exams. Master it by solving medical testing and spam filter problems.
• Combinatorics Link: Probability often combines with counting principles (e.g., $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$). Review permutations/combinations if needed.
• Real-World Connection: Relate problems to scenarios like risk assessment, machine learning (Naive Bayes), or game theory.

---

Exam Tip:

• Time Management: Probability questions in TU exams often carry 5–10 marks. Spend ~5 minutes defining events and ~5 minutes calculating.
• Show Work: Even if you struggle, partial credit is given for correct steps. For example:
  • Writing $P(A|B)=\frac{P(A\cap B)}{P(B)}$ earns marks.
  • Expanding $P(B)$ using the Law of Total Probability earns additional marks.
• Avoid Assumptions: If a problem doesn’t specify replacement, assume no replacement (dependent events). If it’s ambiguous, state your assumption.