Statistical Inference & Hypothesis Testing: Tests, Confidence, p-Values

TAKEAWAYS:

• Statistical inference lets us estimate population parameters from sample data using confidence intervals and point estimators.
• Hypothesis testing uses p-values and critical values to decide whether observed data supports a null hypothesis (H₀).
• Z-tests and t-tests compare means; χ²-tests compare distributions or goodness-of-fit; F-tests compare variances.
• Type I error (rejecting H₀ when true) and Type II error (failing to reject H₀ when false) trade off with sample size and significance level (α).
• Business uses these tools for A/B testing (e.g., Daraz promotions), quality checks (NTC network reliability), and financial risk (NEPSE stock performance).
• Always assume H₀ is true unless evidence overwhelmingly contradicts it (proof by contradiction).

1. Statistical Inference: Estimating Population Parameters

Statistical inference bridges the gap between sample data and population conclusions. It includes:

• Point estimation: Single-value estimates (e.g., sample mean $\bar{X}$ for population mean $\mu$).
• Interval estimation: Confidence intervals (e.g., $\mu \pm z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}$ for 95% CI).

Key Concepts

• Confidence Interval (CI): A range $[L,U]$ where the true parameter lies with confidence $1-\alpha$ (e.g., 95% CI).

  • Formula for mean ($\sigma$ known): $$\bar{X}\pm z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}$$
  • For small samples ($n<30$) or unknown $\sigma$, use t-distribution: $$\bar{X}\pm t_{\alpha /2}\cdot \frac{s}{\sqrt{n}}$$

• Margin of Error (ME): Half the width of the CI: $$ME=z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}\text{(for large }n\text{)}$$

Worked Example: Daraz’s Promotional Effectiveness

Scenario: Daraz runs a discount campaign and collects a sample of 100 orders. Before the campaign, the average order value was Rs 1,200 (σ = Rs 300). After the campaign, the sample mean is Rs 1,300. Test if the campaign increased order value at 95% confidence.

Steps:

1. Hypotheses:
   • $H_0:\mu =1200$ (no effect)
   • $H_1:\mu >1200$ (campaign helped)
2. Test Statistic (Z-test for large $n$): $$Z=\frac{\bar{X}-{\mu }_0}{\sigma /\sqrt{n}}=\frac{1300-1200}{300/\sqrt{100}}=\frac{100}{30}\approx 3.33$$
3. Critical Value: For 95% CI, $z_{0.05}=1.645$.
   • Since $3.33>1.645$, reject $H_0$.
4. 95% Confidence Interval: $$1300\pm 1.645\cdot \frac{300}{\sqrt{100}}=1300\pm 49.35⟹[1250.65,1349.35]$$ The entire interval is above 1200, confirming the campaign’s effect.

Visual: Interpretation: Daraz can claim the campaign increased order values with 95% confidence.

2. Hypothesis Testing: Framework and Steps

Hypothesis testing evaluates claims about populations using sample data. Steps:

1. State hypotheses:
   • $H_0$: Null hypothesis (status quo, e.g., "no effect").
   • $H_1$: Alternative hypothesis (research claim, e.g., "there is an effect").
2. Choose significance level ($\alpha$): Commonly 0.05 (5%).
3. Select test statistic: Z, t, χ², or F based on data and hypotheses.
4. Compute test statistic and p-value (probability of observing data if $H_0$ is true).
5. Decision rule:
   • Reject $H_0$ if p-value < $\alpha$ or test statistic > critical value.
   • Fail to reject $H_0$ otherwise.

Types of Errors

Trade-off: Lowering $\alpha$ reduces Type I errors but increases Type II errors (and vice versa).

Visual:

flowchart TD
    A["Start: Assume H₀ true"] --> B["Collect sample data"]
    B --> C["Calculate test statistic"]
    C --> D{"Is p-value < α?"}
    D -->|"Yes"| E["Reject H₀: Evidence against H₀"]
    D -->|"No"| F["Fail to reject H₀: Insufficient evidence"]
    E --> G["Conclude effect exists"]
    F --> G

3. Types of Hypothesis Tests

Worked Example: NEPSE Stock Performance

Scenario: An investor claims NEPSE’s average daily return is 0.5% (σ = 1.2%). Over 30 trading days, the sample mean return is 0.7%. Test at 95% confidence.

Steps:

1. Hypotheses:
   • $H_0:\mu =0.5\%$
   • $H_1:\mu \ne 0.5\%$ (two-tailed test)
2. Test Statistic (Z-test): $$Z=\frac{0.7-0.5}{1.2/\sqrt{30}}=\frac{0.2}{0.22}\approx 0.909$$
3. Critical Values: For 95% CI, $z_{\alpha /2}=\pm 1.96$.
   • Since $|0.909|<1.96$, fail to reject $H_0$.
4. p-value: $P(Z>0.909)+P(Z<-0.909)\approx 0.364$ (from Z-table).
   • p-value > 0.05 → no significant evidence to reject $H_0$.

Visual:

Interpretation: The investor’s claim lacks statistical support. The data does not prove the average return differs from 0.5%.

4. Comparing Two Populations

Independent Samples (Two-Sample t-test)

Tests if two population means are equal:

• Equal variances: Pooled variance $s_p^2=\frac{(n_1-1)s_1^2+(n_2-1)s_2^2}{n_1+n_2-2}$ $$t=\frac{{\bar{X}}_1-{\bar{X}}_2}{s_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}$$
• Unequal variances: Welch’s t-test (uses separate variances).

Worked Example: NTC vs. Ncell Call Quality Scenario: NTC claims its call drop rate is lower than Ncell’s. Samples:

• NTC: $n_1=100$, ${\bar{X}}_1=2\%$, $s_1=0.5\%$
• Ncell: $n_2=100$, ${\bar{X}}_2=3\%$, $s_2=0.6\%$

Steps:

1. Hypotheses:
   • $H_0:{\mu }_{NTC}={\mu }_{Ncell}$
   • $H_1:{\mu }_{NTC}<{\mu }_{Ncell}$ (one-tailed)
2. Pooled Variance: $$s_p^2=\frac{99\cdot (0.5{)}^2+99\cdot (0.6{)}^2}{200-2}=\frac{24.5+35.67}{198}\approx 0.297$$
3. t-statistic: $$t=\frac{2-3}{0.545\sqrt{0.01+0.01}}=\frac{-1}{0.077}\approx -13.0$$
4. Critical Value: For 95% CI, one-tailed, $t_{0.05,198}\approx 1.65$ (from t-table).
   • Since $-13.0<-1.65$, reject $H_0$.

Visual:

Interpretation: NTC’s call drop rate is significantly lower than Ncell’s at 95% confidence.

Paired Samples (Dependent t-test)

Used when data is paired (e.g., before/after measurements).

• Test statistic: $$t=\frac{\bar{D}}{s_D/\sqrt{n}}$$ where $D=X_1-X_2$ (differences).

Worked Example: Kathmandu Traffic Before/After Flyover Scenario: Traffic speeds (km/h) before and after a flyover were recorded for 20 vehicles:

Steps:

1. Hypotheses:
   • $H_0:{\mu }_D=0$ (no effect)
   • $H_1:{\mu }_D<0$ (flyover improved speed)
2. t-statistic: $$t=\frac{-4}{3/\sqrt{20}}=\frac{-4}{0.67}\approx -5.97$$
3. Critical Value: For 95% CI, one-tailed, $t_{0.05,19}\approx 1.73$.
   • Since $-5.97<-1.73$, reject $H_0$.

Visual:

5. Chi-Square (χ²) Tests

Goodness-of-Fit Test

Tests if observed frequencies match expected frequencies (e.g., die rolls, survey responses).

• Test statistic: $${\chi }^2=\sum \frac{(O_i-E_i{)}^2}{E_i}$$ where $O_i$ = observed, $E_i$ = expected.

Worked Example: Pathao Rider Preferences Scenario: Pathao surveyed 200 riders on preferred pickup times:

Steps:

1. Hypotheses:
   • $H_0$: Rider preferences follow uniform distribution.
   • $H_1$: Preferences are non-uniform.
2. Expected Frequencies: $E_i=200/5=40$ for each slot.
3. χ²-statistic: $${\chi }^2=\frac{(50-40{)}^2}{40}+\frac{(30-40{)}^2}{40}+\frac{(80-40{)}^2}{40}+\frac{(40-40{)}^2}{40}+\frac{(0-40{)}^2}{40}$$ $$=2.5+2.5+20+0+40=65$$
4. Critical Value: For 4 df (k-1) and α=0.05, ${\chi }_{0.05}^2=9.49$.
   • Since $65>9.49$, reject $H_0$.

Visual:

Interpretation: Rider preferences are not uniform; Pathao should adjust surge pricing accordingly.

Test of Independence

Tests if two categorical variables are independent (e.g., gender vs. product choice).

• Contingency table:

  	Product A	Product B	Total
  Male	40	60	100
  Female	30	70	100
  Total	70	130	200

• Test statistic same as above; degrees of freedom = $(r-1)(c-1)$.

6. F-Test: Comparing Variances

Tests if two populations have equal variances (e.g., consistency in loan processing times).

• Test statistic: $$F=\frac{s_1^2}{s_2^2}$$
• Critical values from F-distribution tables.

Worked Example: Loan Processing Times Scenario: Two banks report:

• Bank X: $n_1=50$, $s_1=10$ days
• Bank Y: $n_2=50$, $s_2=15$ days

Steps:

1. Hypotheses:
   • $H_0:{\sigma }_1^2={\sigma }_2^2$
   • $H_1:{\sigma }_1^2\ne {\sigma }_2^2$
2. F-statistic: $$F=\frac{10^2}{15^2}=\frac{100}{225}\approx 0.444$$
3. Critical Values: For 95% CI, two-tailed, $F_{0.025,49,49}\approx 1.77$ and $1/1.77\approx 0.56$.
   • Since $0.444<0.56$, fail to reject $H_0$.

Interpretation: No significant difference in processing time variability between the banks.

In the Real World

1. eSewa’s Transaction Fraud Detection

   • Idea: Uses hypothesis testing to flag unusual transaction patterns (e.g., sudden high-volume transfers).
   • How: Compares observed transaction frequencies to expected distributions (χ²-test). If p-value < 0.05, alerts fraud teams.
   • Example: A user’s transactions spike from Rs 5,000/month to Rs 50,000 in one day. The χ²-test rejects the null hypothesis of "normal usage," triggering a review.

2. Daraz’s A/B Testing for Discounts

   • Idea: Tests if a discount offer increases conversion rates using two-sample t-tests.
   • How: Randomly assigns users to control (no discount) and treatment (20% off) groups. If the mean order value in the treatment group is significantly higher (p < 0.05), the discount is rolled out.
   • Worked Example: Control group mean = Rs 1,200 ($n=1,000$, σ=Rs 300); treatment group mean = Rs 1,350 ($n=1,000$, σ=Rs 350).
     • Z-test: $$Z=\frac{1350-1200}{\sqrt{300^2/1000+350^2/1000}}\approx 10.7$$
     • p-value ≈ 0 → reject $H_0$. Daraz concludes the discount works.

3. NEPSE’s Stock Performance Monitoring

   • Idea: Uses confidence intervals to track if a stock’s return deviates from historical norms.
   • How: Monthly, NEPSE calculates a 95% CI for daily returns. If a stock’s return falls outside the CI (e.g., 0.5% ± 0.5%), analysts investigate anomalies.
   • Example: If a stock’s return is 1.2% and the CI is [0.0%, 1.0%], the upper bound is exceeded, prompting further analysis.

Exam Tip

1. Master the Steps: Always follow the 5-step hypothesis testing framework (state hypotheses → choose α → select test → compute statistic → decide). Partial credit is lost if steps are skipped.
2. Know When to Use Which Test:
   • Z-test: Large samples, σ known.
   • t-test: Small samples or σ unknown.
   • χ²-test: Categorical data (goodness-of-fit or independence).
   • F-test: Compare variances.
   • Paired t-test: Before/after or matched pairs.
3. Calculate p-values or Critical Values:
   • For Z-tests, use standard normal tables.
   • For t-tests, use t-distribution tables (df = $n-1$).
   • For χ², use χ² tables (df = categories - 1).
   • For F-tests, use F-distribution tables (df₁, df₂).
4. Interpret Rejection Regions:
   • Two-tailed: Reject if $|Z|>z_{\alpha /2}$ or $|t|>t_{\alpha /2}$.
   • One-tailed: Reject if $Z>z_{\alpha }$ or $t>t_{\alpha }$.
5. Real-World Context:
   • Always tie your answer to a business scenario (e.g., "This test helps Daraz decide whether to continue a discount campaign").
6. Show Work:
   • For calculations, write out the test statistic formula and substitute values clearly. Partial credit is given for correct setup even if the final answer is wrong.
7. Common Pitfalls:
   • Direction of Alternative Hypothesis: $H_1:\mu >{\mu }_0$ (one-tailed) vs. $H_1:\mu \ne {\mu }_0$ (two-tailed) changes critical values.
   • Degrees of Freedom: For t-tests, df = $n-1$; for χ², df = (rows-1)(cols-1).
   • Assumptions: Always state assumptions (e.g., "Data is normally distributed" for t-tests).

Final Visual Summary:

mindmap
  root((Statistical Inference & Hypothesis Testing))
    Statistical_Inference
      Point_Estimation
      Interval_Estimation
        Confidence_Intervals
          Z-Intervals
          t-Intervals
    Hypothesis_Testing
      Framework
        Hypotheses
        Test_Statistics
        Decision_Rules
      Types
        Z-Tests
        t-Tests
          Independent
          Paired
        Chi-Square
          Goodness-of-Fit
          Independence
        F-Tests
    Real-World_Applications
      eSewa_Fraud_Detection
      Daraz_A_B_Testing
      NEPSE_Stock_Monitoring
    Exam_Tips
      Steps
      Test_Selection
      p-values_vs_Critical_Values
      Assumptions