Non-parametric Tests & Hypothesis Testing: Methods, Assumptions & Applications

Core Concepts

1. Non-parametric Tests: Overview and When to Use

Non-parametric tests are distribution-free statistical methods used when:

• Data violates parametric test assumptions (e.g., normality, homogeneity of variance).
• Data is ordinal or nominal (not continuous).
• Sample size is small (<30).

Key Differences: Parametric vs. Non-parametric

Common Non-parametric Tests

---

2. Hypothesis Testing Framework

Steps for Any Hypothesis Test

1. State Hypotheses:
   • Null Hypothesis (H₀): Default assumption (e.g., "no effect").
   • Alternative Hypothesis (H₁/Hₐ): Research claim (e.g., "there is a difference").
2. Choose Significance Level (α): Typically 0.05 or 0.01.
3. Select Test Statistic: Depends on data type (e.g., U-statistic for Mann-Whitney).
4. Calculate Test Statistic: From sample data.
5. Determine Critical Value/ p-value:
   • Compare test statistic to critical value (from tables).
   • Or compare p-value to α.
6. Make Decision:
   • Reject H₀ if p ≤ α (significant result).
   • Fail to reject H₀ otherwise.

Example: Mann-Whitney U Test

Scenario: Compare IT knowledge scores (ordinal data) between two groups of workers. Data:

• Group A (n₁=5): 5, 8, 2, 7, 6
• Group B (n₂=4): 9, 12, 4, 6

Steps:

1. Rank all data together (ties get average rank):

   Value | Group | Rank
   ---------------------
   2     | A     | 1
   4     | B     | 2
   5     | A     | 3
   6     | A/B   | 4.5 (tie)
   6     | A/B   | 4.5
   7     | A     | 6
   8     | A     | 7
   9     | B     | 8
   12    | B     | 9
   

2. Sum ranks for each group:
   • R₁ (Group A) = 1 + 3 + 4.5 + 6 + 7 = 21.5
   • R₂ (Group B) = 2 + 4.5 + 8 + 9 = 23.5
3. Calculate U-statistic:
   • $U_1=n_1{n}_2+\frac{n_1(n_1+1)}{2}-R_1=5\ast 4+15-21.5=8.5$
   • $U_2=n_1{n}_2-U_1=20-8.5=11.5$
   • Use $U=\min (U_1,U_2)=8.5$.
4. Compare to critical value:
   • For α=0.05, two-tailed, $U_{\text{critical}}=2$ (from Mann-Whitney table for n₁=5, n₂=4).
   • Since 8.5 > 2, fail to reject H₀ (no significant difference).

---

3. Chi-Square Tests for Categorical Data

Chi-Square Goodness-of-Fit Test

Tests if observed frequencies match expected frequencies. Example: Test if email hacking status is independent of email type. Data:

Steps:

1. State H₀: Hacking status is independent of email type.
2. Calculate expected frequencies:
   • $E_{11}=\frac{90\times 100}{200}=45$ (Yahoo/Yes).
3. Compute χ²: $${\chi }^2=\sum \frac{(O-E{)}^2}{E}=\frac{(60-45{)}^2}{45}+\frac{(40-55{)}^2}{55}+\frac{(30-45{)}^2}{45}+\frac{(70-55{)}^2}{55}=13.33$$
4. Critical value: For df = (rows-1)(cols-1) = 1, α=0.05 → χ²₀.₀₅ = 3.841.
   • Since 13.33 > 3.841, reject H₀ (dependence exists).

Chi-Square Test of Independence

Same as above but for association between two categorical variables.

---

4. Applications in CS/IT

---

5. Advantages and Limitations

Advantages of Non-parametric Tests

• Robustness: Work with non-normal data.
• Flexibility: Handle ordinal/nominal data.
• Simplicity: Fewer assumptions than parametric tests.

Limitations

• Lower power: Harder to detect true effects.
• Less precise: Provide ranks, not exact values.
• Tie sensitivity: Affects Mann-Whitney/Kruskal-Wallis results.

When to Use Which Test

---

6. Worked Example: Kruskal-Wallis Test

Scenario: A manufacturer tests 3 catalysts (C1, C2, C3) for alcohol distillation. Data (cc):

• C1: 380, 430, 410
• C2: 290, 350, 270, 250, 270
• C3: 400, 380, 450

Steps:

1. Rank all data (n=11):

   Value | Group | Rank
   ------------------------
   250   | C2    | 1
   270   | C2    | 2.5 (tie)
   270   | C2    | 2.5
   290   | C2    | 4
   350   | C2    | 5
   380   | C1    | 6.5
   380   | C3    | 6.5
   400   | C3    | 8
   410   | C1    | 9
   430   | C1    | 10
   450   | C3    | 11
   

2. Sum ranks per group:
   • R₁ (C1) = 6.5 + 9 + 10 = 25.5
   • R₂ (C2) = 1 + 2.5 + 2.5 + 4 + 5 = 15
   • R₃ (C3) = 6.5 + 8 + 11 = 25.5
3. Calculate H-statistic: $$H=\frac{12}{N(N+1)}\sum \frac{R_i^2}{n_i}-3(N+1)$$ Where $N=11$, $n_i$ = group sizes (3, 5, 3). $$H=\frac{12}{11\times 12}\left(\frac{25.5^2}{3}+\frac{15^2}{5}+\frac{25.5^2}{3}\right)-33=7.05$$
4. Critical value: For df = 3-1 = 2, α=0.05 → χ²₀.₀₅ = 5.991.
   • Since 7.05 > 5.991, reject H₀ (significant differences exist).

---

Exam Tip

What Examiners Look For

1. Correct Test Selection:

   • Always justify why a non-parametric test is appropriate (e.g., "data is ordinal" or "normality violated").
   • Common mistakes: Using t-test for non-normal data or Mann-Whitney for paired samples.

2. Hypothesis Formulation:

   • Clearly state H₀ and H₁ (e.g., "H₀: μ₁ = μ₂ vs. H₁: μ₁ ≠ μ₂").
   • Specify directionality (one-tailed vs. two-tailed).

3. Step-by-Step Calculations:

   • Show ranking, U-statistic, or χ² formula explicitly.
   • For Mann-Whitney: Always calculate both U₁ and U₂.
   • For Kruskal-Wallis: Verify degrees of freedom (k-1).

4. Interpretation:

   • Link p-values to α (e.g., "p = 0.03 < 0.05 → reject H₀").
   • State conclusions in context (e.g., "Catalyst C1 performs significantly better than C2/C3").

5. Assumptions Check:

   • For non-parametric tests, confirm:
     • Data is independent.
     • No extreme outliers (affects ranks).
   • For chi-square: Expected frequencies ≥5 in most cells (use Fisher’s exact test if violated).

Common Pitfalls

• Ignoring ties: In Mann-Whitney/Kruskal-Wallis, ties must be averaged.
• Wrong degrees of freedom: For Kruskal-Wallis, df = k-1 (not N-1).
• Misapplying tests:
  • Wilcoxon for independent samples (use Mann-Whitney instead).
  • Chi-square for ordinal data (use Spearman’s rho).
• Overlooking multiple testing: If running multiple tests, adjust α (e.g., Bonferroni correction).

Quick Reference Table for Exams

Sample Exam Question Breakdown

Question: A survey of 200 students classified by class and opinion on a core curriculum change yielded:

Expected Answer Structure:

1. State H₀/H₁:
   • H₀: Opinion is independent of class.
   • H₁: Opinion depends on class.
2. Calculate expected frequencies (e.g., for Freshman/Favoring: $E=\frac{235\times 205}{615}$).
3. Compute χ²: $${\chi }^2=\sum \frac{(O-E{)}^2}{E}$$
4. Critical value: df = (3-1)(2-1) = 2 → χ²₀.₀₅ = 5.991.
5. Decision: If χ² > 5.991, reject H₀.
6. Conclusion: "There is significant association between class and opinion on the curriculum change (p < 0.05)."

---

Note Length: ~1,800 words Key Coverage: Definitions, test selection, step-by-step procedures, comparisons, applications, and exam-specific tips. All subtopics from the syllabus are addressed with practical examples.