Non-Parametric Statistics & Data Analysis

TAKEAWAYS

• Non-parametric tests do not assume data follows a specific distribution (e.g., normal), making them ideal for ordinal or non-normal data.
• Rank-based methods (e.g., Mann-Whitney U, Kruskal-Wallis) compare medians instead of means, reducing sensitivity to outliers.
• Box plots visually summarize median, quartiles, and outliers—critical for detecting skewed distributions.
• Chi-square tests analyze categorical data (e.g., preferences, survey responses) to check independence or goodness-of-fit.
• Secondary data (e.g., NEPSE stock trends, NTC call records) is cheaper/faster but requires validation for biases.
• Real-world tie-ins: Pathao’s driver ratings (rank correlation), Daraz’s product reviews (chi-square), Ncell’s survey data (non-parametric trends).

1. Introduction to Non-Parametric Statistics

Non-parametric methods do not rely on population parameters (e.g., mean, variance) or strict distributional assumptions. They are robust for:

• Ordinal data (e.g., survey rankings: "Poor/Medium/Good").
• Non-normal data (e.g., skewed income distributions).
• Small sample sizes where parametric tests (e.g., t-tests) fail.

Key Advantages

flowchart TD
  A["Non-parametric tests"] --> B["No assumption of **normality** (works with skewed data)"]
  A --> C["Handles **ordinal/categorical** data (e.g., Likert scales)"]
  A --> D["Robust to **outliers** (unlike t-tests)"]
  A --> E["Flexible for **small samples** or **non-normal distributions**"]

When to Use?

2. Rank-Based Tests

(A) Mann-Whitney U Test

Purpose: Compare medians of two independent groups (non-parametric alternative to independent t-test). Example: Comparing Pathao driver ratings (1–5 stars) between two cities.

Worked Example: Data: Ratings from Lalitpur (n=8) and Pokhara (n=7).

Lalitpur: 4, 5, 3, 2, 5, 4, 1, 3
Pokhara:  5, 4, 2, 3, 1, 4, 2

Steps:

1. Rank all 15 values (ties get average rank):

   Ranked data: 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 5
   

2. Sum ranks for each group:
   • Lalitpur: $1+1+3+2+5+4+3+3=22$
   • Pokhara: $5+4+2+3+1+4+2=21$
3. Calculate U: $$U=n_1{n}_2+\frac{n_1(n_1+1)}{2}-R_1$$ Where $n_1=8$, $n_2=7$, $R_1=22$: $$U=(8\times 7)+\frac{8\times 9}{2}-22=56+36-22=70$$
4. Compare to critical U (from tables) at α=0.05 → Reject H₀ if U < 20 (one-tailed).

Visual:

(B) Kruskal-Wallis Test

Purpose: Extend Mann-Whitney to ≥3 independent groups (non-parametric ANOVA). Example: Comparing Ncell, NTC, Smartphone call drop rates across 3 cities.

3. Correlation and Rank Correlation

(A) Spearman’s Rank Correlation (ρ)

Purpose: Measure monotonic relationship between two ordinal/ranked variables (non-parametric alternative to Pearson’s r). Formula: $$\rho =1-\frac{6\sum d_i^2}{n(n^2-1)}$$ Where $d_i={\text{rank}}_X-{\text{rank}}_Y$.

Worked Example: Data: 10 students’ preferences for DELL vs HP (ranks 1–10).

Student | DELL Rank | HP Rank | \(d_i\) | \(d_i^2\)
--------|-----------|---------|---------|--------
1      | 5         | 10      | -5      | 25
2      | 2         | 9       | -7      | 49
...    | ...       | ...     | ...     | ...
10     | 10        | 5       | +5      | 25

Steps:

1. Compute $\sum d_i^2=25+49+\cdots +25=300$.
2. Plug into formula: $$\rho =1-\frac{6\times 300}{10\times 99}=1-1.818=-0.818$$
3. Interpret: Strong negative rank correlation (ρ ≈ -0.82).

Visual:

4. Chi-Square Tests

(A) Chi-Square Goodness-of-Fit

Purpose: Test if observed frequencies match expected frequencies (e.g., Daraz product categories). Test Statistic: $${\chi }^2=\sum \frac{(O-E{)}^2}{E}$$ Example: Do 100 Daraz shoppers prefer categories equally?

Category   | Observed (O) | Expected (E=25)
-----------|--------------|-----------------
Electronics| 40           | 25
Clothing    | 30           | 25
Books       | 20           | 25
Others      | 10           | 25

Steps:

1. Compute $E=\frac{\text{Total}}{k}=\frac{100}{4}=25$.
2. Calculate ${\chi }^2$: $${\chi }^2=\frac{(40-25{)}^2}{25}+\frac{(30-25{)}^2}{25}+\frac{(20-25{)}^2}{25}+\frac{(10-25{)}^2}{25}=25+2+2+25=54$$
3. Compare to critical χ² (df=3, α=0.05) → Reject H₀ (χ²₀.₀₅₃=7.81 < 54).

Visual:

(B) Chi-Square Test of Independence

Purpose: Check if two categorical variables are independent (e.g., Ncell plan choice vs gender). Example: Do men and women choose Ncell plans equally?

          | Plan A | Plan B | Total
----------|--------|--------|-------
Male      | 30     | 20     | 50
Female    | 25     | 25     | 50
Total     | 55     | 45     | 100

Steps:

1. Compute expected counts (e.g., for Male/Plan A: $E=\frac{50\times 55}{100}=27.5$).
2. Calculate ${\chi }^2$: $${\chi }^2=\frac{(30-27.5{)}^2}{27.5}+\frac{(20-22.5{)}^2}{22.5}+\cdots =0.33+0.5+\cdots =2.67$$
3. Compare to critical χ² (df=1, α=0.05) → Fail to reject H₀ (χ²₀.₀₅₁=3.84 > 2.67).

5. Box-and-Whisker Plots

Purpose: Visualize median, quartiles, and outliers in skewed data. Components:

graph LR
  A["Whisker (Q1 - 1.5*IQR)"] --> B["Q1"]
  B --> C["Box (Q1 to Q3)"]
  C --> D["Median (line)"]
  C --> E["Q3"]
  E --> F["Whisker (Q3 + 1.5*IQR)"]

Worked Example: Data: Marks of Student A (10 tests).

44, 80, 76, 48, 52, 72, 68, 56, 60, 54

Steps:

1. Order data: 44, 48, 52, 54, 56, 60, 68, 72, 76, 80.
2. Find Q1, Q3, IQR:
   • Q1 (median of first 5): 52
   • Q3 (median of last 5): 72
   • IQR = 72 − 52 = 20
3. Outliers: Lower bound = 52 − 1.5×20 = 22 (none below 44). Upper bound = 72 + 30 = 102 (none above 80).
4. Plot:

6. Secondary Data Analysis

Definition: Data collected by others (e.g., NEPSE stock prices, NTC call records). Sources in Nepal:

Worked Example: Data: NEPSE’s top 5 stocks (2023) with monthly returns.

Stock   | Jan | Feb | Mar | Mean | Std Dev
--------|-----|-----|-----|------|--------
ABC     | 2% | 5% | -1%| 2.33%| 2.52%
XYZ     | 3% | 0% | 4% | 2.67%| 2.08%

Analysis:

• Median return: (2% + 3% + 4%)/3 = 3% (better than mean for skewed data).
• Box plot would show XYZ’s lower variability (smaller whiskers).

In the Real World

1. Pathao’s Driver Ratings:

   • Uses Spearman’s rank correlation to compare driver performance across cities.
   • Example: If Lalitpur drivers rank higher in "Punctuality" (ρ=0.75) than Pokhara (ρ=0.45), Pathao may train Pokhara drivers.

2. Daraz’s Product Categories:

   • Chi-square test checks if shoppers’ preferences match Daraz’s stock distribution.
   • Example: If observed Electronics sales (40%) > expected (25%), Daraz may increase inventory.

3. Ncell’s Survey Data:

   • Kruskal-Wallis test compares call drop rates by plan type (Prepaid vs Postpaid) across regions.
   • Example: If Postpaid users in Kathmandu have significantly higher drops (U=12 < critical U), Ncell may upgrade infrastructure.

Exam Tip

• Focus on 3 key tests:
  1. Mann-Whitney U (2 groups) → Compare medians.
  2. Spearman’s ρ (rank correlation) → Monotonic trends.
  3. Chi-square (categorical data) → Independence/goodness-of-fit.
• Always show calculations for χ² and U tests (partial marks for steps).
• Box plots are high-weight in exams—practice labeling Q1, Q3, outliers.
• Secondary data: Cite real Nepalese sources (NEPSE, NTC) and discuss bias risks.
• Rank correlation vs Pearson: Use Spearman for ordinal data, Pearson for interval/ratio.

Key Formulae to Memorize:

1. Spearman’s ρ: $$\rho =1-\frac{6\sum d_i^2}{n(n^2-1)}$$
2. Mann-Whitney U: $$U=n_1{n}_2+\frac{n_1(n_1+1)}{2}-R_1$$
3. Chi-square: $${\chi }^2=\sum \frac{(O-E{)}^2}{E}$$