Regression Modelling: Types, Assumptions, and Applications

TAKEAWAYS:

• Regression models predict outcomes by fitting a line (linear) or curve (nonlinear) to data, with R² measuring fit quality.
• Linear regression assumes linearity, independence, homoscedasticity, and normality; violations require transformations or robust methods.
• Logistic regression models binary outcomes (e.g., "buy/no-buy") using probabilities via the logit link function.
• Multicollinearity (correlated predictors) inflates coefficient variance; use VIF or stepwise selection to mitigate it.
• Residual analysis (plots of residuals vs. fitted values) detects heteroscedasticity, outliers, or nonlinearity.
• Businesses use regression for pricing (Daraz), risk (banks), and demand forecasting (NTC).

What is Regression Modeling?

Regression modeling predicts a dependent variable (Y) using one or more independent variables (X₁, X₂, ...). It answers:

• "How does advertising spend (X) affect sales (Y)?"
• "What factors influence loan defaults?"

Key Types

1. Linear Regression

   • Models a linear relationship: $Y={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\cdots +ϵ$
   • Simple linear: 1 predictor (e.g., house price vs. size).
   • Multiple linear: ≥2 predictors (e.g., salary vs. experience + education).

2. Logistic Regression

   • Models binary outcomes (e.g., "churn/no-churn") using probabilities: $P(Y=1)=\frac{1}{1+e^{-({\beta }_0+{\beta }_{1}X)}}$
   • Outputs odds ratios (e.g., "A 1% increase in ad spend doubles churn odds").

3. Nonlinear Regression

   • Fits curves (e.g., polynomial, exponential) when relationships are nonlinear.
   • Example: Gompertz growth model for population trends.

How Linear Regression Works: Step-by-Step

1. Model Assumptions

Regression relies on 4 critical assumptions (violations require fixes):

2. Estimating Coefficients

Use Ordinary Least Squares (OLS) to minimize the sum of squared residuals (SSR): $$\text{SSR}=\sum _{i=1}^n(Y_i-{\hat{Y}}_i{)}^2$$ Example: Predicting monthly sales (Y) of a Daraz seller based on ad spend (X₁) and seasonality (X₂).

Model: $$\hat{Y}=100+0.2X_1+150X_2$$

• Interpretation:
  • A ₹1,000 increase in ad spend raises sales by ₹200.
  • A 0.1 increase in seasonality (e.g., festival month) raises sales by ₹15.

Visualizing Regression

1. Residual Plots (Diagnostics)

graph LR
    A["Scatter Plot: Y vs. X"] --> B["Residual Plot: Residuals vs. Fitted"]
    B --> C["Normal Q-Q Plot"]
    B --> D["Homoscedasticity Check"]

• Good fit: Residuals randomly scatter around 0.
• Bad fit: Patterns (curves, funnels) → nonlinearity or heteroscedasticity.

2. Real-World Example: NTC’s Revenue Prediction

Problem: NTC wants to predict monthly revenue (Y) using:

• Number of subscribers (X₁)
• Average call duration (X₂)

Data:

Model: $$\hat{Y}=200M+100X_1+5000X_2$$

• Prediction for March: 5.5M subscribers, 140 sec duration → $\hat{Y}=200M+100(5.5M)+5000(140)=\ast \ast ₹1,020M\ast \ast$.

Logistic Regression: Binary Outcomes

How It Works

• Outputs probabilities (0 to 1) using the logit link: $$\text{logit}(P)=\ln \left(\frac{P}{1-P}\right)={\beta }_0+{\beta }_{1}X$$
• Example: Predicting if a Khalti user will default on a loan based on:
  • Income (X₁)
  • Loan amount (X₂)

Model: $$P(\text{Default})=\frac{1}{1+e^{-(-5+0.001X_1-0.005X_2)}}$$

• Interpretation:
  • A ₹10,000 increase in income reduces default odds by 5% (since $e^{-0.001\times 10000}\approx 0.95$).
  • A ₹100,000 loan increases default odds by 30% (since $e^{-0.005\times 100000}\approx 0.7$).

Common Pitfalls and Fixes

## In the Real World

1. Daraz’s Demand Forecasting

   • Uses multiple linear regression to predict product demand based on:
     • Price (X₁)
     • Seasonality (X₂)
     • Competitor promotions (X₃)
   • Example: If Daraz lowers the price of a laptop by ₹5,000, sales increase by 12% (coefficient ${\beta }_1=0.012$).

2. Ncell’s Churn Prediction

   • Uses logistic regression to identify customers likely to cancel:
     • Call duration (X₁)
     • Data usage (X₂)
     • Customer service complaints (X₃)
   • Example: A user with <100 mins/month** and **>3 complaints has a 70% chance of churning.

3. Nepal Rastra Bank’s Loan Risk

   • Banks use probit/logit models to assess loan defaults:
     • Borrower income (X₁)
     • Loan-to-value ratio (X₂)
     • Credit history (X₃)
   • Example: A borrower with a 60% LTV and poor credit has a 40% default probability.

## Exam Tip

1. Always check assumptions before interpreting results. Examiners test this!

   • Plot residuals vs. fitted values.
   • Use Durbin-Watson for autocorrelation.

2. Interpret coefficients correctly:

   • Linear: "A 1-unit increase in X → Y changes by $\beta$."
   • Logistic: "A 1-unit increase in X → odds of Y change by $e^{\beta }$."

3. Compare models using:

   • R² (higher = better fit, but can overfit).
   • Adjusted R² (penalizes extra predictors).
   • AIC/BIC (for nonlinear models).

4. Real-world applications are gold:

   • Link regression to business decisions (e.g., "NTC should increase rates by 5% to boost revenue by ₹200M").
   • Use Nepali examples (Ncell, Daraz, banks) to stand out.

Example: House price vs. size in Kathmandu (Image: Thomas.haslwanter, CC BY-SA 3.0, via Wikimedia Commons) Example: Khalti loan default probability (Image: Qef (talk), Public domain, via Wikimedia Commons)

graph TD
    A["Collect Data"] --> B["Check Assumptions"]
    B --> C["Fit Model (OLS/Logit)"]
    C --> D["Diagnose Residuals"]
    D -->|"Good"| E["Interpret Coefficients"]
    D -->|"Bad"| F["Transform Data or Use Robust Methods"]
    E --> G["Make Predictions"]

Example: Daraz sales model output (Image: Jtneill, Public domain, via Wikimedia Commons)