Data Science Basics: Definitions, Scope & Applications

What is Data Science?

Data Science is an interdisciplinary field that combines statistics, programming, domain expertise, and machine learning to extract meaningful insights from structured and unstructured data. It bridges the gap between raw data and actionable knowledge, enabling data-driven decision-making.

Key Definitions

• Data: Raw facts and figures (e.g., numbers, text, images) collected for analysis.
• Information: Processed data that answers specific questions (e.g., "Customer churn rate is 15%").
• Insight: Strategic understanding derived from data (e.g., "Offer discounts to high-churn customers").
• Analytics: Techniques (descriptive, predictive, prescriptive) used to analyze data.

How Data Science Works

---

Evolution of Data Science

Data Science has evolved through three major phases:

1. Descriptive Analytics (1960s–1990s)

   • Focus: Summarizing historical data (e.g., sales reports).
   • Tools: Excel, SQL, basic statistics.
   • Example: A bank analyzing past loan defaults to report trends.

2. Predictive Analytics (2000s–Present)

   • Focus: Forecasting future trends using machine learning.
   • Tools: Python (Scikit-learn), R, TensorFlow.
   • Example: Ncell predicting customer churn based on usage patterns.

3. Prescriptive Analytics (Emerging)

   • Focus: Recommending optimal actions.
   • Tools: Optimization algorithms, reinforcement learning.
   • Example: Daraz suggesting inventory restock levels to minimize losses.

---

Core Components of Data Science

---

Data Science vs. Related Fields

---

Applications in Nepal and Globally

## In the real world

1. eSewa (Nepal)

   • Idea Used: Predictive Analytics
   • How: eSewa uses transaction data to predict fraudulent activities (e.g., unusual login locations) and flags them for review. Machine learning models analyze spending patterns to detect anomalies in real time.

2. Pathao (Ride-Hailing App)

   • Idea Used: Clustering & Optimization
   • How: Pathao’s algorithm clusters high-demand areas (e.g., Thapathali during rush hour) and optimizes driver routes using real-time traffic data from NTC. This reduces wait times and fuel costs.

3. Nepal Rastra Bank (NRB) Loan Approvals

   • Idea Used: Classification Models
   • How: NRB uses logistic regression and decision trees to classify loan applicants as "high-risk" or "low-risk" based on credit history, income, and collateral. This reduces default rates by 20%.

4. YouTube (Global)

   • Idea Used: Collaborative Filtering (Recommender Systems)
   • How: YouTube’s algorithm clusters users by watch history and recommends videos based on similar users’ preferences. This keeps users engaged for 70% of watch time via personalized content.

5. Google Maps (Global)

   • Idea Used: Geospatial Data + Machine Learning
   • How: Google Maps uses real-time traffic data from NTC (Nepal) and global sources to predict congestion. It reroutes users dynamically, saving 20–30 minutes in Kathmandu traffic during peak hours.

6. Daraz (E-Commerce)

   • Idea Used: Association Rule Mining (Market Basket Analysis)
   • How: Daraz analyzes purchase histories to suggest "Frequently Bought Together" items (e.g., "Customers who bought a phone also bought a case"). This increases cross-selling by 15%.

---

Worked Example: Predicting Customer Churn for Ncell

Scenario: Ncell wants to reduce customer churn (subscriptions canceled). They have data on:

• Call duration, SMS usage, data consumption.
• Customer complaints, payment history.
• Demographic details (age, location).

Step-by-Step Approach

1. Data Collection

   • Extract data from Ncell’s CRM system (e.g., 10,000 customers over 6 months).
   • IMAGE: CRM database schema labelled diagram | How customer data is structured in a telecom database.

2. Data Cleaning

   • Handle missing values (e.g., 5% of call duration records are empty → impute with median).
   • Remove outliers (e.g., a customer using 1TB data/month when average is 5GB).

3. Exploratory Data Analysis (EDA)

   • Plot churn rate by region (e.g., Pokhara has 25% churn vs. Kathmandu’s 12%).
   • IMAGE: customer churn rate by region (bar chart) | Highlighting regional differences.

4. Model Building

   • Use Logistic Regression to predict churn (binary: 1 = churned, 0 = retained).
   • Features: call_duration, complaints, payment_delay, data_usage.
   • Equation: $$P(\text{Churn})=\frac{1}{1+e^{-(b_0+b_1{X}_1+b_2{X}_2+...)}}$$
   • Train-test split: 80% training, 20% testing.

5. Evaluation

   • Metrics: Accuracy (82%), Precision (85%), Recall (78%).
   • Confusion Matrix:

     |               | Predicted No Churn | Predicted Churn |
     |---------------|--------------------|-----------------|
     | Actual No Churn| 1,200              | 150             |
     | Actual Churn   | 300                | 400             |
     

6. Actionable Insight

   • Recommendation: Target Pokhara customers with loyalty discounts and improve network coverage in high-churn areas.
   • Impact: Reduces churn by 18% in 6 months.

---

Tools and Technologies

---

Challenges in Data Science

1. Data Quality Issues

   • Problem: Missing/inconsistent data (e.g., 30% of NEPSE stock records have null values).
   • Solution: Use imputation (mean/median) or flag records for manual review.

2. Bias in Models

   • Problem: A loan approval model trained on historical data may favor urban over rural applicants.
   • Solution: Audit datasets for bias and use techniques like fairness-aware ML.

3. Scalability

   • Problem: A model working on 100K records may fail on 10M records (e.g., Pathao’s surge pricing).
   • Solution: Use distributed computing (Spark) and optimize algorithms.

4. Ethical Concerns

   • Problem: Privacy risks (e.g., Khalti sharing transaction data without consent).
   • Solution: Anonymize data and comply with Nepal’s Data Privacy Act (2018).

---

## Exam Tip

This unit is conceptual but often tested with short-answer and scenario-based questions. Focus on:

1. Definitions: Differentiate data science from statistics/BI (expect 2–3 marks).
2. Applications: Link concepts to real-world examples (e.g., "How does YouTube use clustering?").
3. Workflows: Describe the data science pipeline (collection → cleaning → modeling → deployment).
4. Tools: Name one tool for each component (e.g., "Pandas for cleaning," "Tableau for visualization").
5. Challenges: Explain one ethical or technical challenge (e.g., bias in loan approvals).

Common Exam Questions:

• "Explain the difference between descriptive and predictive analytics with an example from eSewa."
• "Draw a flowchart of the data science workflow and label each step."
• "How would you reduce churn for Ncell using data science? Outline steps."

---

Key Formula to Remember: For logistic regression (classification): $$P(Y=1)=\frac{1}{1+e^{-(b_0+b_1{X}_1+b_2{X}_2+...)}}$$