Recommendation Systems: Models, Algorithms & Applications

TAKEAWAYS:

• Recommendation systems predict user preferences using collaborative filtering (user-item interactions) or content-based filtering (item features).
• Matrix factorization and deep learning improve accuracy by uncovering latent patterns in sparse data.
• Hybrid systems combine multiple approaches to balance accuracy and scalability.
• Evaluation metrics like precision@k, recall@k, and RMSE measure performance.
• Real-world applications include eSewa’s transaction suggestions, Daraz’s product recommendations, and Ncell’s personalized offers.

What Are Recommendation Systems?

Recommendation systems are algorithms that predict user preferences for items (e.g., products, videos, or news) based on past behavior or item features. They are widely used in e-commerce, streaming services, and social media to personalize user experiences and increase engagement.

Why Are They Important?

• Increase revenue: Suggest relevant products (e.g., Daraz, Amazon).
• Improve user satisfaction: Reduce decision fatigue (e.g., YouTube’s "Recommended for You").
• Enhance discovery: Introduce users to new but relevant items (e.g., Spotify’s "Discover Weekly").

Types of Recommendation Systems

Recommendation systems can be broadly classified into three categories:

1. Collaborative Filtering (CF)

Uses user-item interactions (e.g., ratings, clicks, purchases) to predict preferences.

• User-User CF: Recommends items liked by similar users.
• Item-Item CF: Recommends items similar to those a user already liked.

classDiagram
    class User {
        +UserID
        +PastInteractions
    }
    class Item {
        +ItemID
        +Features
    }
    class CollaborativeFiltering {
        +PredictRating(User, Item)
        +FindSimilarUsers(User)
        +FindSimilarItems(Item)
    }
    User "1" --> "many" CollaborativeFiltering : uses
    Item "1" --> "many" CollaborativeFiltering : uses

Example:

• eSewa recommends financial services (e.g., bill payments) based on users who frequently pay electricity or phone bills.
• Ncell’s "My Offers" suggests plans based on data usage patterns of similar customers.

Advantages: ✔ No need for item features (works even with unstructured data). ✔ Captures complex user preferences.

Disadvantages: ✖ Cold-start problem: New users/items have no interaction history. ✖ Scalability issues: Computationally expensive for large datasets.

2. Content-Based Filtering (CBF)

Recommends items similar to those a user liked in the past, using item features (e.g., keywords, categories).

• Works well for cold-start scenarios (new users/items).
• Relies on text mining, NLP, or metadata.

classDiagram
    class UserProfile {
        +UserID
        +PreferredFeatures
    }
    class ItemFeatures {
        +ItemID
        +Attributes (e.g., genre, keywords)
    }
    class ContentBasedFiltering {
        +RecommendItems(UserProfile)
        +ComputeSimilarity(ItemFeatures)
    }
    UserProfile "1" --> "1" ContentBasedFiltering : uses
    ItemFeatures "many" --> "1" ContentBasedFiltering : uses

Example:

• Daraz recommends products based on category preferences (e.g., if you buy phone accessories, it suggests earphones).
• YouTube recommends videos based on watch history and tags.

Advantages: ✔ Works for new users/items. ✔ Explainable (users understand why an item is recommended).

Disadvantages: ✖ Over-specialization: Only recommends similar items (no serendipitous discoveries). ✖ Requires high-quality item features.

3. Hybrid Recommendation Systems

Combine collaborative and content-based filtering to leverage strengths of both.

• Weighted Hybrid: Combines predictions from CF and CBF (e.g., 70% CF + 30% CBF).
• Feature Combination: Merges user and item features before applying CF.
• Cascade Hybrid: Uses CBF first, then refines with CF.

Example:

• Netflix uses a hybrid approach: content-based (genre preferences) + collaborative (ratings from similar users).
• Amazon combines purchase history (CF) with product descriptions (CBF).

Advantages: ✔ Balances accuracy and diversity. ✔ Mitigates cold-start and over-specialization issues.

Disadvantages: ✖ More complex to implement. ✖ Requires careful tuning of weights.

Advanced Techniques in Recommendation Systems

1. Matrix Factorization (MF)

• Problem: User-item interaction matrices are sparse (most users rate few items).
• Solution: Decompose the matrix into latent factors (hidden features) using Singular Value Decomposition (SVD) or Alternating Least Squares (ALS).

graph LR
    A["User-Item Matrix"] -->|"Decompose"| B["User Factors"]
    A -->|"Decompose"| C["Item Factors"]
    B -->|"Multiply"| D["Predicted Ratings"]
    C -->|"Multiply"| D

Example:

• Spotify’s "Discover Weekly" uses MF to predict music preferences based on latent factors like "energy level" or "tempo."

Advantages: ✔ Handles sparsity better than traditional CF. ✔ Captures latent patterns (e.g., users who like action movies also like sci-fi).

Disadvantages: ✖ Requires matrix operations, which can be slow for large datasets.

2. Deep Learning for Recommendations

• Uses neural networks (e.g., Autoencoders, DeepFM, Neural Collaborative Filtering) to learn non-linear relationships.
• Example Architectures:
  • Wide & Deep Learning (Google): Combines memorization (CF) and generalization (deep learning).
  • YouTube’s DeepFM: Uses feature crossing to capture complex interactions.

Example:

• Facebook’s "People You May Know" uses graph neural networks (GNNs) to recommend friends based on social connections.
• TikTok’s "For You Page" uses reinforcement learning to personalize video recommendations.

Advantages: ✔ Captures complex patterns in data. ✔ Works well with multi-modal data (text, images, videos).

Disadvantages: ✖ High computational cost. ✖ Requires large datasets for training.

3. Reinforcement Learning (RL) for Recommendations

• Goal: Maximize long-term user engagement (not just short-term clicks).
• How it works:
  • The system learns an optimal policy (e.g., what to recommend next) by rewarding user actions (e.g., watch time, purchases).
  • Used in sequential recommendation (e.g., YouTube, Netflix).

Example:

• YouTube’s recommendation system uses RL to maximize watch time, not just clicks.
• Ncell’s "Smart Bundle" adjusts data offers based on real-time usage patterns.

Advantages: ✔ Optimizes for long-term user satisfaction. ✔ Adapts to changing user preferences.

Disadvantages: ✖ Exploration vs. exploitation trade-off (balancing new vs. known recommendations). ✖ Requires careful reward modeling.

Evaluation Metrics for Recommendation Systems

To measure performance, we use:

Example:

• Daraz might evaluate its system using Precision@10 to ensure the top 10 recommendations are highly relevant.
• Ncell could use RMSE to check how accurately it predicts data usage.

Real-World Applications in Nepal

1. eSewa: Transaction Recommendations

• Technique: Collaborative Filtering + Hybrid Approach
• How it works:
  • If User A frequently pays electricity bills, eSewa recommends phone bills (since many users who pay electricity also pay phone bills).
  • Uses content-based filtering for new users (e.g., "You might need to pay your internet bill").
• Visual:

  sequenceDiagram
      User->>eSewa: Pays Electricity Bill
      eSewa->>Database: Log Interaction
      Database->>CF: Find Similar Users
      CF->>eSewa: "Recommend Phone Bill"
      eSewa->>User: Show Suggestion

2. Daraz: Product Recommendations

• Technique: Content-Based + Collaborative Filtering
• How it works:
  • If you buy a smartphone, Daraz recommends cases, screen guards, and chargers (content-based).
  • If many users who bought iPhones also bought AirPods, it recommends AirPods (collaborative).
• Example Trace:
  • Input: User buys Samsung Galaxy S23.
  • CF Step: Finds users who bought S23 also bought Samsung earbuds.
  • CBF Step: Checks product description (S23 is a flagship phone → recommends high-end accessories).
  • Output: Recommends Samsung earbuds, tempered glass screen protector, and a fast charger.

3. Ncell: Personalized Offers

• Technique: Matrix Factorization + Reinforcement Learning
• How it works:
  • MF: Predicts which users will upgrade to 4G/5G based on past data usage.
  • RL: Adjusts recommendations in real-time (e.g., if a user frequently uses data at night, it promotes night bundles).
• Example:
  • User Profile: High data usage (30GB/month), mostly in evenings.
  • Recommendation: "Evening Unlimited" data plan (instead of a general "Unlimited" offer).

4. Pathao: Ride & Delivery Recommendations

• Technique: Collaborative Filtering + Location-Based Filtering
• How it works:
  • If many users in Thapathali order pizza at night, Pathao recommends pizza delivery to new users in that area.
  • Uses real-time demand-supply matching (like Uber’s surge pricing but for recommendations).

Worked Example: Rocchio Algorithm for Recommendation (From Past Exam)

Problem: Given two classes:

• Class 1 (Operating System): Documents = {"process scheduling", "memory management", "CPU scheduling"}
• Class 2 (Automata): Documents = {"finite automata", "regular expressions", "pushdown automata"}

Task: Classify the document "process scheduling" using the Rocchio algorithm (a text classification technique).

Step 1: Represent Documents as Vectors

Assume TF-IDF vectors (term frequency-inverse document frequency) for each document.

Step 2: Compute Class Centroids

• Centroid for OS (Class 1): $$C_{OS}=\frac{1}{3}\left(\left[\begin{array}{c}1\\ 1\\ 0\\ 0\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ 1\\ 1\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 1\\ 0\\ 0\\ 1\end{array}\right]\right)=\left[\begin{array}{c}1/3\\ 2/3\\ 1/3\\ 1/3\\ 1/3\end{array}\right]$$
• Centroid for Automata (Class 2): $$C_{Auto}=\frac{1}{3}\left(\left[\begin{array}{c}0\\ 0\\ 0\\ 0\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ 0\\ 0\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ 0\\ 0\\ 0\end{array}\right]\right)\text{ (simplified for example)}$$ (Note: In reality, we’d use TF-IDF weights here.)

Step 3: Apply Rocchio Formula

Rocchio updates the query vector $Q$ as: $$Q_{new}=\alpha \cdot Q+\beta \cdot \frac{1}{|D_{pos}|}\sum _{d\in D_{pos}}d-\gamma \cdot \frac{1}{|D_{neg}|}\sum _{d\in D_{neg}}d$$ Where:

• $\alpha =1,\beta =1,\gamma =1$ (default weights).
• $D_{pos}$ = documents in the same class as the query.
• $D_{neg}$ = documents in the other class.

For "process scheduling" (assuming it belongs to OS class):

• Positive examples: All OS documents.
• Negative examples: All Automata documents.

New query vector: $$Q_{new}=\left[\begin{array}{c}1\\ 1\\ 0\\ 0\\ 0\end{array}\right]+\frac{1}{3}\left(\left[\begin{array}{c}1\\ 1\\ 0\\ 0\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ 1\\ 1\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 1\\ 0\\ 0\\ 1\end{array}\right]\right)-\frac{1}{3}\left(\left[\begin{array}{c}0\\ 0\\ 0\\ 0\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ 0\\ 0\\ 0\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ 0\\ 0\\ 0\end{array}\right]\right)$$ (Simplified: Just add positive centroid and subtract negative centroid.)

Result: The updated vector will be closer to the OS centroid, confirming classification as "Operating System".

In the Real World

1. eSewa: Transaction Recommendations

• Idea Used: Collaborative Filtering
• How:
  • If User A pays electricity bills frequently, eSewa recommends phone bills because many users who pay electricity also pay phone bills.
  • Cold-start solution: For new users, it uses content-based rules (e.g., "You might need to pay your internet bill").

2. Daraz: "Frequently Bought Together"

• Idea Used: Item-Item Collaborative Filtering
• How:
  • If Product X is frequently bought with Product Y, Daraz shows "Customers who bought this also bought" suggestions.
  • Example: If users who buy iPhone 15 also buy AirPods Pro, Daraz recommends AirPods when a user views the iPhone.

3. Ncell: Dynamic Data Plan Offers

• Idea Used: Matrix Factorization + Reinforcement Learning
• How:
  • MF predicts which users will upgrade to 4G/5G based on past data usage.
  • RL adjusts recommendations in real-time:
    • If a user uses 90% of data by 8 PM, Ncell promotes "Evening Unlimited" plans.
    • If a user rarely uses data at night, it suggests "Daytime Special" offers.

4. Pathao: Ride & Delivery Hotspots

• Idea Used: Location-Based Collaborative Filtering
• How:
  • If Thapathali has high demand for pizza at 8 PM, Pathao recommends pizza delivery to users in that area.
  • Uses real-time demand-supply matching (like Uber’s surge pricing but for recommendations).

Exam Tip

What Examiners Look For

1. Definitions:

   • Clearly distinguish between collaborative vs. content-based filtering.
   • Explain hybrid systems and matrix factorization with examples.

2. Algorithms:

   • For Rocchio, show the vector update formula and classification steps.
   • For collaborative filtering, explain user-user vs. item-item methods.

3. Real-World Applications:

   • Link concepts to Nepali companies (eSewa, Daraz, Ncell) or global platforms (YouTube, Netflix).
   • Example: "Daraz uses item-item CF to recommend accessories with smartphones."

4. Evaluation Metrics:

   • Know when to use Precision@k vs. RMSE.
   • Example: "For a music recommendation system, Precision@10 is better than RMSE because we care about relevance, not exact ratings."

5. Diagrams:

   • Draw user-item matrix decomposition for matrix factorization.
   • Show a hybrid system flowchart (e.g., CF + CBF).

Common Mistakes to Avoid

❌ Confusing CF and CBF: Always explain which one uses user behavior vs. item features. ❌ Ignoring cold-start problem: Mention how content-based filtering helps new users. ❌ Overlooking hybrid systems: Many real-world systems (e.g., Netflix) are not purely CF or CBF. ❌ Skipping evaluation metrics: Exams often ask to choose the right metric for a scenario.

Summary Table: Recommendation System Types

Final Thoughts

Recommendation systems are everywhere—from eSewa’s bill suggestions to YouTube’s video recommendations. Understanding the trade-offs between collaborative vs. content-based filtering and knowing when to use hybrid approaches is key to acing this unit.

Key Takeaways for Exams:

1. Collaborative Filtering = user behavior (e.g., ratings, clicks).
2. Content-Based Filtering = item features (e.g., keywords, categories).
3. Hybrid Systems = best of both worlds (used by Netflix, Amazon).
4. Matrix Factorization = solves sparsity in user-item matrices.
5. Deep Learning = for complex patterns (used by YouTube, TikTok).
6. Evaluation Metrics = Precision@k for relevance, RMSE for ratings.

Practice:

• Given a user-item matrix, explain how matrix factorization would work.
• For a new user, how would content-based filtering help?
• Why does YouTube use reinforcement learning instead of just collaborative filtering?