User Research Methods: Techniques, Tools & Real-World Impact

TAKEAWAYS:

• User research is the foundation of UX design, ensuring solutions meet real user needs (not assumptions).
• Interviews and surveys uncover qualitative/quantitative insights, while observations reveal unspoken behaviors.
• Usability testing (e.g., think-aloud protocols) identifies pain points in prototypes, but requires careful moderation.
• Analytics tools (e.g., Google Analytics) track real user behavior, but lack context without qualitative data.
• Poor research leads to designs that fail in the market (e.g., a Daraz checkout flow with unclear CTAs).
• Ethical guidelines (e.g., anonymity, informed consent) are critical to build trust and avoid bias.

---

1. What Is User Research?

User research is the systematic study of end-users, their needs, behaviors, and motivations to inform design decisions. It bridges the gap between assumptions and evidence, ensuring designs solve real problems—not hypothetical ones.

Why It Matters in UX:

• Reduces rework by validating ideas early.
• Increases user satisfaction by addressing pain points.
• Justifies design choices to stakeholders (e.g., "Users prefer this layout because 70% completed tasks faster").

---

1.1 Types of User Research

User research falls into two broad categories:

---

1.2 The Research Cycle

User research is iterative, following this loop:

---

2. Core User Research Methods

Each method has strengths, weaknesses, and best use cases. Below are the five most critical for UX designers.

---

2.1 Interviews

Definition: Structured or unstructured conversations with users to explore their needs, frustrations, and goals. Can be one-on-one or group-based (focus groups).

How It Works:

1. Recruit participants (e.g., via social media, user communities).
2. Prepare questions (open-ended, e.g., "Tell me about your last time using [product].").
3. Conduct (record with consent; use a moderator’s guide).
4. Analyze for themes (e.g., "Users hate step 3 of checkout").

Example in Nepal: eSewa uses interviews to understand why users abandon transactions. A key finding: small transaction fees confused users, leading to a simplified fee display.

Advantages:

• Deep insights into motivations and context.
• Flexible (can pivot questions based on responses).

Disadvantages:

• Bias (users may not articulate true needs).
• Time-consuming (requires skilled moderators).

Worked Example: Daraz Checkout Pain Points Problem: Daraz’s checkout had a 30% abandonment rate. Research: Interviews with 20 users revealed:

• Users didn’t understand shipping costs until the final step.
• The "Add to Cart" button was too small on mobile. Solution: Simplified cost breakdowns and enlarged CTAs → 15% reduction in abandonment.

---

2.2 Surveys

Definition: Structured questionnaires to collect quantitative data (e.g., ratings, multiple-choice answers) from a large group.

How It Works:

1. Design questions (avoid leading questions; use Likert scales for sentiment).
2. Distribute (via email, social media, or embedded in apps).
3. Analyze with tools like Google Forms or SurveyMonkey.

Example in Nepal: Pathao used surveys to ask riders: "How often do you cancel a ride after booking?" Result: 42% canceled due to unclear driver availability. This led to a "Live Drivers Nearby" feature.

Advantages:

• Scalable (reach hundreds of users quickly).
• Quantifiable (e.g., "80% prefer dark mode").

Disadvantages:

• Low response rate (users may skip or lie).
• Lacks depth (no follow-up questions).

Survey Question Types:

---

2.3 Observations

Definition: Watching users in their natural environment (or lab) to see how they actually interact with a product.

Types:

• Naturalistic Observation (users in their home/workplace).
• Controlled Observation (users in a lab with a prototype).

Example in Nepal: NTC observed call centers to see how users struggled with IVR menus. They found:

• Users pressed buttons randomly when lost.
• Long hold times increased frustration. Solution: Simplified menu options and added a "Speak to Agent" option.

Advantages:

• Reveals unspoken behaviors (e.g., users clicking "Back" 5 times).
• No bias from self-reporting.

Disadvantages:

• Invasive (users may act differently when observed).
• Hard to scale (requires physical presence).

---

2.4 Usability Testing

Definition: Evaluating a prototype or live product with real users to identify usability issues (e.g., confusing navigation, slow load times).

Methods:

• Think-Aloud Protocol: Users verbalize their thoughts while using the interface.
• A/B Testing: Compare two versions (e.g., Button A vs. Button B).
• First-Click Testing: Measure how quickly users find key actions.

Example in Nepal: Khalti tested its payment gateway with users:

• Task: "Pay for a Daraz order."
• Finding: 60% users didn’t see the "Confirm Payment" button.
• Fix: Changed button color to red (high contrast) → 20% increase in completions.

Advantages:

• Direct feedback on real-world use.
• Prioritizes fixes (e.g., "This error message confuses 80% of users").

Disadvantages:

• Requires prototypes (can’t test ideas without a mockup).
• Moderation skills needed to avoid leading questions.

Usability Testing Checklist:

---

2.5 Analytics & Data-Driven Research

Definition: Using tools like Google Analytics, Hotjar, or Mixpanel to track user behavior after launch.

Key Metrics:

Example in Nepal: Ncell used analytics to find:

• Users exited the app at the "Top-Up" screen.
• Heatmap showed they scrolled past the CTA. Solution: Added a fixed "Top-Up Now" button → 30% increase in top-ups.

Advantages:

• Real-time data (no need to wait for user feedback).
• Scalable (works for millions of users).

Disadvantages:

• Lacks context (e.g., why a user dropped off?).
• Requires technical setup (tracking codes, etc.).

---

3. Choosing the Right Method

Not all methods are equal. The table below helps decide:

When to Combine Methods:

• Example: Start with surveys to identify pain points, then interview top respondents for depth, and test prototypes with usability studies.

---

4. Ethical Considerations in User Research

User research must respect privacy, consent, and fairness. Key guidelines:

1. Informed Consent:

   • Users must know why they’re being studied and how data will be used.
   • Example: eSewa’s research form includes:

     "Your responses will be anonymous. You can withdraw at any time."

2. Anonymity & Confidentiality:

   • Avoid collecting personally identifiable information (PII) unless necessary.
   • Example: Pathao uses screen recordings but blurs faces.

3. Avoiding Bias:

   • Leading questions (e.g., "Don’t you think our app is great?") skew results.
   • Diverse participants (e.g., test with users of different ages/abilities).

4. Compensation:

   • Pay users for their time (e.g., Rs. 500 for a 30-minute interview).

---

5. Real-World Applications

In the Real World

1. eSewa: Reducing Transaction Confusion

   • Method: Surveys + usability testing.
   • Problem: Users cancelled payments due to unclear fees.
   • Solution: Added a "Fee Breakdown" tooltip and simplified language.
   • Result: 25% reduction in drop-offs.

2. Daraz: Optimizing Checkout Flow

   • Method: Observations + A/B testing.
   • Problem: 30% abandonment at the payment step.
   • Finding: Users didn’t trust "Buy Now Pay Later" options.
   • Solution: Added trust badges (e.g., "Secure by Daraz") and clear interest rates.
   • Result: 18% increase in conversions.

3. NTC: Simplifying IVR Menus

   • Method: Naturalistic observation.
   • Problem: Users got stuck in IVR loops.
   • Solution: Replaced text menus with voice-guided options (e.g., "Press 1 for bill inquiry").
   • Result: 40% faster call resolution.

---

6. Common Pitfalls & How to Avoid Them

---

7. Exam Tips

1. Compare Methods:

   • The exam often asks to compare two methods (e.g., interviews vs. surveys).
   • Structure your answer like this:

     | **Aspect**       | **Interviews**               | **Surveys**                  |
     |------------------|-----------------------------|-----------------------------|
     | **Depth**        | High (qualitative)          | Low (quantitative)          |
     | **Scalability**  | Low (1:1)                   | High (100s of users)        |
     | **Bias Risk**    | Moderate (moderator influence) | High (leading questions)    |
     

2. Link Methods to Stages of UX Process:

   • Discovery: Interviews + surveys.
   • Prototyping: Usability testing.
   • Launch: Analytics.
   • Example Answer:

     "In the discovery phase, interviews help uncover user pain points, while surveys quantify preferences. During prototyping, usability testing validates if the design solves those problems. Post-launch, analytics track real-world behavior to identify further improvements."

3. Use Real-World Examples:

   • Always tie answers to Nepali companies (e.g., Daraz, Khalti) or global apps (e.g., WhatsApp’s "Last Seen" feature was tested via surveys).
   • Example:

     "WhatsApp used A/B testing to compare two "Last Seen" options. The version showing time ranges (e.g., "3 hours ago") performed better than exact times, as users preferred privacy."

4. Discuss Trade-offs:

   • Exams test if you understand when to use each method.
   • Example:

     "While surveys are fast and scalable, they lack depth. For example, a survey might reveal that users ‘hate’ a feature, but an interview could uncover they actually don’t understand its purpose—leading to a better redesign."

5. Mention Tools:

   • Know specific tools for each method:
     • Surveys: Google Forms, Typeform
     • Usability Testing: UserTesting.com, Maze
     • Analytics: Google Analytics, Hotjar
   • Example:

     "Tools like Hotjar provide session recordings, which reveal where users get stuck—unlike surveys, which only give post-hoc feedback."

---

8. Worked Example: Designing a New Bank App

Scenario: A Nepali bank wants to redesign its mobile app for first-time users.

Research Plan:

1. Interviews (5 users):

   • Question: "What’s the hardest part about using a bank app?"
   • Finding: Users didn’t know how to link their account to eSewa.

2. Surveys (100 users):

   • Question: "How often do you use mobile banking?"
   • Result: 60% never linked their account to other apps.

3. Usability Test (Prototype):

   • Task: "Link your account to eSewa."
   • Problem: 70% users couldn’t find the "Link Account" button.

4. Analytics (Post-Launch):

   • Heatmap showed users scrolled past the CTA.
   • Solution: Added a "Get Started" banner at the bottom.

Final Design:

• Added a "First-Time User" guide in the onboarding flow.
• Highlighted the "Link Account" button with a green badge.
• Added tooltips explaining eSewa integration.

Result: 45% increase in account linking within 3 months.

---

9. Key Takeaways for the Exam

1. User research is not optional—it’s the difference between a product that flops and one that succeeds.
2. No single method is perfect—combine qualitative (interviews) + quantitative (surveys/analytics) for robust insights.
3. Ethics matter—always get consent, ensure anonymity, and avoid leading questions.
4. Real-world examples score high—know how Daraz, eSewa, or NTC used research to improve their products.
5. Exams test comparisons—be ready to contrast interviews vs. surveys vs. usability testing with pros/cons.
6. Link research to design outcomes—show how findings directly improved a product (e.g., button placement, error messages).

---

10. Quick Review Questions (Practice for Exams)

1. Why would a designer choose observations over surveys for researching a new grocery app?

   • Answer: Observations reveal how users actually shop (e.g., scanning barcodes, abandoning carts), while surveys only capture self-reported preferences.

2. How did Ncell use analytics to improve its app?

   • Answer: They found users exited at the top-up screen via drop-off analytics and added a fixed "Top-Up Now" button.

3. Compare the strengths of interviews and usability testing.

   • Answer:

     Aspect	Interviews	Usability Testing
     Purpose	Explore why users think.	Test how users interact.
     When to Use	Early-stage research.	Late-stage (prototype testing).

4. What ethical issue might arise in a survey about sensitive topics (e.g., financial habits)?

   • Answer: Privacy concerns—users may lie about income/spending if they feel judged. Fix: Anonymize responses and assure confidentiality.

5. How would you select a research method for a new feature in Khalti’s payment gateway?

   • Answer:
     1. Start with surveys (100 users) to quantify pain points (e.g., "Do you trust 'Buy Now Pay Later'?").
     2. Interview top respondents to dig deeper (e.g., "Why do you hesitate?").
     3. Test a prototype with usability testing to see if the redesign fixes issues.