Design Evaluation & Iteration: Testing, Feedback, and Refinement

TAKEAWAYS:

• Evaluation is iterative: Formative tests (early stages) fix usability flaws, while summative tests (final stages) validate the product’s success.
• Feedback loops: User testing, surveys, and analytics reveal pain points (e.g., Khalti’s checkout drop-off rates).
• Prototyping refinement: Low-fidelity sketches evolve into high-fidelity prototypes with micro-interactions (e.g., eSewa’s loading animations).
• Accessibility checks: Tools like screen readers and contrast analyzers ensure designs work for all users (e.g., NEPSE’s mobile app for visually impaired traders).
• Data-driven decisions: Heatmaps and session recordings pinpoint where users struggle (e.g., Daraz’s product search filters).
• Continuous improvement: Iteration turns "good" designs into "great" ones by addressing real-world usage (e.g., Pathao’s ride-sharing route optimizations).

1. Why Evaluation Matters in UI/UX

Evaluation is the bridge between design and user needs. Without it, even well-intentioned interfaces fail—like Ncell’s early mobile app, which had confusing navigation until usability tests revealed users struggled to find data plans.

Key questions evaluation answers:

• Are users finding what they need?
• Do they understand how to interact?
• Are they completing tasks efficiently?
• Are edge cases (e.g., slow connections) handled?

1.1 Formative vs. Summative Evaluation

Evaluation happens at two critical stages:

flowchart TD
    A["Formative Evaluation"] -->|"Early stages"| B["Identify usability issues"]
    B --> C["Fix problems before final design"]
    D["Summative Evaluation"] -->|"Final stages"| E["Validate if design meets goals"]
    E --> F["Release to users"]
    A -->|"Ongoing"| G["Iterate based on feedback"]
    G -->|"Loop"| A

Comparison Table:

1.2 Common Evaluation Methods

1. Usability Testing

• What it is: Observing real users complete tasks while thinking aloud.
• How it works:
  • Users interact with a prototype while a moderator notes struggles.
  • Example: Testing eSewa’s mobile app for elderly users to verify button sizes.
• Tools:
  • Moderated testing (in-person or remote via Zoom).
  • Unmoderated testing (tools like UserTesting.com or Hotjar).
• Example Worked Trace:
  • Problem: Users abandon Khalti’s payment page at the "Enter OTP" step.
  • Test: 20 users attempt a mock payment. 60% fail to locate the OTP field.
  • Fix: Added a floating "Need help?" button with a video tutorial.

2. Heuristic Evaluation

• What it is: Experts (e.g., UX designers) check a design against Nielsen’s 10 Usability Heuristics (e.g., "Visibility of system status").
• Example: Auditing NTC’s website for missing error messages during form submissions.

3. Surveys & Questionnaires

• What it is: Structured feedback via Likert scales (e.g., "How easy was this task?").
• Example: Ncell’s app survey revealed users wanted a "Favorites" tab for quick access.

4. Analytics & Heatmaps

• What it is: Data-driven insights (e.g., Google Analytics, Hotjar).
  • Click maps: Show where users click (often vs. intended).
  • Session recordings: Replay user journeys to spot drop-offs.
• Example: Daraz’s heatmap showed users ignored the "Filter by Price" option, leading to a redesign.

5. A/B Testing

• What it is: Comparing two versions (A vs. B) to see which performs better.
• Example: eSewa tested two checkout button colors—blue (original) vs. green. Green increased conversions by 12%.

1.3 Measuring Success: Key Metrics

2. Iteration: Turning Feedback into Improvements

Iteration is the cycle of test → analyze → fix → retest. Here’s how it works in practice:

sequenceDiagram
    participant User
    participant Designer
    participant Prototype

    User->>Designer: "I can’t find the ‘Save’ button!"
    Designer->>Prototype: Analyze heatmap (button ignored)
    Designer->>Prototype: Move ‘Save’ to top-right (high-visibility)
    Prototype-->>Designer: Updated prototype
    Designer->>User: "Try the new layout!"
    User->>Designer: "Now it’s easier!"

Example: Refining Daraz’s Search Function

1. Problem: Users struggled to filter by price range.
2. Test: Heatmap showed 70% ignored the filter dropdown.
3. Fix: Added a slider and quick-access price tags (e.g., "Under ₹1,000").
4. Result: Filter usage increased by 45%.

2.1 Low-Fidelity vs. High-Fidelity Prototypes

Prototypes evolve from rough sketches to interactive models. Each serves a purpose:

2.2 Making Prototypes Realistic

High-fidelity prototypes mimic the final product. Key techniques:

• Micro-interactions: Small animations that guide users (e.g., eSewa’s loading spinner).
• Feedback: Confirmations (e.g., "Payment successful!" toast in Khalti).
• Error Handling: Clear messages (e.g., "Invalid OTP. Try again." in Ncell).
• Consistency: Uniform buttons, icons, and spacing (e.g., Daraz’s "Add to Cart" button style).

Example: Pathao’s Ride Confirmation

• Before: Users got a static "Ride booked!" message.
• After: Added a countdown timer and driver’s photo for transparency.
• Result: User satisfaction increased by 20%.

3. Accessibility in Evaluation

Accessibility ensures designs work for all users, including those with disabilities. Key checks:

mindmap
  root((Accessibility Evaluation))
    - Visual Impairments
      - Sufficient contrast (WCAG AA)
      - Alt text for images
    - Hearing Impairments
      - Captions for videos
      - Visual alerts (e.g., flashing buttons)
    - Motor Impairments
      - Keyboard navigation
      - Large tap targets (≥48px)
    - Cognitive Impairments
      - Simple language
      - Predictive text

Example: NEPSE’s Mobile App for Traders

• Problem: Visually impaired traders struggled to read stock charts.
• Fix:
  • Added high-contrast mode.
  • Implemented screen reader compatibility.
• Result: 90% of visually impaired users could now trade independently.

WCAG Checklist for Evaluation:

4. Real-World Examples: How Nepali Apps Use Evaluation

1. Pathao: Ride-Sharing Route Optimization

• Problem: Users complained about long wait times for drivers.
• Evaluation:
  • Heatmaps showed users abandoned the app after 30 seconds.
  • Surveys revealed confusion over fare estimates.
• Iteration:
  • Added a "Driver ETA" counter.
  • Implemented real-time fare adjustments.
• Result: Wait times dropped by 25%, and app ratings improved.

2. Daraz: Checkout Flow Redesign

• Problem: 40% of users abandoned carts at payment.
• Evaluation:
  • Session recordings showed users hesitated at the "Enter Card" step.
  • A/B test: Compared one-step vs. two-step payment.
• Iteration:
  • Simplified to one-step payment with auto-fill.
  • Added multiple payment options (eSewa, Khalti, credit card).
• Result: Checkout completion rose from 60% to 85%.

3. NEPSE: Accessible Trading for All Users

• Problem: Elderly and visually impaired traders struggled with the desktop site.
• Evaluation:
  • Usability testing with screen readers revealed navigation issues.
  • Analytics showed high drop-off on mobile.
• Iteration:
  • Developed a mobile app with high-contrast mode.
  • Added voice-guided trading for visually impaired users.
• Result: Mobile app adoption grew by 50%.

5. Common Pitfalls in Evaluation

6. Exam Tips for Unit 8

1. Link evaluation to real apps:

   • Example: "Pathao used session recordings to find that users abandoned the app when the driver’s ETA exceeded 5 minutes, leading to a redesign of the wait-time display."
   • Tip: Always cite Nepali examples (Pathao, Daraz, eSewa) or global apps (WhatsApp, Google Maps).

2. Compare methods with a table:

   • Example:

     Method	Best For	Limitations
     Usability Testing	Observing real user behavior	Expensive, requires moderation
     Surveys	Large-scale feedback	May not reveal "why" users struggle

3. Explain iteration with a cycle diagram:

   • Show test → analyze → fix → retest (like the sequence diagram above).
   • Tip: Use Pathao’s ride confirmation or Khalti’s payment flow as examples.

4. Address accessibility with WCAG:

   • Mention contrast ratios, keyboard navigation, and alt text in your answers.
   • Example: "NEPSE’s mobile app improved accessibility by adding high-contrast mode and screen reader support, reducing drop-off for visually impaired traders."

5. Connect to past exam questions:

   • If asked about navigation models, tie it to evaluation: "Flat navigation (like Daraz’s homepage) is easy to evaluate with heatmaps, but hierarchical models (like Pathao’s driver search) require card sorting tests to ensure users find categories."

6. Use metrics in your answers:

   • Example: "The task success rate for eSewa’s payment flow improved from 70% to 95% after reducing the number of steps from 5 to 3."

Final Worked Example: Redesigning Khalti’s Payment Page

Scenario: Khalti’s payment page had a 30% drop-off rate at the OTP step.

Evaluation Steps:

1. Heatmap Analysis:
   • Found users ignored the OTP field and clicked the "Back" button instead.
2. Usability Testing:
   • 15 users struggled to locate the OTP input; 80% made errors.
3. Survey:
   • Users wanted visual feedback (e.g., a timer) and error guidance.

Iteration:

• Added a countdown timer ("OTP expires in 30s").
• Included a "Resend OTP" button with a 60-second cooldown.
• Changed the OTP field to a larger, outlined input for better visibility.

Result:

• Drop-off rate dropped to 10%.
• User satisfaction score (SUS) rose from 65 to 88.

Why This Works for Exams:

• Shows data-driven iteration.
• Cites real metrics (drop-off rate, SUS).
• Uses Nepali context (Khalti).