Software Testing: Types, Techniques & Strategies

TAKEAWAYS:

• Testing is not just bug-finding: It validates correctness, reliability, and usability through systematic evaluation.
• Black-box vs. white-box: Black-box tests behavior (e.g., user inputs/outputs), while white-box tests internal logic (e.g., code paths).
• Test case design: Equivalence partitioning and boundary value analysis reduce test cases while maximizing coverage.
• Automation saves time: Scripts for regression testing (e.g., Daraz’s order validation) or performance testing (e.g., Ncell’s load handling).
• Agile vs. V-model: Agile integrates testing early (continuous feedback), while V-model separates phases strictly.
• Metrics matter: Defect density, test coverage, and mean time to repair (MTTR) measure quality.

1. What is Software Testing?

Software testing is the process of verifying and validating that a software product meets requirements and works as intended. It ensures:

• Correctness: The software behaves as specified.
• Reliability: It handles errors gracefully (e.g., eSewa’s transaction retries).
• Usability: Users can navigate it easily (e.g., Pathao’s ride-booking flow).
• Performance: It scales under load (e.g., NEPSE’s trading platform during market hours).

Why test?

• Cost: Fixing bugs early costs 100x less than post-release (IEEE data).
• Reputation: Poor testing (e.g., a bank app crash) loses user trust.
• Compliance: Industries like healthcare (e.g., Nepal’s health management systems) require rigorous testing.

1.1 Types of Testing

Testing is classified by scope, level, and technique. Here’s a breakdown:

1.1.1 By Level

1.1.2 By Technique

1.2 Test Case Design Techniques

Goal: Design minimal yet effective test cases to catch defects.

1.2.1 Equivalence Partitioning

Divide input data into partitions where each partition is expected to behave the same. Test one value per partition.

Example: Testing a loan eligibility function in a bank app.

• Valid partitions:
  • Income ≥ 50,000 NPR (should approve).
  • Income < 50,000 NPR (should reject).
• Invalid partitions:
  • Negative income (edge case).
  • Non-numeric input (e.g., "abc").

Test Cases:

1.2.2 Boundary Value Analysis

Test boundaries of input ranges (where errors often occur).

• Example: A hotel booking system where rooms are booked for 1–30 days.
  • Test cases: 0, 1, 30, 31 days.
  • Expected: 0 and 31 should fail; 1 and 30 should succeed.

2. Black-Box vs. White-Box Testing

3. Test Automation

Manual testing is slow and error-prone. Automation:

• Saves time: Run regression tests overnight (e.g., Daraz’s daily sales checks).
• Improves accuracy: No human fatigue.
• Supports CI/CD: Tools like Jenkins integrate tests into development pipelines.

3.1 When to Automate?

3.2 Popular Automation Tools

4. Testing in Software Development Models

How testing fits into SDLC models:

4.1 V-Model

• Waterfall approach: Testing phases mirror development phases.
• Pros: Structured, good for regulated industries (e.g., NTC’s billing systems).
• Cons: Late defect detection.

4.2 Agile Model

• Testing is continuous: Developers write tests alongside code (TDD).
• Pros: Early defect detection, faster feedback.
• Cons: Requires discipline; less documentation.

Example: Google’s Test-Driven Development (TDD) for YouTube’s recommendation engine.

1. Write a failing test for a new feature.
2. Write minimal code to pass the test.
3. Refactor and repeat.

5. Non-Functional Testing

Ensures quality attributes beyond functionality:

In the Real World

1. eSewa’s Transaction Testing

   • What’s tested: Black-box testing for payment flows (e.g., "Transfer 500 NPR to a friend").
   • How: Equivalence partitioning for amounts (valid/invalid), boundary values (0.01 NPR), and error cases (insufficient balance).
   • Automation: Selenium scripts verify the UI; Postman tests the API.

2. Daraz’s Order Queue

   • What’s tested: System testing for race conditions (multiple users checking out simultaneously).
   • How: Load testing with 10,000 concurrent users to simulate Black Friday.
   • Tool: JMeter simulates traffic; Jenkins triggers tests post-deployment.

3. Ncell’s Network Stability

   • What’s tested: Performance testing for call drops during peak hours (e.g., 7–9 PM).
   • How: Inject 50,000 calls/minute; measure drop rate and latency.
   • Tool: LoadRunner monitors KPIs like mean time between failures (MTBF).

4. Nepal Rastra Bank’s Core Banking System

   • What’s tested: Security testing for SQL injection and data breaches.
   • How: Penetration testing by ethical hackers; OWASP ZAP scans for vulnerabilities.
   • Compliance: Must pass ISO 27001 audits.

6. Test Metrics

Quantify testing effectiveness:

7. Common Testing Pitfalls

• Over-testing: Wasting resources on trivial cases (e.g., testing "1+1=2" in a calculator).
• Under-testing: Skipping edge cases (e.g., not testing empty carts in Daraz).
• Ignoring non-functional tests: Assuming "if it works, it’s fast" (e.g., a slow NEPSE login page).
• No test automation: Manual regression testing is unscalable for apps like Pathao.

Exam Tip

What Examiners Look For

1. Definitions: Know the difference between unit vs. integration testing, black-box vs. white-box.
2. Examples: Relate concepts to Nepalese apps (e.g., "How would you test eSewa’s payment flow?").
3. Diagrams: Draw V-model vs. Agile testing phases or equivalence partitioning tables.
4. Tools: Name at least 2 tools for each test type (e.g., Selenium for UI, JMeter for load).
5. Metrics: Calculate defect density or test coverage from given data.
6. Real-world scenarios: Explain how automation helps Daraz or how boundary testing catches bugs in Ncell’s billing.

High-Scoring Answers Include:

• Step-by-step test case design (e.g., "For a login system, I’d test: valid credentials, wrong password, empty fields").
• Pros/cons comparisons (e.g., "V-model is rigid but thorough; Agile is flexible but requires discipline").
• Visuals: Always sketch a test matrix or flowchart if the question asks for a plan.

Practice Question

Scenario: You’re testing a food delivery app (like Pathao) for a TU exam. Design 3 black-box test cases using equivalence partitioning and boundary value analysis for the "Add to Cart" feature.

Solution Outline:

1. Valid partitions:
   • Quantity = 1 (minimum).
   • Quantity = 5 (normal).
2. Invalid partitions:
   • Quantity = 0 (edge case).
   • Quantity = 101 (max exceeded).
3. Boundary values:
   • Quantity = 0 (should show "Add at least 1 item").
   • Quantity = 100 (max allowed; should accept).
   • Quantity = 101 (should reject with "Max 100 items").

Test Cases:

flowchart TD
    A["Unit Tests"] -->|"Most Frequent"| B["Integration Tests"]
    B -->|"Less Frequent"| C["UI/Acceptance Tests"]
    caption Testing Pyramid (Adapted from Mike Wacker, Public Domain)