Software Testing: Types, Techniques & Strategies

TAKEAWAYS:

• Software testing ensures software meets requirements, identifies defects early, and improves quality through systematic validation and verification.
• Test types (unit, integration, system, acceptance) target different levels of software granularity, each requiring distinct strategies.
• Test techniques like black-box (input/output-based) and white-box (code structure-based) are chosen based on accessibility to system internals.
• Test case design follows standards (e.g., equivalence partitioning, boundary value analysis) to maximize defect detection with minimal effort.
• Automation (e.g., Selenium, JUnit) speeds up regression testing but requires careful script maintenance and tool selection.
• Real-world applications (e.g., eSewa’s transaction validation, Daraz’s order processing) rely on rigorous testing to ensure reliability and user trust.

1. Introduction to Software Testing

Software testing is a quality assurance process that evaluates whether software behaves as expected. It involves executing a program with specific inputs and comparing actual outputs to expected results. Testing is not just about finding bugs—it also validates requirements, improves usability, and ensures compliance with standards.

Why is Testing Important?

• Defect Detection: Identifies errors early, reducing costs (fixing a bug in design costs 100x less than in production).
• Quality Assurance: Ensures software meets user and business needs.
• Risk Mitigation: Prevents failures in critical systems (e.g., banking, healthcare).
• Compliance: Meets industry standards (e.g., ISO/IEC 25010 for software quality).

Testing vs. Debugging

2. Types of Software Testing

Testing is classified based on scope, level, and technique. Below are the key categories:

2.1 By Test Level

sequenceDiagram
    participant User
    participant Daraz_API
    participant Inventory_DB
    participant Notification_Service

    User->>Daraz_API: Place Order (Item ID: 123)
    Daraz_API->>Inventory_DB: Check Stock
    Inventory_DB-->>Daraz_API: Stock Available
    Daraz_API->>Notification_Service: Send Confirmation
    Notification_Service-->>User: Order Confirmed
    alt Inventory Unavailable
        Daraz_API-->>User: Out of Stock
    end

a) Unit Testing

• Tests individual components (functions, methods, classes) in isolation.
• Example: Testing a calculateDiscount() function in an e-commerce app.
• Tools: JUnit (Java), pytest (Python), NUnit (.NET).
• Worked Example: Suppose a bank’s calculateInterest() function is tested with inputs:
  • Principal = 100,000; Rate = 5%; Time = 1 year → Expected: 5,000.
  • Edge case: Principal = 0 → Expected: 0 (no interest).

b) Integration Testing

• Tests interactions between integrated units (e.g., database + API).
• Example: Testing if Daraz’s order processing module correctly updates inventory and sends notifications.
• Types:
  • Big Bang: All units integrated at once (risky).
  • Incremental: Step-by-step integration (e.g., top-down, bottom-up).

c) System Testing

• Tests the entire system against functional and non-functional requirements.
• Example: Testing eSewa’s end-to-end transaction flow (login → payment → confirmation).
• Subtypes:
  • Functional Testing: Validates features (e.g., user login).
  • Non-Functional Testing: Performance, security, usability.

d) Acceptance Testing

• Performed by end-users or clients to validate if the system meets business needs.
• Example: NTC testing a new billing software with sample users before full deployment.
• Types:
  • User Acceptance Testing (UAT): Users test in a real-world scenario.
  • Operational Acceptance Testing (OAT): IT team checks deployment readiness.

2.2 By Test Technique

a) Black-Box Testing

• Tests without knowing internal code (focuses on inputs/outputs).
• Techniques:
  • Equivalence Partitioning: Divides inputs into groups (e.g., valid/invalid emails).
  • Boundary Value Analysis: Tests edge cases (e.g., max/min limits).
• Example: Testing WhatsApp’s login with invalid passwords (e.g., empty field, special characters).

b) White-Box Testing

• Tests internal logic (requires code access).
• Techniques:
  • Statement Coverage: Ensures every line is executed.
  • Branch Coverage: Tests all decision paths (e.g., if-else conditions).
• Example: Checking if a loop in Pathao’s ride-matching algorithm terminates correctly.

c) Grey-Box Testing

• Combines black-box and white-box (partial knowledge of internals).
• Example: Testing a bank’s ATM system where testers know some backend logic but not all.

2.3 By Test Type

3. Test Case Design

A test case is a set of inputs, execution conditions, and expected results designed to validate a specific feature.

stateDiagram-v2
    [*] --> Valid_Credentials
    Valid_Credentials --> Dashboard
    Invalid_Credentials --> Error_Page
    Error_Page --> [*]
    Error_Page --> Retry
    Retry --> Valid_Credentials
    Retry --> Invalid_Credentials

Steps to Design Test Cases

1. Identify Test Objectives: What is being tested? (e.g., login functionality).
2. Define Test Inputs: Valid and invalid data (e.g., correct/incorrect email formats).
3. Specify Expected Results: What should happen? (e.g., "System should show ‘Invalid email’").
4. Execute and Compare: Run the test and verify actual vs. expected results.

Example: Testing a Login System

Techniques for Test Case Design

4. Test Automation

Automation uses scripts/tools to execute tests repeatedly with minimal human intervention.

When to Automate?

• Regression Testing: Re-run tests after code changes.
• Repetitive Tasks: E.g., logging in 100 times to test session handling.
• Performance Testing: Simulate thousands of users (e.g., load testing for NEPSE’s trading platform).

Tools for Automation

Advantages of Automation

• Speed: Faster execution than manual testing.
• Cost-Effective: Reduces long-term manual effort.
• Accuracy: Eliminates human errors in repetitive tasks.
• Reusability: Scripts can be reused across projects.

Challenges

• Initial Setup Cost: Writing and maintaining scripts.
• Tool Dependency: Requires expertise in automation tools.
• Not All Tests Can Be Automated: Exploratory testing needs human intuition.

5. Software Testing Life Cycle (STLC)

STLC is a structured approach to testing, aligned with the software development life cycle (SDLC).

flowchart TD
    A["Requirements Analysis"] --> B["Test Planning"]
    B --> C["Test Case Design"]
    C --> D["Test Environment Setup"]
    D --> E["Test Execution"]
    E --> F["Defect Reporting"]
    F --> G["Test Cycle Closure"]
    G -->|"Re-test if needed"| C

Phases of STLC

1. Requirements Analysis: Review software requirements to identify testable items.
2. Test Planning: Define scope, resources, and schedule.
3. Test Case Design: Create test cases using techniques like equivalence partitioning.
4. Test Environment Setup: Prepare hardware/software (e.g., staging servers for eSewa).
5. Test Execution: Run tests and log defects.
6. Defect Reporting: Document bugs using tools like JIRA or Bugzilla.
7. Test Cycle Closure: Analyze test results and generate reports.

6. Defect Management

A defect (or bug) is any discrepancy between expected and actual results.

Defect Life Cycle

stateDiagram-v2
    [*] --> New
    New --> Assigned
    Assigned --> Fixed
    Fixed --> Retest
    Retest --> Reopen
    Reopen --> Fixed
    Fixed --> Closed
    Retest --> Closed
    Closed --> [*]

Steps in Defect Management

1. Defect Logging: Record details (steps to reproduce, severity).
2. Defect Triage: Prioritize (e.g., critical vs. minor).
3. Fixing: Developers resolve the issue.
4. Retesting: QA verifies the fix.
5. Closure: Defect is marked resolved or reopened if needed.

Severity vs. Priority

7. Real-World Applications

Example 1: eSewa’s Transaction Testing

• Problem: eSewa must ensure 100% accuracy in financial transactions.
• Testing Applied:
  • Unit Testing: Validate processPayment() function.
  • Integration Testing: Check interaction between payment gateway and database.
  • Regression Testing: Ensure new features (e.g., QR payments) don’t break existing ones.
• Outcome: Reduces fraud and improves user trust.

Example 2: Daraz’s Order Processing

• Problem: Daraz handles millions of orders daily; delays or errors cause losses.
• Testing Applied:
  • Performance Testing: Simulate 10,000 concurrent users to check server response time.
  • Usability Testing: Ensure mobile app works on low-end devices.
  • Security Testing: Penetration tests to prevent hacking (e.g., credit card theft).
• Outcome: Faster deliveries and fewer order cancellations.

Example 3: NTC’s Billing Software

• Problem: NTC must bill millions of customers accurately without errors.
• Testing Applied:
  • Acceptance Testing: Users (e.g., telecom agents) verify billing reports.
  • Regression Testing: After tax law changes, ensure calculations update correctly.
• Outcome: Reduces customer complaints and revenue leaks.

8. Common Testing Mistakes to Avoid

• Skipping Test Planning: Leads to ad-hoc testing and missed defects.
• Ignoring Edge Cases: Focus only on happy paths (e.g., not testing empty inputs).
• Over-Automating: Not all tests (e.g., exploratory testing) can be automated.
• Poor Defect Reporting: Vague bug descriptions waste developers’ time.
• Testing Too Late: Fixing defects in later stages is costly.

9. Exam Tip

How This Unit is Examined

1. Theory Questions (30-40%):

   • Define black-box vs. white-box testing.
   • Explain STLC phases with a diagram.
   • Compare unit vs. integration testing.

2. Scenario-Based Questions (30-40%):

   • Given a login system, design 3 test cases using boundary value analysis.
   • Describe how you would test Daraz’s order processing for performance.
   • Explain how automation reduces regression testing effort.

3. Diagram-Based Questions (20-30%):

   • Draw the defect life cycle or STLC flowchart.
   • Sketch a test case table for a given scenario (e.g., ATM withdrawal).

Key Focus Areas for Full Marks

• Memorize: Definitions (e.g., "Software testing is the process of verifying and validating...").
• Apply: Use real-world examples (e.g., eSewa, Daraz) in answers.
• Draw: STLC, defect life cycle, or test case tables in exams.
• Compare: Tables for black-box vs. white-box, unit vs. system testing.
• Calculate: Simple test case design (e.g., "Design 2 test cases for a search function").

Sample Exam Question & Answer

Question: "Explain the difference between functional and non-functional testing with examples from Nepalese software products. Design a test case for Khalti’s ‘Add Money’ feature using boundary value analysis."

Answer: Functional Testing verifies specific functions of the software (e.g., does Khalti’s "Add Money" button work?). Non-Functional Testing checks quality attributes like performance, security, and usability (e.g., does Khalti’s app load within 2 seconds?).

Test Case for "Add Money" (Boundary Value Analysis):

In the real world

• eSewa’s transaction validation: Uses system testing to verify end-to-end flows (user login → payment → confirmation) with real-world scenarios like failed OTP retries and bank API timeouts.
• Daraz’s order processing: Employs integration testing to ensure inventory updates, payment gateways (Khalti), and notification services (SMS/email) sync correctly without race conditions.
• NTC’s billing software: Relies on user acceptance testing (UAT) where sample customers (e.g., Ncell users) validate invoice accuracy and payment gateway integrations before full deployment.