Software Quality: Metrics, Assurance & Control

TAKEAWAYS:

• Quality ≠ correctness: ISO 25010 defines 8 quality characteristics (functional suitability, reliability, usability, etc.) measured via metrics like cyclomatic complexity and defect density.
• Verification vs. Validation: Verification checks if the product meets specs (e.g., code reviews); validation checks if it meets user needs (e.g., beta testing with eSewa users).
• Inspections > Testing: Formal Fagan inspections (4–6 people) catch 60–90% of defects early—critical for safety-critical systems like NTC’s billing software.
• Configuration Management (CM): Tracks changes via baselines, version control (Git), and build automation—used by Daraz to roll back faulty releases.
• Ethics in SE: Violations (e.g., selling user data like Pathao’s past leaks) breach ACM/IEEE codes; whistleblowing is legally protected in Nepal’s IT Act 2008.
• COCOMO Model: Predicts effort (person-months) for projects; embedded mode (e.g., Ncell’s billing system) requires 2.4× more effort than organic mode.

1. Defining Software Quality

Software quality is the degree to which a system satisfies stated and implied needs (ISO/IEC 25010). It’s not just bug-free code—it’s about reliability, usability, and maintainability.

Key Quality Characteristics (ISO 25010)

Why it matters:

• eSewa’s transaction system must guarantee functional suitability (correct deductions) and security (no fraud).
• Pathao’s app prioritizes usability (intuitive UI) and performance (fast ride matching).

2. Quality Metrics: Measuring What Matters

Metrics quantify quality attributes. Two critical types:

A. Product Metrics (Static Analysis)

Measured without executing the code. Examples:

1. McCabe’s Cyclomatic Complexity (V(G))

   • Measures logical complexity in code.
   • Formula: $$V(G)=E-N+2P$$ where $E$ = edges, $N$ = nodes, $P$ = connected components.
   • Threshold: < 10 (simple), 10–20 (moderate), >20 (complex/refactor).
   • Example:

     if (x > 0) { // Node 1
       if (y < 5) { // Node 2
         z = a + b; // Edge 1-2
       } else {
         z = a - b; // Edge 1-3
       }
     } else {
       z = 0; // Edge 1-4
     }
     

     Here, $V(G)=5-4+2=3$ (low complexity).

2. Halstead’s Software Science

   • Predicts effort and bugs using:
     • $n_1$ = unique operators (e.g., +, if)
     • $n_2$ = unique operands (variables)
     • $N_1$ = total operators
     • $N_2$ = total operands
   • Program Length: $N=N_1+N_2$
   • Vocabulary: $n=n_1+n_2$
   • Volume: $V=N{\log }_{2}n$
   • Example: For int sum = a + b;, $n_1=2$ (int, +), $n_2=3$ (sum, a, b), $N_1=2$, $N_2=3$.

B. Process Metrics (Dynamic Analysis)

Measured during development/testing:

• Defect Density: Defects per KLOC (thousand lines of code).
  • Example: If a 100-KLOC system has 50 defects, density = 0.5 defects/KLOC.
  • Industry Benchmark: < 1.0 (high quality).
• Mean Time Between Failures (MTBF): Critical for NTC’s network software or Ncell’s call routing.

3. Verification vs. Validation: The Critical Distinction

Real-World Trace:

• Nepal Rastra Bank’s core banking system:
  • Verification: Auditors check if interest calculations (e.g., $\text{SI}=P\times R\times T$) match RBI guidelines.
  • Validation: Bank staff test if loan officers can process applications without errors.

4. Inspection Process: Catching Defects Early

Fagan Inspection (a formal review process) involves 4–6 people and 6 steps:

stateDiagram-v2
  [*] --> Planning: "Moderator + author select documents"
  Planning --> Overview: "Team reviews goals"
  Overview --> Preparation: "Individuals study code"
  Preparation --> Meeting: "Discuss defects"
  Meeting --> Rework: "Author fixes issues"
  Rework --> Follow-up: "Verify fixes"
  Follow-up --> [*]

Why inspections work:

• eSewa’s payment gateway uses inspections to catch SQL injection flaws before deployment.
• Defect removal efficiency: Inspections find 60–90% of defects vs. 30% via testing.

5. Configuration Management (CM): Controlling Change

CM ensures consistency across software versions. Key activities:

1. Version Control: Track changes (e.g., Git for Daraz’s app updates).
2. Baselining: Freeze a version (e.g., NTC’s billing software v1.2).
3. Build Automation: Tools like Jenkins compile/test code automatically.
4. Release Management: Control deployments (e.g., Pathao’s nightly rollouts).

Example: Daraz’s Order Queue System

• Problem: A bug in v3.2 caused order duplicates.
• CM Solution:
  • Rollback: Reverted to v3.1 using Git tags.
  • Hotfix: Patched v3.2 with a new baseline.

6. Software Ethics: Protecting Users and Developers

Ethical violations in Nepal:

• Pathao (2018): Sold user location data to third parties → ACM Code Violation (privacy breach).
• Nepal Stock Exchange (NEPSE): Insider trading via unethical access to trading algorithms.

ACM/IEEE Ethical Guidelines:

1. Public: Avoid harm to users (e.g., Ncell’s net neutrality).
2. Client/Employer: Honor contracts (e.g., eSewa’s data-sharing agreements).
3. Product: Ensure software is reliable and secure.
4. Judgment: Disclose conflicts of interest (e.g., a Daraz dev owning a competing app).

Whistleblowing in Nepal:

• Protected under IT Act 2008 (Section 49).
• Example: An NTC engineer reporting billing software fraud is legally safe.

7. COCOMO Model: Estimating Effort and Time

Constructive Cost Model (COCOMO) predicts effort for projects. Three modes:

Worked Example: A project is 320 KLOC (Kilo Lines of Code).

• Organic Mode:
  • Effort = $2.4\times 320^{1.05}=2.4\times 338.5=812.4$ person-months.
  • Time = $2.5\times 812.4^{0.38}=2.5\times 10.5=26.25$ months.
• Embedded Mode:
  • Effort = $3.6\times 320^{1.20}=3.6\times 423.6=1525$ person-months.
  • Time = $2.5\times 1525^{0.32}=2.5\times 12.8=32$ months.

Real-World Tie-In:

• Ncell’s billing system (embedded mode) took 32 months vs. eSewa’s organic-mode MVP (26 months).

8. Risk Management in Quality

Quality risks arise from:

• Poor requirements (e.g., NEPSE’s delayed trading system).
• Technical debt (e.g., Pathao’s legacy code crashes).
• External dependencies (e.g., Daraz relying on Ncell’s API).

Risk Analysis Steps:

1. Identify: List risks (e.g., "Database corruption").
2. Assess: Probability × Impact (e.g., 0.7 × High = Critical).
3. Mitigate: Actions like backups (NTC), load testing (eSewa).

In the Real World

1. eSewa’s Transaction Validation

   • Quality Idea: Validation ensures users can pay bills without errors.
   • How: Beta tests with 500+ users before full rollout.
   • Metric Used: Defect density < 0.3/KLOC in production.

2. NTC’s Network Reliability

   • Quality Idea: Reliability (MTBF) must exceed 99.99%.
   • How: Fagan inspections on routing protocols; automated regression tests.
   • Real Picture: IMAGE: "NTC fiber optic cable installation" | Underground fiber links ensuring low latency.

3. Pathao’s Ride-Matching Algorithm

   • Quality Idea: Performance efficiency (response time < 2s).
   • How: McCabe complexity kept below 10 for critical modules.
   • Ethics Violation: 2018 data leak breached ACM’s "Public" principle.

Exam Tip

1. COCOMO Questions:

   • Always show step-by-step calculations (e.g., $320^{1.05}$).
   • Memorize mode multipliers (Organic: 2.4, Embedded: 3.6).

2. Verification vs. Validation:

   • Verification = Specs (e.g., "Does the code match the design?").
   • Validation = Users (e.g., "Can farmers use eSewa on feature phones?").

3. Inspection Process:

   • 6 steps: Planning → Overview → Preparation → Meeting → Rework → Follow-up.
   • Defect catch rate: 60–90% (vs. 30% for testing).

4. Ethics:

   • ACM Code: Public, Client, Product, Judgment.
   • Nepal Law: IT Act 2008 protects whistleblowers.

5. Metrics:

   • Cyclomatic Complexity: $V(G)=E-N+2P$.
   • Defect Density: Defects/KLOC (<1.0 is good).

6. CM Activities:

   • Version Control (Git), Baselining, Build Automation (Jenkins).

Pro Tip: For short-answer questions, use bullet points and real examples (e.g., "Like eSewa’s validation tests..."). For long answers, structure with headings and diagrams. Always tie theory to Nepali apps (eSewa, NTC, Daraz).