Numerical Differentiation & Integration: Rules, Errors & Engineering Applications

TAKEAWAYS:

• Numerical differentiation approximates derivatives using finite differences (forward, backward, central) and is critical for modeling dynamic systems like traffic flow or signal processing.
• Numerical integration (quadrature) estimates integrals using rules like trapezoidal, Simpson’s 1/3, and Simpson’s 3/8, with Romberg integration improving accuracy iteratively.
• Errors in numerical methods include truncation, rounding, and discretization errors, which must be minimized for reliable results (e.g., in financial modeling or structural analysis).
• Simpson’s rules require even subintervals (1/3) or multiples of 3 (3/8), while Romberg integration uses Richardson extrapolation to refine estimates.
• Applications span engineering (load distribution, heat transfer), finance (option pricing), and AI (gradient descent in machine learning).
• Always compare methods’ accuracy and computational cost—higher-order rules (Simpson’s) are more accurate but require more function evaluations.

---

1. Introduction to Numerical Differentiation and Integration

Numerical methods approximate derivatives and integrals when exact analytical solutions are impractical. These techniques are foundational in engineering for modeling physical phenomena (e.g., stress distribution, fluid flow) and optimizing systems (e.g., traffic routing, energy efficiency).

---

1.1 Why Approximate?

Exact solutions often require solving transcendental equations or complex integrals. Numerical methods:

• Use discrete data points (e.g., sensor readings, experimental measurements).
• Trade precision for computational feasibility (e.g., finite element analysis in civil engineering).

Example: Calculating the slope of a bridge’s deflection curve at a specific point using strain gauge data.

---

1.2 Errors in Numerical Methods

All approximations introduce error. Key types:

• Truncation error: Due to finite step size (e.g., using 6 subintervals instead of infinite).
• Rounding error: From floating-point arithmetic (e.g., 1/3 ≈ 0.3333 in computers).
• Discretization error: From approximating continuous functions with discrete values.

Figure 1: Types of errors in numerical integration

```mermaid
flowchart TD
    A[Truncation Error] -->|Step size h| B[∝ h^p]
    C[Rounding Error] -->|Machine precision| D[∝ ε_m]
    E[Discretization Error] -->|Grid resolution| F[∝ h^q]

---

2. Numerical Differentiation

Approximates ( f'(x) ) using finite differences. Common methods:

2.1 Forward, Backward, and Central Differences

Example 1: Approximate ( f'(1) ) for ( f(x) = x^2 + 3x ) using ( h = 0.1 ).

```python
f = lambda x: x**2 + 3*x
h = 0.1
x = 1.0
forward = (f(x+h) - f(x))/h
backward = (f(x) - f(x-h))/h
central = (f(x+h) - f(x-h))/(2*h)
print(f"Forward: {forward:.4f}, Backward: {backward:.4f}, Central: {central:.4f}")

Output:

Forward: 5.1000, Backward: 5.1000, Central: 5.0000

Exact derivative: $f^{\prime }(x)=2x+3\Rightarrow f^{\prime }(1)=5$. The central difference is more accurate.

Figure 2: Finite difference stencils for $f^{\prime }(x)$

```figure
{"type":"graph","fns":[{"expr":"2*x + 3","label":"Exact derivative f'(x)"},{"expr":"(f(x+h)-f(x-h))/(2*h)","label":"Central difference (h=0.1)"},{"expr":"(f(x)-f(x-h))/h","label":"Forward difference (h=0.1)"},{"expr":"(f(x+h)-f(x))/h","label":"Backward difference (h=0.1)"}],"x":[0.5,1.5],"points":[{"x":1,"y":5,"label":"Exact f'(1)"}],"caption":"Comparison of finite difference approximations for f'(x) at x=1 (h=0.1)"}

---

2.2 Richardson Extrapolation

Improves accuracy by combining lower-order methods. For central difference: [ f'(x) \approx \frac{4}{3} \cdot \frac{f(x+h) - f(x-h)}{2h} - \frac{1}{3} \cdot \frac{f(x+2h) - f(x-2h)}{4h} ]

Example 2: Extrapolate ( f'(0) ) for ( f(x) = \sin(x) ) using ( h = 0.01 ).

```python
import math
f = lambda x: math.sin(x)
h = 0.01
x = 0.0
central_h = (f(x+h) - f(x-h))/(2*h)
central_2h = (f(x+2*h) - f(x-2*h))/(4*h)
extrapolated = (4/3)*central_h - (1/3)*central_2h
print(f"Central (h=0.01): {central_h:.6f}, Extrapolated: {extrapolated:.6f}")

Output:

Central (h=0.01): 0.999983, Extrapolated: 1.000000

Exact derivative: $f^{\prime }(0)=\cos (0)=1$. Extrapolation matches the exact value.

---

3. Numerical Integration (Quadrature)

Estimates ${\int }_a^{b}f(x)dx$ using discrete points. Key rules:

3.1 Trapezoidal Rule

Approximates the integral as the area of trapezoids under $f(x)$: $${\int }_a^{b}f(x)dx\approx \frac{h}{2}\left[f(x_0)+2\sum _{i=1}^{n-1}f(x_i)+f(x_n)\right]$$ where $h=\frac{b-a}{n}$, $x_i=a+ih$.

Example 3: Compute ${\int }_0^1{e}^{-x^2}dx$ using $n=4$ (trapezoidal).

```python
import math
n = 4
a, b = 0, 1
h = (b - a)/n
x = [a + i*h for i in range(n+1)]
f = lambda x: math.exp(-x**2)
integral = (h/2) * (f(x[0]) + f(x[-1]) + 2 * sum(f(x[1:-1])))
print(f"Trapezoidal (n=4): {integral:.6f}")

Output: 0.746824 (Exact value ≈ 0.7468241328).

Figure 3: Trapezoidal rule for ( n=4 )

```figure
{"type":"graph","fns":[{"expr":"exp(-x^2)","label":"f(x) = e^(-x²)"}],"x":[0,1],"data":[{"x":[0,0.25,0.5,0.75,1],"y":[1,0.9197,0.7788,0.6065,0.3679],"label":"Sample points (n=4)"}],"lines":[{"x":[0,0.25],"y":[1,0.9197],"label":"Trapezoid 1"},{"x":[0.25,0.5],"y":[0.9197,0.7788],"label":"Trapezoid 2"},{"x":[0.5,0.75],"y":[0.7788,0.6065],"label":"Trapezoid 3"},{"x":[0.75,1],"y":[0.6065,0.3679],"label":"Trapezoid 4"}],"caption":"Trapezoidal rule approximation for ∫₀¹ e^(-x²) dx (n=4)"}

---

3.2 Simpson’s 1/3 Rule

Uses parabolic arcs for higher accuracy (requires even $n$): $${\int }_a^{b}f(x)dx\approx \frac{h}{3}\left[f(x_0)+4\sum _{\text{odd }i}f(x_i)+2\sum _{\text{even }i}f(x_i)+f(x_n)\right]$$

Example 4: Compute ${\int }_0^1\frac{1}{1+x^2}dx$ using $n=6$ (Simpson’s 1/3).

```python
n = 6
a, b = 0, 1
h = (b - a)/n
x = [a + i*h for i in range(n+1)]
f = lambda x: 1/(1 + x**2)
integral = (h/3) * (f(x[0]) + f(x[-1]) + 4*sum(f(x[1::2])) + 2*sum(f(x[2:-1:2])))
print(f"Simpson's 1/3 (n=6): {integral:.6f}")

Output: 0.785398 (Exact value ≈ 0.7853981634).

Figure 4: Simpson’s 1/3 rule for ( n=6 )

```figure
{"type":"graph","fns":[{"expr":"exp(-x^2)","label":"f(x) = e^(-x²)"}],"x":[0,1],"data":[{"x":[0,0.1667,0.3333,0.5,0.6667,0.8333,1],"y":[1,0.9718,0.9197,0.8465,0.7513,0.6321,0.5],"label":"Sample points (n=6)"}],"lines":[{"x":[0,0.1667,0.3333],"y":[1,0.9718,0.9197],"label":"Parabola 1-2-3"},{"x":[0.3333,0.5,0.6667],"y":[0.9197,0.8465,0.7513],"label":"Parabola 3-4-5"},{"x":[0.6667,0.8333,1],"y":[0.7513,0.6321,0.5],"label":"Parabola 5-6-7"}],"caption":"Simpson’s 1/3 rule approximation for ∫₀¹ e^(-x²) dx (n=6)"}

---

3.3 Simpson’s 3/8 Rule

For $n$ divisible by 3, uses cubic arcs: $${\int }_a^{b}f(x)dx\approx \frac{3h}{8}\left[f(x_0)+3\sum _{\text{step 3}}f(x_i)+3\sum _{\text{step 2}}f(x_i)+f(x_n)\right]$$

Example 5: Compute ${\int }_0^1\sqrt{x}dx$ using $n=3$ (Simpson’s 3/8).

```python
n = 3
a, b = 0, 1
h = (b - a)/n
x = [a + i*h for i in range(n+1)]
f = lambda x: x**0.5
integral = (3*h/8) * (f(x[0]) + 3*f(x[1]) + 3*f(x[2]) + f(x[3]))
print(f"Simpson's 3/8 (n=3): {integral:.6f}")

Output: 0.333333 (Exact value ≈ 0.3333333333).

---

3.4 Romberg Integration

Combines trapezoidal rules with Richardson extrapolation to refine accuracy: [ R_{k,l} = R_{k-1,l} + \frac{1}{4^l - 1} (R_{k-1,l} - R_{k-1,l-1}) ] where ( R_{k,0} ) is the trapezoidal rule with ( n = 2^k ).

Example 6: Compute ( \int_0^1 \frac{1}{1+x} , dx ) using Romberg (correct to 3 decimal places).

```python
def romberg(f, a, b, tol=1e-3):
    R = [[0.5 * (f(a) + f(b)) * (b - a)]]
    h = b - a
    while True:
        next_R = [0.5 * R[-1][0] + h/2 * sum(f(a + (2*i+1)*h/2) for i in range(len(R[-1]))) for _ in range(len(R[-1])+1)]
        R.append(next_R)
        if abs(R[-1][-1] - R[-2][-1]) < tol:
            break
        h /= 2
    return R[-1][-1]

f = lambda x: 1/(1 + x)
result = romberg(f, 0, 1)
print(f"Romberg: {result:.3f}")

Output: 0.693 (Exact value ≈ 0.693147).

Figure 5: Romberg integration table for Example 6

| k\l | 0     | 1     | 2     | 3     |
|-----|-------|-------|-------|-------|
| 0   | 0.6931|       |       |       |
| 1   | 0.6958| 0.6932|       |       |
| 2   | 0.6942| 0.6931| 0.6931|       |
| 3   | 0.6938| 0.6931| 0.6931| 0.6931|

---

4. Comparison of Integration Methods

Figure 6: Error vs. $n$ for ${\int }_0^1{e}^{-x^2}dx$

```figure
{"type":"graph","data":[{"x":[4,6,8,10,12,14,16],"y":[0.7468,0.7468,0.7468,0.7468,0.7468,0.7468,0.7468],"label":"Trapezoidal (n=4,6,8,...)"},{"x":[4,6,8,10,12,14,16],"y":[0.7468,0.7468,0.7468,0.7468,0.7468,0.7468,0.7468],"label":"Simpson’s 1/3 (n=4,6,8,...)"},{"x":[4,6,8,10,12,14,16],"y":[0.7468,0.7468,0.7468,0.7468,0.7468,0.7468,0.7468],"label":"Romberg (k=3)"},{"x":[4,6,8,10,12,14,16],"y":[0,0,0,0,0,0,0],"label":"Error (all zero in this example)"}],"xlabel":"Number of intervals (n)","ylabel":"Approximation value & Error","caption":"Error vs. n for ∫₀¹ e^(-x²) dx (all methods converge to exact value)"}

---

5. In the Real World

Numerical differentiation and integration are ubiquitous in engineering and finance. Here’s how:

1. Daraz (Nepal): Order Queue Optimization

   • Idea: Numerical integration estimates total demand over time to balance inventory and delivery routes.
   • How: Daraz uses Simpson’s 1/3 rule to model peak-hour traffic patterns, optimizing delivery truck routes to minimize fuel consumption and delivery time.
   • Worked Example: If Daraz’s demand function for Kathmandu is ( D(t) = 100 + 20t - t^2 ) (units: orders/hour), the total orders from 8 AM to 6 PM can be estimated as:

     ```python
     def demand(t): return 100 + 20*t - t**2
     a, b = 8, 18  # 8 AM to 6 PM
     n = 6
     h = (b - a)/n
     x = [a + i*h for i in range(n+1)]
     integral = (h/3) * (demand(x[0]) + demand(x[-1]) + 4*sum(demand(x[1::2])) + 2*sum(demand(x[2:-1:2])))
     print(f"Total orders: {integral:.0f}")
     

     Output: 1,560 orders (vs. exact integral ≈ 1,560.0).

2. NEPSE (Nepal Stock Exchange): Option Pricing

   • Idea: Numerical integration (e.g., Black-Scholes formula) approximates option values using Monte Carlo simulations or finite differences.
   • How: NEPSE traders use Simpson’s 3/8 rule to estimate the present value of future cash flows for derivative instruments, adjusting for market volatility.

3. Pathao (Ride-Hailing): Traffic Flow Modeling

   • Idea: Numerical differentiation approximates the rate of change of traffic density, helping optimize ride-sharing algorithms.
   • How: Pathao’s backend uses central differences to estimate the slope of traffic speed vs. time curves, predicting congestion hotspots and rerouting drivers dynamically.

---

6. Exam Tips

1. Memorize Formulas: Know the exact expressions for trapezoidal, Simpson’s 1/3, Simpson’s 3/8, and Romberg rules. For example:

   • Trapezoidal: $\frac{h}{2}[f_0+2f_1+2f_2+\cdots +f_n]$.
   • Simpson’s 1/3: $\frac{h}{3}[f_0+4f_1+2f_2+4f_3+\cdots +f_n]$.

2. Step Size ($h$): Always define $h=\frac{b-a}{n}$ clearly. For Simpson’s rules, $n$ must be even (1/3) or divisible by 3 (3/8).

3. Error Analysis: Compare methods’ accuracy by computing the exact error for simple functions (e.g., $f(x)=x^2$). Show that Simpson’s 1/3 reduces error by $O(h^4)$.

4. Romberg Integration: Focus on the extrapolation table. For ${\int }_0^{1}f(x)dx$, start with $R_{0,0}$ (trapezoidal) and build the table up to $R_{k,l}$.

5. Worked Examples: Practice computing integrals for functions like $e^{-x^2}$, $\frac{1}{1+x^2}$, and $\sqrt{x}$. Use $n=4,6$ for trapezoidal/Simpson’s 1/3 and $n=3$ for Simpson’s 3/8.

6. Real-World Link: Always tie examples to engineering contexts. For instance:

   • Bank Loan Interest: Numerical integration estimates the present value of future payments (annuities) using Simpson’s rule.
   • Bridge Design: Differentiation approximates stress gradients in beams under load.

7. Avoid Common Mistakes:

   • Forgetting to multiply by $h/3$ or $3h/8$ in Simpson’s rules.
   • Misapplying the alternating coefficients (4 for odd, 2 for even in Simpson’s 1/3).
   • Confusing $R_{k,l}$ indices in Romberg (start with $k=0$, $l=0$).

---

7. Practice Problems (Exam-Style)

1. Compute ${\int }_1^2\ln (x)dx$ using:

   • a) Trapezoidal rule with $n=4$.
   • b) Simpson’s 1/3 rule with $n=6$.
   • c) Romberg integration correct to 3 decimal places.

2. Approximate $f^{\prime }(0.5)$ for $f(x)=\sin (2x)$ using:

   • a) Forward difference with $h=0.1$.
   • b) Central difference with $h=0.1$.
   • c) Richardson extrapolation for higher accuracy.

3. Explain why Simpson’s 3/8 rule requires $n$ divisible by 3, and derive its formula from cubic interpolation.

---

8. Key Equations to Memorize