Numerical Differentiation – Forward, Backward, Central, Error Analysis

1. Introduction

Numerical differentiation is the process of estimating the derivative of a function $f(x)$ when only discrete values of $f$ are available. It is indispensable in data analysis, engineering simulations, and real‑time control systems where analytical derivatives are unavailable or expensive to compute.

The basic idea is to replace the limit definition

$$f^{\prime }(x)=\underset{h\to 0}{\lim }\frac{f(x+h)-f(x)}{h}$$

by a finite difference that uses a small but finite step $h$.

2. Forward, Backward, and Central Difference Formulas

The central difference is generally preferred because its truncation error is two orders higher.

The figure shows the sine function and the points used in the central difference formula.

3. Error Sources

The total error is approximately the sum of truncation and round‑off errors. An optimal step size $h_{\text{opt}}$ balances the two:

$$h_{\text{opt}}\approx \sqrt{\frac{2{\epsilon }_{\text{mach}}}{|f^{\prime \prime }(x)|}}$$

4. Worked Example – Analytical Derivative of $\sin x$

Compute $f^{\prime }(x)$ for $f(x)=\sin x$ at $x=\pi /4$ using central difference with $h=0.1$.

Exact derivative: $f^{\prime }(\pi /4)=\cos (\pi /4)=\frac{\sqrt{2}}{2}\approx 0.70710678$.

Central difference:

$$f_{\text{num}}^{\prime }\approx \frac{\sin (\pi /4+0.1)-\sin (\pi /4-0.1)}{0.2}$$

Numerical values:

$$\sin (0.785398+0.1)=\sin (0.885398)=0.774167,\sin (0.785398-0.1)=\sin (0.685398)=0.632456$$

$$f_{\text{num}}^{\prime }\approx \frac{0.774167-0.632456}{0.2}=0.706555$$

Error: $|0.706555-0.70710678|=5.52\times 10^{-4}$.

The truncation error is $O(h^2)=0.01^2=10^{-4}$, close to the observed error.

5. Worked Example – Real‑World Application: Speed Estimation from GPS

A Pathao delivery bike records its position every second.
Positions (in meters) at times $t=0,1,2$ s:

$$x_0=0,x_1=10,x_2=25$$

We want the instantaneous speed at $t=1$ s.

Using central difference:

$$v(1)\approx \frac{x_2-x_0}{2\Delta t}=\frac{25-0}{2\times 1}=12.5\text{m/s}$$

If the actual speed (from a high‑precision sensor) is $15\text{m/s}$, the error is $2.5\text{m/s}$ (≈16.7 %).

The error arises from both truncation (due to non‑linear acceleration) and measurement noise.

6. Algorithm for Numerical Differentiation

flowchart TD
    "Start" --> "Input f(x), x, h"
    "Choose scheme" --> "If central then"
    "Compute f(x+h)" --> "Compute f(x-h)"
    "Compute derivative" --> "Output f'(x)"
    "If forward/backward then"
    "Compute f(x±h)" --> "Compute derivative"
    "Output f'(x)"
    "End"

The algorithm is straightforward: evaluate the function at the required points and apply the chosen finite‑difference formula.

7. C Implementation

#include <stdio.h>
#include <math.h>

/* Forward difference */
double forward_diff(double (*f)(double), double x, double h) {
    return (f(x + h) - f(x)) / h;
}

/* Backward difference */
double backward_diff(double (*f)(double), double x, double h) {
    return (f(x) - f(x - h)) / h;
}

/* Central difference */
double central_diff(double (*f)(double), double x, double h) {
    return (f(x + h) - f(x - h)) / (2.0 * h);
}

/* Example function: sin(x) */
double func(double x) {
    return sin(x);
}

int main(void) {
    double x = M_PI / 4.0;
    double h = 0.1;
    printf("Forward: %f\n", forward_diff(func, x, h));
    printf("Backward: %f\n", backward_diff(func, x, h));
    printf("Central: %f\n", central_diff(func, x, h));
    return 0;
}

Compile with gcc -std=c11 -O2 -lm diff.c -o diff and run.

8. Error Analysis – Truncation vs Round‑off

The truncation error for central difference is

$$E_T=\frac{h^2}{6}{f}^{\prime \prime \prime }(x)+O(h^4)$$

The round‑off error, assuming machine epsilon ${\epsilon }_{\text{mach}}$, is

$$E_R\approx \frac{{\epsilon }_{\text{mach}}}{h}|f(x)|$$

Total error: $E\approx E_T+E_R$.

Plotting $E$ vs $h$ shows a U‑shaped curve: decreasing $h$ reduces truncation error until round‑off dominates.

The minimum of the black curve gives the optimal $h$.

9. Applications

In the real world

• Pathao delivery bikes: GPS positions sampled every second are differentiated to compute instantaneous speed, enabling dynamic routing and driver performance monitoring.
• eSewa transaction monitoring: The number of transactions per minute is recorded; a forward difference gives the rate of change, helping detect sudden surges or fraud.
• NEPSE price charts: Daily closing prices are differentiated to estimate the daily return rate, used by traders for momentum strategies.

10. Summary

Numerical differentiation transforms discrete data into derivative estimates using finite differences. The central difference formula offers the best accuracy for smooth functions, but practical use requires balancing truncation and round‑off errors by selecting an appropriate step size. The concepts are directly applicable to everyday technologies such as GPS navigation, financial analytics, and health monitoring systems.

In the real world

1. Pathao – GPS coordinates of delivery vehicles are differentiated to compute speed and acceleration, enabling real‑time traffic management.
2. eSewa – Transaction counts per minute are differentiated to detect abnormal spikes, indicating potential security incidents.
3. NEPSE – Daily closing prices are differentiated to calculate daily returns, informing trading decisions.

Exam tip

• Memorize the forward, backward, and central difference formulas and their truncation error orders.
• Be able to derive the truncation error using Taylor series.
• Practice computing derivatives for simple functions and for real‑world data sets.
• Understand how to choose an optimal step size by balancing truncation and round‑off errors.
• For coding questions, write clear, modular C functions for each difference scheme.

Device that records position over time, derivative gives velocity. (Image: Siarhei Besarab, CC BY-SA 4.0, via Wikimedia Commons)