1 Definition and basic ideas
Approximation error is the discrepancy between a true quantity and the value used to represent it. In practice, this gap arises whenever a calculation, model, or measurement replaces an exact result with a simpler or more manageable one. The concept is fundamental in numerical work because many problems cannot be solved exactly, or an exact treatment would be too costly to use.
1.1 Exact value and approximate value
The exact value is the ideal quantity being sought, such as a mathematical constant, an analytic solution, or a physically defined parameter. The approximate value is the substitute obtained from a formula, algorithm, measurement, or model. The usefulness of an approximation depends not only on how close it is to the exact value, but also on whether the residual difference is acceptable for the intended purpose.
1.2 Error as a difference
In its simplest form, approximation error is the difference between exact and approximate values. This difference may be positive or negative depending on convention, though its magnitude is often the main concern. Because the same approximate value can be good in one context and inadequate in another, error is usually interpreted relative to a target accuracy or application need.
1.3 Sources of approximation error
Approximation error can arise from simplifying assumptions, finite computational precision, truncated formulas, discretized domains, or imperfect data. It also appears when continuous objects are represented by finite sets of numbers or when complex functions are replaced by simpler surrogates. In many procedures, several sources act together and must be analyzed separately when possible.
1.4 Distinction from measurement error
Approximation error is not identical to measurement error. Measurement error concerns the inaccuracy of an observed or recorded value, while approximation error concerns the gap between an approximation and the exact quantity it is meant to represent. The two may interact, especially when measured data are used inside a computational model, but they describe different stages of uncertainty.
2 Types of approximation error
Different fields describe approximation error in terms suited to their methods. Some forms emphasize the size of the discrepancy, while others focus on how errors are generated or how they behave under computation.
2.1 Absolute error
Absolute error is the numerical difference between the exact value and the approximate value, usually taken in magnitude. It is easy to interpret because it uses the same units as the quantity being approximated. This measure is most informative when the scale of the quantity is fixed or when a direct tolerance is specified.
2.2 Relative error
Relative error compares the absolute error with the size of the exact value or a reference value. It is useful when quantities vary widely in scale, since the same absolute difference may be minor for a large quantity but significant for a small one. Relative error helps indicate proportional accuracy.
2.3 Percentage error
Percentage error is relative error expressed as a percentage. It is commonly used in applied contexts because it gives an intuitive sense of how large the discrepancy is compared with the quantity itself. Like relative error, it is most meaningful when the reference value is nonzero.
2.4 Forward and backward error
Forward error measures the difference between the computed output and the true output of a problem. Backward error asks how much the input would need to be changed so that the computed output becomes exact for a nearby problem. This distinction is especially important in numerical analysis, where an algorithm may produce a good result even if direct error is not small, provided the underlying perturbation is tiny.
2.5 Truncation error
Truncation error occurs when an infinite or exact process is cut off after finitely many steps. Examples include replacing an infinite series with a partial sum or approximating derivatives with finite formulas. The size of truncation error often depends on how much of the neglected tail remains and on the smoothness of the underlying function.
2.6 Discretization error
Discretization error results from replacing a continuous problem with a discrete one. This happens when a domain is sampled at finitely many points, such as on a grid or mesh. The error generally decreases as the discretization becomes finer, though this improvement can increase computational cost.
3 Error measures and notation
To analyze approximation quality, mathematics uses standardized measures and symbols. These tools make it possible to compare methods, prove convergence, and derive limits on performance.
3.1 Norm-based error measures
Norms provide a way to measure the size of an error vector or function. Common choices include the maximum norm, which captures the largest pointwise deviation, and integral-based norms, which summarize average or total discrepancy over a domain. The choice of norm depends on which features of the error matter most for the application.
3.2 Pointwise versus global error
Pointwise error describes the discrepancy at a particular location or instance. Global error aggregates errors over an entire interval, region, or data set. A method may have small pointwise error almost everywhere but still show a larger global error if the deviations accumulate across many points.
3.3 Error bounds
Error bounds provide guaranteed limits on how large an approximation error can be. They are central to assessing reliability, since they transform an approximate result into an estimate with a known uncertainty range. Bounds may be derived from theoretical analysis, inequality arguments, or numerical properties of the method.
3.3.1 Upper bounds
An upper bound states that the error does not exceed a specified quantity. Such bounds are valuable for safety and verification because they establish a worst-case limit. In practice, upper bounds may be conservative, but they still offer a rigorous criterion for acceptability.
3.3.2 Asymptotic bounds
Asymptotic bounds describe how error behaves as some parameter grows or shrinks, often the step size in a numerical method. They identify the dominant rate at which error declines, rather than an exact finite-size estimate. These bounds are especially useful for comparing methods by their long-term refinement behavior.
3.4 Error estimates
Error estimates are calculated or inferred values intended to approximate the true error. Unlike strict bounds, they may not guarantee enclosure, but they often provide practical guidance on accuracy. Many algorithms use estimates adaptively to decide whether further refinement is needed.
4 Approximation error in numerical methods
Numerical methods routinely replace exact analytic operations with finite procedures. Approximation error is therefore built into their design and must be studied to ensure dependable results.
4.1 Polynomial approximation
Polynomial approximation represents a function by a polynomial chosen to match selected features of the original function. The approximation error depends on the degree of the polynomial, the smoothness of the function, and the region where the approximation is used. Polynomials are widely used because they are easy to evaluate and differentiate.
4.2 Interpolation error
Interpolation error is the difference between a function and the interpolant that matches it at chosen data points. It depends on the spacing of nodes and the behavior of the function between them. Interpolation can be highly accurate when the function is smooth and the sampling points are suitably arranged.
4.2.1 Linear interpolation
Linear interpolation uses straight-line segments between known values. Its error is typically small when the function changes gradually over each interval, but it can become noticeable when curvature is strong. The method is simple and fast, which makes it a common baseline technique.
4.2.2 Higher-order interpolation
Higher-order interpolation uses polynomials or other smooth forms with more degrees of freedom. It can achieve much smaller error than linear interpolation, especially for smooth functions. However, more elaborate interpolants may be sensitive to node placement or may exhibit oscillatory behavior in some settings.
4.3 Numerical differentiation
Numerical differentiation estimates derivatives from nearby values rather than from exact calculus formulas. Since differentiation amplifies variation, it often magnifies approximation error and noise. The choice of step size is crucial: too large gives poor resolution, while too small can intensify round-off effects.
4.4 Numerical integration
Numerical integration approximates an integral by summing values at discrete points. The error depends on the smoothness of the integrand and the quadrature rule used. Higher-order methods can reduce error rapidly for well-behaved functions, though singularities or rapid oscillations may require special treatment.
4.5 Solution of differential equations
Approximating differential equations usually involves discretizing time, space, or both. The resulting error reflects the gap between the exact solution and the computed discrete solution. Analysis of this error is essential in simulations that model physical, biological, or engineered systems.
4.5.1 Initial value problems
In initial value problems, the solution evolves from a specified starting condition. Approximation error can accumulate as the solution is advanced step by step, making long-time behavior particularly important. Method choice often balances local accuracy against cumulative drift.
4.5.2 Boundary value problems
Boundary value problems impose conditions at multiple points, often at the ends of an interval or along the boundary of a region. Their approximation error depends strongly on how the domain is discretized and how the boundary conditions are enforced. Fine meshes and carefully designed schemes are commonly used to improve fidelity.
5 Convergence and accuracy
Convergence describes whether an approximation gets closer to the exact quantity as the method is refined. Accuracy concerns how small the error is for a given level of refinement.
5.1 Convergence of approximations
A convergent approximation method produces results that approach the true solution as resolution increases or step size decreases. Convergence does not necessarily mean that every intermediate approximation is highly accurate, only that the trend is correct in the limit. The rate of convergence is often as important as the fact of convergence itself.
5.2 Order of accuracy
Order of accuracy describes how rapidly the error decreases as the approximation parameter is refined. A higher order usually indicates a faster reduction in error for smooth problems. This concept helps compare methods with different computational costs and levels of sophistication.
5.3 Consistency and stability
Consistency means that a discrete method correctly reflects the continuous problem as the step size tends to zero. Stability means that small perturbations do not grow uncontrollably during computation. Together, these properties help determine whether approximation error remains controlled and whether convergence is likely.
5.4 Refinement and mesh size effects
Refinement improves resolution by using smaller step sizes or denser meshes. As the mesh becomes finer, discretization and interpolation errors often shrink, although at increased computational expense. In some cases, overly fine refinement can expose round-off limitations or create inefficient computations.
6 Error propagation
Errors rarely remain isolated. They can spread through formulas, algorithms, and iterative procedures, sometimes becoming much larger than the original perturbation.
6.1 Propagation through arithmetic operations
When approximate numbers are added, multiplied, or divided, their errors combine according to the structure of the operation. Some calculations are relatively forgiving, while others can cause cancellation or amplification. Understanding these effects is essential for designing reliable algorithms.
6.2 Sensitivity to input perturbations
Sensitivity describes how strongly an output changes when inputs are altered slightly. A highly sensitive problem can produce large output errors even from tiny input inaccuracies. This property is distinct from algorithmic quality, since the problem itself may be inherently delicate.
6.3 Condition number
The condition number measures how sensitive a problem is to perturbations in its input. Large condition numbers indicate that small changes in data may lead to large changes in the result. Conditioning is a property of the mathematical problem, not just of the chosen numerical method.
6.4 Round-off error interaction
Round-off error arises from finite precision representation of numbers in computers. It can interact with approximation error in subtle ways, sometimes offsetting it and sometimes worsening it. In long computations, round-off may accumulate or interfere with convergence, especially in unstable algorithms.
7 Modeling and approximation theory
Approximation is not limited to numerical algorithms; it also underlies how functions and phenomena are represented mathematically. Approximation theory studies how well simpler objects can stand in for more complex ones.
7.1 Approximate representations of functions
Functions may be represented by formulas, basis expansions, tabulated values, or surrogate models. Each representation introduces error by capturing only selected features of the original behavior. The quality of the representation depends on the function class, the basis used, and the intended domain of application.
7.2 Least-squares approximation
Least-squares approximation chooses parameters that minimize the sum of squared deviations between a model and data or between a function and a simplified representation. It is widely used because it yields stable, computationally tractable solutions. The resulting approximation error reflects both model flexibility and data variability.
7.3 Series truncation
Series truncation replaces an infinite expansion with a finite number of terms. Common examples include power series, Fourier series, and orthogonal expansions. The truncation error depends on how quickly the omitted terms decay and whether the series converges uniformly or pointwise.
7.4 Finite-dimensional approximations
Finite-dimensional approximations restrict an infinite-dimensional problem to a space with finitely many degrees of freedom. This approach is central to methods such as finite elements, spectral schemes, and reduced-order models. The main challenge is balancing simplicity against the accuracy needed to represent the original system faithfully.
8 Applications
Approximation error matters wherever exact computation is unavailable or unnecessary. Its analysis supports decision-making, design, and interpretation across many technical fields.
8.1 Scientific computation
In scientific computation, approximation error helps assess the trustworthiness of simulations, numerical solvers, and computational experiments. Researchers use error analysis to choose step sizes, compare algorithms, and determine whether results are sufficiently accurate for scientific conclusions. The concept is especially important in large-scale models where exact solutions are out of reach.
8.2 Engineering analysis
Engineering applications rely on approximate methods to evaluate stress, flow, heat transfer, circuits, and control systems. Error analysis guides design margins and supports efficient use of computational resources. By quantifying uncertainty in calculated values, engineers can judge whether a model is precise enough for safe and effective use.
8.3 Data fitting and regression
In data fitting and regression, approximation error measures how well a model represents observed data. A small error may indicate a good fit, though it must be interpreted alongside generalization and model complexity. The balance between underfitting and overfitting is often understood through the behavior of approximation error on data not used to build the model.
8.4 Simulation and forecasting
Simulation and forecasting use approximate models to project system behavior over time. Approximation error can influence both short-term outputs and long-range predictions, especially when models are iterated repeatedly. Careful error analysis helps identify which uncertainties arise from the model structure, which from numerical methods, and which from the limits of available information.