1 Foundations
Approximation theory studies how a complicated object can be replaced by a simpler one while preserving essential features. The simpler model may be a polynomial, trigonometric sum, spline, rational function, or a finite-dimensional element of a function space. The central questions are how close the approximation is, how it is constructed, and what limits govern its accuracy.
1.1 Basic concepts of approximation
At its core, approximation compares a target object with a surrogate that is easier to analyze, compute, or store. The target may be a function, data set, operator, or geometric shape. The approximation is judged by a chosen notion of distance or error, which depends on the context and the intended use.
1.1.1 Error and deviation
The error is the difference between the exact object and its approximation. It may be measured point by point, in an integral sense, or by a worst-case criterion. Deviation describes how far the approximation departs from the original and often determines whether the simplified model is acceptable for a given task.
1.1.2 Exact versus approximate representation
Exact representation gives a complete description without loss of information, while approximate representation retains only selected features. In many problems, exact formulas are unavailable or impractical, so approximation becomes the main tool. The trade-off is that the simpler model may introduce error, but it can often be evaluated or manipulated far more efficiently.
1.2 Historical development
Approximation theory grew out of classical analysis, where mathematicians sought to represent functions by algebraic or trigonometric expressions. Over time, the field expanded into a broad discipline linked to computation, numerical methods, and functional analysis. Its development has been driven by both theoretical questions and practical demands.
1.2.1 Classical analysis roots
Early work on approximation is closely tied to polynomial interpolation, trigonometric series, and the study of smooth functions. Results such as the approximation of continuous functions by polynomials established that many complicated functions can be approximated arbitrarily well under suitable conditions. These ideas helped form the foundation of modern analysis.
1.2.2 Modern computational influence
With the rise of digital computation, approximation theory became essential for algorithms that must work with finite precision. Numerical methods for simulation, modeling, and data processing rely on approximating continuous objects by finite representations. This computational perspective broadened the subject beyond classical formulas to include stability, efficiency, and error control.
1.3 Goals and applications
The field aims to understand both what can be approximated and how accurately it can be done. It also seeks methods that are efficient enough for practical use. These goals connect approximation theory to science, engineering, statistics, and scientific computing.
1.3.1 Simplification of complex functions
A major purpose of approximation is to replace a complicated function by one with a simpler structure. This can make qualitative analysis easier and reveal underlying behavior such as trends, oscillation, or smoothness. Simplified representations also support theoretical proofs and symbolic manipulation.
1.3.2 Numerical computation and modeling
In computation, approximations are used to evaluate integrals, solve differential equations, interpolate data, and compress information. Models in applied science often begin with idealized formulas and then use approximation to make them numerically tractable. The quality of the approximation directly affects the reliability of the final result.
2 Metrics and measures of approximation quality
The study of approximation depends on how closeness is measured. Different metrics emphasize different aspects of the discrepancy, such as the largest error, average error, or behavior over specific subsets. A suitable measure is chosen according to the structure of the problem and the desired interpretation.
2.1 Norms and distances
Norms provide a systematic way to assign size to errors and to compare functions or vectors. Distances derived from norms make it possible to speak precisely about convergence and optimality. In approximation theory, the choice of norm strongly influences both results and methods.
2.1.1 Uniform norm
The uniform norm measures the maximum absolute deviation over a domain. It is especially useful when one wants control at every point, not just on average. Because it captures the worst-case error, it plays a central role in best approximation and interpolation.
2.1.2 Lp norms
Lp norms measure error using an integral of the p-th power of the absolute deviation. They are suited to problems where average behavior is more important than pointwise accuracy. Different values of p lead to different geometries and often different approximation properties.
2.2 Convergence concepts
Convergence describes the behavior of a sequence of approximations as it improves. Various notions of convergence capture different strengths of agreement between the approximants and the target. The choice of concept depends on the space in which the approximation is studied.
2.2.1 Pointwise convergence
Pointwise convergence means that the approximating sequence converges separately at each point. It is a weak form of convergence and may hold even when global control is poor. Although useful, it does not by itself guarantee uniform or stable approximation.
2.2.2 Uniform convergence
Uniform convergence requires the error to become small across the whole domain at once. This form of convergence is stronger than pointwise convergence and preserves many important properties of the limit. It is especially important in analysis and in approximation by polynomials or trigonometric functions.
2.2.3 Mean convergence
Mean convergence measures improvement in an averaged sense, often using an Lp norm. It is weaker than uniform convergence but stronger than mere pointwise agreement in many settings. This concept is common in Fourier analysis, statistics, and numerical approximation.
2.3 Error bounds
Error bounds quantify how large an approximation error can be. They may be exact, explicit, or asymptotic, and they often reveal how the quality of approximation depends on smoothness or other structural features. Such bounds are central to both theory and computation.
2.3.1 Absolute error
Absolute error is the magnitude of the difference between the exact value and the approximation. It is a direct and intuitive measure, especially when the scale of the quantity is fixed. In many applications, an absolute tolerance determines whether a result is sufficiently accurate.
2.3.2 Relative error
Relative error compares the absolute error with the size of the exact quantity. It is useful when values vary widely in magnitude, since a small absolute error may still be significant for a small target. Relative error is common in numerical analysis and scientific computing.
2.3.3 Asymptotic estimates
Asymptotic estimates describe how error behaves as the approximation becomes increasingly refined. They often reveal rates of convergence in terms of the number of terms, degree, or discretization scale. Such estimates are valuable for comparing methods and predicting performance.
3 Polynomial approximation
Polynomial approximation is one of the most important and well-studied parts of the subject. Polynomials are simple to evaluate, differentiate, and integrate, which makes them useful in both theory and computation. Many fundamental results show that polynomials can approximate a wide class of functions very effectively.
3.1 Algebraic polynomials
Algebraic polynomials are finite sums of powers of a variable with constant coefficients. Their simplicity makes them a natural basis for approximation on bounded intervals. They are central to classical theorems and to numerical algorithms.
3.1.1 Weierstrass approximation theorem
The Weierstrass approximation theorem states that every continuous function on a closed interval can be uniformly approximated by polynomials. This result is a cornerstone of approximation theory because it guarantees the density of polynomials in a broad function class. It also motivates the search for constructive approximation methods.
3.1.2 Bernstein polynomials
Bernstein polynomials provide an explicit constructive proof of polynomial approximation on an interval. They are built from values of the target function at equally spaced points and have strong positivity properties. Their shape-preserving behavior makes them useful in analysis and in geometric design.
3.2 Best polynomial approximation
Best approximation asks for the polynomial that minimizes the error under a chosen norm. This turns approximation into an optimization problem and leads to deep connections with geometry and convexity. The answer depends on both the function and the metric.
3.2.1 Minimax approximation
Minimax approximation minimizes the maximum error over the domain. It is also called uniform best approximation and often produces nearly equioscillating error patterns. This approach is important when the worst-case deviation must be controlled.
3.2.2 Least-squares approximation
Least-squares approximation minimizes the average squared error. It is widely used because it is computationally convenient and naturally compatible with data fitting. In many applications, it provides a stable compromise between accuracy and simplicity.
3.3 Orthogonal polynomials
Orthogonal polynomials arise when polynomials are organized according to orthogonality with respect to an inner product or weight. They provide efficient bases for expansion and approximation. Their recurrence relations and extremal properties make them especially useful in numerical methods.
3.3.1 Legendre polynomials
Legendre polynomials are orthogonal on a finite interval with respect to a constant weight. They appear in polynomial expansions, spectral methods, and classical physics. Their smooth behavior and recurrence structure support efficient computation.
3.3.2 Chebyshev polynomials
Chebyshev polynomials are closely associated with near-optimal uniform approximation. They minimize certain extremal quantities and are widely used in numerical algorithms. Their roots and oscillatory patterns make them particularly effective for interpolation and approximation.
3.4 Polynomial interpolation
Polynomial interpolation constructs a polynomial that matches given function values at selected points. It is a basic technique for representing data and building approximants from discrete information. The choice of nodes strongly affects accuracy and stability.
3.4.1 Lagrange interpolation
Lagrange interpolation expresses the interpolating polynomial as a sum of basis polynomials that isolate each node value. This formula is conceptually simple and useful for theoretical work. It also clarifies how each data point contributes to the final interpolant.
3.4.2 Newton interpolation
Newton interpolation uses divided differences and builds the interpolating polynomial incrementally. This form is convenient when points are added one by one or when a nested representation is desired. It is often preferred in computational settings.
3.4.3 Interpolation error
Interpolation error measures the difference between the target function and its interpolating polynomial. It depends on the spacing of the nodes and the smoothness of the function. Understanding this error is essential for selecting reliable interpolation schemes.
4 Approximation by trigonometric functions
Trigonometric approximation is suited to periodic phenomena and oscillatory behavior. Sines and cosines provide natural building blocks for representing repeating patterns. This area is closely tied to Fourier analysis and signal processing.
4.1 Fourier series
Fourier series expand a periodic function into an infinite sum of trigonometric terms. They reveal frequency content and often simplify analysis by converting problems into coefficient relations. The series plays a foundational role in both pure and applied mathematics.
4.1.1 Fourier coefficients
Fourier coefficients measure how strongly each frequency contributes to the function. They are obtained by projecting the function onto sine and cosine components. The decay of these coefficients often reflects smoothness and regularity.
4.1.2 Convergence of Fourier series
The convergence of a Fourier series may occur pointwise, in mean, or uniformly under additional hypotheses. The behavior near discontinuities can involve overshoot and slower convergence. This topic illustrates how approximation quality depends on both function structure and the chosen mode of convergence.
4.2 Trigonometric interpolation
Trigonometric interpolation constructs a periodic trigonometric polynomial that matches a function at selected points. It is the periodic analogue of algebraic interpolation and is widely used in computation. The method is particularly efficient when data are sampled on an evenly spaced grid.
4.2.1 Periodic function approximation
Periodic function approximation is useful whenever the underlying phenomenon repeats over time or space. Trigonometric polynomials often approximate such functions very effectively because they align with the geometry of periodicity. They also support fast numerical evaluation.
4.2.2 Discrete Fourier methods
Discrete Fourier methods compute approximate frequency information from sampled data. They underpin many algorithms in signal analysis, spectral computation, and numerical simulation. Their efficiency has made them one of the most important tools in modern applied mathematics.
4.3 Harmonic analysis connections
Approximation by trigonometric functions is deeply connected with harmonic analysis. This connection links the approximation of functions to the study of frequencies, transforms, and representation in function spaces. It also provides a powerful language for understanding regularity.
4.3.1 Approximation in function spaces
Function spaces organize approximants and target functions according to norms and structural properties. Harmonic analysis provides tools for studying how well functions can be represented by finite frequency sums. This viewpoint is central to many advanced approximation results.
4.3.2 Frequency-domain interpretation
The frequency-domain interpretation describes a function through its spectral components rather than its pointwise values. Approximation then becomes the process of retaining the most relevant frequencies and discarding the rest. This perspective is essential in signal processing and computational analysis.
5 Approximation in normed and inner product spaces
Abstract spaces provide a general framework in which approximation can be studied beyond specific formulas. Normed and inner product spaces capture geometric ideas such as distance, angle, and projection. These concepts unify many approximation problems.
5.1 Linear approximation
Linear approximation uses elements from a subspace to model a given object. The central idea is to replace a complicated element with a linear combination of simpler basis elements. This is a natural setting for projection methods and numerical schemes.
5.1.1 Projections onto subspaces
Projection onto a subspace selects the closest element in that subspace to the target, according to a given norm or inner product. Projections are fundamental because they formalize the process of best linear approximation. They also appear in least-squares fitting and spectral methods.
5.1.2 Projection theorems
Projection theorems guarantee the existence and sometimes uniqueness of best approximants under suitable conditions. In Hilbert spaces, such results are especially strong and elegant. They provide the theoretical basis for many practical algorithms.
5.2 Best approximation in Banach spaces
Banach spaces extend the study of approximation to complete normed spaces. Best approximation in these spaces can be more subtle than in Euclidean settings because orthogonality may not be available. Nonetheless, the framework is broad enough to include many function spaces of interest.
5.2.1 Existence and uniqueness
Existence and uniqueness of best approximations depend on the geometry of the space and the subset used for approximation. In some settings, a closest point always exists, while in others it may not be unique. These questions are central to understanding the reliability of approximation schemes.
5.2.2 Convexity methods
Convexity methods exploit the geometric properties of convex sets to study best approximation. Since convex subsets often behave well under minimization, they support general existence and stability results. This approach is widely used in optimization and functional analysis.
5.3 Hilbert space methods
Hilbert spaces are distinguished by their inner product structure, which enables geometric reasoning similar to Euclidean geometry. Approximation in these spaces often reduces to orthogonal projection. This makes them especially convenient for analysis and computation.
5.3.1 Orthogonality principles
Orthogonality principles characterize best approximants by requiring the residual to be orthogonal to the approximating subspace. This condition translates optimization into a geometric statement. It is one of the most powerful ideas in linear approximation.
5.3.2 Least-squares projection
Least-squares projection is the Hilbert space formulation of minimizing squared error. It is widely used in data fitting, numerical linear algebra, and spectral methods. The method combines theoretical clarity with computational practicality.
6 Interpolation and spline methods
Interpolation and splines are practical tools for constructing approximations from discrete information. Interpolation forces agreement at selected points, while splines combine local polynomial pieces to improve flexibility and stability. These methods are widely used in graphics, modeling, and numerical computation.
6.1 Interpolating functions
Interpolating functions match prescribed values, and sometimes derivative values, at chosen nodes. They are used to reconstruct a smooth curve or surface from sampled data. Their success depends heavily on node placement and the degree of smoothness required.
6.1.1 Node selection
Node selection influences both accuracy and numerical stability. Poorly chosen nodes can lead to oscillations or large errors, while well-chosen nodes improve performance. This issue is especially important in high-degree polynomial interpolation.
6.1.2 Interpolation polynomials
Interpolation polynomials are the simplest interpolating functions and are built to pass exactly through the data points. They are easy to define but can become unstable for large numbers of nodes. Their behavior motivates the use of more flexible piecewise methods.
6.2 Spline functions
Splines are piecewise polynomial functions joined smoothly at specified points. They offer a compromise between global simplicity and local flexibility. Because they can adapt to data while maintaining smoothness, they are a standard tool in approximation theory.
6.2.1 Piecewise polynomials
Piecewise polynomials break the domain into intervals and use a separate polynomial on each one. This local structure improves control over shape and reduces the risk of global oscillation. It also makes splines suitable for large data sets.
6.2.2 Cubic splines
Cubic splines are among the most common spline functions because they provide a good balance of smoothness and computational efficiency. They typically match function values and first and second derivatives at the joins. This makes them useful for curve fitting and numerical modeling.
6.3 Approximation properties of splines
Splines are valued not only for their construction but also for their approximation quality. Their local support and smoothness conditions can yield strong error bounds. These properties make them effective in both theory and applications.
6.3.1 Smoothness conditions
Smoothness conditions specify how derivatives must match across the boundaries of spline pieces. Greater smoothness usually improves the quality of the approximation and the visual appearance of the curve. These conditions also affect the algebraic structure of spline spaces.
6.3.2 Error estimates
Error estimates quantify how closely a spline approximates the target function. They often depend on the size of the intervals and the smoothness of the function being approximated. Such estimates guide the design of adaptive and efficient spline methods.
7 Rational and nonlinear approximation
Rational and nonlinear methods extend approximation beyond polynomials and linear subspaces. They are especially useful when the target function has sharp features, poles, or other structures that are hard to capture with polynomials alone. These methods can achieve high accuracy with relatively few parameters.
7.1 Rational functions
Rational functions are quotients of polynomials and can represent more complex behavior than polynomial approximants. They are particularly effective near singularities and in modeling functions with rapid variation. Their flexibility makes them important in theoretical and computational settings.
7.1.1 Padé approximation
Padé approximation replaces a function by a rational function whose power series matches as many coefficients as possible. It often provides better local behavior than truncating a series. This method is widely used in analysis, physics, and numerical computation.
7.1.2 Rational interpolation
Rational interpolation constructs a rational function that matches prescribed data values. It can offer improved accuracy compared with polynomial interpolation, especially when the target has poles or steep gradients. Stability, however, can be more delicate than in the polynomial case.
7.2 Nonlinear approximation
Nonlinear approximation allows the approximating family to depend on the target function in a nonlinear way. This can produce more efficient representations than fixed linear subspaces. It is especially important in sparse modeling and compression.
7.2.1 Sparse representation
Sparse representation aims to describe a function or signal using only a small number of significant terms. The selected terms may come from a large dictionary, but most coefficients are set to zero or ignored. This strategy is useful in data compression and modern signal analysis.
7.2.2 Best n-term approximation
Best n-term approximation selects the optimal set of n building blocks from a larger collection. It often gives better error rates than approximating within a fixed n-dimensional subspace. The concept is central to nonlinear approximation theory.
7.3 Approximation of singular behavior
Some functions contain singularities, sharp corners, or rapid transitions that challenge standard approximation schemes. Specialized methods seek to capture these features more efficiently. The analysis of singular behavior is important in both theory and numerical practice.
7.3.1 Endpoint singularities
Endpoint singularities occur near the boundary of an interval, where the function may fail to be smooth. Standard polynomial methods can struggle in such regions, while rational or adaptive techniques may perform better. Understanding these singularities is key to accurate modeling.
7.3.2 Analytic continuation ideas
Analytic continuation extends the domain of a function beyond the region where it is initially given. In approximation, this idea helps explain why certain rational methods can achieve strong performance. It also connects approximation with complex analysis.
8 Approximation in functional spaces
Functional spaces provide a broad setting for understanding approximation in terms of structure, density, and smoothness. Different spaces emphasize different kinds of regularity and error measurement. This framework is central to modern analysis.
8.1 Approximation in C[a,b]
The space C[a,b] consists of continuous functions on a closed interval. It is a natural setting for uniform approximation because continuity and compactness support strong density results. Many classical approximation theorems are formulated here.
8.1.1 Dense subspaces
A dense subspace is one whose elements can approximate every function in the larger space arbitrarily well. Dense subspaces are important because they identify simple building blocks that are theoretically sufficient. Polynomials are a classical example in C[a,b].
8.1.2 Uniform approximation
Uniform approximation in C[a,b] requires the maximum error to be small across the interval. This is one of the strongest and most intuitive forms of approximation. It is especially useful when precise control is needed everywhere, not merely on average.
8.2 Approximation in Lp spaces
Lp spaces measure functions by integrable powers of their magnitude. They are central in analysis because they capture both regularity and average size. Approximation in these spaces is often aligned with integral-based error criteria.
8.2.1 Density results
Density results in Lp spaces show that simple functions, polynomials, or other structured families can approximate broad classes of functions. Such results justify the use of finite expansions and numerical discretizations. They are often essential for existence proofs and algorithm design.
8.2.2 Integral-based error measures
Integral-based error measures assess approximation by averaging deviations over the domain. They are well suited to applications where localized errors are less important than overall fidelity. These measures are common in statistical fitting and numerical integration.
8.3 Sobolev space approximation
Sobolev spaces incorporate both a function and its derivatives into the notion of regularity. This makes them especially useful in partial differential equations and variational methods. Approximation in Sobolev spaces often reflects smoothness in a more refined way than ordinary normed spaces.
8.3.1 Smoothness and regularity
Smoothness and regularity determine how well a function can be approximated and how rapidly the error decreases. Functions with more derivatives or stronger regularity typically admit better approximations. Sobolev spaces provide a framework for quantifying this relationship.
8.3.2 Weak derivatives
Weak derivatives extend the concept of differentiation to functions that may not be classically differentiable. They allow approximation theory to treat broader classes of functions encountered in applications. This concept is fundamental in modern analysis and PDE theory.
9 Quantitative theory
Quantitative approximation theory seeks explicit rates and inequalities that describe how approximation improves with increased complexity. It connects smoothness properties of functions with the decay of approximation error. This part of the subject is concerned with precise estimates rather than only existence.
9.1 Moduli of continuity
The modulus of continuity measures how rapidly a function can change over small distances. It provides a quantitative way to describe smoothness without requiring derivatives. Such measures are often used in direct approximation estimates.
9.1.1 Smoothness measures
Smoothness measures capture the degree of regularity of a function in a numerical form. They help predict how well the function can be approximated by polynomials, splines, or trigonometric sums. Stronger smoothness usually leads to faster convergence.
9.1.2 Jackson-type inequalities
Jackson-type inequalities bound approximation error in terms of smoothness. They are among the most important direct estimates in classical approximation theory. These inequalities show how regularity controls achievable accuracy.
9.2 Inverse theorems
Inverse theorems infer smoothness from the rate of approximation. They provide a converse to direct results by asking what can be deduced when approximations converge quickly. This direction is essential for understanding the meaning of observed error rates.
9.2.1 Converse approximation results
Converse approximation results explain which regularity properties must hold if a function is approximated rapidly. They help complete the relationship between smoothness and approximation quality. Such results are key to a full characterization of function classes.
9.2.2 Saturation phenomena
Saturation phenomena occur when an approximation method cannot improve beyond a certain rate for a broad class of functions. This sets an intrinsic limit on the method’s performance. Recognizing saturation is important for choosing more powerful approximation schemes when needed.
9.3 Direct theorems
Direct theorems start from known regularity and derive approximation rates. They show how the structure of a function determines the speed at which the error decreases. These theorems are a core tool for predicting performance.
9.3.1 Approximation rates
Approximation rates describe how quickly the error shrinks as the approximating family becomes richer. They may depend on degree, number of terms, mesh size, or other complexity measures. Such rates are used to compare competing methods.
9.3.2 Smoothness classes
Smoothness classes group functions according to shared regularity properties. Different classes admit different approximation rates and error bounds. Identifying the relevant class helps determine the most effective approximation strategy.
10 Computational and applied aspects
Approximation theory has extensive practical importance in computation, data analysis, and simulation. Modern applications require methods that are not only accurate but also stable and efficient. Theoretical approximation results often guide the design of algorithms used in practice.
10.1 Numerical algorithms
Algorithms implement approximation methods in finite arithmetic. Their quality depends on both mathematical accuracy and numerical stability. Efficient algorithms are essential when working with large data sets or high-dimensional models.
10.1.1 Fast transforms
Fast transforms compute coefficients or representations quickly, often by exploiting structure such as periodicity or sparsity. They are widely used in spectral methods and signal processing. Their efficiency has transformed many areas of computation.
10.1.2 Stable evaluation methods
Stable evaluation methods reduce the impact of rounding errors and other numerical artifacts. They ensure that approximations remain reliable when implemented on computers. Stability is especially important for interpolation, polynomial evaluation, and iterative procedures.
10.2 Approximation in scientific computing
Scientific computing depends on approximating continuous models by discrete, computable objects. The accuracy of these approximations affects simulations, predictions, and parameter estimation. Approximation theory supplies the mathematical foundation for many of these techniques.
10.2.1 Data fitting
Data fitting constructs an approximate model from observed values. It may use least squares, interpolation, splines, or more flexible nonlinear methods. The goal is to capture the main pattern of the data while limiting noise and overfitting.
10.2.2 Numerical integration
Numerical integration approximates definite integrals by finite sums. Quadrature methods rely on approximation of functions by polynomials, trigonometric expressions, or other basis elements. Error analysis is crucial for ensuring dependable results.
10.2.3 Differential equation solvers
Differential equation solvers replace continuous unknowns with finite approximations. Common methods include finite differences, finite elements, spectral schemes, and spline-based techniques. Approximation theory helps explain convergence, stability, and accuracy.
10.3 Modern extensions
Recent developments have broadened approximation theory into areas influenced by multiscale analysis and data-driven methods. These extensions address complex structures that are difficult to represent with classical bases alone. They continue to expand the reach of the field.
10.3.1 Wavelet approximation
Wavelet approximation uses localized basis functions at multiple scales. This makes it effective for representing features such as edges, bursts, and local irregularity. Wavelets are widely used in compression, denoising, and multiresolution analysis.
10.3.2 Machine-assisted approximation
Machine-assisted approximation uses computational tools to identify effective surrogate models, basis elements, or fitting schemes. It includes automated symbolic search, regression methods, and other data-driven techniques. These methods extend classical approximation ideas into modern algorithmic environments.