Mercer's condition is a fundamental requirement in functional analysis and the theory of integral equations, specifying when a symmetric continuous kernel \( K(x, y) \) defined on a compact metric space is positive-definite. Formally, a kernel satisfies Mercer’s condition if it can be expanded in a uniformly convergent series of eigenfunctions and non‑negative eigenvalues, which is the content of Mercer’s theorem. This condition guarantees that the associated integral operator is compact, self‑adjoint, and positive, and it underpins the construction of reproducing kernel Hilbert spaces (RKHS). In modern applications, Mercer's condition is central to kernel methods in machine learning, such as support vector machines and Gaussian processes.
1 Definition
1.1 Formal statement
Let \(X\) be a compact metric space and let \(K: X \times X \to \mathbb{R}\) be a continuous symmetric function, i.e., \(K(x,y)=K(y,x)\) for all \(x,y\in X\). The kernel \(K\) is said to satisfy Mercer's condition if for every finite set of points \(\{x_1,\dots,x_n\}\subseteq X\) and any real coefficients \(c_1,\dots,c_n\), the quadratic form \[ \sum_{i=1}^n \sum_{j=1}^n c_i c_j K(x_i,x_j) \ge 0. \] In other words, the matrix \(\big(K(x_i,x_j)\big)_{i,j=1}^n\) is positive semidefinite for every choice of points. This property is equivalent to \(K\) being a positive-definite kernel.
1.2 Integral operator formulation
Associated with the kernel \(K\) is the integral operator \(T_K\) defined on the Hilbert space \(L^2(X,\mu)\) (where \(\mu\) is a Borel measure on \(X\), typically the volume measure) by \[ (T_K f)(x) = \int_X K(x,y) f(y)\, d\mu(y). \] Mercer's condition is equivalent to the operator \(T_K\) being compact, self‑adjoint, and positive semidefinite: \(\langle f, T_K f\rangle \ge 0\) for all \(f\in L^2(X,\mu)\). This functional analytic viewpoint connects the kernel directly to spectral theory.
2 Historical background
2.1 Mercer’s theorem (1909)
James Mercer (1883–1932) proved the theorem that bears his name in 1909. He showed that for a continuous symmetric positive-definite kernel on a compact interval \([a,b]\), there exists an orthonormal sequence of eigenfunctions \(\{\phi_n\}\) in \(L^2([a,b])\) and a sequence of non‑negative eigenvalues \(\{\lambda_n\}\) such that \[ K(x,y) = \sum_{n=1}^\infty \lambda_n \phi_n(x)\phi_n(y), \] with the series converging absolutely and uniformly on \([a,b]\times[a,b]\). This laid the theoretical foundation for the spectral representation of positive-definite kernels.
2.2 Generalizations and extensions
Subsequent work extended Mercer's theorem to more general spaces: compact topological spaces, metric spaces, and even non‑compact domains under certain integrability conditions. The theory of reproducing kernel Hilbert spaces, developed by Aronszajn (1950), provided a natural functional setting for Mercer kernels. More recent generalizations include kernels on non‑Euclidean domains such as graphs and manifolds, and the extension to operator-valued kernels.
3 Properties
3.1 Necessity and sufficiency
Mercer's condition is both necessary and sufficient for a continuous symmetric kernel to be representable as a uniformly convergent series of eigenfunctions with non‑negative eigenvalues. If the kernel fails to be positive-definite, the eigenvalues of the integral operator may be negative, and the series expansion may diverge or converge only conditionally.
3.2 Continuity and integrability requirements
The classic Mercer theorem requires \(K\) to be continuous on the compact domain \(X\times X\). For non‑compact domains or discontinuous kernels, the condition is often replaced by the requirement that the integral operator is a Hilbert–Schmidt operator with positive eigenvalues. In practice, many kernels used in machine learning (e.g., the Gaussian kernel) are continuous and satisfy the condition on compact subsets.
3.3 Eigenvalue expansion
3.3.1 Convergence of the series
The uniform convergence of the series is a key consequence of Mercer's theorem. It ensures that the kernel can be approximated uniformly by a finite sum of eigenfunctions, which is essential for numerical applications. The convergence is absolute and uniform on the product space.
3.3.2 Non‑negativity of eigenvalues
The eigenvalues \(\lambda_n\) are all non‑negative. If the kernel is strictly positive-definite, then all eigenvalues are positive. The trace of the integral operator satisfies \(\sum_{n} \lambda_n = \int_X K(x,x)\, d\mu(x) < \infty\), linking the spectral decomposition to the diagonal of the kernel.
4 Applications
4.1 Integral equations
Mercer's theorem provides a method for solving Fredholm integral equations of the second kind: \[ f(x) = g(x) + \mu \int_X K(x,y) f(y)\,dy. \] By expanding the kernel in its eigenfunctions, the integral equation reduces to an algebraic system in the coefficients of the eigenfunction expansion, often leading to closed-form solutions.
4.2 Kernel methods in machine learning
In machine learning, a kernel satisfying Mercer's condition is called a Mercer kernel. It defines an inner product in a high‑dimensional feature space without explicitly mapping data points, enabling nonlinear algorithms.
4.2.1 Support vector machines
Support vector machines (SVMs) use Mercer kernels to transform input data into a feature space where a linear separating hyperplane is constructed. The kernel trick relies entirely on the positive-definiteness of the kernel. Common choices include polynomial kernels and the radial basis function kernel.
4.2.1.1 Radial basis function kernel example
The radial basis function (RBF) kernel is given by \[
| K(x,y) = \exp\left(-\frac{\|x-y\|^2}{2\sigma^2}\right), |
|---|
\] which is continuous, symmetric, and positive-definite on any compact subset of \(\mathbb{R}^d\). It satisfies Mercer's condition and is widely used in SVMs and Gaussian processes for its ability to capture local similarity.
4.2.2 Gaussian processes
In Gaussian process regression, a Mercer kernel defines the covariance function of the process. The positive-definiteness ensures that the covariance matrix of any finite set of points is positive semidefinite, a necessary condition for a valid stochastic process. The eigendecomposition of the kernel underlies the formulation of Gaussian process priors and posterior predictions.
4.3 Reproducing kernel Hilbert spaces (RKHS)
An RKHS is a Hilbert space \(\mathcal{H}\) of functions on \(X\) such that the evaluation functional \(\delta_x: f \mapsto f(x)\) is continuous. Every RKHS has a unique reproducing kernel \(K\) satisfying \(f(x) = \langle f, K(x,\cdot)\rangle_{\mathcal{H}}\) for all \(f\in\mathcal{H}\). Conversely, every positive-definite kernel defines an RKHS.
4.3.1 Connection to Mercer kernels
A kernel satisfying Mercer's condition is precisely the reproducing kernel of the RKHS generated by the eigenfunctions \(\{\sqrt{\lambda_n}\,\phi_n\}\). The RKHS norm corresponds to the weighted sum of squared coefficients in the eigenfunction expansion. This connection is the foundation of the representer theorem, which justifies the use of kernel methods in regularization problems.
5 See also
- Positive-definite kernel
- Mercer's theorem
- Reproducing kernel Hilbert space
- Kernel method
- Integral operator
- Fredholm integral equation