1 Definition and notation
The Hessian is the matrix formed from the second-order partial derivatives of a scalar-valued function. It generalizes the familiar second derivative from single-variable calculus to functions of several variables. In multivariable analysis, it is used to describe how a function bends near a point and to help determine whether that point is locally minimal, maximal, or saddle-shaped.
1.1 Scalar-valued functions
For a function \(f(x_1, x_2, \dots, x_n)\) taking real inputs and producing a real output, the Hessian is defined only when second partial derivatives exist. The function may represent a surface, an energy, a cost, or any quantity that varies with several variables. Because the output is a single scalar, the second derivative information can be arranged in a square matrix.
1.2 Hessian matrix entries
If \(f\) is twice differentiable, its Hessian at a point is written as an \(n \times n\) matrix. The entry in row \(i\) and column \(j\) is the second partial derivative \(\frac{\partial^2 f}{\partial x_i \partial x_j}\). This matrix records both the curvature in each coordinate direction and the interactions between different variables.
1.2.1 Second partial derivatives
The diagonal entries of the Hessian are the pure second partial derivatives. They measure how rapidly the slope of the function changes as one variable is varied while the others are held fixed. Large positive values indicate upward bending in that direction, while large negative values indicate downward bending.
1.2.2 Mixed partial derivatives
The off-diagonal entries are mixed second partial derivatives. They measure how change in one variable affects the rate of change in another. These terms capture coupling between coordinates and are central to understanding how a function behaves away from axis-aligned directions.
1.3 Hessian as a matrix of curvature
The Hessian is often described as a curvature matrix because it summarizes second-order behavior near a point. While first derivatives describe slope, the Hessian describes how the slope itself changes. This makes it a natural tool for local approximation and for identifying geometric features of a function.
2 Basic properties
The Hessian has several structural properties that make it especially useful in analysis and optimization. Its behavior depends strongly on differentiability assumptions and on whether mixed partial derivatives agree.
2.1 Symmetry conditions
When a function has sufficiently smooth second derivatives, the Hessian is symmetric. Symmetry means the entry in position \((i,j)\) equals the entry in position \((j,i)\). This property simplifies computation and ensures real eigenvalues in many standard settings.
2.1.1 Equality of mixed partials
Under common smoothness conditions, the mixed partial derivatives satisfy \[ \frac{\partial^2 f}{\partial x_i \partial x_j} = \frac{\partial^2 f}{\partial x_j \partial x_i}. \] This equality is a classical result of calculus and is often used to confirm symmetry of the Hessian. It may fail when derivatives are not sufficiently regular.
2.2 Dependence on differentiability assumptions
The Hessian is well behaved when the function is twice continuously differentiable. If only partial derivatives exist in a weaker sense, the matrix may not be symmetric, or some entries may not be defined at all. Thus, the usefulness of the Hessian depends on the smoothness of the underlying function.
2.3 Relation to the gradient and Jacobian
The Hessian is the Jacobian matrix of the gradient. Since the gradient is a vector of first partial derivatives, differentiating it once more produces the matrix of second partial derivatives. This relationship shows that the Hessian is the natural second-order analogue of the gradient.
3 Geometric interpretation
The Hessian gives local geometric information about a scalar function. Near a point, it helps distinguish whether the graph bends upward, downward, or in different ways along different directions.
3.1 Local curvature
At a point, the Hessian encodes the local curvature of the function. If the curvature is positive in all directions, the graph tends to curve upward like a bowl. If it is negative in all directions, the graph curves downward like an inverted bowl. Mixed signs indicate saddle-like behavior.
3.2 Quadratic approximation
Near a point \(a\), a smooth function can be approximated by a Taylor expansion whose second-order term involves the Hessian: \[ f(x) \approx f(a) + \nabla f(a)\cdot (x-a) + \frac12 (x-a)^T H(a)(x-a). \] This quadratic approximation is one of the most important uses of the Hessian. It provides a local model that is often accurate enough for analysis and numerical methods.
3.3 Principal directions and eigenvalues
The eigenvalues and eigenvectors of the Hessian describe the principal directions of curvature. Each eigenvector points along a direction in which the quadratic approximation simplifies to a pure second-order change. Positive eigenvalues correspond to upward curvature, negative eigenvalues to downward curvature, and zero eigenvalues indicate flat or degenerate directions.
4 Hessian determinant
The determinant of the Hessian is often used as a compact summary of second-order structure. In low-dimensional problems, it plays a central role in classifying critical points.
4.1 Computation
To compute the Hessian determinant, one first forms the Hessian matrix and then takes its determinant. For a function of two variables, this is especially simple: \[ \det(H) = f_{xx}f_{yy} - (f_{xy})^2. \] In higher dimensions, the determinant is computed by the usual linear algebra methods.
4.2 The Hessian test for critical points
At a critical point, where the gradient vanishes, the Hessian determinant and related quantities can help classify the point. The test is most familiar in two-variable calculus, though the general idea extends through definiteness in higher dimensions.
4.2.1 Positive definite case
If the Hessian is positive definite at a critical point, the function has a local minimum there. In this case, the quadratic approximation bends upward in every nonzero direction.
4.2.2 Negative definite case
If the Hessian is negative definite at a critical point, the function has a local maximum there. The graph bends downward in all directions near that point.
4.2.3 Indefinite case
If the Hessian is indefinite, the function has a saddle point. Some directions curve upward while others curve downward, so the point is neither a local minimum nor a local maximum.
5 Hessians in optimization
In optimization, the Hessian provides second-order information that improves both theoretical analysis and computational methods. It is especially valuable when finding extrema of nonlinear functions.
5.1 Newton’s method
Newton’s method for multivariable optimization uses the Hessian to refine an approximate solution. The method combines gradient and curvature information to produce faster convergence than first-order techniques in many cases. When the Hessian is invertible and well conditioned, it can yield efficient updates.
5.2 Second-order sufficient conditions
Second-order sufficient conditions use the Hessian to confirm that a critical point is a strict local extremum. If the Hessian is positive definite, the point is a strict local minimum; if negative definite, a strict local maximum. These conditions are stronger than first-order tests because they account for curvature.
5.3 Convexity and concavity
For twice-differentiable functions, the Hessian is closely tied to convexity and concavity. A positive semidefinite Hessian indicates convex behavior, while a negative semidefinite Hessian indicates concavity. This connection is fundamental in mathematical optimization, where convex problems are often easier to analyze and solve.
6 Higher-dimensional and generalized settings
The Hessian extends naturally beyond two or three variables. It also appears in more abstract forms when coordinates change or when the function is defined on geometric objects rather than on ordinary Euclidean space.
6.1 Hessians of functions of many variables
For functions with many inputs, the Hessian becomes a large square matrix. Its size grows with the number of variables, but its interpretation remains the same: it measures all second-order interactions among the coordinates. In high-dimensional problems, it may be sparse or structured, which can make computation more efficient.
6.2 Hessian bilinear form
The Hessian can also be viewed as a bilinear form acting on pairs of vectors. In this form, it describes the second directional derivative of a function along given directions. This viewpoint is useful in differential geometry and in the study of curvature on manifolds.
6.3 Hessian in coordinate transformations
Under a change of coordinates, the Hessian does not generally transform as simply as the gradient. Its expression can involve additional terms unless the coordinate change is linear. This makes coordinate-dependent calculations more delicate, especially in geometric applications.
7 Examples
Concrete examples help show how the Hessian captures local behavior. Even simple functions can display a range of curvature patterns.
7.1 Functions of two variables
For \(f(x,y)=x^2+y^2\), the Hessian is \[ \begin{pmatrix} 2 & 0\\ 0 & 2 \end{pmatrix}, \] which is positive definite. The origin is a local minimum. For \(f(x,y)=x^2-y^2\), the Hessian is \[ \begin{pmatrix} 2 & 0\\ 0 & -2 \end{pmatrix}, \] which is indefinite, so the origin is a saddle point.
7.2 Functions of three or more variables
For \(f(x,y,z)=x^2+2y^2+3z^2\), the Hessian is diagonal with positive entries. This indicates upward curvature in every coordinate direction. If cross terms such as \(xy\) or \(xz\) are included, the off-diagonal Hessian entries become nonzero and the curvature directions may rotate away from the coordinate axes.
7.3 Worked critical-point classifications
At a critical point of a smooth function of two variables, one may compute \(f_{xx}\), \(f_{yy}\), and \(f_{xy}\), then evaluate \[ D = f_{xx}f_{yy} - (f_{xy})^2. \] If \(D>0\) and \(f_{xx}>0\), the point is a local minimum. If \(D>0\) and \(f_{xx}<0\), it is a local maximum. If \(D<0\), the point is a saddle. If \(D=0\), the test is inconclusive.
8 Applications
The Hessian appears in many areas where second-order behavior matters. Its role ranges from theoretical analysis to practical computation.
8.1 Mathematics
In calculus and analysis, the Hessian is used to study extrema, stability, and local approximations. It also plays a role in differential geometry, where it helps describe curvature and the structure of smooth surfaces and manifolds.
8.2 Physics
In physics, the Hessian often appears in energy minimization and stability analysis. Near equilibrium, it helps determine whether a configuration is stable or unstable by examining the curvature of the energy function.
8.3 Economics and machine learning
In economics, the Hessian can describe the curvature of utility or cost functions and support comparative analysis of optimization problems. In machine learning, it is used in training models, especially for understanding the shape of loss functions and for designing second-order optimization algorithms.