1 Statement and Notation
1.1 Setup: primal space, dual pairing, and convex conjugate
Let \(X\) be a real vector space equipped with a dual space \(Y\), together with a bilinear dual pairing \(\langle y,x\rangle\) for \(x\in X\), \(y\in Y\). Consider a function \[ f:X\to \mathbb{R}\cup\{+\infty\}. \] The Legendre–Fenchel (convex conjugate) of \(f\) is defined by \[ f^\*(y)=\sup_{x\in X}\bigl(\langle y,x\rangle - f(x)\bigr). \] This definition is central: it packages all affine lower bounds to \(f\) into a single dual object.
1.2 Fenchel–Young inequality (primal form)
For all \(x\in X\) and all \(y\in Y\), \[ f(x) + f^\*(y) \ge \langle y,x\rangle, \] where the convention is understood that \(f(x)\) or \(f^\*(y)\) may take the value \(+\infty\), in which case the inequality holds in the extended sense.
1.3 Conditions on \(f\): properness, lower semicontinuity, convexity
The inequality is typically stated for a proper, lower semicontinuous, convex function \(f\). Concretely:
- Convexity ensures that the conjugate captures supporting affine behavior rather than arbitrary slopes.
- Properness means \(f\not\equiv +\infty\) and \(f\) never takes the value \(-\infty\).
- Lower semicontinuity (in the appropriate topology) ensures regularity so that conjugacy and duality theory behave well, especially when one studies equality conditions and attainment of suprema.
2 Geometric and Variational Interpretation
2.1 Supporting hyperplanes and epigraph geometry
Geometrically, the conjugate inequality reflects how the graph of \(f\) sits relative to affine functions. Since \(f^\*(y)\) is a supremum of expressions \(\langle y,x\rangle - f(x)\), the inequality \[ f(x)\ge \langle y,x\rangle - f^\*(y) \] says that for each dual vector \(y\), the affine function \(x\mapsto \langle y,x\rangle - f^\*(y)\) lies below \(f\). In epigraph terms, this is equivalent to the statement that \(f\) admits supporting hyperplanes governed by its conjugate data.
2.2 Interpretation via Legendre transforms (intuition)
When \(f\) is smooth and strictly convex, the Legendre–Fenchel transform reduces to the classical Legendre transform. In that setting, \(y\) plays the role of the “slope” variable: at corresponding primal–dual points, \(y\) matches the gradient of \(f\). The Fenchel–Young inequality then becomes the analytic expression of “a tangent plane lies below a convex function,” with equality at tangency.
2.3 Equality case and tight bounds
The inequality is generally strict. Equality occurs precisely when the chosen pair \((x,y)\) matches the subdifferential structure of \(f\) and \(f^\*\).
2.3.1 Equality through subdifferentials
A standard characterization is: \[ f(x) + f^\*(y)=\langle y,x\rangle \quad\Longleftrightarrow\quad y\in \partial f(x) \quad\Longleftrightarrow\quad x\in \partial f^\*(y), \] where \(\partial f(x)\) denotes the subdifferential of \(f\) at \(x\). This converts a purely numerical inequality into a geometric exactness condition.
2.3.2 Equality via optimality of dual variables
Equality also corresponds to optimal solutions in the definitions of \(f^\*(y)\) and related supremum problems. If equality holds, then the supremum in \[ f^\*(y)=\sup_{u}\bigl(\langle y,u\rangle - f(u)\bigr) \] is attained at \(u=x\). Likewise, equality links to the attainment of the supporting affine function that touches \(f\) at \(x\).
3 Connections to Subgradients and Optimality
3.1 Subgradient form of equality
If \(y\in \partial f(x)\), then by definition of subgradient, \[ f(z)\ge f(x)+\langle y,z-x\rangle\quad\text{for all }z. \] Rearranging yields \[ \langle y,z\rangle - f(z)\le \langle y,x\rangle - f(x), \] so the supremum over \(z\) equals \(\langle y,x\rangle - f(x)\). Substituting into the conjugate definition gives \[ f^\*(y)=\langle y,x\rangle - f(x), \] which is exactly the equality case of Fenchel–Young.
3.2 Maximizers/minimizers in conjugate definitions
Because \(f^\*(y)\) is defined via a supremum, dual attainment is not automatic in all topological vector spaces. However, when equality holds, the relevant supremum is tight at the corresponding \(x\). In this way, Fenchel–Young provides a bridge between:
- the maximization viewpoint in \(f^\*\) (supremum of linearized surplus), and
- the minimization viewpoint in primal–dual optimization (minimization of \(f(x)\) under constraints).
3.3 First-order optimality implications
In convex optimization, Fenchel–Young underlies many first-order conditions. When one introduces a Lagrange multiplier or an auxiliary dual variable \(y\), the inequality supplies a universal lower bound to the primal objective plus dual objective. Equality indicates that the dual variable is not merely feasible but optimal (or that primal and dual solutions are matched through subgradients).
4 Proofs and Derivations
4.1 Proof using the definition of convex conjugate
Start from the definition \[ f^\*(y)=\sup_{u\in X}\bigl(\langle y,u\rangle - f(u)\bigr). \] For a fixed \(x\in X\), the supremum dominates the specific value at \(u=x\): \[ f^\*(y)\ge \langle y,x\rangle - f(x). \] Rearranging yields \[ f(x)+f^\*(y)\ge \langle y,x\rangle, \] which is Fenchel–Young. This argument does not require more than the defining supremum property; the regularity assumptions become relevant mainly for equality characterization and the behavior of duality constructions.
4.2 Proof via Fenchel duality framework
Consider the optimization problem of minimizing the expression \(\,f(u)-\langle y,u\rangle\) over \(u\). By definition, \[ f^\*(y)= -\inf_{u\in X}\bigl(f(u)-\langle y,u\rangle\bigr). \] Then for any \(x\), \[ -\inf_{u}\bigl(f(u)-\langle y,u\rangle\bigr)\ge -\bigl(f(x)-\langle y,x\rangle\bigr), \] so \[ f^\*(y)\ge \langle y,x\rangle - f(x), \] and the inequality follows again by rearrangement. This proof emphasizes that Fenchel–Young is a consequence of weak duality between a function and its conjugate-based dual.
4.3 Proof variants (finite-dimensional vs. general locally convex spaces)
In finite-dimensional spaces, suprema defining \(f^\*\) often coincide with maxima under mild coercivity/regularity assumptions, making equality and attainment more straightforward. In general locally convex spaces, one must be careful with:
- the topology governing lower semicontinuity,
- the topology underlying the dual pairing, and
- the potential lack of attainment in supremum definitions.
Even then, the inequality itself remains valid because it relies only on the supremum dominating the value at a chosen point.
5 Examples and Special Cases
5.1 Quadratic function and its conjugate
Let \(X=\mathbb{R}^n\) and \[
| f(x)=\tfrac12 \|x\|^2 |
|---|
\] (with the Euclidean norm and dual pairing given by the dot product). The conjugate is \[
| f^\*(y)=\tfrac12 \|y\|^2. |
|---|
\] Fenchel–Young becomes \[
| \tfrac12\|x\|^2+\tfrac12\|y\|^2\ge x\cdot y, |
|---|
\]
| which is equivalent to \(\|x-y\|^2\ge 0\). Equality holds when \(x=y\). |
|---|
5.2 Norms and dual norms
| For a norm \(\|\cdot\|\) on \(X\), define |
|---|
\[
| f(x)=\|x\|. |
|---|
\]
| Its conjugate is the indicator of the unit ball of the dual norm \(\|\cdot\|_*\): |
|---|
\[ f^\*(y)= \begin{cases}
| 0, & \|y\|_*\le 1,\\ |
|---|
| +\infty, & \|y\|_*>1. |
\end{cases} \]
| Fenchel–Young then yields a familiar bound: when \(\|y\|_*\le 1\), the inequality reduces to \(\|x\|\ge \langle y,x\rangle\); if \(\|y\|_*>1\), the conjugate is infinite and the inequality is trivial in the extended-real sense. |
|---|
5.3 Indicator functions and support functions
Let \(f=\delta_C\) be the indicator function of a convex set \(C\subset X\): \[ \delta_C(x)= \begin{cases} 0, & x\in C,\\ +\infty, & x\notin C. \end{cases} \] Then \[ f^\*(y)=\sigma_C(y):=\sup_{x\in C}\langle y,x\rangle, \] the support function of \(C\). Fenchel–Young becomes \[ \delta_C(x)+\sigma_C(y)\ge \langle y,x\rangle. \] When \(x\in C\), this is exactly the defining upper bound \(\sigma_C(y)\ge \langle y,x\rangle\).
5.4 Exponential/entropy-type examples (light applications)
In many applications, convex functions related to entropy or log-partition functions appear. For instance, on \(\mathbb{R}\), \[ f(x)=e^x \] has a conjugate that can be expressed in terms of \(y\log y - y\) for \(y>0\) (and \(+\infty\) for \(y\le 0\)), reflecting the constraint that the conjugate variable must fall in the natural domain induced by exponential growth. Fenchel–Young then yields inequalities that resemble those used in variational formulations and divergence bounds, with equality occurring at matching primal–dual pairs determined by subgradient conditions.
6 Applications in Convex Optimization
6.1 Deriving duality bounds from the inequality
Consider minimizing a convex objective involving \(f\) over \(x\), and form a dual expression involving \(f^\*\). Fenchel–Young provides a universal estimate: \[ f(x)+f^\*(y)\ge \langle y,x\rangle. \] When the optimization problem is written so that \(\langle y,x\rangle\) becomes part of the Lagrangian or dual objective, summing Fenchel–Young across components produces weak duality: the dual objective never exceeds the primal optimum.
6.2 Constructing certificates of optimality
If primal and dual variables \((x,y)\) satisfy equality in Fenchel–Young (or its multi-term analogue), then \(y\) is in \(\partial f(x)\) and the pair typically certifies optimality. In practice, equality provides a checkable condition: one confirms that the dual variable corresponds to a supporting hyperplane touching the primal function at the primal point.
6.3 Role in primal–dual algorithms (conceptual)
Many primal–dual methods use proximal operators of \(f\) and \(f^\*\), and the logic behind these updates is tied to conjugacy. Fenchel–Young helps interpret these iterations as enforcing consistency between primal and dual descriptions, steering the algorithm toward pairs \((x,y)\) where the inequality becomes tight.
7 Related Inequalities and Theoretical Context
7.1 Fenchel duality and conjugate calculus
Fenchel–Young is the basic inequality that supports Fenchel duality and conjugate calculus rules. It underpins statements such as:
- the relationship between primal minimization and dual maximization,
- the use of conjugates to compute envelopes and regularizations, and
- the appearance of subgradient conditions as “optimality transfer” mechanisms between primal and dual spaces.
7.2 Young’s inequality for products (classical connection)
Classical Young’s inequality for products, \[ ab\le \Phi(a)+\Psi(b) \] for conjugate convex functions \(\Phi\) and \(\Psi\), can be viewed as a special case of Fenchel–Young by choosing \(f\) and \(f^\*\) corresponding to \(\Phi\) and \(\Psi\), and letting \(\langle y,x\rangle\) represent the product. Thus Fenchel–Young provides the general convex-analytic origin of the product inequality.
7.3 Relation to Legendre–Fenchel transform properties
The inequality also reflects broader properties of the transform \(f\mapsto f^\*\), such as:
- order-reversing behavior under conjugation,
- the involution phenomenon under suitable closure (e.g., \(f^{\*\*}\) yielding the closed convex hull of \(f\)),
- and the tight coupling between support functions, indicator functions, and gauge-type constructions.
These aspects are frequently combined with Fenchel–Young to translate geometric information into algebraic dual expressions.
8 Common Pitfalls and Technical Details
8.1 Domain issues and effective domains
Because \(f\) and \(f^\*\) can take the value \(+\infty\), one must interpret inequalities on the effective domain where quantities are finite. If \(x\) lies outside \(\operatorname{dom} f:=\{x: f(x)<+\infty\}\), then \(f(x)=+\infty\) and the inequality becomes vacuous. Similarly, if \(y\) lies outside \(\operatorname{dom} f^\*\), then \(f^\*(y)=+\infty\).
8.2 Properness and cases where conjugates may be infinite
If \(f\) fails to be proper, the conjugate may behave pathologically (for example, becoming identically \(+\infty\) or undefined in a way that breaks the intended interpretation). Properness ensures the conjugate reflects meaningful supporting affine bounds rather than degenerate extremes.
8.3 Subdifferential existence and closure considerations
Equality characterization uses subdifferentials \(\partial f(x)\) and \(\partial f^\*(y)\). Subdifferentials may be empty at boundary points where supporting hyperplanes do not exist in the required topology. Lower semicontinuity and convexity help guarantee that conjugacy and subgradient relationships are consistent with the closure of epigraphical data, but one must still check assumptions when working in general topological vector spaces or with extended-real functions.