1 Definition and basic setup
1.1 Fenchel conjugate via supremum
Let \(X\) and \(Y\) be real vector spaces equipped with a bilinear pairing \(\langle x,y\rangle\). For an extended-real-valued function \(f:X\to \overline{\mathbb{R}}:=\mathbb{R}\cup\{+\infty,-\infty\}\), its Fenchel conjugate (or convex conjugate) is defined by \[ f^{*}(y)=\sup_{x\in X}\bigl(\langle x,y\rangle - f(x)\bigr). \] The supremum represents the best affine minorant interpretation: among all expressions of the form \(\langle x,y\rangle - c\), \(f^*(y)\) collects the maximal shift compatible with subtracting the value of \(f\). When \(f\) is proper (not identically \(+\infty\) and never \(-\infty\)), the conjugate typically captures convex structure by effectively “enveloping” \(f\) through supporting linear behavior.
1.2 Choice of pairing and ambient spaces
The conjugation depends on the pairing. In finite dimensions, one often takes \(X=\mathbb{R}^n\) and \(Y=\mathbb{R}^n\) with \(\langle x,y\rangle=x^\top y\). In functional settings, \(Y\) is commonly taken as the continuous dual \(X'\), with \(\langle x,y\rangle\) denoting the action of a functional on a vector. Different choices of topology or dual space can change what “continuous” linear perturbations are allowed, which in turn affects properties such as lower semicontinuity of the conjugate and the validity of biconjugation identities.
1.3 Extended-real-valued functions
To include constraints and nonsmooth terms, \(f\) is allowed to take the values \(+\infty\) and \(-\infty\). In practice, one usually restricts to the class
- Proper: \(f(x)>-\infty\) for all \(x\) and \(f\not\equiv +\infty\),
- Convex: \(f(tx_1+(1-t)x_2)\le tf(x_1)+(1-t)f(x_2)\),
- Lower semicontinuous (lsc): epigraph is closed in the chosen topology.
Extended values are crucial: indicator functions of feasible sets are modeled by setting \(f(x)=0\) on the set and \(+\infty\) outside, and conjugation then translates constraints into support-function-like quantities.
2 Properties of the Fenchel conjugate
2.1 Convexity and lower semicontinuity
For any function \(f\) (with no assumptions beyond being extended-real-valued), the conjugate \(f^*\) is always convex as a supremum of affine functions in \(y\). Under standard regularity assumptions (commonly when \(f\) is proper and convex and the pairing is compatible with the topology), \(f^*\) is also lower semicontinuous with respect to the topology induced on \(Y\). This convex-lsc structure is one reason conjugation is a central tool in convex analysis: the transform systematically produces well-behaved objects from potentially irregular inputs.
2.2 Order-reversing and transformation behavior
The conjugation operation reverses inequalities: if \(f\le g\) pointwise, then \[ f^* \ge g^*. \] Intuitively, decreasing \(f\) increases the quantity \(\langle x,y\rangle-f(x)\), hence increases the supremum. Conjugation also interacts with translation-like operations and affine changes, leading to systematic “rule” formulas later used in calculus of conjugates.
2.3 Fenchel–Young inequality
A foundational inequality relates \(f\) and \(f^*\): \[ f(x)+f^*(y)\ge \langle x,y\rangle \quad \text{for all }x,y. \] It follows directly from the definition of \(f^*(y)\) as a supremum: for any fixed \(x\), \[ f^*(y)=\sup_{u}\bigl(\langle u,y\rangle-f(u)\bigr)\ge \langle x,y\rangle-f(x), \] and rearranging gives the inequality. This relation is fundamental in duality theory, optimization certificates, and characterizations of optimality.
2.4 Effective domains and conjugate domain relationships
The effective domain of \(f\) is \(\mathrm{dom}\,f=\{x\in X: f(x)<+\infty\}\). The conjugate’s domain reflects which linear perturbations yield finite supremum: \[ \mathrm{dom}\,f^*=\{y\in Y: f^*(y)<+\infty\}. \] If \(f\) has “steep” growth, its conjugate may be finite only on a smaller set of directions. Conversely, if \(f\) permits large negative offsets (or is too flat), \(f^*\) can become \(+\infty\) on wide regions. These domain interactions are central when interpreting dual problems as optimization over feasible dual variables.
3 Computation techniques and examples
3.1 Conjugates of affine and indicator functions
Two classes are repeatedly used.
1 Definition and basic setup
\[ f^*(y)= \begin{cases} -b, & y=a,\\ +\infty, & y\ne a, \end{cases} \] provided the supremum is taken over the whole space where the linear part can match \(y\).
2 Properties of the Fenchel conjugate
\[ \delta_C(x)= \begin{cases} 0, & x\in C,\\ +\infty, & x\notin C. \end{cases} \] Its conjugate becomes the support function of \(C\): \[ \delta_C^*(y)=\sup_{x\in C}\langle x,y\rangle. \] This transforms geometric constraints into linear-growth functionals on the dual side.
3.2 Conjugates of norms and norm powers
| Norm-based regularizers yield explicit conjugates, often via dual norms and standard power laws. Let \(\|\cdot\|\) be a norm on \(X\) with dual norm \(\|\cdot\|_*\) on \(Y\). For many exponents, the conjugate reduces to a corresponding dual power. |
|---|
3.2.1 Conjugate of the \(\ell_p\) norm
| For \(1<p<\infty\), consider \(f(x)=\frac{1}{p}\|x\|_p^p\) on \(X=\mathbb{R}^n\) (or \(\ell_p\)). Let \(q\) be the conjugate exponent with \(1/p+1/q=1\). Then |
|---|
\[
| f^*(y)=\frac{1}{q}\|y\|_q^q. |
|---|
\] This identity is a direct manifestation of Hölder-type duality and the Young inequality for scalar powers. It underpins dual formulations of regularized optimization and many sparsity/regularization models.
3.3 Quadratic functions and completing the square
| Quadratic functions admit conjugates computed by maximizing a concave quadratic expression. For example, with \(f(x)=\frac{1}{2}\|x\|^2\) under an inner product \(\langle \cdot,\cdot\rangle\), one obtains \(f^*(y)=\frac{1}{2}\|y\|^2\). More generally, if \(f(x)=\frac{1}{2}\langle x,Ax\rangle+\langle b,x\rangle+c\) with \(A\) symmetric positive definite, then conjugation produces |
|---|
\[ f^*(y)=\frac{1}{2}\langle y-b, A^{-1}(y-b)\rangle - c, \] again obtainable by completing the square and using the maximizing first-order condition.
3.4 Conjugates of piecewise-linear functions
Piecewise-linear convex functions have conjugates that often become indicator functions or support functions of polyhedral sets. Their computation typically relies on identifying where the supremum is achieved and translating slope constraints into set membership in the dual variable.
3.4.1 Max/min-type formulas and support functions
If \(f\) is constructed as a maximum of affine functions, then conjugation relates to intersections and convex hulls through support functions. In many cases, the conjugate of a max becomes an indicator of a set of subgradients, reflecting that the transform “records” allowable slopes. Conversely, the conjugate of a sum of indicators (or of functions with constrained epigraph geometry) often yields a support-function-like object, with infimal convolution producing the appropriate combination of dual constraints.
4 Fenchel duality and optimization
4.1 Primal–dual construction
Fenchel duality starts from a primal problem expressed as a minimization of a convex function (or sum of convex terms). A typical form is \[ \inf_{x\in X}\bigl(f(x)+g(Ax)\bigr), \] where \(A:X\to Z\) is linear and \(f,g\) are convex. Introducing conjugates via identities of the type \[ g(Ax)=\sup_{y\in Z'}\bigl(\langle Ax,y\rangle - g^*(y)\bigr) \] and exchanging infimum/supremum under suitable conditions leads to a dual problem that typically takes the form \[ \sup_{y\in Z'}\bigl(-f^*( -A^\top y)-g^*(y)\bigr), \] or an equivalent expression depending on sign conventions.
4.2 Weak and strong duality in the Fenchel framework
The Fenchel–Young inequality yields weak duality: the dual objective value never exceeds the primal infimum. Under additional regularity (often expressed as a Slater-type condition on relative interior points of domains), strong duality holds, meaning the optimal values coincide and no duality gap occurs.
4.3 Optimality conditions using subgradients
Equality in Fenchel–Young underpins a KKT-style characterization. For convex \(f\), one has \[ f(x)+f^*(y)=\langle x,y\rangle \quad\Longleftrightarrow\quad y\in \partial f(x) \quad\Longleftrightarrow\quad x\in \partial f^*(y), \] where \(\partial f(x)\) denotes the subdifferential. In duality, these relationships identify how primal optimal variables and dual optimal multipliers match through conjugacy: the dual variable represents a supporting hyperplane slope of the primal function at the optimizer.
4.4 Slater-type conditions (qualitative regularity)
Strong duality generally requires a qualification ensuring that the supremum and infimum interchange without leaving gaps. A common qualitative statement is that there exists an \(x\) satisfying the primal constraints with enough room (e.g., lying in an appropriate relative interior of domains), so that the feasible epigraph intersection is not “tangent-only.” Such conditions are often framed in terms of nonempty intersection of relative interiors of relevant domains of \(f\) and \(g\circ A\).
5 Subgradients and conjugacy relations
5.1 Subdifferential correspondence (KKT-style)
The conjugacy transform creates a mirror relation between subgradients of \(f\) and \(f^*\). If \(y\in\partial f(x)\), then the supporting affine function at \(x\) with slope \(y\) achieves equality in Fenchel–Young, and the dual variable satisfies \(x\in\partial f^*(y)\). In optimization, these relations become the mathematical content behind KKT conditions: they express stationarity and complementary slackness in a form compatible with nonsmooth analysis.
5.2 Characterizations of equality in Fenchel–Young
Beyond the subgradient criterion, equality can be expressed through epigraph geometry. The inequality \(f(x)+f^*(y)\ge \langle x,y\rangle\) becomes tight exactly when the affine function \(u\mapsto \langle u,y\rangle - f^*(y)\) supports \(f\) at \(x\). Equivalently, \(y\) is a normal vector to the epigraph of \(f\) at the point \((x,f(x))\). These perspectives are useful for deriving optimality conditions and for understanding why conjugation yields dual certificates.
5.3 Biconjugation and Fenchel–Moreau theorem
Define the biconjugate \(f^{**}=(f^*)^*\). Always, one has \(f^{}\le f\) in general, because conjugation is order-reversing and taking conjugates “regularizes” the function. The Fenchel–Moreau theorem states that if \(f\) is proper, convex, and lower semicontinuous (with respect to the chosen topology), then \[ f^{}=f. \] Thus the biconjugate returns the greatest lower semicontinuous convex minorant of \(f\), effectively removing nonconvexity or lack of lower semicontinuity.
5.3.1 Conditions for \(f^{\*\*}=f\)
The equality \(f^{**}=f\) holds under the standard requirement that \(f\) is convex and lsc (and proper) relative to the pairing-induced topology. If \(f\) fails lsc, the biconjugate coincides with its lsc convex envelope; if \(f\) fails convexity, the biconjugate similarly provides the convex hull in the epigraph sense. This “envelope” behavior is often used to replace an ill-behaved cost with a convex approximation preserving dual interpretations.
6 Transform rules and calculus
6.1 Scaling and translation rules
Conjugates respond predictably to simple transformations.
- Vertical shifts: If \(f(x)\) is replaced by \(f(x)+c\), then \((f+c)^*(y)=f^*(y)-c\).
- Scaling in the argument: For \(f(ax)\) with scalar \(a\neq 0\) (in suitable spaces), conjugation rescales the dual variable so that \(\langle ax,y\rangle=\langle x, a y\rangle\), producing an explicit relationship between \(f^*\) and \((f\circ a)^*\).
These rules simplify computation in structured models where regularizers are shifted or rescaled.
6.2 Sums, infimal convolution, and conjugation
The conjugate of a sum is not generally the sum of conjugates. Instead, one obtains an infimal convolution identity: \[ (f+g)^*(y)=\inf_{y_1+y_2=y}\bigl(f^*(y_1)+g^*(y_2)\bigr), \] under standard convexity/properness assumptions. Conversely, \[ (f^*\square g^*)(x)=(f^* \square g^*) (x) := \inf_{u+v=x}\bigl(f^*(u)+g^*(v)\bigr) \] relates to \((f+g)^{**}\) through duality. This calculus is central when modeling regularization as a sum of penalties and translating it into separable terms in the dual.
6.3 Composition with linear maps
If \(A:X\to Z\) is linear and \(g:Z\to\overline{\mathbb{R}}\), consider \(g\circ A\). Its conjugate involves a supremum over \(x\) subject to \(z=Ax\) behavior and typically becomes a constrained support-like expression on the dual side. In many common settings (e.g., when \(A\) is continuous between Banach spaces and appropriate regularity holds), one can express \((g\circ A)^*\) in terms of \(g^*\) evaluated at dual variables and with constraints reflecting the range of \(A\). Exact formulas depend on whether \(Y\) is the full dual or a restricted continuous dual and on closure operations.
6.4 Pointwise maxima/minima and conjugate effects
For convex functions, pointwise maxima have epigraphs given by intersections, and conjugation turns those geometric operations into dual counterparts involving infimal convolution and closure. While direct min formulas can be more delicate (minima may destroy convexity), max of affine/convex pieces is common in applications, and conjugation often translates the max structure into a manageable dual description via support functions and indicator-like conjugates.
7 Applications in analysis and variational methods
7.1 Regularization and dual representations
Conjugates provide a principled method for rewriting regularized objectives. For instance, a primal problem minimizing \(f(x)+\lambda r(x)\) can be reformulated into a dual problem involving \(f^*\) and \(r^*\), often yielding alternative algorithms or better-conditioned subproblems. This is especially useful when \(r\) is a norm or gauge with a conjugate that can be computed explicitly.
7.2 Variational inequalities and energy functionals
In variational methods, energies are frequently expressed as sums of convex terms, possibly including nonsmooth pieces or constraints. Fenchel conjugation converts these energies into dual functionals whose optimality conditions correspond to force balance or subgradient inclusions. The conjugate framework can thus characterize solutions to variational inequalities through complementary slackness-like equalities derived from Fenchel–Young.
7.3 Connections to Legendre transforms in smooth settings
When \(f\) is differentiable and strictly convex, the Fenchel conjugate aligns with the classical Legendre transform used in thermodynamics and mechanics. The map between primal and dual variables becomes \(y=\nabla f(x)\) and \(x=\nabla f^*(y)\), with the identity \(f(x)+f^*(y)=\langle x,y\rangle\) holding at corresponding points.
7.3.1 Recovering classical thermodynamic potentials (general idea)
In thermodynamic models, one often starts from an energy expressed in one set of variables (e.g., extensive variables), then uses a Legendre-type transform to obtain potentials in conjugate variables (e.g., intensive variables). The Fenchel perspective generalizes this procedure beyond smoothness and strict convexity, ensuring that even when the classical differentiable assumptions fail, one can still define dual potentials via supremum constructions and interpret them as convex conjugates.
8 Common pitfalls and practical considerations
8.1 Handling empty domains and \(\infty\)-values
Extended-real arithmetic can lead to misleading computations if one ignores domains. If \(f\) takes \(+\infty\) on most of the space, the supremum defining \(f^*\) may effectively range over a restricted subset. Similarly, improper functions (taking \(-\infty\) somewhere) can render conjugates meaningless or produce trivial \(+\infty\) outcomes. Careful bookkeeping of where functions are finite is essential.
8.2 When strong duality may fail
Even when both primal and dual problems are well-defined, strong duality can fail without suitable regularity. Typical failure modes include boundary-only feasibility (no interior point) or incompatible domain geometry leading to a nonzero duality gap. In such cases, one still retains weak duality, and the biconjugate \(f^{**}\) reveals how much convex-lsc “regularization” is missing.
8.3 Numerical implications for dual problems
In numerical optimization, moving to the dual can improve separability or produce linear constraints that are easier to handle, but it also introduces sensitivity to domain restrictions. When \(f^*\) is finite only on a subset, numerical methods must enforce dual feasibility carefully. Additionally, infimal convolution or closure operations hidden in the calculus rules can complicate implementation if one relies on naive algebraic manipulation.
8.4 Choosing function classes for well-posed conjugates
Well-posed conjugation typically requires convexity and properness of the primal function and compatibility between the chosen pairing and topology. Selecting function classes that are lsc convex and ensure nonempty relative interior intersections enables robust use of strong duality and biconjugation. When working outside these classes, one should interpret \(f^*\) and \(f^{**}\) as convex envelopes or generalized dual representations rather than direct equalities with the original data.
9 Further generalizations
9.1 Conjugates on Banach vs. Hilbert spaces
The definition of Fenchel conjugates applies broadly, but the analysis changes with the underlying space. In Hilbert spaces, Riesz representation can identify a space with its dual under an inner product, simplifying notation and enabling gradient-based interpretations. In Banach spaces, one must treat \(X\) and its continuous dual \(X'\) separately, and subdifferentials are expressed using duality mappings and support properties rather than direct gradients.
9.2 Conjugates under different topologies
Conjugation depends on what linear functionals are allowed and on closure properties tied to the topology. Using a stronger topology can shrink the dual space of continuous functionals, altering the conjugate and the validity of the Fenchel–Moreau theorem. Consequently, specifying the topology (for example, norm topology versus weak topology) is not merely technical—it can determine whether \(f^{**}\) recovers the same function or only its closure/regularization.
9.3 Relation to gauge/support functions
Fenchel conjugation is closely tied to geometric functionals. Support functions arise as conjugates of indicator functions of convex sets. Gauge (Minkowski) functionals are similarly connected to conjugates of polar sets: the polar of a set describes admissible dual directions, and the conjugate expresses how the primal gauge limits those directions. These relations unify convex geometry and optimization, showing that conjugacy often encodes feasibility and scaling geometry in dual language.