1 Basic Definitions
1.1 Convex Sets
A subset \(C\) of a real vector space is convex if, for any two points \(x,y\in C\) and any \(t\in[0,1]\), the convex combination \(tx+(1-t)y\) also lies in \(C\). Geometrically, this means that the straight segment connecting any two points in the set stays entirely inside the set. Convexity is preserved under taking convex combinations, making it a natural framework for optimization and approximation.
1.2 Convex Functions
A function \(f\) defined on a convex set \(C\) is convex if for all \(x,y\in C\) and \(t\in[0,1]\), \[ f\big(tx+(1-t)y\big)\le tf(x)+(1-t)f(y). \] This inequality expresses that the graph of \(f\) lies below the straight line segment joining the points \((x,f(x))\) and \((y,f(y))\). Convexity is fundamentally about how function values compare to averages of inputs.
1.3 Strict and Strong Convexity
Strict convexity strengthens the definition by requiring strict inequality whenever \(x\neq y\) and \(t\in(0,1)\): \[ f\big(tx+(1-t)y\big) < tf(x)+(1-t)f(y). \] Strong convexity introduces a quadratic term. One common form: \(f\) is \(\mu\)-strongly convex (\(\mu>0\)) if \[
| f\big(tx+(1-t)y\big)\le tf(x)+(1-t)f(y)-\frac{\mu}{2}t(1-t)\|x-y\|^2. |
|---|
\] Strong convexity implies not only uniqueness of minimizers (under appropriate conditions) but also quantitative curvature control.
1.4 Jensen-Type Inequalities for Convex Functions
A prototypical consequence of convexity is Jensen’s inequality: for a convex \(f\) and suitable random variable \(X\) with values in the domain, \[ f(\mathbb{E}[X])\le \mathbb{E}[f(X)]. \] More generally, Jensen-type statements compare \(f\) at an average to an average of \(f\). They are a central tool in analysis because they convert geometric convexity into inequalities for expectations, integrals, and mixtures.
2 Geometric Characterizations
2.1 Line Segment Criteria
For sets, the defining criterion is already geometric: any segment between two points must lie in the set. For functions, convexity can be interpreted by comparing the value at a point on a chord with the chord itself: the function at an interior convex combination cannot exceed the linear interpolation of endpoint values. This “chord-under-graph” viewpoint links convexity directly to geometry of graphs and epigraphs.
2.2 Epigraph and Hypograph
A key method for converting function properties into set properties is to consider regions in the product space.
2.2.1 Epigraph Formulation
The epigraph of \(f\) is \[ \operatorname{epi}(f)=\{(x,\alpha)\,:\,\alpha\ge f(x)\}. \] A function \(f\) is convex if and only if its epigraph is a convex set. This equivalence is often used because many operations on convex functions correspond to set operations on their epigraphs.
2.2.2 Hypograph and Concavity Relation
Similarly, the hypograph is \[ \operatorname{hypo}(f)=\{(x,\alpha)\,:\,\alpha\le f(x)\}. \] Concavity of \(f\) is equivalent to convexity of the hypograph. Thus convexity and concavity are dual geometric notions, differing only by flipping the inequality direction in the defining region.
2.3 Supporting Hyperplanes and Lines
Convexity is tightly connected to the existence of affine functions that touch or underestimate the graph.
2.3.1 Separation Theorems (Geometric Form)
A central geometric principle is that disjoint convex sets can often be separated by hyperplanes. In functional terms, separating an epigraph point from a lower region yields inequalities of the form “there exists an affine function lying below \(f\)”. Such theorems are foundational in convex analysis and underpin duality and optimality conditions.
2.3.2 Supporting Affine Minorants and Majorants
For a convex function, a supporting affine minorant at a point \(x_0\) is an affine function \(a(x)=\langle p,x\rangle+b\) satisfying \(a(x)\le f(x)\) for all \(x\) and \(a(x_0)=f(x_0)\). When it exists, it captures how the function behaves locally in a global inequality sense. Concave functions admit supporting affine majorants analogously.
3 Analytic Characterizations
3.1 Differentiable Convex Functions
If \(f\) is differentiable on an open convex set, convexity can be expressed via first-order upper bounds. A standard characterization is: \[ f(y)\ge f(x)+\langle \nabla f(x),\, y-x\rangle \quad \text{for all }x,y. \] Thus the tangent plane at any point lies below the graph. This condition is equivalent to convexity under regularity assumptions and provides a practical tool for verifying convexity.
3.2 Second-Order Conditions
Second-order characterizations describe convexity through curvature.
3.2.1 Hessian-Based Tests
If \(f\) is twice continuously differentiable, then \(f\) is convex on a convex region if and only if its Hessian matrix is positive semidefinite everywhere on that region: \[ \nabla^2 f(x)\succeq 0. \] Positive semidefiniteness means the quadratic form \(v^\top \nabla^2 f(x)\, v\) is nonnegative for all directions \(v\), reflecting curvature outward.
3.2.2 One-Dimensional Second Derivative Criterion
On an interval, convexity is equivalent to nonnegativity of the second derivative where it exists. More precisely, if \(f\) is twice differentiable on an open interval, then \(f\) is convex iff \(f''(x)\ge 0\) for all \(x\) in the interval. This one-dimensional criterion generalizes to multi-dimensional settings via restriction to lines.
3.3 Subgradients and Nonsmooth Convexity
Convexity does not require differentiability. Subgradients replace gradients.
3.3.1 Subdifferential Definition
For a convex function \(f\) on \(C\), the subdifferential at \(x\) is the set \[ \partial f(x)=\{p:\ f(y)\ge f(x)+\langle p,\, y-x\rangle \ \text{for all }y\in C\}. \] Any \(p\in \partial f(x)\) gives an affine function that supports the epigraph at \((x,f(x))\).
3.3.2 Characterizing Convexity via Subgradients
Under mild assumptions, convexity can be characterized by the global supporting property of subgradients: if for every point in a domain one can find supporting affine functions with slopes in \(\partial f(x)\), the function must be convex. Conversely, convexity ensures that subgradients exist at points in the relative interior of the effective domain, linking geometric separation and analytic inequalities.
4 Closure Properties and Constructions
4.1 Preservation Under Operations
Convexity behaves well under many operations used in analysis:
- Adding a convex function to another convex function yields a convex function.
- Scaling by a nonnegative constant preserves convexity.
- Taking the pointwise supremum of convex functions (under mild assumptions, such as upper boundedness at some point) can produce a convex function.
These rules allow complex models to be built from simpler convex components.
4.2 Pointwise Limits and Epigraph Convergence
Limits can destroy regularity, so the relevant notion of convergence must be chosen carefully.
4.2.1 Monotone Convergence for Convex Functions
If a sequence of convex functions increases pointwise to a limit and the limit is finite on the region of interest, then the limit is convex. Monotonicity aligns well with epigraph geometry: increasing functions correspond to decreasing epigraphs, preserving convexity under suitable limiting behavior.
4.2.2 Lower Semicontinuity and Convexity
Lower semicontinuity is closely tied to convexity because many convex analysis tools rely on closedness of epigraphs. A typical statement: a convex function that is the pointwise limit of convex functions in a suitable sense often becomes lower semicontinuous automatically, and the lower semicontinuous closure is frequently the relevant object.
4.3 Transformations
Convexity is stable under many coordinate changes and algebraic manipulations.
4.3.1 Affine Changes of Variables
If \(A\) is an affine map and \(f\) is convex, then \(f\circ A\) is convex on the appropriate domain. Affine maps preserve convex combinations, so they preserve the chord-below-graph property.
4.3.2 Composition with Affine Maps
A common special case is linear transformations: if \(f\) is convex on a set \(C\) and \(L\) is linear, then \(x\mapsto f(Lx)\) is convex provided the domain is mapped into \(C\). This supports reduction of multidimensional problems to lower-dimensional parameterizations.
4.4 Sums, Maxima, and Regularization
A sum of convex functions remains convex, and the pointwise maximum of convex functions is also convex. Regularization introduces stability by adding a strongly convex term (often a quadratic) or by taking a lower semicontinuous envelope. Such steps are standard in optimization and in variational analysis because they ensure better behavior of minimizers and facilitate numerical schemes.
5 Examples and Canonical Models
5.1 Linear and Affine Functions
Affine functions \(f(x)=\langle a,x\rangle + b\) are both convex and concave. Their epigraphs are half-spaces, so they represent the boundary case of zero curvature: the function equals its chords exactly.
5.2 Norms and Power Functions
Many classical convex functions are built from norms and exponents.
5.2.1 The Euclidean Norm
| The map \(x\mapsto \|x\|\) is convex. More generally, the norm satisfies the triangle inequality, which translates into the chord-below-graph property for its graph. Squared Euclidean norm \(x\mapsto \|x\|^2\) is even strongly convex on \(\mathbb{R}^n\). |
|---|
5.2.2 Absolute Value and p-Norms
| On \(\mathbb{R}\), \( | x | \) is convex. For \(p\ge 1\), the p-norm \(\|x\|_p=(\sum_i | x_i | ^p)^{1/p}\) is convex; for \(p>1\) it is strictly convex on many spaces. These examples are central in functional analysis and optimization. |
|---|
5.3 Exponential and Logarithmic Functions
Classical special functions provide canonical curvature examples.
5.3.1 Exponential as Convex
The exponential \(e^x\) is convex on \(\mathbb{R}\). Since its second derivative equals \(e^x>0\), it exhibits strictly positive curvature. As a result, compositions like \(x\mapsto e^{\langle a,x\rangle}\) are convex.
5.3.2 Logarithm and Concavity
The logarithm \(\log x\) is concave on \((0,\infty)\) and thus corresponds to convexity of its hypograph. Many inequalities in analysis can be recast using this concavity property via Jensen-type arguments.
5.4 Quadratic Forms and Distance Functions
| A quadratic form \(x\mapsto x^\top Q x\) is convex precisely when \(Q\) is positive semidefinite. Distance functions of the form \(x\mapsto \|x-x_0\|\) are convex because norms are convex. These models often appear in least-squares estimation and variational problems. |
|---|
6 Duality and Conjugates (Convex Analysis Core)
6.1 Legendre–Fenchel Conjugate
The Legendre–Fenchel conjugate of a function \(f\) (often extended-valued) is \[ f^*(y)=\sup_x \{\langle y,x\rangle - f(x)\}. \] Conjugation converts minimization of \(f\) into maximization of an associated functional, and it exchanges geometric curvature information between primal and dual formulations. The conjugate is always convex, even when \(f\) is not well-behaved.
6.2 Conjugate Relationships for Standard Functions
For many classical functions, conjugates can be computed explicitly, illustrating the general duality machinery.
6.2.1 Duality for Quadratics
| For \(f(x)=\frac12\|x\|^2\) on a Hilbert space, the conjugate satisfies \(f^*(y)=\frac12\|y\|^2\). With general positive definite quadratics, conjugation transforms the quadratic by inverting the associated operator, reflecting a precise dual relationship. |
|---|
6.2.2 Indicator Functions and Support Functions
If \(f\) is the indicator function of a set \(C\), equal to \(0\) on \(C\) and \(+\infty\) outside, then its conjugate is the support function: \[ \sigma_C(y)=\sup_{x\in C}\langle y,x\rangle. \] This connects constraints in primal problems to geometry of feasible sets and their supporting directions.
6.3 Fenchel–Young Inequality
A foundational inequality relates \(f\) and \(f^*\): \[ \langle y,x\rangle \le f(x)+f^*(y). \] Equivalently, \(\langle y,x\rangle - f(x) \le f^*(y)\) for all \(x,y\). Equality holds under subgradient conditions linking \(x\in \partial f^*(y)\) and \(y\in \partial f(x)\).
6.4 Biconjugation and Lower Semicontinuous Envelopes
The biconjugate \(f^{**}\) is defined as \((f^*)^*\). For a broad class of functions, \(f^{}\) equals the lower semicontinuous convex hull** of \(f\), meaning it is the greatest lower semicontinuous convex function not exceeding the original epigraph information. This provides a canonical “convex and closed” replacement for potentially irregular functions.
7 Convexity in Measure and Integration Contexts
7.1 Convexity Under Integration
When convexity holds pointwise in a parameter, it often persists after integrating with respect to a measure. A common scenario: if \(f(x,\omega)\) is convex in \(x\) for each \(\omega\) and integrability conditions are satisfied, then \(F(x)=\int f(x,\omega)\,d\mu(\omega)\) is convex. This principle is useful for variational models involving expectations or marginalization.
7.2 Jensen’s Inequality in Integral Form
Jensen’s inequality extends from expectations of random variables to integrals with respect to measures. For a convex \(f\) and an integrable function \(g\), \[ f\!\left(\frac{\int g\,d\mu}{\int 1\,d\mu}\right)\le \frac{\int f(g)\,d\mu}{\int 1\,d\mu}, \] under normalization and finiteness assumptions. It converts distributional averaging into deterministic inequalities.
7.3 Convex Functions and Moment Bounds
Convexity can control moments via inequalities like Markov-type bounds and logarithmic moment generating function techniques. When \(f\) is convex, \(f(\mathbb{E}[X])\) provides an upper bound for \(\mathbb{E}[f(X)]\) or vice versa depending on the direction and whether one uses convexity or concavity. In probability and analysis, this yields quantitative estimates from qualitative curvature.
8 Applications in Analysis and Optimization
8.1 Convex Minimization Problems
A convex minimization problem seeks to minimize a convex objective over a convex set. Under mild compactness or coercivity conditions, minimizers exist; under strict or strong convexity, they are unique. Convexity ensures that local information often aligns with global structure, preventing spurious local minima.
8.2 First-Order Optimality Conditions
For differentiable convex objectives, first-order conditions are both necessary and sufficient. If \(x^\star\) is a minimizer of a convex differentiable function \(f\) on a convex set, then \[ \langle \nabla f(x^\star),\, x-x^\star\rangle \ge 0 \quad \text{for all feasible } x, \] equivalently \(-\nabla f(x^\star)\) lies in an appropriate normal cone. For nonsmooth functions, gradients are replaced by subgradients.
8.3 Gradient Methods and Convex Descent
Gradient-based algorithms exploit the fact that convex functions have no “downward curvature traps.” Methods such as gradient descent generate sequences that, under step-size choices and regularity conditions, converge to minimizers. The rate depends on smoothness and strong convexity, connecting algorithmic performance to the analytical properties described earlier.
8.4 Convexity in Inequalities
Many inequality frameworks can be viewed as consequences of convexity combined with Jensen-type arguments or supporting-line techniques.
8.4.1 Karamata-Type Frameworks (Conceptual Use)
Karamata’s inequality is a classical result related to majorization and convex functions. Conceptually, it states that when one vector is more “spread out” in the sense of majorization, applying a convex function and summing yields an inequality reflecting that increased spread. While the majorization definition is separate, convexity supplies the curvature mechanism driving the comparison.
8.4.2 Hermite–Hadamard Inequality (Integral Bounds)
For a convex function \(f\) on an interval \([a,b]\), \[ f\!\left(\frac{a+b}{2}\right)\le \frac{1}{b-a}\int_a^b f(x)\,dx \le \frac{f(a)+f(b)}{2}. \] This gives bounds comparing the average value of \(f\) to endpoint values and to its midpoint value. The result is a direct manifestation of “graph below chords” paired with integral averaging.
9 Common Pitfalls and Technical Conditions
9.1 Domain Issues and Effective Domain
Convexity is defined on a domain. For extended-real-valued functions, the effective domain where the function is finite matters: properties may fail or change if one ignores where \(f\) takes finite values. Many theorems require the domain to be convex and often require nonemptiness of the relative interior for subgradient results.
9.2 Lower Semicontinuity Requirements
Even if a function satisfies a convex inequality, its epigraph might not be closed, and some duality or existence results depend on closedness. Lower semicontinuity ensures that epigraph limits do not introduce “hidden gaps,” stabilizing minimization and conjugation arguments.
9.3 Non-differentiability and Subgradient Interpretation
Convex functions can have corners or kinks. In such cases, gradients may not exist at all points, but subgradients still provide meaningful supporting directions. Treating nondifferentiable points as if gradients must exist is a common mistake; instead, one uses the set-valued subdifferential.
9.4 When “Convex” Does Not Imply Differentiable or Smooth
Convexity alone does not guarantee differentiability, higher-order derivatives, or continuity beyond lower semicontinuity. A convex function may be continuous on the interior of its effective domain while remaining nonsmooth there. Stronger regularity requires additional assumptions such as smoothness, strict convexity with appropriate structure, or conditions related to Lipschitz continuity of subgradients.