1 Definition and geometric intuition
1.1 Convex sets and the “line segment” characterization
A subset \(C\) of a real vector space is convex if for any two points \(x,y\in C\) and any \(t\in[0,1]\), the point \((1-t)x+ty\) also lies in \(C\). Geometrically, this means the straight line segment joining any two points of the set stays entirely within the set. For functions, the same “line segment” idea is applied to the graph or to the regions lying above or below the graph.
1.2 Concave functions and sign conventions
A real-valued function \(f\) defined on an interval (or more generally on a convex domain) is convex if, for all \(x,y\) in the domain and \(t\in[0,1]\), \[ f((1-t)x+ty)\le (1-t)f(x)+tf(y). \] It is concave if the inequality is reversed: \[ f((1-t)x+ty)\ge (1-t)f(x)+tf(y). \] Sign conventions often lead to the practical rule: \(-f\) is convex whenever \(f\) is concave, and \(-f\) is concave whenever \(f\) is convex. This duality is a recurring theme in both theory and computation.
1.3 Jensen midpoint inequality interpretation
A common starting form uses \(t=\tfrac12\). For a convex function, \[ f\!\left(\frac{x+y}{2}\right)\le \frac{f(x)+f(y)}{2}, \] which can be read as: the value at the midpoint lies below the midpoint of the values. Concave functions reverse the inequality. This “midpoint” perspective foreshadows Jensen’s inequality, which generalizes the midpoint statement from two points to arbitrary convex combinations.
2 Equivalent characterizations
2.1 Supporting hyperplanes (subgradients/supergradients)
For convex functions, a geometric characterization uses supporting hyperplanes. In finite dimensions, \(f\) is convex on a convex set \(C\) exactly when, at each point \(x\) in the (effective) domain, there exists an affine function that lies below \(f\) everywhere and touches it at \(x\). When derivatives may not exist, the relevant objects are subgradients: vectors \(g\) such that \[ f(y)\ge f(x)+\langle g, y-x\rangle \] for all \(y\) in the domain. Concave functions admit analogous supporting affine functions from above using supergradients.
2.2 Epigraph and hypograph viewpoints
Two standard sets encode a function’s curvature. The epigraph of \(f\) is \[ \mathrm{epi}(f)=\{(x,t): t\ge f(x)\}, \] and the hypograph is \[ \mathrm{hypo}(f)=\{(x,t): t\le f(x)\}. \] A function \(f\) is convex if and only if its epigraph is a convex set; it is concave if and only if its hypograph is convex. This viewpoint is especially useful in optimization and in studying nonsmooth functions.
2.3 First-derivative (slope) criteria
In one variable, convexity can be characterized through behavior of secant slopes and derivatives. Informally, a convex function has nondecreasing “slope as you move right”: for \(x<y\), the slope of the chord \(\frac{f(y)-f(x)}{y-x}\) increases as the interval slides rightward. Where \(f'\) exists, the derivative itself is nondecreasing for convex functions and nonincreasing for concave ones.
2.4 Second-derivative criteria in one variable
For twice differentiable functions \(f\) on an open interval, convexity is equivalent to nonnegativity of the second derivative: \[ f \text{ convex} \iff f''(x)\ge 0, \] and concavity corresponds to \[ f \text{ concave} \iff f''(x)\le 0. \] This is the most familiar calculus test. At points where derivatives do not exist or the function is not smooth, the second-derivative test may fail even when curvature still has a generalized meaning.
2.5 Second-derivative and Hessian criteria in several variables
For a twice continuously differentiable function \(f:\mathbb{R}^n\to\mathbb{R}\), convexity can be checked using the Hessian matrix \(\nabla^2 f(x)\). The function is convex on a convex region if and only if the Hessian is positive semidefinite everywhere on that region (and concave if it is negative semidefinite). In practice, positive semidefiniteness means \[ v^\top \nabla^2 f(x)\, v \ge 0 \] for all directions \(v\).
3 Basic calculus tools for concavity/convexity
3.1 Monotonicity of the derivative and secant slopes
For differentiable functions on an interval, convexity implies that the derivative increases with the argument. Equivalently, for any \(a<b<c\), the secant slopes satisfy \[ \frac{f(b)-f(a)}{b-a}\le \frac{f(c)-f(b)}{c-b} \] for convex \(f\). These relations can be derived from supporting line inequalities or from the monotonicity of \(f'\). Concavity yields the reverse inequalities.
3.2 Tangent-line upper/lower bounds
If \(f\) is convex and differentiable at \(x\), then the tangent line at \(x\) lies below the graph: \[ f(y)\ge f(x)+f'(x)(y-x)\quad \text{for all }y. \] For concave functions, tangents lie above: \[ f(y)\le f(x)+f'(x)(y-x). \] These bounds extend to multivariate settings via supporting hyperplanes. They are central in gradient-based methods and in proofs of Jensen-type inequalities.
3.3 Relationships with Lipschitz continuity of derivatives (smooth case)
| When \(f\) is twice differentiable and its Hessian is bounded in operator norm, derivatives inherit Lipschitz continuity. For instance, if \(\|\nabla^2 f(x)\|\le L\) on a region, then |
|---|
\[
| \|\nabla f(x)-\nabla f(y)\|\le L\|x-y\|. |
|---|
\] This link connects curvature properties (convexity through semidefiniteness) with quantitative smoothness (through bounded second derivatives). Such estimates yield error bounds for first-order approximations.
3.4 Strict vs non-strict concavity/convexity
A function is strictly convex if the defining inequality is strict for distinct \(x,y\) and for all \(t\in(0,1)\). Similarly, strict concavity reverses the strictness. Strictness implies uniqueness of minimizers/maximizers in many settings, though it requires care: a function may be convex but still linear on some region, losing strictness there.
4 Examples and canonical function families
4.1 Polynomials and power functions
Many polynomial behaviors can be inferred from the sign of derivatives. For example, \(x^p\) for \(p\ge 1\) is convex on \((0,\infty)\), while \(x^p\) for \(0<p\le 1\) is concave on \((0,\infty)\). Quadratic functions \(ax^2+bx+c\) are convex when \(a>0\) and concave when \(a<0\). Higher-degree polynomials can be convex only on restricted intervals, where the second derivative maintains a fixed sign.
4.2 Exponential, logarithm, and reciprocal functions
The exponential function \(e^x\) is convex on all \(\mathbb{R}\), while the logarithm \(\log x\) is concave on \((0,\infty)\). The reciprocal \(1/x\) is convex on \((0,\infty)\) and concave on \((-\infty,0)\). These examples form a basic toolkit for modeling curvature in both theory and applications.
4.3 Norms and absolute value
| The absolute value \( | x | \) is convex on \(\mathbb{R}\). More generally, norms are convex: for a norm \(\|\cdot\|\) on a vector space, the mapping \(x\mapsto \|x\|\) is convex. Squared Euclidean norm \(\|x\|_2^2\) is also convex (and strictly convex), a feature commonly used in regularization and least-squares formulations. |
|---|
4.4 Trigonometric examples on restricted intervals
Sine and cosine are not globally convex on their full natural domains, but they can be convex or concave on restricted intervals where their second derivatives keep a consistent sign. For instance, \(\sin x\) is convex on intervals where \(-\sin x\ge 0\) holds, and concave where the sign reverses. Such localized convexity is useful in approximation and bounding arguments.
4.5 Piecewise-defined functions and “kinks”
| Piecewise linear functions are convex exactly when their slopes do not decrease across breakpoints. At a “kink,” derivatives jump while a generalized supporting-line property still holds. The absolute value function \( | x | \) is a canonical example: it is linear on each side of the origin but convex overall because the slope increases from \(-1\) to \(+1\). Non-differentiability is therefore compatible with strong convexity properties. |
|---|
5 Operations and closure properties
5.1 Sums and positive scalar multiples
Convexity is preserved under sums: if \(f\) and \(g\) are convex on a common convex domain, then \(f+g\) is convex. Likewise, multiplying by a nonnegative scalar preserves convexity: for \(\alpha\ge 0\), \(\alpha f\) is convex. When the scalar is negative, convexity flips to concavity, reflecting the earlier sign convention.
5.2 Pointwise maxima and minima
The pointwise maximum of convex functions is convex. Similarly, the pointwise minimum of concave functions is concave. These results follow from the epigraph/hypograph interpretation: the epigraph of \(\max(f,g)\) can be expressed as an intersection of convex sets. Pointwise minimum of convex functions need not be convex and may be neither convex nor concave, except in special cases.
5.3 Composition rules (with domain/range restrictions)
Composition can preserve convexity when outer and inner functions have compatible monotonicity and curvature. A typical smooth rule: if \(f\) is convex and nondecreasing, and \(g\) is convex, then \(f\circ g\) is convex on the relevant domain. If \(f\) is convex and nonincreasing, then \(f\circ g\) is convex when \(g\) is concave. Similar patterns apply for concavity. Careful attention to monotonicity and range constraints is essential.
5.4 Perspective transforms (where applicable)
The perspective transform provides a powerful closure property in optimization: for a convex function \(f\), the function \[ (x,t)\mapsto t\, f(x/t) \] is jointly convex on appropriate domains where \(t>0\). This technique is commonly used to derive convex formulations from simpler ones, though its applicability depends on the functional form and domain scaling behavior.
5.5 Limits, uniform convergence, and preservation of convexity
Convexity behaves well under limits. For example, if a sequence of convex functions converges pointwise to a finite limit on the interior of the domain, the limit is convex under suitable conditions. Uniform convergence on compact sets can also ensure preservation of convexity and related regularity properties. These results justify approximating complicated convex functions by simpler ones.
6 Inequalities derived from concavity/convexity
6.1 Jensen’s inequality
Jensen’s inequality states that for a convex function \(f\) and points \(x_i\) with weights \(\lambda_i\ge 0\) summing to 1, \[ f\Big(\sum_i \lambda_i x_i\Big)\le \sum_i \lambda_i f(x_i). \] For concave functions the inequality reverses. Jensen’s inequality is often interpreted probabilistically: if \(X\) is a random variable and \(f\) is convex, then \(f(\mathbb{E}[X])\le \mathbb{E}[f(X)]\), provided expectations are well defined.
6.2 Tangent line inequality and its corollaries
Supporting-line inequalities yield many corollaries. In the differentiable case, convexity gives \[ f(y)\ge f(x)+f'(x)(y-x), \] which can be rearranged into bounds comparing function values with linear approximations. These statements become tools for estimating integrals, proving inequalities, and deriving upper or lower envelopes.
6.3 Hermite–Hadamard-type bounds (where applicable)
For a convex function \(f\) on an interval \([a,b]\), the function value at the midpoint and the average value of \(f\) are ordered: \[ f\Big(\frac{a+b}{2}\Big)\le \frac{1}{b-a}\int_a^b f(x)\,dx \le \frac{f(a)+f(b)}{2}. \] The inequalities reverse for concave functions. These bounds link pointwise evaluations to integral means and are useful in numerical analysis and estimation.
6.4 Karamata majorization connection (high level)
Karamata’s inequality connects convexity to majorization order. Roughly, if one vector is more “spread out” than another in a majorization sense, then applying a convex function and summing yields an inequality. This perspective generalizes Jensen’s inequality and provides a structured method for proving many classical inequalities.
7 Convexity in optimization contexts (analytic perspective)
7.1 Local vs global optimality for convex problems
Convex optimization benefits from a fundamental geometric fact: any local minimum of a convex function over a convex feasible region is a global minimum. Similarly, any local maximum of a concave objective is global. This reduces the risk of spurious optima and underpins the reliability of many algorithms.
7.2 First-order optimality conditions and supporting hyperplanes
At an unconstrained optimum \(x^\ast\) of a differentiable convex function, the gradient vanishes: \(\nabla f(x^\ast)=0\). In constrained or nonsmooth settings, optimality can be expressed using subgradients and supporting hyperplanes: a point is optimal when there exists an appropriate supporting affine function that majorizes or minorizes the objective over feasible directions.
7.3 Role of convexity in constrained settings (conceptual)
When constraints define a convex feasible set and the objective is convex (for minimization), the overall problem remains convex. This structure allows separation and duality ideas to work cleanly, often enabling the transformation of difficult primal problems into more tractable dual ones. Even without detailing proofs, the guiding principle is that convexity preserves tractable geometry under formulation.
7.4 Special functions used as losses (example patterns)
Many common loss functions in statistics and machine learning are convex in their prediction argument, such as squared error, absolute error, and logistic-type losses. Convexity supports stable optimization, enabling gradient methods and guaranteeing that a descent step does not move toward artificial local minima. The specific functional form determines whether convexity is global or only holds under constraints like domain restrictions.
8 Generalizations and related concepts
8.1 Quasi-convexity and quasi-concavity
A function is quasi-convex if its sublevel sets \(\{x: f(x)\le \alpha\}\) are convex for every real \(\alpha\). Equivalently, along any line segment, the function value does not exceed the maximum of endpoint values. Quasi-concavity is defined similarly with convex superlevel sets. These notions relax strict “midpoint bowing” into set-based curvature.
8.2 Strong convexity and strong concavity
Strong convexity strengthens convexity by adding a quadratic term. A function \(f\) is \(\mu\)-strongly convex if \[
| f((1-t)x+ty)\le (1-t)f(x)+tf(y)-\frac{\mu}{2}t(1-t)\|x-y\|^2 |
|---|
\] for \(\mu>0\). Strong concavity is the analogous reversed inequality. This property implies not only uniqueness of minimizers but also quantitative error bounds and faster convergence rates in optimization algorithms.
8.3 Smoothness assumptions vs non-smooth convex analysis
Classical derivative tests require smoothness, but convexity can be studied without differentiability. Tools from convex analysis—subgradients, normal cones, and separation theorems—extend the theory to kinks and absolute-value-type behavior. Many modern applications rely on these nonsmooth techniques because real models often produce nondifferentiable objectives.
8.4 Subdifferentials and superdifferentials overview
For convex functions, the subdifferential at a point \(x\) is the set of subgradients—vectors defining supporting affine underestimates. For concave functions, superdifferentials capture supporting affine overestimates. These sets generalize the gradient: when \(f\) is differentiable, the subdifferential collapses to the single vector \(\nabla f(x)\). Existence and boundedness of subgradients depend on domain properties and regularity conditions.
8.5 Biconvex and separately convex functions (basic)
A function of two variables \(f(x,y)\) is biconvex if it is convex in \(x\) for each fixed \(y\) and convex in \(y\) for each fixed \(x\), though it need not be jointly convex. Similarly, separate convexity holds when convexity applies in each coordinate or block while keeping others fixed. Biconvex problems often have more complex optimization behavior than jointly convex problems, but the definition provides a structured intermediate class.
9 Practical checks and common pitfalls
9.1 Verifying convexity from derivatives vs from inequalities
Derivative-based checks are convenient when smoothness holds: verify sign of second derivatives in one dimension or positive semidefiniteness of Hessians in higher dimensions. If derivatives fail, use inequality-based tests such as midpoint convexity, supporting-line inequalities, or epigraph convexity. For many piecewise functions, convexity can be inferred from slope monotonicity even when differentiability breaks at finitely many points.
9.2 Handling domain restrictions and endpoints
Convexity is always relative to the domain where the function is defined. A function may be convex on \((0,\infty)\) but not on \(\mathbb{R}\). Endpoints matter: if a function is defined only on a closed interval, one-sided checks or extension conventions may be required. Misinterpreting the domain is a frequent source of incorrect classification.
9.3 Mistakes with sign changes and function negation
Negating a function flips convexity to concavity. Thus, it is not unusual to accidentally apply convex rules to \(-f\) without adjusting inequalities. Similarly, composing with a decreasing function may reverse curvature behavior under composition rules, so monotonicity must be tracked carefully.
9.4 Dealing with non-differentiability (subgradient viewpoint)
At nondifferentiable points, second-derivative tests break down and naive derivative computations may be invalid. The correct approach is to use supporting line inequalities via subgradients. For example, absolute value and hinge-like losses are convex despite lacking derivatives at a kink. Recognizing this often turns a “failure” of smooth tests into a correct convex classification.
10 Further reading and references
10.1 Core textbooks in convex/concave analysis
Classical references include introductory and advanced texts on convex analysis, optimization, and functional inequalities. Standard choices emphasize geometric interpretations (epigraphs/hypographs), duality, and nonsmooth tools like subgradients.
10.2 Classic inequalities and proof techniques
For inequality-driven development, readers often consult materials covering Jensen’s inequality, tangent-line methods, and majorization. Proof techniques commonly rely on convex combinations, supporting hyperplanes, and careful use of monotonicity or integral averaging.
10.3 Suggested exercises and verification problems
Effective practice problems include: classifying standard functions on various intervals, checking convexity after algebraic operations (sums, maxima, scalar multiples), testing composition rules with different monotonicity patterns, and constructing counterexamples where an operation does not preserve convexity. Verification tasks also benefit from comparing derivative-based tests with epigraph-based definitions.