In applied mathematics and machine learning, a loss function (also known as a cost function or objective function) quantifies the discrepancy between the predicted output of a model and the actual target value. It serves as a scalar measure of error that guides the optimization process: by minimizing the loss over a training dataset, models learn to make accurate predictions. Loss functions are central to supervised learning algorithms, including regression, classification, and ranking tasks, and their design directly influences model behavior, convergence speed, and robustness. Different loss functions encode different assumptions about noise, outliers, and the desired trade-off between bias and variance.
1 Mathematical foundations
1.1 Definition and notation
Let a dataset consist of input–output pairs \((x_i, y_i)\) for \(i=1,\dots,n\), where \(x_i \in \mathcal{X}\) is the feature vector and \(y_i \in \mathcal{Y}\) is the true label or value. A model produces a prediction \(\hat{y}_i = f(x_i; \theta)\) parameterized by \(\theta\). A loss function \(L(y, \hat{y})\) maps a true–predicted pair to a nonnegative real number, with \(L(y, \hat{y}) = 0\) indicating a perfect prediction. The total loss over the dataset is typically the average \(\frac{1}{n}\sum_{i=1}^n L(y_i, \hat{y}_i)\).
1.2 Properties of loss functions
1.2.1 Convexity
A loss function is convex if its Hessian matrix is positive semidefinite, ensuring that any local minimum is global. Convex losses (e.g., squared error, hinge loss) are preferred for optimization because gradient descent converges reliably. Non-convex losses (e.g., zero-one loss, neural network losses with hidden units) may have multiple local minima, requiring careful initialization and advanced optimization strategies.
1.2.2 Symmetry and asymmetry
A loss is symmetric if it penalizes overestimation and underestimation equally (e.g., squared error). Asymmetric losses assign different penalties to errors of opposite sign, useful when undershooting or overshooting has different practical consequences (e.g., quantile loss).
1.2.3 Differentiability and subgradients
Many optimization algorithms require gradients. Smooth losses (e.g., squared error) are twice differentiable everywhere. Non-smooth losses (e.g., absolute error, hinge loss) are differentiable almost everywhere but have points of nondifferentiability; subgradients or subdifferentials extend the notion of a gradient to these points.
1.3 Relationship to risk and empirical risk minimization
In statistical learning theory, the risk \(R(f) = \mathbb{E}_{(x,y)\sim P}[L(y, f(x))]\) is the expected loss under the true data distribution. Since the distribution is unknown, we minimize the empirical risk \(\hat{R}(f) = \frac{1}{n}\sum_i L(y_i, f(x_i))\). This is the principle of empirical risk minimization (ERM). The choice of loss function determines how well the empirical risk approximates the true risk and influences generalization.
2 Regression loss functions
2.1 Squared error (L2 loss)
2.1.1 Mathematical form
The squared error loss is defined as \(L(y, \hat{y}) = (y - \hat{y})^2\). For multiple outputs, the sum or mean of squared errors is used. It is convex and differentiable, leading to closed-form solutions for linear models (ordinary least squares).
2.1.2 Sensitivity to outliers
Because errors are squared, large deviations are penalized heavily. This makes the L2 loss sensitive to outliers: a single extreme value can dominate the average loss and skew the model fit.
2.2 Absolute error (L1 loss)
2.2.1 Robustness properties
| The absolute error \(L(y, \hat{y}) = | y - \hat{y} | \) penalizes all deviations linearly, reducing the influence of outliers compared to squared error. The median of the data minimizes the sum of absolute errors, whereas the mean minimizes the sum of squared errors. |
|---|
2.2.2 Nondifferentiability at zero
The absolute value function is not differentiable at \(y = \hat{y}\). In practice, subgradient methods or smooth approximations (e.g., Huber loss) are used during gradient-based optimization.
2.3 Huber loss
2.3.1 Combination of L1 and L2
The Huber loss is defined piecewise: for small residuals (below a threshold \(\delta\)), it behaves like squared error; for larger residuals, it transitions to linear absolute error. This gives a smooth, differentiable function that is less sensitive to outliers than L2 loss while remaining convex.
2.3.2 Tuning parameter delta
The parameter \(\delta > 0\) controls the transition point. Smaller \(\delta\) makes the loss more robust (closer to L1), while larger \(\delta\) makes it behave more like L2 for a wider range of residuals. The choice is often data-dependent.
2.4 Quantile loss
2.4.1 Asymmetric penalties
For a given quantile \(\tau \in (0,1)\), the quantile loss is \(L(y,\hat{y}) = \max(\tau(y-\hat{y}), (\tau-1)(y-\hat{y}))\). It penalizes underestimation and overestimation asymmetrically, with relative weight \(\tau\) and \(1-\tau\).
2.4.2 Use in quantile regression
Minimizing the quantile loss yields the conditional \(\tau\)-quantile of the target distribution. This is useful for estimating prediction intervals and analyzing heteroscedasticity.
2.5 Log-cosh loss
2.5.1 Smooth approximation to L1
The log-cosh loss is defined as \(L(y,\hat{y}) = \log(\cosh(y - \hat{y}))\). It approximates the absolute error for large residuals but is twice differentiable, making it amenable to Newton-type optimizers.
2.5.2 Numerical stability
| For very large residuals, \(\cosh\) can overflow. Implementations often use the identity \(\log(\cosh(z)) = | z | + \log(1+e^{-2 | z | }) - \log 2\), which is numerically stable. The loss offers a smooth alternative to Huber without an extra tuning parameter. |
|---|
3 Classification loss functions
3.1 Zero-one loss
3.1.1 Non-convexity and computational intractability
Zero-one loss assigns 0 for correct classification and 1 for incorrect. It is non-convex and discontinuous, making direct minimization NP-hard in general. It is the true objective for classification accuracy but is rarely used directly in optimization.
3.1.2 Surrogate losses
Because zero-one loss is intractable, convex surrogates (hinge, logistic, exponential) are used during training. These upper-bound the zero-one loss and are consistent under appropriate conditions: minimizing the surrogate also minimizes zero-one loss asymptotically.
3.2 Hinge loss
3.2.1 Support vector machines
The hinge loss for binary classification (\(y \in \{-1,1\}\)) is \(L(y,\hat{y}) = \max(0, 1 - y\hat{y})\), where \(\hat{y}\) is the raw score. The support vector machine (SVM) minimizes the sum of hinge losses plus a regularization term. It produces a maximum-margin separating hyperplane.
3.2.2 Margin maximization interpretation
The hinge loss encourages the classifier to produce a score with magnitude at least 1 for correct classification. Points with \(y\hat{y} \ge 1\) incur zero loss; points within the margin incur a linear penalty, driving the decision boundary away from the data.
3.3 Logistic loss (cross-entropy)
3.3.1 Probabilistic interpretation
For binary classification with labels \(y \in \{0,1\}\) and predicted probability \(\hat{p}\), logistic loss is \(L(y,\hat{p}) = -[y\log\hat{p} + (1-y)\log(1-\hat{p})]\). Minimizing this is equivalent to maximizing the likelihood under a Bernoulli model. The loss is smooth and convex in the linear predictor.
3.3.2 Binary and multi-class formulations
The binary case extends to multi-class using the softmax function: \(L(y,\hat{\mathbf{p}}) = -\sum_{k=1}^K y_k \log \hat{p}_k\), where \(\hat{p}_k\) is the predicted probability for class \(k\). This is known as categorical cross-entropy and is standard for neural network classifiers.
3.4 Exponential loss
3.4.1 Connection to AdaBoost
The exponential loss is \(L(y,\hat{y}) = e^{-y\hat{y}}\) for \(y \in \{-1,1\}\). It is used in the AdaBoost algorithm, where the loss drives sequential reweighting of training examples. Minimizing exponential loss leads to a classifier that approximates the log-odds.
3.4.2 Sensitivity to label noise
Exponential loss penalizes misclassifications very heavily (exponentially) and is sensitive to outliers and label noise. It does not have a probabilistic interpretation that accounts for uncertainty, making it less robust than logistic loss in noisy settings.
3.5 Perceptron loss
3.5.1 Online learning context
The perceptron loss is \(L(y,\hat{y}) = \max(0, -y\hat{y})\). It is a variant of hinge loss without a margin requirement. The perceptron algorithm minimizes it online by updating weights only on misclassified examples.
3.5.2 Lack of margin
Unlike hinge loss, perceptron loss does not enforce a margin; a correctly classified point incurs zero loss even if it is arbitrarily close to the decision boundary. This leads to solutions that may not generalize as well as margin-based classifiers.
4 Ranking and structured prediction loss functions
4.1 Pairwise ranking loss
4.1.1 Margin ranking loss
For a pair of items \((i,j)\) where \(i\) should be ranked higher than \(j\), the margin ranking loss is \(L = \max(0, m - (s_i - s_j))\), where \(s_i, s_j\) are scores and \(m>0\) is a margin. This encourages the correct ordering with at least a margin difference.
4.1.2 Bayesian personalized ranking
Bayesian Personalized Ranking (BPR) uses a pairwise loss derived from a probabilistic model of user preferences. The BPR loss is \(-\log \sigma(s_i - s_j)\), which is smooth and encourages ordering without an explicit margin.
4.2 Listwise ranking loss
4.2.1 NDCG and MAP surrogates
Normalized Discounted Cumulative Gain (NDCG) and Mean Average Precision (MAP) are standard evaluation metrics for ranking. Surrogate losses approximating these metrics (e.g., smooth versions of ranking measures) are used in learning to rank because the true metrics are non-differentiable.
4.2.2 LambdaRank
LambdaRank defines a gradient for each document pair based on the change in NDCG or MAP from swapping their positions. The loss function is not explicitly defined; instead, gradients (lambda gradients) are computed directly. This approach achieved state-of-the-art ranking performance.
4.3 Structured perceptron loss
For structured prediction (e.g., sequence labeling, parsing), the structured perceptron loss extends the binary perceptron loss to joint input–output spaces. The loss for a true structure \(y\) and predicted \(\hat{y}\) is \(L = \Phi(x,y) - \Phi(x,\hat{y})\), where \(\Phi\) is a feature function. The algorithm updates weights when the score of the true structure is not higher than that of the predicted.
4.4 Conditional random field loss
| Conditional Random Fields (CRFs) use the negative log-likelihood as the loss: \(L = -\log P(y | x) = \log Z(x) - \sum_k \theta_k F_k(x,y)\), where \(Z(x)\) is the partition function. This loss is convex and allows incorporating dependencies between output variables. Inference and gradient computation often require dynamic programming. |
|---|
5 Regularization and composite losses
5.1 Loss plus regularization (Lagrangian form)
Many learning problems minimize a composite objective: total loss + \(\lambda R(\theta)\), where \(R\) is a regularization term and \(\lambda > 0\) controls the trade-off. This can be interpreted as a Lagrangian for a constrained optimization problem.
5.1.1 L1 and L2 penalties
| L2 regularization (ridge) adds \(\|\theta\|_2^2\), encouraging small weights and reducing variance. L1 regularization (lasso) adds \(\|\theta\|_1\), promoting sparsity (some weights exactly zero). L1–L2 combinations form the elastic net. |
|---|
5.1.2 Elastic net
| The elastic net penalty is \(\lambda_1 \|\theta\|_1 + \lambda_2 \|\theta\|_2^2\). It combines the sparsity of L1 with the grouping effect of L2, useful when there are correlated features. |
|---|
5.2 Adversarial loss
5.2.1 Generating adversarial examples
Adversarial training augments the loss with a term that penalizes sensitivity to small worst-case perturbations of the input: \(L_{\text{adv}} = L(y, f(x+\delta))\) where \(\delta\) is chosen to maximize the loss within a norm ball.
5.2.2 Robust optimization viewpoint
This approach treats training as a robust optimization problem, minimizing the worst-case loss over a neighborhood of inputs. It improves model robustness against adversarial attacks and often leads to smoother decision boundaries.
5.3 Curriculum loss weighting
Curriculum learning dynamically weights training examples based on their difficulty. Early in training, easier examples receive higher weight; later, harder examples are emphasized. The loss becomes \(\sum_i w_i(t) L(y_i, \hat{y}_i)\), where \(w_i(t)\) changes with iteration \(t\). This can improve convergence and final accuracy.
6 Custom and application-specific loss functions
6.1 Focal loss (class imbalance)
6.1.1 Modulating factor gamma
Focal loss modifies the cross-entropy loss for binary classification: \(L = -\alpha (1-p_t)^\gamma \log(p_t)\), where \(p_t\) is the probability of the true class. The modulating factor \((1-p_t)^\gamma\) down-weights well-classified examples, focusing training on hard, misclassified samples. The parameter \(\gamma \ge 0\) controls the down-weighting rate.
6.1.2 Object detection applications
Focal loss was introduced for dense object detectors (e.g., RetinaNet) to handle extreme class imbalance between foreground and background. It enabled high accuracy without a separate sampling stage.
6.2 Dice loss (segmentation)
Dice loss is derived from the Dice coefficient (F1 score) for segmentation tasks: \(L = 1 - \frac{2\sum y\hat{y}}{\sum y + \sum \hat{y}}\), where \(y\) and \(\hat{y}\) are binary or continuous maps. It is robust to class imbalance and directly optimizes overlap. Variants include generalized Dice and Tversky loss.
6.3 Triplet loss (metric learning)
6.3.1 Anchor-positive-negative formulation
Triplet loss operates on triplets: an anchor, a positive (same class), and a negative (different class). It enforces that the distance between anchor and positive is smaller than the distance between anchor and negative by a margin: \(L = \max(0, d(a,p) - d(a,n) + m)\).
6.3.2 Margin hyperparameter
The margin \(m>0\) controls how much separation is enforced. Too large a margin makes training difficult; too small may not produce well-separated embeddings.
6.4 Contrastive loss (siamese networks)
Contrastive loss is used in siamese networks for similarity learning. For pairs, \(L = \frac{1}{2}(1-y)d^2 + \frac{1}{2}y\max(0, m-d)^2\), where \(y=1\) for dissimilar pairs, \(y=0\) for similar, and \(d\) is the Euclidean distance between embeddings. It pulls similar pairs together and pushes dissimilar pairs apart beyond a margin.
7 Optimization and convergence considerations
7.1 Gradient descent variants
7.1.1 Stochastic and mini-batch
Full-batch gradient descent uses the entire dataset per update, which can be slow for large data. Stochastic gradient descent (SGD) uses one sample at a time, introducing noise that can help escape local minima. Mini-batch GD is a compromise, using a small random subset each iteration, balancing speed and stability.
7.1.2 Adaptive methods (Adam, RMSprop)
Adaptive optimizers maintain per-parameter learning rates based on gradient history. RMSprop scales learning rates by the root mean square of recent gradients; Adam combines momentum with RMSprop-like scaling. These methods are effective for non-convex deep learning losses.
7.2 Smoothness and Lipschitz constants
The smoothness of a loss function (bounded second derivatives) affects convergence rates. Lipschitz continuity of the gradient ensures that gradient descent with appropriate step size converges linearly for convex functions. Non-smooth losses require subgradient methods, which converge more slowly.
7.3 Loss landscape visualization
7.3.1 Local minima and saddle points
The loss landscape for deep networks contains many local minima and saddle points. Saddle points (zero gradient but not extremum) are more numerous than local minima in high dimensions. Adaptive methods and momentum help navigate these structures.
7.3.2 Mode connectivity
Recent studies show that local minima found by SGD are often connected by low-loss pathways. This suggests that the loss landscape is not as fragmented as previously thought. Mode connectivity allows interpolation between solutions with only modest loss increase.
8 Evaluation and selection criteria
8.1 Consistency with the evaluation metric
The loss function should be aligned with the final evaluation metric (e.g., accuracy, F1, AUC). Surrogate losses that are convex and differentiable are often necessary for optimization, but their minimization should ideally lead to good performance on the true metric. Consistency (or calibration) ensures that minimizing the surrogate yields optimal for the target.
8.2 Computational efficiency
Loss functions must be computable quickly, especially for large datasets. Closed-form gradients, batch-friendly implementations, and vectorization are important. Some losses (e.g., CRF loss with partition function) may require approximate inference for tractability.
8.3 Robustness to noise and outliers
The presence of label noise or outliers can degrade model performance. Loss functions like Huber or L1 are more robust than squared error. In classification, logistic loss is more robust than exponential loss. Robust losses often have bounded influence functions.
8.4 Interpretability of loss values
In practice, loss values are monitored during training. Losses that are on an intuitive scale (e.g., mean absolute error in original units) aid debugging. Cross-entropy losses have a probabilistic interpretation; raw values can be compared to theoretical minima (e.g., entropy of the data distribution).