1 Definition
Machine epsilon is a measure of the spacing of floating-point numbers near 1. It is used to describe the precision of a number system and to estimate the smallest relative change that can be represented around unity. In numerical analysis, the term is central because it helps quantify how much rounding can alter computed results.
1.1 Floating-point representation
A floating-point system stores numbers in a finite format built from a significand and an exponent. Because only finitely many values can be represented, real numbers are rounded to nearby machine-representable values. The density of these values varies across the number line, and the spacing near 1 is especially important for defining machine epsilon.
1.2 Formal definition
Machine epsilon is commonly defined as the smallest positive number ε such that 1 + ε is distinguishable from 1 in a given floating-point system. In other words, adding ε to 1 produces a value that the machine can represent as different from 1. This definition reflects the granularity of arithmetic near unity, where relative accuracy is often assessed.
1.3 Alternative conventions
The term machine epsilon is not used uniformly in all texts and programming contexts. Some authors define it as the distance from 1 to the next larger representable number, while others use it in connection with the maximum relative rounding error. As a result, the same word can refer to related but distinct quantities.
1.3.1 Gap above 1
Under one convention, machine epsilon is the gap between 1 and the next representable floating-point number greater than 1. This value is directly tied to the representational spacing of the format. It is often the quantity returned by language libraries that expose a precision constant.
1.3.2 Unit roundoff
Another convention defines epsilon as unit roundoff, the largest relative error introduced by rounding to nearest. In many binary systems this is half the gap above 1. This distinction matters because formulas in textbooks or software documentation may assume one convention or the other.
2 Computation
The value of machine epsilon can be derived from the base of the number system and the precision of the significand. It can also be found experimentally by repeatedly halving a number until adding it to 1 no longer changes the result. Both approaches reveal the finite resolution of the format.
2.1 Derivation from radix and precision
For a normalized floating-point system with radix β and precision p, the spacing near 1 is determined by the least significant digit of the significand. The gap above 1 is typically β^(1-p). When rounding to nearest is used, unit roundoff is often half of that amount, depending on the adopted convention.
2.2 Machine epsilon in common formats
Standard floating-point formats have well-known epsilon values because their base and precision are fixed. These values are important for portability, since numerical behavior differs across systems and data types.
2.2.1 Single-precision
In IEEE 754 single-precision binary floating-point, the gap above 1 is 2^-23. If unit roundoff is used instead, the value is 2^-24. Single-precision arithmetic therefore provides modest accuracy, sufficient for many graphics, simulations, and embedded computations.
2.2.2 Double-precision
In IEEE 754 double-precision binary floating-point, the gap above 1 is 2^-52. Under the unit-roundoff convention, it is 2^-53. Double precision is widely used because it offers substantially finer resolution and lower rounding error than single precision.
2.2.3 Extended precision
Extended-precision formats provide more significand bits than double precision, yielding a smaller epsilon. The exact value depends on the architecture and format specification. Such precision is useful in intermediate calculations, scientific software, and hardware that supports widened registers.
2.3 Empirical determination
A common experimental method starts with ε = 1 and repeatedly divides by 2 until 1 + ε equals 1 in floating-point arithmetic. The last value for which the sum still changes 1 is then related to machine epsilon under the chosen convention. This procedure is simple, but the result depends on rounding behavior and the definition being used.
3 Numerical significance
Machine epsilon plays a key role in understanding why computed values differ from exact mathematical results. It provides a baseline for the expected size of rounding effects and is often used when analyzing the reliability of algorithms.
3.1 Rounding error
Every floating-point operation may introduce rounding error because the exact result may not be representable. Machine epsilon gives a scale for how large such errors can be near 1. Although errors can accumulate across many operations, epsilon helps characterize the smallest meaningful perturbation.
3.2 Relative error bounds
Many numerical estimates are expressed as relative error bounds proportional to machine epsilon. For well-behaved operations, the computed result often differs from the exact value by a factor on the order of ε. This is why epsilon is frequently used in derivations of floating-point error models.
3.3 Conditioning and stability
The sensitivity of a problem to small input changes is described by its conditioning, while the behavior of an algorithm is described by its stability. Machine epsilon is used in both contexts as a reference scale for perturbations. A stable algorithm typically controls the propagation of errors so that results remain close to the ideal computation.
4 Relation to floating-point arithmetic
Machine epsilon is inseparable from the structure of floating-point arithmetic. It depends on how numbers are encoded, how rounding is performed, and how extreme values such as subnormals are treated.
4.1 Significand and exponent
A floating-point number is represented by a significand, which controls precision, and an exponent, which controls scale. Epsilon is determined by the number of significand bits rather than the exponent range. Thus, changing the precision alters epsilon, while changing only the exponent range does not.
4.2 Rounding modes
Different rounding modes can affect how one interprets epsilon in practice. Round-to-nearest is the most common mode and underlies many textbook formulas. Directed rounding modes, such as rounding toward zero or toward infinity, can change the size and direction of rounding error even though the underlying spacing of representable numbers remains the same.
4.3 Subnormal numbers
Subnormal numbers fill the gap near zero with reduced precision. They do not alter machine epsilon near 1, but they extend the representable range for very small magnitudes. Their presence is important in applications where gradual underflow helps preserve tiny values instead of flushing them immediately to zero.
5 Algorithms and programming
Programming languages and numerical libraries often expose machine epsilon or a related constant. Developers use it when comparing values, choosing tolerances, and writing code that behaves consistently across platforms.
5.1 Language-specific constants
Many languages provide a built-in constant for floating-point precision, often in standard libraries or type metadata. The exact name and meaning vary by language and data type. Care is needed to determine whether the constant corresponds to the gap above 1 or to unit roundoff.
5.2 Library functions
Some numerical libraries offer functions that compute the machine epsilon for a given floating-point type. These functions are useful for portable code because they adapt to the active representation. They are also helpful in generic programming, where code may operate on multiple precisions.
5.3 Comparison strategies
Because floating-point values are approximate, direct equality testing can be unreliable for results of arithmetic. Machine epsilon is often used to design comparisons that tolerate small discrepancies. The appropriate strategy depends on the magnitudes being compared and the source of the error.
5.3.1 Equality testing
Exact equality is suitable for values that should match bit-for-bit, such as copied constants or identical computations. For quantities produced by different arithmetic paths, exact comparison may fail even when results are effectively the same. Epsilon-based reasoning helps explain why such failures occur.
5.3.2 Tolerance-based tests
A tolerance test accepts two values as equal if their difference is smaller than a chosen threshold. That threshold may be based on machine epsilon, often scaled by the size of the numbers being compared. Good practice typically combines relative and absolute tolerances rather than relying on a single fixed value.
6 Applications
Machine epsilon appears in many practical tasks involving numerical computation. It serves as a compact summary of the precision available in a machine representation.
6.1 Error estimation
Analysts use epsilon to estimate how much roundoff may affect computed quantities. It provides a first-order benchmark for deciding whether observed differences are meaningful. Such estimates are especially important in iterative methods and in calculations with many dependent operations.
6.2 Algorithm design
Algorithms are often designed to minimize sensitivity to rounding. Knowledge of machine epsilon helps in choosing numerically safe formulas, scaling inputs, and avoiding subtraction of nearly equal values. It also guides decisions about precision requirements for a given task.
6.3 Termination criteria
Iterative algorithms commonly stop when successive approximations change by less than a small threshold. That threshold is often related to machine epsilon to avoid unnecessary iteration beyond the useful precision of the format. In practice, the stopping rule must also account for problem scale and convergence behavior.
7 Common misconceptions
Machine epsilon is frequently misunderstood because it is a technical quantity with several conventions. Clarifying what it does and does not represent is important for correct numerical reasoning.
7.1 Confusing epsilon with smallest number
Machine epsilon is not the smallest positive representable number. It measures spacing near 1, not the minimum magnitude in the system. The smallest positive number depends on the exponent range and, for subnormals, on special underflow handling.
7.2 Confusing epsilon with rounding error
Machine epsilon is not the actual error in a particular calculation. Instead, it is a characteristic scale that helps bound or estimate such error. The true rounding error may be smaller, larger in accumulated computations, or influenced by the structure of the problem.
7.3 Assuming a universal value
Epsilon is not the same across all computers or all numeric types. It varies with precision and representation, so single precision, double precision, and decimal formats each have different values. Portable software should not hard-code a single universal constant without checking the active type.
8 Examples
Simple examples help illustrate how machine epsilon depends on the underlying format. The exact values are determined by the spacing of representable numbers near 1.
8.1 Binary floating-point example
In a binary system with 24 bits of precision, the next number above 1 may differ by 2^-23. If 1 is written with a 1.0 significand and the least significant bit changes by one unit in the last place, the representable gap is fixed by that precision. This is the basis for the standard single-precision epsilon convention.
8.2 Decimal floating-point example
In a decimal system with a finite number of significant digits, the next number above 1 depends on the number of stored decimal places. For a format with seven decimal digits of precision, the spacing near 1 is 10^-6 under the gap-above-1 convention. Decimal formats are sometimes preferred in financial applications because they align more naturally with base-10 quantities.
</INTERNAL_LINK_CANDIDATES> Floating-point representation (a finite encoding of real numbers using significand and exponent) IEEE 754 (the standard defining common binary floating-point formats and rounding behavior) Significand (the part of a floating-point number that carries its precision) Exponent (the part of a floating-point number that controls scale) Rounding mode (the rule used to choose a representable result) Unit roundoff (the maximum relative error from rounding to nearest) Subnormal number (a very small floating-point value with reduced precision) Radix (the base of a floating-point number system) Precision (the number of significant digits or bits in a format) Single-precision (a 32-bit floating-point format) Double-precision (a 64-bit floating-point format) Extended precision (a floating-point format with more significand bits than double precision) Machine epsilon (the spacing measure near 1 in a floating-point system) Rounding error (the difference caused by rounding an exact result) Conditioning (a measure of how sensitive a problem is to input changes) Stability (a measure of how errors propagate in an algorithm) Floating-point comparison (methods for deciding whether two computed values are sufficiently close) Tolerance (an allowed threshold for approximate equality) Iterative algorithm (a procedure that repeats until a stopping condition is met) Underflow (the loss of magnitude when values become too small to represent normally)