Exponential distribution

1 Properties and Definition

1.1 Probability density function

The exponential distribution is the continuous distribution on nonnegative time values \(t \ge 0\) with rate parameter \(\lambda>0\). Its probability density function is \[ f(t)=\lambda e^{-\lambda t}, \quad t\ge 0, \] and \(f(t)=0\) for \(t<0\). The single parameter \(\lambda\) governs how quickly the density decays as time increases.

1.2 Cumulative distribution function

The cumulative distribution function is obtained by integrating the density: \[ F(t)=P(T\le t)= \begin{cases} 0, & t<0,\\[4pt] 1-e^{-\lambda t}, & t\ge 0. \end{cases} \] Equivalently, the survival function is \(P(T>t)=e^{-\lambda t}\), which is often used directly in applications involving “time until an event.”

1.3 Rate parameter vs. mean (scale) parameter

The mean waiting time is \(E[T]=1/\lambda\). In practice, some references reparameterize the distribution using a scale parameter \(\theta=1/\lambda\), writing \(f(t)=\frac{1}{\theta}e^{-t/\theta}\) for \(t\ge 0\). The two forms are interchangeable: increasing \(\lambda\) shortens the typical waiting time, while increasing \(\theta\) lengthens it.

1.4 Support and basic constraints

The exponential distribution has support only on \([0,\infty)\). Any modeling assumption that produces negative waiting times is incompatible with the standard exponential model without additional transformation (e.g., shifting). The rate must satisfy \(\lambda>0\); at \(\lambda=0\) the model is not well-defined as a probability distribution.

2 Memorylessness

2.1 Formal statement of the memoryless property

A nonnegative random variable \(T\) is memoryless if, for all \(s,t\ge 0\), \[ P(T&gt;s+t \mid T&gt;s)=P(T&gt;t). \] For the exponential distribution, this identity holds exactly with \(P(T&gt;t)=e^{-\lambda t}\).

2.2 Intuition and practical meaning

The practical interpretation is that, given survival up to a time \(s\), the additional waiting time beyond \(s\) has the same distribution as the original waiting time. There is no “aging” in the process: the event propensity does not diminish or intensify with elapsed time.

2.3 Relation to hazard rate

The hazard rate for an exponential distribution is constant. Define the hazard rate as \[ h(t)=\frac{f(t)}{P(T&gt;t)}. \] For \(f(t)=\lambda e^{-\lambda t}\) and \(P(T&gt;t)=e^{-\lambda t}\), one obtains \(h(t)=\lambda\) for all \(t\ge 0\). This constant hazard is equivalent to memorylessness for continuous waiting-time models under standard regularity conditions.

3 Connections to Poisson Processes

3.1 Inter-arrival times in a Poisson process

In a homogeneous Poisson process with event rate \(\lambda\), inter-arrival times between consecutive events are exponentially distributed with the same \(\lambda\). If \(N(t)\) counts events up to time \(t\), the waiting time to the next event has the exponential law because the probability of no events in a given interval decays exponentially.

3.2 Deriving exponential waiting times from Poisson counts

Let \(T\) be the time until the next event after time 0. Then \[ P(T&gt;t)=P(\text{no events occur in }(0,t])=P(N(t)=0). \] For a Poisson process, \(P(N(t)=0)=e^{-\lambda t}\), so \(P(T&gt;t)=e^{-\lambda t}\), yielding the exponential survival function and density after differentiation.

3.3 Superposition and thinning relationships

Superposition: If two independent Poisson processes operate with rates \(\lambda_1\) and \(\lambda_2\), their union is also a Poisson process with rate \(\lambda_1+\lambda_2\). Consequently, the time to the next event in the combined system is exponential with rate \(\lambda_1+\lambda_2\).

Thinning: If each event of a Poisson process is retained independently with probability \(p\), the retained events form a Poisson process with rate \(p\lambda\). The waiting time between retained events is therefore exponential with rate \(p\lambda\).

3.4 Minimum of independent exponential variables

If \(T_1,\dots,T_k\) are independent exponential variables with rates \(\lambda_1,\dots,\lambda_k\), then the minimum \(M=\min_i T_i\) is exponential with rate \(\lambda_1+\cdots+\lambda_k\). Moreover, the probability that a particular \(T_j\) is the minimizer equals \(\lambda_j/(\sum_i \lambda_i)\), reflecting how higher-rate components tend to “trigger” first.

4 Moments and Summary Statistics

4.1 Expected value

The expected waiting time is \[ E[T]=\frac{1}{\lambda}. \] This follows directly from integrating \(t f(t)\) over \([0,\infty)\) or from standard properties of exponential families.

4.2 Variance

The variance is \[ \operatorname{Var}(T)=\frac{1}{\lambda^2}. \] The standard deviation matches the mean, which is a common rule-of-thumb for checking whether a fitted exponential model has plausible variability.

4.3 Higher moments

For \(n\ge 1\), the \(n\)-th raw moment is \[ E[T^n]=\frac{n!}{\lambda^n}. \] From these, one can compute central moments and characterize skewness and kurtosis if needed. The distribution is right-skewed, with substantial probability mass in larger waiting times relative to its mean.

4.4 Median and quantiles

The median \(m\) solves \(P(T\le m)=1/2\), giving \[ m=\frac{\ln 2}{\lambda}. \] More generally, the \(q\)-quantile satisfies \(P(T\le t_q)=q\), so \[ t_q=\frac{-\ln(1-q)}{\lambda}, \quad 0&lt;q&lt;1. \] Quantile formulas are convenient for comparing model predictions against observed distribution percentiles.

5 Parameter Estimation

5.1 Maximum likelihood estimation (MLE)

Given independent observations \(t_1,\dots,t_n\) assumed i.i.d. from \(\text{Exponential}(\lambda)\), the likelihood is \[ L(\lambda)=\prod_{i=1}^n \lambda e^{-\lambda t_i} =\lambda^n e^{-\lambda \sum_{i=1}^n t_i}. \] Maximizing with respect to \(\lambda\) yields \[ \hat{\lambda}_{\text{MLE}}=\frac{n}{\sum_{i=1}^n t_i} =\frac{1}{\bar{t}}, \] where \(\bar{t}\) is the sample mean. This reflects the reciprocal relationship between \(\lambda\) and the mean waiting time.

5.2 Method of moments

The method of moments matches the sample mean to the theoretical mean: \[ \bar{t} \approx \frac{1}{\lambda}. \] Solving gives the same estimator as the MLE: \[ \hat{\lambda}_{\text{MoM}}=\frac{1}{\bar{t}}. \] Because the exponential distribution has a single parameter and a direct moment equation, both approaches coincide for \(\lambda\).

5.3 Estimating rate from sample data

When the observed times are already measured in the natural time unit of the process, the estimate \(\hat{\lambda}=1/\bar{t}\) directly corresponds to a rate per unit time. If times were recorded in different units (e.g., milliseconds rather than seconds), rescaling is required: changing the time unit by a factor \(c\) changes the estimated rate by \(1/c\).

5.4 Bias and consistency (high level)

For the exponential MLE \(\hat{\lambda}=n/\sum t_i\), the estimator is generally biased for finite samples, though it is consistent: as \(n\) grows, \(\hat{\lambda}\) converges in probability to the true \(\lambda\). Bias-corrected or alternative estimators may be used in small-sample settings, but consistency under repeated sampling supports reliable long-run inference.

6 Related Distributions and Transformations

6.1 Gamma distribution as a sum of exponentials

A gamma distribution with shape parameter \(k\) (often integer in this relationship) and rate parameter \(\lambda\) can represent the sum of \(k\) i.i.d. exponential variables: \[ T=\sum_{i=1}^k X_i,\quad X_i \sim \text{Exponential}(\lambda) \ \text{independent}. \] The resulting \(T\) has gamma distribution, explaining why gamma models arise in “waiting for the \(k\)-th event” contexts.

6.2 Exponential as a special case of Weibull

The Weibull distribution with shape parameter \(k\) includes the exponential distribution as the special case \(k=1\). Under Weibull parameterizations, \(k=1\) yields survival \(P(T&gt;t)=e^{-(t/\theta)}\), which matches exponential behavior. This connection helps compare models where the hazard may increase or decrease over time (Weibull with \(k\neq 1\)).

6.3 Scaling and time rescaling

If \(T\sim\text{Exponential}(\lambda)\) and \(Y=cT\) for constant \(c>0\), then \(Y\sim\text{Exponential}(\lambda/c)\). Rescaling time by \(c\) therefore adjusts the rate inversely, consistent with \(E[Y]=cE[T]\).

6.4 Conditional distributions and truncation

Conditioning on \(T\) exceeding a threshold produces a shifted distribution with the same memoryless form: \[ (T-s \mid T&gt;s)\sim \text{Exponential}(\lambda),\quad s\ge 0. \] If observations are truncated (e.g., only times below a maximum are recorded), the resulting conditional distribution changes and likelihood-based inference must account for the truncation to avoid systematic distortion.

7 Simulation and Computation

7.1 Generating samples via inverse transform

To simulate \(T\sim\text{Exponential}(\lambda)\), use the inverse CDF method. Draw \(U\sim\text{Uniform}(0,1)\) and set \[ T = -\frac{1}{\lambda}\ln(1-U). \] Since \(1-U\) is also uniform, an equivalent form is \(T=-(1/\lambda)\ln(U)\). This produces exact samples from the target distribution.

7.2 Using random number generators

Most computing environments provide uniform random number generators; converting them to exponential draws via the inverse transform is standard. Care should be taken to ensure the uniform generator does not produce exact endpoint values that would cause \(\ln(0)\) (some implementations avoid zero or handle it safely).

7.3 Numerical considerations and overflow/underflow

For very small \(\lambda\) or large simulated values, exponentials and logarithms can challenge floating-point precision. Using the inverse-transform formula avoids direct computation of \(e^{-\lambda t}\), but computing \(-\ln(U)\) may still underflow or overflow if \(U\) is extremely close to 0 or 1. Double precision is usually adequate for typical simulation sizes, though safeguards may be needed in extreme parameter regimes.

7.4 Assessing simulated vs. theoretical behavior

Simulation results are commonly checked with summary statistics such as sample mean, sample variance, and empirical quantiles, then compared to theoretical values \(1/\lambda\), \(1/\lambda^2\), and \(-\ln(1-q)/\lambda\). Graphical diagnostics (e.g., empirical CDF overlays) can reveal discrepancies due to coding errors, incorrect parameterization, or dependence in generated data.

8 Applications (Modeling Contexts)

8.1 Queueing and service-time waiting

In queueing theory, exponential distributions often model service times or inter-arrival times in memoryless systems (commonly leading to Markovian structures). While real service processes may deviate from exponential assumptions, the model is valuable as a tractable baseline and as an approximation when events occur randomly and independently with a roughly constant hazard.

8.2 Reliability engineering and lifetimes

In reliability settings, exponential lifetimes correspond to components with constant failure rate. If a device’s failure mechanism does not “wear out” or improve over time, exponential modeling can represent the time until failure as a memoryless waiting time.

8.3 Survival analysis interpretation

In survival analysis, exponential models describe time-to-event with constant hazard. This interpretation aligns the exponential distribution with the survival function \(S(t)=e^{-\lambda t}\), enabling straightforward computation of survival probabilities and event-free horizons under the constant-risk assumption.

8.4 Event-time modeling in networks

Networked systems sometimes use exponential waiting times to represent random durations between events such as packet arrivals, user actions, or link-state transitions under simplifying assumptions. When event rates are approximately stable and event timings can be treated as independent, the exponential model provides interpretable metrics like expected time to the next occurrence.

9 Common Pitfalls and Assumptions

9.1 Interpreting “rate” correctly

The parameter \(\lambda\) is a rate (events per unit time), not a probability. Misinterpreting \(\lambda\) as a fraction or using the wrong time unit can produce fitted models that are numerically correct in form but incorrect in meaning.

9.2 When memorylessness does not hold

Real systems often exhibit aging, learning, wear-out, or threshold effects. When the hazard changes with time—whether increasing or decreasing—the exponential assumption fails and alternative distributions (such as Weibull or lognormal) may better capture the observed survival curve.

9.3 Mixing multiple processes (non-exponential outcomes)

If waiting times arise from a mixture of regimes (different underlying rates that switch between contexts), the resulting distribution typically deviates from a single exponential. Mixtures can produce heavier tails or altered curvature in survival plots, signaling that the one-rate assumption is insufficient.

9.4 Unit consistency and data preprocessing

Data preprocessing steps such as censoring handling, time alignment, and conversions between measurement units can strongly affect inference. Exponential modeling requires nonnegative times; preprocessing must ensure that event times are expressed on the correct scale and that any offset or truncation is handled consistently in both estimation and evaluation.

10 Visualizations and Intuition Aids

10.1 Shape changes with the rate parameter

Increasing \(\lambda\) shifts mass toward smaller times and makes the density decay more rapidly. Decreasing \(\lambda\) stretches the distribution, raising the likelihood of longer waiting times. Visual comparisons of multiple densities for different rates help interpret what “faster” or “slower” event streams mean.

10.2 Survival function plots

Because the survival function is \(S(t)=e^{-\lambda t}\), plotting survival probability versus time provides an immediate view of exponential decay. On a linear scale, the curve is concave downward; on a semi-log scale (log survival versus time), it becomes a straight line with slope \(-\lambda\).

10.3 Hazard rate interpretation

Plotting the hazard rate for different candidate models clarifies whether the constant-hazard assumption is reasonable. For the exponential distribution, the hazard is a horizontal line at height \(\lambda\), offering a direct visual check against time-varying failure propensities.

10.4 Example walkthroughs with small numeric cases

Consider a rate \(\lambda=2\) per unit time. Then \(E[T]=1/2=0.5\). The probability of waiting more than \(t=0.25\) is \(P(T>0.25)=e^{-2(0.25)}=e^{-0.5}\approx 0.607\). The median is \(m=\ln 2/2\approx 0.347\), meaning half the simulated waiting times should be below roughly 0.347. Working through a few such calculations builds intuition for how the rate affects probabilities at concrete times.