1 Definitions and notation

CP decomposition, short for CANDECOMP/PARAFAC decomposition, is a tensor factorization method that represents a multidimensional array as a sum of simple rank-one terms. It extends the familiar idea of decomposing a matrix into latent factors to tensors of order three or higher. In practice, the method is used to uncover hidden structure, compress data, and separate interacting patterns across multiple modes.

1.1 Tensors and multilinear algebra

A tensor is a multidimensional array whose entries are indexed by two or more indices. The order of a tensor refers to the number of indices needed to locate one element. Vectors and matrices are special cases of first-order and second-order tensors, respectively. Multilinear algebra studies operations that act separately along each mode, providing the language used to describe tensor factorizations.

1.2 Rank-one tensors

A rank-one tensor is formed by the outer product of vectors, one from each mode. Such a tensor has a highly separable structure: each entry is the product of the corresponding vector components. Rank-one tensors are the basic building blocks of CP decomposition, much as one-dimensional basis elements are in simpler linear models.

1.3 CP decomposition model

In the CP model, a tensor is approximated as a sum of rank-one tensors. Each component is typically interpreted as a latent factor contributing across all modes. The number of components is often denoted by a rank parameter, which controls the complexity of the representation.

1.3.1 Factor matrices

The factors of a CP model are usually arranged in matrices, with one matrix per tensor mode. Each column of a factor matrix corresponds to one latent component. Together, these matrices encode how strongly each component appears along each dimension of the data.

1.3.2 Outer-product representation

The outer-product form expresses each component as the product of one vector from each factor matrix. Summing these terms gives the full approximation of the tensor. This representation makes the model compact and conceptually simple, while also revealing how interactions among modes are captured.

1.4 Tensor rank and CP rank

The CP rank of a tensor is the smallest number of rank-one terms needed to represent it exactly. Unlike matrix rank, tensor rank can behave irregularly and is often difficult to compute. For many tensors, the rank is not obvious from inspection, and even deciding whether a given rank is minimal can be challenging.

2 Mathematical properties

The mathematical study of CP decomposition focuses on whether a decomposition exists, whether it is unique, and how stable it is under perturbations. These properties are central to both theory and applications, since they determine whether the extracted factors can be meaningfully interpreted.

2.1 Existence of decomposition

Every finite tensor can be approximated by sums of rank-one tensors, though exact representation may require many terms. For some tensors, a finite exact CP decomposition exists at a particular rank. In other cases, the best low-rank approximation may be difficult to define cleanly because the set of tensors of bounded rank is not always closed.

2.2 Uniqueness and identifiability

A major advantage of CP decomposition is that, under suitable conditions, the factorization can be essentially unique. This means the latent components can often be recovered up to trivial ambiguities. Such identifiability is one reason CP methods are valued in applications where interpretability matters.

2.2.1 Kruskal's condition

Kruskal's condition gives a sufficient criterion for uniqueness based on the column dependencies of the factor matrices. It uses the so-called Kruskal ranks of the factors to establish when a decomposition is identifiable. Although not necessary in all cases, it has become one of the best-known uniqueness results for CP models.

2.2.2 Generic uniqueness

Generic uniqueness refers to uniqueness that holds for almost all tensors of a given size and rank. In many settings, a randomly chosen tensor will satisfy uniqueness conditions even when special structured examples do not. This notion is useful in understanding the typical behavior of CP decompositions beyond exceptional cases.

2.3 Scaling and permutation ambiguities

CP decompositions are not unique under simple rescalings and reordering of components. A factor in one mode may be multiplied by a constant if another factor in the same component is divided by the same constant. Likewise, the order of the rank-one terms can be permuted without changing the reconstructed tensor. These are considered intrinsic ambiguities rather than failures of the model.

2.4 Degeneracy and non-closedness

Tensor factorization can exhibit degeneracy, where intermediate components grow large while their sum remains bounded. This phenomenon is closely tied to the fact that sets of tensors with bounded CP rank are not always closed. As a result, numerical algorithms may approach a limit only through increasingly unstable parameter values.

3 Computational methods

Computing a CP decomposition usually requires iterative numerical methods. The choice of algorithm depends on tensor size, desired rank, noise level, and whether constraints are imposed. No single technique is optimal in every setting.

3.1 Alternating least squares

Alternating least squares, often abbreviated ALS, is one of the most common CP algorithms. It updates one factor matrix at a time while holding the others fixed, cycling through the modes repeatedly. The method is simple to implement and widely used, though it may converge slowly or become trapped in poor local solutions.

3.1.1 Initialization strategies

Because ALS is sensitive to starting values, initialization can strongly influence the result. Common choices include random factors, spectral estimates, and initial guesses derived from matricized data. Good initialization often improves convergence speed and reduces the chance of numerical difficulties.

3.1.2 Convergence behavior

ALS typically decreases the approximation error at each step, but the overall convergence landscape can be complicated. The algorithm may stagnate near saddle points or move slowly when components are nearly dependent. In practice, convergence is assessed through changes in fit, factor updates, or objective value.

3.2 Gradient-based optimization

Gradient-based methods treat CP fitting as a nonlinear optimization problem. They use derivatives of the loss function to update all factors simultaneously or in structured blocks. Compared with ALS, these methods can offer better control over step size and regularization, especially in large-scale problems.

3.3 Algebraic and direct methods

Algebraic approaches attempt to recover factors through explicit formulas, matrix identities, or polynomial techniques. These methods are often faster when they apply, but they usually require restrictive assumptions such as exact data, specific rank conditions, or small tensor dimensions. They are therefore less general than iterative optimization methods.

3.4 Constraints and regularization

Constraints and penalties are often added to improve interpretability or numerical stability. They can reflect prior knowledge about the factors, such as positivity, smoothness, or sparsity. Regularization is also used to reduce overfitting and to control ill-conditioning.

3.4.1 Nonnegativity constraints

Nonnegative CP decomposition restricts factor entries to be zero or positive. This is useful when the data represent quantities that cannot meaningfully be negative, such as concentrations or intensities. The resulting factors are often easier to interpret because they resemble additive parts rather than canceling terms.

3.4.2 Sparsity constraints

Sparsity constraints encourage many factor entries to become exactly or approximately zero. They can produce more compact models and highlight localized structure. Sparse CP representations are especially helpful when the underlying sources affect only a limited set of modes or features.

4 Model selection and interpretation

A CP model is useful only if the chosen rank and factor structure reflect the data reasonably well. Model selection therefore plays a central role in practical analysis. Interpretation of the factors is often the main goal, especially in scientific applications.

4.1 Choosing the number of components

Selecting the number of components is a model selection problem balancing simplicity against explanatory power. Too few components may miss important structure, while too many can fit noise or create unstable estimates. Common approaches include inspecting residual error, using validation data, and comparing models across several candidate ranks.

4.2 Goodness-of-fit measures

Goodness-of-fit measures quantify how closely the decomposition reproduces the observed tensor. Typical measures include relative error, explained variance, and likelihood-based scores when a probabilistic model is used. These statistics help determine whether additional components materially improve the fit.

4.3 Overfitting and underfitting

Overfitting occurs when the model captures random fluctuations rather than genuine patterns. Underfitting arises when the chosen rank is too small to describe the main structure. Both problems are common in tensor factorization, so analysts often combine quantitative criteria with domain knowledge.

4.4 Interpretation of factors

The columns of the factor matrices are often interpreted as latent patterns, sources, or profiles. In applications such as chemometrics or neuroscience, each component may correspond to a meaningful underlying process. Interpretation depends on scaling conventions and on the extent to which the factors are unique and stable.

5 Numerical issues

Practical CP computation is influenced by numerical stability, missing values, and problem size. Even when a model is mathematically well defined, floating-point computation may introduce additional difficulty. Careful implementation is therefore important.

5.1 Ill-conditioning

Ill-conditioning arises when small changes in the data lead to large changes in the estimated factors. It is often associated with nearly dependent components or very uneven scaling. In such cases, optimization becomes sensitive to noise and may produce unreliable estimates.

5.2 Scaling conventions

Because CP terms can be rescaled without changing the tensor, a convention is usually adopted to normalize factors. Common choices include fixing the norm of selected columns or absorbing weights into one matrix. Standardization makes results easier to compare across components and across runs.

5.3 Missing data and noisy observations

Real data often contain missing entries or measurement noise. CP methods can be adapted by optimizing only over observed values or by incorporating probabilistic noise models. These extensions are important in applied settings where complete, exact tensors are rarely available.

5.4 Computational complexity

The cost of CP algorithms grows with tensor dimensions, rank, and the number of iterations required. Large tensors may need specialized implementations that exploit sparsity, low storage formats, or parallel computation. Efficiency considerations often determine whether a method is practical at all.

6 Variants and extensions

Many tensor models are related to CP decomposition but incorporate additional structure. These variants broaden the range of data types and constraints that can be represented. They also connect CP methods to other factorization frameworks.

6.1 Constrained CP decomposition

Constrained CP decomposition adds prior restrictions to the factor matrices or weights. Examples include orthogonality, smoothness, nonnegativity, and sparsity. Such constraints can improve interpretability, reflect known physics, or regularize unstable problems.

6.2 Coupled matrix and tensor factorization

Coupled matrix and tensor factorization jointly models a tensor together with related matrices that share latent factors. This approach is useful when multiple data sources describe the same underlying entities in different formats. By linking the factorizations, the method can improve estimation and consistency across datasets.

6.3 Block-term decomposition

Block-term decomposition generalizes CP by allowing each component to be a small structured block rather than a single rank-one term. This can capture richer interactions while retaining some of the interpretive benefits of factorized models. It is often used when CP is too restrictive but more general tensor models are needed.

6.4 Tucker decomposition comparison

Tucker decomposition represents a tensor using a core array multiplied by factor matrices along each mode. Unlike CP, which sums rank-one terms, Tucker allows interactions among components through the core. CP is often more parsimonious and easier to interpret, while Tucker can be more flexible.

7 Applications

CP decomposition has been adopted in many fields because it can separate overlapping patterns in multidimensional data. Its usefulness comes from combining compression, latent-variable modeling, and interpretability. The specific meaning of the components depends on the application domain.

7.1 Chemometrics

In chemometrics, CP decomposition is used to analyze spectral and concentration data. It can separate mixtures into constituent profiles and help identify underlying chemical sources. The method is especially valuable for excitation-emission fluorescence data and other multiway measurements.

7.2 Signal processing

In signal processing, CP models help recover sources from arrays of measurements collected across sensors, frequencies, or time windows. They are used in blind source separation, array processing, and multi-channel analysis. The tensor structure can reveal relationships that are difficult to see in flattened data.

7.3 Machine learning

Machine learning applications include latent feature extraction, recommender systems, and compression of high-dimensional parameter arrays. CP decomposition can reduce model size while preserving essential structure. It is also used as a building block in tensor-based neural network methods and statistical learning pipelines.

7.4 Psychometrics

Psychometrics uses CP decomposition to study patterns in multiway behavioral or survey data. The method can represent how responses vary across individuals, items, and contexts. Its factor structure may support the interpretation of latent traits or interaction effects.

7.5 Neuroscience

In neuroscience, CP models are applied to recordings that vary across neurons, time, trials, or conditions. They can identify coordinated activity patterns and temporal signatures in population data. The decomposition helps summarize complex recordings in a compact and interpretable form.

7.6 Computer vision

Computer vision problems often produce tensors from color channels, frames, scales, or feature maps. CP decomposition can compress these data and isolate recurring visual patterns. It is also used in tasks involving multimodal image analysis and representation learning.

CP decomposition belongs to a broader family of tensor and matrix factorization methods. Comparing it with related models clarifies both its strengths and its limitations. Several neighboring concepts share common algebraic ideas but differ in structure and flexibility.

8.1 Matrix singular value decomposition

The singular value decomposition is the matrix analogue most often used as a point of comparison. It expresses a matrix as a sum of orthogonal rank-one terms and plays a central role in linear algebra. CP decomposition generalizes the rank-one-sum idea beyond two dimensions, though without the same orthogonality guarantees.

8.2 Tensor train decomposition

Tensor train decomposition represents a tensor as a sequence of interconnected low-dimensional cores. It is designed for high-order tensors where direct CP models may become impractical. Compared with CP, tensor train formats often offer better numerical scalability for very large dimensions.

8.3 Tucker model

The Tucker model factors a tensor into a core and mode-specific factor matrices. It allows interactions among latent components and can capture more complex internal structure than CP. The two models are often compared in terms of interpretability, compression, and computational convenience.

8.4 PARAFAC and CANDECOMP nomenclature

PARAFAC and CANDECOMP are historical names for the same basic decomposition. PARAFAC stands for parallel factors, while CANDECOMP refers to canonical decomposition. The acronym CP has become the most common modern shorthand, especially in contemporary mathematical and data-analytic literature.