Tucker rank - Ciallo Wiki

1 Tucker decomposition and mode-wise factorization

1.1 Tensor notation and mode definitions

A tensor is a multi-way array generalizing vectors and matrices. For an \(N\)-way tensor \(\mathcal{X} \in \mathbb{R}^{I_1 \times I_2 \times \cdots \times I_N}\), each index position corresponds to a tensor *mode*. The \(n\)-th mode ranges over \(I_n\) values, and operations can be defined by treating one index direction as the “row/column” direction while the others are grouped.

Mode-related terminology is central because Tucker rank assigns a separate notion of dimensionality to each mode. Consequently, any analysis must specify which mode unfolding or factor matrix corresponds to which axis of the original data.

1.2 Tucker decomposition overview

The Tucker decomposition represents \(\mathcal{X}\) using a smaller *core tensor* multiplied by factor matrices along each mode. In its common form, \[ \mathcal{X} \approx \mathcal{G} \times_1 U^{(1)} \times_2 U^{(2)} \cdots \times_N U^{(N)}, \] where \(\mathcal{G}\) is the core tensor and \(U^{(n)}\) is a factor matrix for mode \(n\). The factor matrices typically have orthonormal columns in the standard higher-order SVD (HOSVD) setting, though other choices exist.

The decomposition separates *multilinear structure* (captured by factor matrices) from *interaction structure* (captured by the core).

1.3 Core tensor interpretation

The core tensor \(\mathcal{G} \in \mathbb{R}^{r_1 \times r_2 \times \cdots \times r_N}\) holds coefficients describing how reduced coordinates across different modes combine. Intuitively:

\(r_n\) determines how many basis directions are retained along mode \(n\),
the core stores the resulting mixed-mode relationships among these retained directions.

When Tucker rank is written as \((r_1,\dots,r_N)\), the tuple directly indicates the size of the core in each mode. This links the algebraic notion of rank to the practical dimensionality of the compressed representation.

2 Definition of Tucker rank

2.1 Tucker rank as a rank tuple (r1, r2, …, rN)

For an \(N\)-way tensor, the Tucker rank is the tuple \[ (r_1, r_2, \ldots, r_N), \] where \(r_n\) characterizes the dimensionality of the tensor along mode \(n\) under Tucker-style factorization. In many formulations, \(r_n\) can be identified with the rank of the mode-\(n\) unfolding (a matrix representation obtained by rearranging indices so that mode \(n\) forms one axis).

Thus, Tucker rank is not a single scalar but a vector of mode-wise complexities. This reflects that a tensor may be “simple” in one direction and complex in another.

2.2 Relationship to mode-n unfoldings

The \(n\)-mode unfolding \(X_{(n)}\) is a matrix derived from \(\mathcal{X}\) by fixing the \(n\)-th index as the row index and stacking the remaining indices into the column index. The Tucker rank component \(r_n\) is commonly taken as: \[ r_n = \operatorname{rank}(X_{(n)}). \] This definition connects Tucker rank to familiar matrix rank, but applied to each mode-specific matricization rather than the whole tensor at once.

When the tensor is represented via Tucker decomposition with factor matrices \(U^{(n)}\) having \(r_n\) columns, the resulting model can be understood as restricting the range of \(\mathcal{X}\) along each mode to an \(r_n\)-dimensional subspace.

2.3 Minimality and non-uniqueness considerations

The tuple \((r_1,\dots,r_N)\) captures the smallest dimensions along each mode needed to represent \(\mathcal{X}\) exactly in a Tucker form. However, Tucker decompositions are generally not unique: different choices of basis within each mode can produce different factor matrices and cores while representing the same tensor.

This non-uniqueness means that, although Tucker rank components are determined by unfolding ranks under the standard definition, the *specific* factor matrices and the core are not. In practice, algorithms often return one convenient set of factors rather than all possible ones.

3 Computing Tucker rank in practice

3.1 Using exact SVDs for small-to-medium tensors

When tensors are modest in size, computing mode-\(n\) unfoldings and applying exact singular value decompositions can yield exact ranks. The workflow is:

form each unfolding \(X_{(n)}\),
compute \(\operatorname{rank}(X_{(n)})\) by counting nonzero singular values (up to numerical tolerance),
assemble the tuple \((r_1,\dots,r_N)\).

Exact SVDs can be computationally heavy because unfoldings can become very large matrices, especially when mode sizes are uneven. Still, for controlled problem sizes they provide a direct route to mode-wise rank.

3.2 Approximate methods and truncation heuristics

For real data, exact ranks are often unstable because tensors contain noise, leading to many small but nonzero singular values. In that setting, practical computation replaces “rank as number of nonzero singular values” with “effective rank as number of singular values exceeding a threshold.”

Approximation methods include:

truncating SVDs to a chosen number of components,
computing approximate multilinear subspaces using randomized or iterative algorithms,
estimating rank components from compressed representations rather than full unfoldings.

The resulting Tucker rank becomes an *estimated* rank tuple, reflecting modeling decisions rather than a strict mathematical property of an exact tensor.

3.3 Choosing ranks from singular value spectra

A common approach is to examine the singular value spectrum of each unfolding \(X_{(n)}\). Several heuristics are used:

Energy-based selection: choose \(r_n\) such that the retained singular values capture a specified proportion of total squared energy.
Gap-based selection: choose \(r_n\) near a noticeable drop (“elbow”) in the spectrum.
Thresholding: set \(r_n\) by comparing singular values to an absolute or relative tolerance (e.g., relative to the largest singular value).

These rules often yield different rank tuples across modes, which is consistent with Tucker rank’s role as a mode-wise complexity descriptor.

3.4 Software and algorithmic workflows

Computing Tucker ranks or their approximations is typically embedded in tensor decomposition libraries. Typical workflows involve:

forming unfoldings implicitly within the algorithm to avoid explicit large matrices,
using HOSVD or higher-order orthogonal iteration,
returning either the factor matrices and core or only the estimated mode ranks.

Algorithm choices affect runtime and numerical stability. Iterative methods can be faster for large data but may require careful tolerance settings to avoid overestimating \(r_n\).

4 Properties and relationships

4.1 Invariance under multilinear basis changes

Under changes of basis in each mode—multiplying factor matrices by invertible transforms—the underlying multilinear structure remains the same, though the specific core tensor entries change. Consequently, Tucker rank components defined via unfolding ranks are preserved by nonsingular mode transformations.

This invariance is valuable because it means Tucker rank reflects intrinsic subspace dimensionality rather than a particular coordinate system.

4.2 Connections to multilinear subspace dimensions

Each \(r_n\) can be interpreted as the dimension of the mode-\(n\) subspace spanned by slices of the tensor after the appropriate unfolding. In a Tucker model, the factor matrix \(U^{(n)}\) spans that subspace.

This connection links Tucker rank to geometry: it describes how many independent directions are needed along each axis to express the tensor data without loss (in the exact-rank setting).

4.3 Comparison with CP rank and matrix rank

Tucker rank is different from CP (CANDECOMP/PARAFAC) rank. CP rank represents the smallest number of rank-one tensors whose sum equals \(\mathcal{X}\). In contrast, Tucker rank is mode-wise and tied to ranks of unfoldings. As a result:

Tucker rank typically gives a more direct handle on compression along each mode,
CP rank is often harder to compute but can yield very compact representations for certain structured tensors.

For \(N=2\), a tensor reduces to a matrix, and Tucker rank collapses to the usual matrix rank.

4.4 Bounds and inequalities across modes

Because Tucker rank is assembled from unfolding ranks, the components are constrained by the sizes of modes: \[ r_n \le I_n \quad \text{for each } n. \] Further, relationships can exist between unfoldings depending on the tensor’s structure. While a universal closed-form inequality between different \(r_n\) values is not generally simple, the tuple always reflects consistent linear-algebraic constraints stemming from how unfoldings share information through the original tensor.

In computations, these bounds help validate outputs (for example, ensuring no estimated \(r_n\) exceeds the corresponding mode dimension).

5 Applications in applied mathematics and data analysis

5.1 Tensor compression and low-rank approximation

Tucker rank underlies Tucker-based compression. Given a target tuple \((r_1,\dots,r_N)\), one can approximate \(\mathcal{X}\) with a smaller core and fewer degrees of freedom in factor matrices. This reduces storage and can speed up downstream computations.

Such approximations are widely used when data exhibit low multilinear structure, such as correlated measurements across multiple axes (time, sensors, spatial positions, or experimental conditions).

5.2 Denoising and dimensionality reduction

When observed tensors contain noise, the singular spectra of unfoldings often show that only a subset of components carries signal. Choosing ranks based on these spectra yields a denoised approximation by projecting onto the dominant mode subspaces.

The resulting reduction in rank tuple acts like a form of dimensionality reduction: each mode is compressed to fewer effective coordinates, while the core captures the remaining interactions.

5.3 Feature extraction in multi-way data

In machine learning pipelines, Tucker decompositions can produce interpretable features:

columns of factor matrices represent latent components for each mode,
the core tensor and its entries summarize cross-mode coupling.

Tucker rank selection influences feature granularity: smaller ranks yield coarse but robust summaries; larger ranks may capture finer patterns at the cost of increased variance and sensitivity to noise.

6 Model selection and rank estimation

6.1 Selecting Tucker ranks for approximation quality

Rank tuple selection is a balance between accuracy and compactness. In practice, one chooses \((r_1,\dots,r_N)\) so that the approximation error is acceptably small while the model size remains manageable. Since each mode has its own \(r_n\), selection can be asymmetric—useful when only certain modes contain strong structure.

The chosen tuple can be determined by increasing ranks gradually until additional components yield diminishing returns.

6.2 Error metrics for rank selection

Common objectives include:

Frobenius norm error between the original tensor and its Tucker approximation,
relative error to normalize by the tensor’s scale,
performance metrics in predictive tasks when the tensor model is part of a larger system.

Error is typically computed on held-out data or through reconstruction tests depending on the task.

6.3 Regularization and stability issues

Rank estimation can be unstable when spectra decay slowly or when noise levels are high. Regularization strategies include:

limiting ranks to a maximum tuple size,
using thresholded or weighted truncation rather than hard cutoffs,
incorporating priors or smoothness constraints in iterative tensor factorization algorithms.

Stability also depends on scaling: if the tensor entries vary widely in magnitude, singular value thresholds must be chosen carefully to avoid biasing rank estimates.

6.4 Cross-validation and practical decision rules

A practical decision rule is to evaluate candidate rank tuples using cross-validation: split data (or observations) so that reconstruction or downstream prediction error can be measured on unseen parts. Because brute-force search over all tuples can be expensive, methods often:

search over one mode rank at a time,
use grid sampling with coarse steps,
refine around promising tuples.

In many workflows, a combination of spectral heuristics (for an initial guess) and cross-validation (for confirmation) is used.

7 Special cases and examples

7.1 Rank-1 and reduced-dimension Tucker forms

A Tucker rank tuple \((1,1,\dots,1)\) corresponds to a form where each mode is restricted to a single basis direction, producing a very low-dimensional approximation. More generally, when some \(r_n=1\), the tensor is compressed along those modes to effectively one-dimensional variation.

Reduced-dimension cases help illustrate the role of each mode: setting \(r_n\) small forces the model to ignore variation along that axis, which can be appropriate for data with strong mode-wise redundancy.

7.2 Symmetric and partially symmetric tensors

For tensors with symmetry properties, mode ranks can be related or sometimes identical because the unfolding matrices share analogous structure. Partially symmetric tensors—symmetric under permutations of certain modes—may admit Tucker decompositions where factor matrices for symmetric modes are constrained or tied.

Even when exact rank components are not forced to match, symmetry can simplify interpretation and guide rank selection by reducing the effective search space.

7.3 Worked example: selecting ranks for a three-way tensor

Consider a three-way tensor \(\mathcal{X}\in \mathbb{R}^{I_1\times I_2\times I_3}\). To estimate Tucker rank:

Compute the singular values of each unfolding \(X_{(1)}, X_{(2)}, X_{(3)}\).
For mode 1, suppose the singular values show that the first \(r_1\) values capture 95% of energy; set \(r_1\) accordingly.
Repeat for modes 2 and 3, yielding \(r_2\) and \(r_3\).
Fit a Tucker model using the selected tuple \((r_1,r_2,r_3)\).
Evaluate reconstruction error (or downstream validation error). If error remains high, increase the rank in the mode with the steepest remaining singular spectrum contributions; if overfitting appears, decrease ranks or tighten truncation thresholds.

This illustrates how Tucker rank selection becomes an iterative modeling choice driven by the spectral behavior of each mode unfolding.

8 Limitations and common pitfalls

8.1 Sensitivity to noise and scaling

With noisy tensors, unfolding ranks may be inflated because many singular values are nonzero (within numerical precision). Effective rank estimation therefore depends on tolerance levels and scaling choices. If the threshold is too low, \(r_n\) is overestimated; if too high, the approximation may discard genuine signal.

A related issue is unit scaling: rescaling one mode can change singular values and influence threshold-based rank decisions unless thresholds are defined relative to appropriate normalization.

8.2 Misinterpretation of mode-wise ranks

A frequent mistake is to treat large \(r_n\) as evidence that the tensor is globally complex, without considering that Tucker rank is mode-specific. A tensor can have a large \(r_1\) but small \(r_2\), meaning complexity concentrates along one axis. Interpretation should therefore align with the meaning of modes in the underlying dataset.

8.3 Overfitting risk with large rank tuples

Using ranks that are too high can lead to memorization of noise and reduced generalization in predictive settings. Even for reconstruction tasks, excessively large tuples can reduce apparent error while increasing variance, especially when the tensor is inferred from incomplete or noisy observations.

Regularization, cross-validation, and constraints on maximum ranks help mitigate this risk.

8.4 Computational cost considerations

Although Tucker rank is defined via mode unfoldings, computing ranks exactly through full unfoldings can be expensive. Memory use can become a bottleneck when mode sizes are large. For large problems, approximate algorithms and implicit computation strategies are typically needed.

Additionally, rank tuple searches can be computationally intensive, since the number of candidate tuples grows quickly with the number of modes. Efficient search strategies and heuristic initialization are important in practice.