Induced operator norm

1 Induced Operator Norm: Definition and Basic Properties

1.1 Setup: normed vector spaces and bounded linear operators

Let \(X\) and \(Y\) be normed vector spaces with norms \(\\|\cdot\\|_X\) and \(\\|\cdot\\|_Y\). For a linear operator \(T:X\to Y\), the induced (operator) norm quantifies the maximal amplification that \(T\) can produce relative to the sizes of vectors measured in \(X\) and \(Y\). In this context, bounded linear operators are precisely those for which such amplification remains finite.

1.2 Definition via the unit ball and unit sphere

The induced operator norm of \(T\) is defined by \[

\\|T\\|:=\sup_{\\|x\\|_X\le 1}\\|Tx\\|_Y.

\] Geometrically, the supremum searches over the image of the unit ball in \(X\) under \(T\) and returns the largest radius, measured in \(Y\). When the supremum is attained, it corresponds to a direction (or set of directions) in which \(T\) stretches vectors most strongly.

1.2.1 Alternative “sup over unit vectors” formulations

If \(X\neq\{0\}\), the supremum over the unit ball can be replaced by a supremum over the unit sphere: \[

\\|T\\|=\sup_{\\|x\\|_X=1}\\|Tx\\|_Y,

because scaling a nonzero vector to unit length does not change the ratio \(\\|Tx\\|_Y/\\|x\\|_X\), and the linearity of \(T\) yields

\frac{\\|Tx\\|_Y}{\\|x\\|_X}=\left\\|T\left(\frac{x}{\\|x\\|_X}\right)\right\\|_Y.

1.2.2 Finite-dimensional equivalence and existence of maximizers

In finite-dimensional spaces, the unit sphere is compact. If \(T\) is continuous (equivalently, linear between finite-dimensional normed spaces), then \(\\|Tx\\|_Y\) attains its maximum on the unit sphere. Consequently, the supremum in the definition becomes a maximum:

\\|T\\|=\max_{\\|x\\|_X=1}\\|Tx\\|_Y.

\] This property is a central practical advantage: numerical maximization is, at least in principle, guaranteed to achieve the operator norm.

1.3 Immediate consequences: positivity, homogeneity, and definiteness

The induced operator norm satisfies basic axioms:

- Positivity: \(\\|T\\|\ge 0\), since \(\\|Tx\\|_Y\ge 0\).

Homogeneity: for any scalar \(\alpha\),

\\|\alpha T\\|=\sup_{\\|x\\|_X\le 1}\\|\alpha Tx\\|_Y=	\alpha	\sup_{\\|x\\|_X\le 1}\\|Tx\\|_Y=	\alpha	\\|T\\|.

- Definiteness: \(\\|T\\|=0\) implies \(Tx=0\) for all \(x\), hence \(T=0\). Indeed, if \(\\|T\\|=0\), then \(\\|Tx\\|_Y\le \\|T\\|\,\\|x\\|_X=0\), forcing \(Tx=0\).

1.4 Subadditivity and the norm axioms for the operator space

For two operators \(S,T:X\to Y\), \[

\\|(S+T)x\\|_Y\le \\|Sx\\|_Y+\\|Tx\\|_Y\le \\|S\\|\,\\|x\\|_X+\\|T\\|\,\\|x\\|_X
=(\\|S\\|+\\|T\\|)\\|x\\|_X

for every \(x\). Taking the supremum over \(\\|x\\|_X\le 1\) gives

\\|S+T\\|\le \\|S\\|+\\|T\\|.

\] Together with positivity and homogeneity, this establishes that the induced operator norm is a genuine norm on the space of bounded linear operators \(B(X,Y)\).

1.5 Characterization of boundedness using induced norms

A linear operator \(T:X\to Y\) is bounded precisely when \(\\|T\\|<\infty\). Moreover, whenever \(\\|T\\|\) is finite, one has the global estimate

\\|Tx\\|_Y\le \\|T\\|\,\\|x\\|_X \quad \text{for all } x\in X.

\] This inequality is often used as the defining practical criterion: it provides a uniform bound controlling the operator on every vector, not only on a set of test vectors.

2 Computation in Finite Dimensions

2.1 Matrix case: induced norms from vector norms

When \(X=\mathbb{R}^n\) or \(\mathbb{C}^n\) with a chosen vector norm and \(Y=\mathbb{R}^m\) or \(\mathbb{C}^m\) with another vector norm, an operator \(T\) can be represented by an \(m\times n\) matrix \(A\). The induced operator norm becomes \[

\\|A\\|=\sup_{\\|x\\|_X\le 1}\\|Ax\\|_Y.

\] Different choices of vector norms lead to different operator norms, each with its own computable formula or characterization.

2.1.1 The \( \ell^2 \) (spectral/2-norm) interpretation

For the Euclidean norm on both domain and codomain, \(\\|x\\|_2\), the induced matrix norm is the spectral norm:

\\|A\\|_2 = \sqrt{\lambda_{\max}(A^*A)},

where \(A^*\) denotes the conjugate transpose and \(\lambda_{\max}\) is the largest eigenvalue. Equivalently, \(\\|A\\|_2\) equals the largest singular value of \(A\). This ties the computation to well-studied linear algebra tools such as eigenvalue and singular value decompositions.

2.1.2 The \( \ell^1 \) induced norm and column-sum formula

With \(\\|x\\|_1=\sum_j	x_j	\) in the domain and \(\\|y\\|_1=\sum_i	y_i	\) in the codomain, the induced matrix norm has the column-sum characterization:

\\|A\\|_1=\max_{1\le j\le n}\sum_{i=1}^m	a_{ij}	.

\] This arises from viewing \((Ax)_i\) as linear combinations of column entries and maximizing over sign patterns that align contributions.

2.1.3 The \( \ell^\infty \) induced norm and row-sum formula

Similarly, if \(\\|x\\|_\infty=\max_j	x_j	\) and the codomain uses \(\\|\cdot\\|_\infty\), then

\\|A\\|_\infty=\max_{1\le i\le m}\sum_{j=1}^n	a_{ij}	.

\] Here the maximizing vector can be chosen to saturate the largest row in absolute-sum magnitude.

2.2 Special operators: rank-one and diagonal operators

Rank-one operators: If \(T\) has the form \(T(x)=\phi(x)\,y\), where \(\phi\) is a linear functional and \(y\in Y\), then the induced norm factors as

\\|T\\|=\\|\phi\\|\,\\|y\\|.

\] This reduction turns norm computation into the computation of a functional norm and a vector norm.

Diagonal operators (in coordinate norms): For diagonal matrices acting on \(\ell^p\) spaces, induced norms often become simple “max modulus” expressions when the same \(p\) governs input and output, especially for \(p=1,\infty,2\). In other norm pairings, the expressions can be more involved but still relate to extremal coordinate or spectral properties.

2.3 Attainment and maximization strategies

In finite dimensions, the maximum exists on the unit sphere. Practical strategies include:

For the \(2\)-norm, compute the largest singular value via SVD or eigenvalue methods on \(A^*A\).
For \(1\)- and \(\infty\)-norms, use the column-sum and row-sum formulas, avoiding iterative maximization.
For more general induced norms, one can still treat the problem as a constrained optimization task, where maximizers satisfy stationarity conditions that can be derived from Lagrange multipliers (though solutions may not have closed forms).

2.4 Relations to singular values and eigenvalues in the \( \ell^2 \) case

In the Euclidean setting, the operator norm is governed by singular values regardless of whether \(A\) is diagonalizable. If \(A\) is normal (i.e., \(A^*A=AA^*\)), singular values correspond to moduli of eigenvalues, yielding a direct eigenvalue interpretation: \[

\\|A\\|_2=\max_i	\lambda_i(A)	.

\] For non-normal matrices, eigenvalues alone may not capture the norm, while singular values still provide the correct measure of maximal stretching.

3 Structural Inequalities and Algebraic Laws

3.1 Submultiplicativity: \( \|ST\| \le \|S\|\|T\| \)

For bounded linear operators \(S:Y\to Z\) and \(T:X\to Y\), \[

\\|ST\\|=\sup_{\\|x\\|_X\le 1}\\|S(Tx)\\|_Z
\le \sup_{\\|x\\|_X\le 1}\\|S\\|\,\\|Tx\\|_Y
\le \\|S\\| \sup_{\\|x\\|_X\le 1}\\|T\\|\,\\|x\\|_X
=\\|S\\|\,\\|T\\|.

\] This inequality expresses that the worst-case amplification of a composition cannot exceed the product of worst-case amplifications of the factors.

3.2 Norm of powers and implications for iterative processes

Applying submultiplicativity repeatedly gives \[

\\|T^k\\|\le \\|T\\|^k \quad \text{for integers } k\ge 1.

This bound is widely used in iterative methods and stability estimates: it translates growth or decay behavior of iterates into information about \(\\|T\\|\). Although \(\\|T\\|^k\) may be a conservative estimate, it provides a reliable global upper bound.

3.3 Bounds for inverses and the role of condition numbers

When \(T:X\to X\) is invertible and bounded with bounded inverse, one can estimate \[

\\|Tx\\|_X\ge \frac{1}{\\|T^{-1}\\|}\\|x\\|_X.

\] A common measure of sensitivity is the condition number \[

\kappa(T)=\\|T\\|\,\\|T^{-1}\\|,

\] which reflects how perturbations in data or operators can lead to amplified changes in solutions. In numerical linear algebra, this quantity often governs the quality of computed results.

3.4 Reverse inequalities and spectral-radius comparisons

Submultiplicativity also motivates the spectral radius comparison. For bounded operators, the spectral radius \(r(T)\) satisfies \[

r(T)\le \\|T\\|.

\] More precisely, one has \[

r(T)=\lim_{k\to\infty}\\|T^k\\|^{1/k},

when considered with standard operator-theoretic definitions. This limit explains why even if \(\\|T\\|\) provides a crude upper bound, the asymptotic behavior of \(\\|T^k\\|\) can be tightly tied to eigenvalue-like quantities.

4 Continuity, Lipschitz Bounds, and Operator Theory

4.1 Continuity of linear maps and induced-norm control

A linear map \(T:X\to Y\) between normed spaces is continuous if and only if it is bounded, which is equivalent to \(\\|T\\|<\infty\). Moreover, the induced norm supplies a Lipschitz constant:

\\|Tx-Ty\\|_Y=\\|T(x-y)\\|_Y\le \\|T\\|\,\\|x-y\\|_X.

Thus, \(\\|T\\|\) controls how variations in inputs translate to variations in outputs.

4.2 Lipschitz constants and stability estimates

Because induced norms yield Lipschitz bounds, they are central in estimating the stability of solutions. For example, if a procedure produces \(u\) from \(f\) via \(u=T(f)\), then \[

\\|u-\tilde u\\|_Y\le \\|T\\|\,\\|f-\tilde f\\|_X,

so \(\\|T\\|\) quantifies worst-case sensitivity. This is a cornerstone in error analysis for both deterministic and approximation-based methods.

4.3 Compactness considerations and limits of induced norms

In infinite-dimensional settings, induced norms may be finite while maximizing sequences need not converge to a true maximizer. Related notions—such as compact operators—interact with induced norms through properties like whether images of bounded sets have compact closure. While induced norms themselves are defined by a supremum, compactness can turn “approach” behavior into “attainment” behavior.

4.4 Completeness: Banach space structure of bounded operators

If \(Y\) is complete (a Banach space) and \(X\) is normed, then the space of bounded linear operators \(B(X,Y)\), equipped with the induced operator norm, is complete. This Banach structure supports fixed-point arguments, convergence theorems, and operator limits such as \(T_n\to T\) whenever \(\\|T_n-T\\|\to 0\).

5 Connections to Other Norms and Metrics

5.1 Comparison between different induced norms

Changing the underlying vector norms changes the induced operator norm. Consequently, two operators may have different operator-norm values depending on the chosen measurement of size in domain and codomain. In applications, the chosen norm often reflects the nature of perturbations or constraints—e.g., whether errors are measured in an entrywise maximum sense (\(\ell^\infty\)) or an overall magnitude sense (\(\ell^2\)).

5.2 Equivalence of norms in finite-dimensional spaces

In finite dimensions, any two vector norms are equivalent: there exist constants \(c,C>0\) such that for all \(x\), \[

c\\|x\\|_a\le \\|x\\|_b\le C\\|x\\|_a.

\] This implies that induced operator norms derived from these vector norms are also equivalent up to multiplicative factors. Practically, this means that while numerical values differ, qualitative behavior (bounded vs. unbounded, convergence rates in broad terms) is preserved.

5.3 Relationship to Schatten norms and traces (where applicable)

In Hilbert space contexts (notably when using the \(\ell^2\)-induced operator norm and operator ideals), one encounters Schatten norms \(\\|A\\|_{S_p}\), defined via singular values:

\\|A\\|_{S_p}=\left(\sum_i s_i^p\right)^{1/p}\quad (1\le p<\infty),

\qquad

\\|A\\|_{S_\infty}=\max_i s_i.

\] The induced operator norm corresponds to \(S_\infty\) (the largest singular value). Schatten norms refine the information by measuring the distribution of singular values rather than only the maximal one.

5.4 Links to quadratic forms and Rayleigh quotients in the \( \ell^2 \) setting

For self-adjoint operators \(A\) on Euclidean spaces, the \(2\)-norm ties into quadratic forms. The Rayleigh quotient \[ \frac{\langle Ax,x\rangle}{\langle x,x\rangle} \] relates to extremal eigenvalues. When \(A\) is positive semidefinite, the operator norm equals the largest eigenvalue, which can be characterized by maximizing the Rayleigh quotient over nonzero \(x\).

6 Examples and Worked Computations

6.1 Example: basic 2×2 matrices under common vector norms

Consider \[ A=\begin{pmatrix}1 & -2\\ 3 & 0\end{pmatrix}. \]

Under the \(\ell^1\) induced norm, compute column sums:

- Column 1: \(	1	+	3	=4\)
- Column 2: \(	-2	+	0	=2\)
Hence \(\\|A\\|_1=\max\{4,2\}=4\).

Under the \(\ell^\infty\) induced norm, compute row sums:

- Row 1: \(	1	+	-2	=3\)
- Row 2: \(	3	+	0	=3\)
Hence \(\\|A\\|_\infty=3\).

Under the \(\ell^2\) induced norm, one may compute the largest singular value via \(\sqrt{\lambda_{\max}(A^*A)}\). For a 2×2 matrix this is often tractable by direct eigenvalue computation.

6.2 Example: bounding an operator norm without exact maximization

Exact maximization can be difficult for nonstandard norm pairs. A common approach is to use known inequalities. For instance, if \(\\|\cdot\\|_Y\le C\\|\cdot\\|_{Y,ref}\) and \(\\|x\\|_X\ge c\\|x\\|_{X,ref}\), then

\\|Tx\\|_Y\le C\\|Tx\\|_{Y,ref}\le C\\|T\\|_{ref}\\|x\\|_{X,ref}\le \frac{C}{c}\\|T\\|_{ref}\\|x\\|_X.

Taking the supremum over \(\\|x\\|_X\le 1\) yields an upper bound on \(\\|T\\|\) without computing a maximizer.

6.3 Example: operator norms of projections and isometries

Projections in the Euclidean setting: An orthogonal projection \(P\) satisfies \(P^2=P\) and \(P=P^*\). Its eigenvalues are \(0\) or \(1\), so

\\|P\\|_2=1

\] unless \(P=0\). This reflects that projections do not increase Euclidean length in the orthogonal sense.

- Isometries: If \(T\) is an isometry for the given norms (i.e., \(\\|Tx\\|_Y=\\|x\\|_X\) for all \(x\)), then every unit vector is stretched by factor \(1\), hence \(\\|T\\|=1\).

6.4 Example: non-attainment in infinite-dimensional settings (conceptual)

In infinite dimensions, the unit sphere need not be compact. Therefore the supremum in \(\\|T\\|=\sup_{\\|x\\|\le 1}\\|Tx\\|\) may fail to be achieved by any vector. One may still find a sequence \((x_n)\) with \(\\|x_n\\|=1\) such that \(\\|Tx_n\\|\to \\|T\\|\), but no single \(x\) satisfies \(\\|Tx\\|=\\|T\\|\). This phenomenon is a qualitative difference from the finite-dimensional case.

7 Practical Considerations in Numerical Analysis

7.1 Estimating induced norms for algorithms

Algorithms often require operator norm estimates to justify step sizes, convergence conditions, or error bounds. In practice:

- If the norm is \(\\| \cdot \\|_1\) or \(\\| \cdot \\|_\infty\), induced norms can be computed by simple summations.
- If the norm is \(\\| \cdot \\|_2\), iterative methods for the largest singular value (or power methods on \(A^*A\)) provide approximations.

For other induced norms, one typically uses optimization heuristics, surrogate norms, or bounding inequalities rather than exact maximization.

7.2 Error propagation using operator norms

If an algorithm computes \(\hat{x}\) with perturbation \(e\) so that \(\hat{x}=x+e\), and a linear model maps \(x\) to \(Tx\), then the propagated output error satisfies \[

\\|T\hat{x}-Tx\\|=\\|Te\\|\le \\|T\\|\,\\|e\\|.

\] This formula explains why induced norms are central: they convert uncertain input errors into guaranteed output error bounds.

7.3 Sensitivity analysis and perturbation bounds

In perturbation theory, induced norms measure how much the solution operator changes when data or operators are perturbed. For invertible operators \(T\), bounds can be expressed using \(\\|T^{-1}\\|\). Coupled with \(\\|T\\|\), this yields estimates that often feature the condition number \(\kappa(T)\), which serves as a practical indicator of stability.

7.4 Computation shortcuts and trade-offs across norms

Different norms trade computational simplicity against tightness of bounds:

\(\ell^1\) and \(\ell^\infty\) induced norms are easy to compute but may be less informative about “directional” worst-case stretching than the \(\ell^2\) norm.
The \(\ell^2\) induced norm is geometrically meaningful but typically more expensive to compute exactly.
In finite dimensions, equivalence of norms means any choice can be related to others up to constants, so one can select a norm that matches both computational budget and the type of perturbations being modeled.