PCA编辑历史

# Principal Component Analysis

Principal Component Analysis (PCA) is a statistical technique used for dimensionality reduction while preserving as much variance in the data as possible. It transforms a set of possibly correlated variables into a smaller number of uncorrelated variables called principal components, ordered by the amount of variance they capture. PCA is widely applied in exploratory data analysis, feature extraction, noise filtering, and data visualization.

## 1 Mathematical foundations

### 1.1 Covariance matrix and eigenvalues

The covariance matrix \(\Sigma\) of a dataset with \(p\) variables is a \(p \times p\) symmetric matrix whose entry \(\sigma_{ij}\) measures the covariance between variable \(i\) and variable \(j\). The diagonal entries are the variances of each variable. PCA seeks linear combinations of the original variables that maximize variance, which leads to the eigenproblem for \(\Sigma\). The eigenvalues of \(\Sigma\) represent the variance explained by each principal component, and the eigenvectors define the directions (principal component axes).

### 1.2 Eigendecomposition

Eigendecomposition decomposes a square matrix into its eigenvectors and eigenvalues. For the covariance matrix \(\Sigma\), the eigendecomposition is \(\Sigma = Q \Lambda Q^{-1}\), where \(Q\) is an orthogonal matrix whose columns are the eigenvectors, and \(\Lambda\) is a diagonal matrix with the eigenvalues \(\lambda_1 \ge \lambda_2 \ge \dots \ge \lambda_p \ge 0\). The eigenvectors are the principal component loadings; the eigenvalues indicate the variance along each component.

#### 1.2.1 Spectral theorem

The spectral theorem states that any real symmetric matrix (such as the covariance matrix) can be diagonalized by an orthogonal matrix. This ensures that the eigenvectors of \(\Sigma\) are mutually orthogonal and that \(\Sigma\) can be expressed as a sum of rank‑1 matrices: \(\Sigma = \sum_{i=1}^p \lambda_i \mathbf{v}_i \mathbf{v}_i^\top\). This propert