Principal Component Analysis
Subject: Computer and Technology noun /ˈprɪn.sə.pəl kəmˈpoʊ.nənt əˈnæl.ə.sɪs/ 1 definition
- machine-learning
- dimensionality-reduction
- linear-algebra
- numerical-computing
-
Definition (noun)
Entry #304 · UpdatedPrincipal Component Analysis (PCA) is an unsupervised statistical and linear-algebra technique that transforms a dataset of potentially correlated features into a smaller set of linearly uncorrelated variables called principal components. After centering the input data—and typically scaling features to unit variance—PCA computes an orthogonal coordinate basis via eigendecomposition of the covariance matrix or Singular Value Decomposition (SVD) of the data matrix, ordering axes so each successive component captures the maximum remaining variance. Projecting observations onto the leading
kprincipal components compresses high-dimensional inputs, eliminates feature multicollinearity, and filters out low-variance noise while preserving the greatest possible linear variance in ak-dimensional subspace. Because PCA relies purely on second-order global variance without consulting target labels, it cannot directly capture nonlinear manifolds and may discard low-variance directions that remain informative for supervised tasks.Common Forms
Standard (Exact) PCA: Computes the exact orthogonal principal axes via full Singular Value Decomposition of the centered data matrix (or eigendecomposition of its covariance matrix), delivering optimal linear variance retention for dense, memory-resident datasets.
Randomized PCA: Uses randomized subspace iteration to approximate the leading
ksingular vectors, drastically cutting runtime and memory overhead when reducing high-dimensional matrices to a much smaller target rankk.Incremental PCA: Processes training observations in fixed-size mini-batches rather than loading the entire matrix into RAM, updating singular vector estimates sequentially for out-of-core or streaming workloads.
Kernel PCA: Implicitly maps inputs into a higher-dimensional feature space using a nonlinear kernel function—such as radial basis function (RBF) or polynomial—before extracting linear principal components to model curved manifolds.
Example
Before training a fraud-detection classifier in scikit-learn on a training set of 100,000 transactions with 256 correlated numeric features, an engineer standardizes each column with
StandardScalerand configuresPCA(n_components=0.95, svd_solver="full")inside aPipeline. Fitting the pipeline projects the100,000 x 256matrix onto 34 uncorrelated principal components, retaining 95% of total variance while eliminating multicollinearity and shrinking downstream memory usage.