Skip to main content
Definition Website Computer and Technology

Principal Component Analysis

Subject: Computer and Technology noun /ˈprɪn.sə.pəl kəmˈpoʊ.nənt əˈnæl.ə.sɪs/ 1 definition

  1. Definition (noun)

    Entry #304 · Updated

    Principal Component Analysis (PCA) is an unsupervised statistical and linear-algebra technique that transforms a dataset of potentially correlated features into a smaller set of linearly uncorrelated variables called principal components. After centering the input data—and typically scaling features to unit variance—PCA computes an orthogonal coordinate basis via eigendecomposition of the covariance matrix or Singular Value Decomposition (SVD) of the data matrix, ordering axes so each successive component captures the maximum remaining variance. Projecting observations onto the leading k principal components compresses high-dimensional inputs, eliminates feature multicollinearity, and filters out low-variance noise while preserving the greatest possible linear variance in a k-dimensional subspace. Because PCA relies purely on second-order global variance without consulting target labels, it cannot directly capture nonlinear manifolds and may discard low-variance directions that remain informative for supervised tasks.

    Common Forms

    • Standard (Exact) PCA: Computes the exact orthogonal principal axes via full Singular Value Decomposition of the centered data matrix (or eigendecomposition of its covariance matrix), delivering optimal linear variance retention for dense, memory-resident datasets.

    • Randomized PCA: Uses randomized subspace iteration to approximate the leading k singular vectors, drastically cutting runtime and memory overhead when reducing high-dimensional matrices to a much smaller target rank k.

    • Incremental PCA: Processes training observations in fixed-size mini-batches rather than loading the entire matrix into RAM, updating singular vector estimates sequentially for out-of-core or streaming workloads.

    • Kernel PCA: Implicitly maps inputs into a higher-dimensional feature space using a nonlinear kernel function—such as radial basis function (RBF) or polynomial—before extracting linear principal components to model curved manifolds.

    Example

    Before training a fraud-detection classifier in scikit-learn on a training set of 100,000 transactions with 256 correlated numeric features, an engineer standardizes each column with StandardScaler and configures PCA(n_components=0.95, svd_solver="full") inside a Pipeline. Fitting the pipeline projects the 100,000 x 256 matrix onto 34 uncorrelated principal components, retaining 95% of total variance while eliminating multicollinearity and shrinking downstream memory usage.