Random Forest
Subject: Computer and Technology noun /ˈræn.dəm ˈfɔːr.ɪst/ 1 definition
- machine-learning
- ensemble-learning
- supervised-learning
- decision-trees
-
Definition (noun)
Entry #299 · UpdatedA Random Forest is an ensemble machine learning algorithm that builds a collection of decision trees during training and aggregates their outputs—taking the majority vote for classification or the mean prediction for regression. Rather than fitting identical trees to the same data, it pairs bootstrap aggregating (bagging), which trains each tree on a random sample drawn with replacement, with random feature subsampling at every node split so dominant predictors cannot force all trees into correlated structures. Unsampled observations from each bootstrap draw supply built-in out-of-bag (OOB) error estimates and permutation feature importance scores without requiring a separate holdout set. Because trees grow independently rather than sequentially as in gradient boosting, a Random Forest parallelizes cleanly across CPU cores and strongly resists overfitting on tabular data, though large ensembles consume substantial memory and sacrifice the direct rule traceability of a single decision tree.
Common Types
Random Forest Classifier: Splits tree nodes using impurity criteria such as Gini impurity or Shannon entropy and predicts discrete class labels via majority voting or averaged class probabilities.
Random Forest Regressor: Optimizes node splits through variance reduction—typically mean squared error (MSE) or mean absolute error (MAE)—and averages continuous leaf values across trees to smooth step-function predictions.
Extremely Randomized Trees (Extra-Trees): Selects split thresholds at random across each candidate feature's range instead of searching for optimal cutpoints—often training on the full dataset without bootstrapping—to reduce variance further and accelerate fitting on high-dimensional data.
Quantile Regression Forest: Retains all observed target values inside each terminal leaf rather than only their mean, enabling full conditional distribution estimates and prediction intervals alongside point predictions.
Example
A fraud-detection pipeline trains a
RandomForestClassifierinscikit-learnwithn_estimators=200,max_features="sqrt", andoob_score=Trueon 1.2 million transaction records spanning 36 tabular features such as hourly spend velocity, merchant category codes, and device age. At inference time, each incoming payment event traverses all 200 trees concurrently across worker threads, averaging their predicted class probabilities in under4 mswhile using out-of-bag estimates to monitor generalization error without a separate validation split.