The Random Forest is an ensemble of decision trees that reduces overfitting by using:
- bagging to reduce variance
- random selection of features from the full set of features at each split to prevent the important features from always being present at the top of each tree
It is important to note that we can’t overfit by putting in too many trees as they just reduce variance; at some point, variance reduction will just stop and we need to take into account computation time.