Society of Actuaries (SOA) PA Practice Exam

What differentiates Random Forests from Boosted Trees in terms of tree construction?

1. Random Forests use residuals to build new trees.

2. Random Forests build trees simultaneously, while Boosted Trees build sequentially.

3. Random Forests rely on a single model for predictions.

4. Random Forests take shorter time to construct each tree.

The correct answer is: Random Forests build trees simultaneously, while Boosted Trees build sequentially.

The distinction between Random Forests and Boosted Trees lies primarily in how they approach the tree construction process. In Random Forests, multiple decision trees are built independently and simultaneously from subsets of the data. This method allows for a diverse set of trees, which contributes to the overall model's robustness and helps reduce overfitting through averaging the predictions of the individual trees. In contrast, Boosted Trees are constructed sequentially, where each new tree is built to correct the errors made by the previous trees. This means that Boosted Trees focus on improving the accuracy of the model incrementally, adjusting to the residuals or errors after each tree is added. Therefore, the key differentiator is that Random Forests build their trees simultaneously, allowing parallel processing and independence among the trees, while Boosted Trees build theirs in a step-wise fashion, focusing progressively on the errors of the earlier trees. This sequential approach in Boosted Trees fosters stronger performance on complex datasets where capturing nuances can lead to better overall accuracy.