Ensemble methods for trees

Ensemble methods for trees

Gradient tree boosting

This applies gradient boosting to tree.

Multiple Additive Regression Trees (MART)

Bayesian Additive Regression Trees (BART)

Extra trees

Random forests

These use bagging techniques with random trees.

At each node, rather than sample the whole data we sample a random selection.

Get \(d\) dimensions, and sample \(m\) of them at each node.

Choose \(m\le \sqrt d\)

Regression forests