"Minimal gain - Decision Trees"

Madcap

Could someone help explain to me minimal gain for decision tree pre-pruning.
How is the gain of a node quantified?

Thanks for any help
-Madcap

varunm1

Hello@Madcap

Gain is calculated based on entropy. Gain will give us the amount of information provided by a splitting a feature.

In case of pre-pruning, we will not build the whole tree as we will start pruning (removing split) while the tree is being built

From my understanding, the minimal gain is where we are setting a cut point (threshold). For example, if the calculated gain for a particular feature split is greater than the set minimal gain then there will be a split else it will just leave that feature as a leaf.