gini index

Gini Index or Gini Impurity is a metric used to measure the frequency with which a randomly chosen element from a dataset would be incorrectly labeled if it were randomly labeled according to the distribution of labels in the subset.

In Machine Learning, it is primarily used as a splitting criterion by the algorithm to build decision trees.

Formula

#

The Formula For a dataset with $C$ total classes, where $p_i$ is the probability of an item belonging to class $i$:

Key Values

#

• $0.0$ (Pure Node): All elements belong to a single class. The split is perfect.
• $0.5$ (Max Impurity for 2 classes): Elements are equally distributed across classes (50/50 split).
• Higher Values: More impurity/randomness. The maximum value approaches $1.0$ as the number of classes increases.

Implementation

#

It can be implemented in numpy as:

def gini_np(y):
    if len(y) == 0:
        return 0.0
    classes, counts = np.unique(y, return_counts=True)
    probabilities = counts / len(y)
    squared_probabilities = probabilities ** 2
    impurity = 1.0 - np.sum(squared_probabilities)
    return float(impurity)

Decision trees commonly use either gini index or entropy at each split.