sklift.metrics.average_squared_deviation

sklift.metrics.metrics.average_squared_deviation(y_true_train, uplift_train, treatment_train, y_true_val, uplift_val, treatment_val, strategy='overall', bins=10)[source]

Compute the average squared deviation.

The average squared deviation (ASD) is a model stability metric that shows how much the model overfits the training data. Larger values of ASD mean greater overfit.

Parameters
  • y_true_train (1d array-like) – Correct (true) target values for training set.

  • uplift_train (1d array-like) – Predicted uplift for training set, as returned by a model.

  • treatment_train (1d array-like) – Treatment labels for training set.

  • y_true_val (1d array-like) – Correct (true) target values for validation set.

  • uplift_val (1d array-like) – Predicted uplift for validation set, as returned by a model.

  • treatment_val (1d array-like) – Treatment labels for validation set.

  • strategy (string, ['overall', 'by_group']) –

    Determines the calculating strategy. Default is ‘overall’.

    • 'overall':

      The first step is taking the first k observations of all test data ordered by uplift prediction (overall both groups - control and treatment) and conversions in treatment and control groups calculated only on them. Then the difference between these conversions is calculated.

    • 'by_group':

      Separately calculates conversions in top k observations in each group (control and treatment) sorted by uplift predictions. Then the difference between these conversions is calculated

  • bins (int) – Determines the number of bins (and the relative percentile) in the data. Default is 10.

Returns

average squared deviation

Return type

float

References

René Michel, Igor Schnakenburg, Tobias von Martens. Targeting Uplift. An Introduction to Net Scores.