The Kolmogorov-Smirnov (KS) test measures the difference in distributions of continuous variables (like credit scores) between different groups. The KS test works by calculating the maximum vertical distance between two distributions. A high KS statistic indicates that the distributions of a variable (like credit scores) are significantly different between the groups. This could suggest potential bias in the underwriting process.
To determine whether the results of the KS test are statistically significant, you primarily look at the p-value.
A p-value less than 0.05 (often used as a standard threshold) suggests that there is less than a 5% probability that the observed difference in distributions is due to random chance. Thus, the result is considered statistically significant, indicating potential issues in fairness between the two groups. Lower thresholds (like 0.01 or 0.001) indicate even higher confidence that a difference in outcomes between groups is not random.
Although there are no concrete fairness thresholds, regulators may find:
