segmented_adverse_impact_ratio#

solas_disparity.disparity.segmented_adverse_impact_ratio(...)#

Calculates within-segment and cross-segment disparate impact and statistical significance statistics for dichotomous outcomes when it is appropriate to aggregate segment results to a single cross-segment measurement of disparity. A segment might be a market, location, type of job, or some other discrete unit.

The Adverse Impact Ratio (AIR) is calculated within each segment and cross-segment.

AIR is defined as the percentage of favorable outcomes of the protected group divided by the percentage of favorable outcomes of the reference group.

\[\text{AIR}_\text{Protected Group} = \frac{\text{% Favorable Outcome}_\text{Protected Group}}{\text{% Favorable Outcome}_\text{Reference Group}}\]

The cross-segment AIR attempts to control for the fact that the protected class may be distributed across segments differently from the reference class. In order to account for this, this method keeps the distribution of protected group members constant across the segments, but reallocates the reference group members according to the protected group members’ distribution. It does keep the protected group and reference group favorable outcome rates constant within each segment. In this way, the method is able to adjust for different distributions of class members by segment, but test whether the overall acceptance rates differ across classes.

Two cross-segment tests for statistical significance are performed: the Cochran-Mantel-Haenszel, CMH, test, which tests whether all segment-level odds ratios are equal to 1.0; and the Breslow-Day test, which tests whether segment level odds ratios are equal to each other.

When one can reject the Breslow-Day test, the p-values from the segment level tests (i.e., the Fisher’s Exact or Chi-Squared tests) are assessed against critical values derived from the Benjamani-Hochberg Procedure for multiple testing, rather than the standard 2-tailed 5% rule.

When one cannot reject the Breslow-Day test, then the CMH practical significance results can be applied to within-segment practical significance. If overwrite_segment_results=True, then the within-segment practical significance results are overwritten with the CMH practical significance results. By setting overwrite_segment_results=False, within-segment results will not be overwritten. For the case where group count is zero in a segment, the practical significance will remain "" and not be overwritten.

Parameters

  • group_data (DataFrame) – Dataframe containing columns for group data.

  • protected_groups (List[str]) – List of protected groups.

  • reference_groups (List[str]) – List of reference groups with the same length as protected_groups.

  • group_categories (List[str]) – List of group categories to which each protected and reference group pair belongs to (e.g. race, gender, age, etc.). Has the same length as protected_groups.

  • outcome (Series) – Boolean outcome series where a value of 1 is assumed to be favorable.

  • air_threshold (float) – Adverse Impact Ratio threshold value.

  • percent_difference_threshold (float) – Percent difference threshold value. For example, a 20% difference is input as percent_difference_threshold=0.2.

  • fdr_threshold (float) – False discovery rate (fdr) threshold value used in calculating whether segment-level results are statistically significant using the Benjamani-Hochberg Procedure.

  • segment (Series) – Segment series.

  • label (Optional[Series], optional) – Boolean label, true outcome, and/or target series evaluated alongside outcome. Defaults to None.

  • sample_weight (Optional[Series], optional) – Sample weight series. Has the same length as group_data. Defaults to None.

  • max_for_fishers (int, optional) – Maximum value of samples for Fisher’s exact test to be used. Defaults to MAX_FOR_FISHERS.

  • shift_zeros (bool, optional) – If True, if any cell count in a contingency table used in the CMH and Breslow-Day tests is zero, then 0.5 is added to all values in the contingency table so an odds ratio is able to be calculated. Note: In large sample sizes, this correction will not make a significant difference. In small sample sizes, this correction has the potential to impact the significance determination. Defaults to True.

Returns

Object containing results of the disparity calculation.

Return type

Disparity