auditing-subgroup-fairness

Auditing Subgroup Fairness

An aggregate pass can hide a group the model fails. For de-identification that failure has a name: under-protection — PHI that leaks more often for one demographic group than another. openmed.eval.fairness_report slices leakage and recall by gold-span group so disparities surface before deployment, not after a breach.

When to use this skill

• You want per-subgroup recall and leakage for a de-id or NER model.
• You suspect (or must rule out) that one group is under-protected.
• You need disparity numbers for a clinical AI governance review.
• You need to document which subgroups you couldn't evaluate (the data gap).

What it measures

For each surrogate group fairness_report returns:

• leakage_rate — fraction of that group's gold PHI characters left exposed (the de-id harm metric).
• recall — fraction of that group's gold spans detected.
• leakage_disparity — max - min leakage across groups (the gap to close).
• worst_group / worst_group_leakage — the most-failed group.

Group membership comes from a group tag in each gold span's metadata (keys group, demographic_group, or surrogate_group); ungrouped spans fall into unspecified.

Quick start

from openmed.eval import fairness_report

# Gold fixtures must tag spans with a surrogate group, e.g.
#   {"start": 4, "end": 12, "label": "PERSON", "metadata": {"group": "female"}}
fair = fairness_report(
    "OpenMed/Privacy-PII-Detection",
    "golden",                 # named suite, or pass a list of fixtures
    device="cpu",
)

print("leakage disparity:", fair.leakage_disparity)
print("worst group      :", fair.worst_group, fair.worst_group_leakage)
for group, m in sorted(fair.per_group.items()):
    print(f"  {group:14s} recall={m.recall:.3f}  leakage={m.leakage_rate:.4f}")

# Under-protection alarm: any group leaking more than the rest.
LEAKAGE_GAP_LIMIT = 0.0       # leakage-first: ideally zero leakage everywhere
assert fair.leakage_disparity <= LEAKAGE_GAP_LIMIT or fair.worst_group_leakage == 0