portfolio = load_portfolio()
audit_subset = (
portfolio.sample(n=min(100_000, len(portfolio)), random_state=RANDOM_STATE, replace=False)
.sort_values("policy_id")
.reset_index(drop=True)
)
audit_subset["driver_age_band"] = pd.cut(
audit_subset["driver_age"],
bins=[17, 30, 55, 120],
labels=["30_and_under", "over_30_to_55", "over_55"],
).astype(str)
audit_path = ARTIFACT_DIR / "protected_audit.parquet"
audit_subset.to_parquet(audit_path, index=False)
assert "driver_age_band" in audit_subset.columns10 Protected-group audit
Azoic keeps protected attributes inside the dataset contract but out of the model features: DatasetSpec(protected_cols=...) validates that a protected column cannot overlap outcome or weight columns, and run_experiment excludes it from candidates while retaining it for held-out subgroup calibration evidence.
This chapter demonstrates the mechanism on freMTPL2 with a derived age-band column declared as protected. This is an illustrative stand-in for the reporting mechanism. Driver age itself remains a rating feature here, so the model still has the information used to form the bands. A real audit would declare the relevant protected attributes supplied by the business and excluded from rating.
audit_config = ExperimentConfig(
name="fremtpl2-protected-audit",
data_path=str(audit_path),
spec=DatasetSpec(
target="claim_amount",
exposure="exposure",
claim_count="claim_count",
protected_cols=["driver_age_band"],
),
features=FEATURES,
split="random",
test_size=0.2,
random_state=RANDOM_STATE,
models={
"tweedie-glm": ModelSpec(
kind="glm",
params={
"family": "tweedie",
"link": "log",
"tweedie_power": 1.5,
"alpha": 0.001,
},
),
},
)
assert "driver_age_band" not in audit_config.features
assert audit_config.spec.protected_cols == ["driver_age_band"]
run = run_experiment(audit_config)result = run.models["tweedie-glm"]
assert set(result.protected_calibration) == {"driver_age_band"}
table = result.protected_calibration["driver_age_band"]
test_frame = audit_subset.iloc[list(run.test_indices)]
np.testing.assert_allclose(
table["exposure"].sum(), test_frame["exposure"].sum()
)
np.testing.assert_allclose(
table["claim_amount"].sum(), test_frame["claim_amount"].sum()
)
display(table.rename(columns={
"claim_amount": "Observed claims", "predicted_claim_amount": "Predicted claims",
"observed_pure_premium": "Observed rate", "predicted_pure_premium": "Predicted rate",
"o_p_ratio": "O/P",
}))| group | exposure | Observed claims | Predicted claims | Observed rate | Predicted rate | O/P | |
|---|---|---|---|---|---|---|---|
| 0 | 30_and_under | 1315.111547 | 214625.63 | 276877.418928 | 163.199563 | 210.535311 | 0.775165 |
| 1 | over_30_to_55 | 6434.634203 | 782258.23 | 785780.770211 | 121.569961 | 122.117396 | 0.995517 |
| 2 | over_55 | 2841.370627 | 223676.85 | 342677.843509 | 78.721462 | 120.603008 | 0.652732 |
Reconciliation checks accounting consistency: all held-out exposure and claims are represented. It does not establish equal calibration across groups. Aggregate balance can conceal subgroup errors, including errors that offset.
With the default 100,000-row subset, overall O/P is about 0.869 and the over-55 group is about 0.653. Both indicate overprediction, which is more pronounced in the older group. This is a reason to review group exposure, claim support, and model structure, not a fairness pass. Different sample settings change these values; compare the current table with this run’s overall result:
print(f"Overall held-out O/P: {result.metrics['op_ratio_test']:.3f}")Overall held-out O/P: 0.869card = model_card(run)
assert "protected columns" in card
assert "driver_age_band" in card
print("\n".join(card.splitlines()[:40]))# Azoic model card -- fremtpl2-protected-audit
## Experiment
- name: `fremtpl2-protected-audit`
- data: `/home/runner/work/azoic/azoic/examples/_artifacts/fremtpl2/protected_audit.parquet`
- split: `random` (test_size=0.2, random_state=42)
- target: `claim_amount` exposure: `exposure`
- data fingerprint: `b192959e2a0e9c06bf654d98b059759dc6a87296daa733ef8fab8fabf09f3cb7`
- rows: 100000 (train 80000 / test 20000)
- features (9): `driver_age`, `vehicle_age`, `bonus_malus`, `density`, `region`, `vehicle_brand`, `veh_power`, `veh_gas`, `area`
- protected columns: `driver_age_band`
### Model: `tweedie-glm` (glm)
- params: `{'family': 'tweedie', 'link': 'log', 'tweedie_power': 1.5, 'alpha': 0.001, 'exposure_col': 'exposure'}`
- gini (train): 0.3442
- gini (test): 0.2997
- O/P ratio (test): 0.8685
- deviance (test): 67.126545
- D² (test): 0.0464
#### Calibration table (test, first 12 rows)
| group | exposure | claim_amount | predicted_claim_amount | observed_pure_premium | predicted_pure_premium | o_p_ratio |
| --- | --- | --- | --- | --- | --- | --- |
| 0.0000 | 1058.5785 | 64398.0400 | 49146.2435 | 60.8344 | 46.4266 | 1.3103 |
| 1.0000 | 1059.4850 | 65528.8000 | 68384.7045 | 61.8497 | 64.5452 | 0.9582 |
| 2.0000 | 1058.8867 | 62277.7800 | 79881.8583 | 58.8144 | 75.4395 | 0.7796 |
| 3.0000 | 1058.7076 | 79990.6900 | 90253.9941 | 75.5550 | 85.2492 | 0.8863 |
| 4.0000 | 1059.5421 | 121716.5300 | 101459.5990 | 114.8765 | 95.7580 | 1.1997 |
| 5.0000 | 1058.9494 | 85355.7000 | 113644.4542 | 80.6041 | 107.3181 | 0.7511 |
| 6.0000 | 1058.9312 | 99185.6800 | 129004.2030 | 93.6658 | 121.8249 | 0.7689 |
| 7.0000 | 1059.6649 | 153029.1600 | 152650.0930 | 144.4128 | 144.0551 | 1.0025 |
| 8.0000 | 1058.4032 | 156507.1400 | 199526.1924 | 147.8710 | 188.5162 | 0.7844 |
| 9.0000 | 1059.9677 | 332571.1900 | 421384.6908 | 313.7560 | 397.5448 | 0.7892 |
#### Protected-group calibration (outer test)
This held-out subgroup evidence is descriptive only; it is not a fairness threshold or legal assessment.
The audit stops at evidence: Azoic reports per-group calibration without universal fairness thresholds or legal conclusions. Reviewing those tables, deciding whether a differential is acceptable, and documenting the judgement remain actuarial and compliance work outside the library.