3  Two reproducible experiments

3.1 Choose the model roles

The direct Tweedie GLM is the tariff baseline. A generalized linear model (GLM) adds feature contributions on a chosen scale. Here its log link means log(expected rate) = intercept + feature contributions: exponentiating gives a positive rate, and coefficients become multiplicative tariff factors. Bins and groups give the GLM discrete rating levels that we can inspect and export.

Tweedie with power 1.5 describes a non-negative cost outcome with both exact zeros and positive continuous values. This suits a portfolio with many no-claim policies and a few costly claims. We model the rate claim_amount / exposure, using exposure as the weight; predictions are expected cost per policy-year. The workflow derives that rate from the aggregate target column below.

The raw-feature GBM is the flexible benchmark. A gradient-boosted tree model (GBM) adds trees sequentially to improve the fit. It can capture nonlinear relationships and interactions without our explicit rating bins, but does not directly give a simple multiplicative tariff. These roles are not a validated model selection; we assess ranking and calibration in the next chapter.

3.2 Declare the data contract

DatasetSpec.target names aggregate policy claim cost, not a pre-divided rate. The feature list deliberately excludes policy IDs, outcomes, exposure, and review-only protected attributes. Special columns remain available to the workflow for weighting and supervised preprocessing, without becoming predictors.

from azoic.data import DatasetSpec
from azoic.workflow import ExperimentConfig, ModelSpec, PreprocessingSpec, run_experiment

portfolio = load_portfolio()
spec = DatasetSpec(
    target="claim_amount", exposure="exposure", claim_count="claim_count",
)
features = [
    "driver_age", "vehicle_age", "bonus_malus", "density", "region",
    "vehicle_brand", "veh_power", "veh_gas", "area",
]

load_portfolio() supplies this book’s cleaned Parquet input. The standalone own-portfolio example below replaces it with your file.

3.3 Set training-only preprocessing

Tree bins pool numeric values; similarity groups pool categorical levels with similar observed loss rates. The exposure and claim-count floors limit sparse groups. All boundaries and memberships are learned after splitting, on training rows only. Detailed merging rules and fitted memberships appear later.

preprocessing = PreprocessingSpec(
    binner={
        "cols": ["driver_age", "vehicle_age", "bonus_malus", "density"],
        "strategy": "tree", "max_bins": 8, "min_exposure": 1000,
        "min_claims": 50, "random_state": 42,
    },
    grouper={
        "cols": ["region", "vehicle_brand", "veh_power", "veh_gas", "area"],
        "strategy": "similarity", "max_groups": 8, "min_exposure": 2000,
        "min_claims": 50,
    },
)

3.4 Name the candidates

The GLM uses the log link and a small coefficient penalty (alpha) to temper large fitted effects. Its Tweedie power stays at 1.5 throughout this comparison.

direct_glm = ModelSpec(
    kind="glm",
    params={"family": "tweedie", "link": "log", "tweedie_power": 1.5, "alpha": 0.001},
)

The benchmark uses 100 trees with at most 31 leaves each; learning_rate controls each tree’s contribution. These are fixed tutorial settings, not a claim that tuning has selected them.

raw_gbm = ModelSpec(
    kind="gbm",
    params={
        "objective": "tweedie", "tweedie_variance_power": 1.5,
        "n_estimators": 100, "num_leaves": 31, "learning_rate": 0.05,
        "random_state": 42,
    },
)

3.5 Assemble and run the experiments

freMTPL2 has no suitable policy-time field. Both experiments therefore use the same random seed and 20% holdout on the same ordered data. The GLM experiment uses bins and groups; the GBM experiment uses raw features. The names identify these two runs and their two candidates in later chapters.

glm_config = ExperimentConfig(
    name="fremtpl2-tariff-structured", data_path=str(PORTFOLIO_PATH),
    spec=spec, features=features, preprocessing=preprocessing,
    split="random", test_size=0.2, random_state=42,
    models={
        "direct-tweedie-glm": direct_glm,
    },
)
gbm_config = ExperimentConfig(
    name="fremtpl2-raw-feature", data_path=str(PORTFOLIO_PATH),
    spec=spec.model_copy(), features=features,
    split="random", test_size=0.2, random_state=42,
    models={"tweedie-lightgbm": raw_gbm},
)

Other chapters use make_experiment_configs from _shared.qmd as the canonical reusable configuration. This check prevents the visible version from drifting:

Verify the shared configuration matches 
assert (glm_config, gbm_config) == make_experiment_configs(str(PORTFOLIO_PATH))

Request fitted estimators explicitly so we can inspect and score with them. The returned Run objects record metrics and the actual training/test positions.

glm_run, glm_estimators = run_experiment(glm_config, return_estimators=True)
gbm_run, gbm_estimators = run_experiment(gbm_config, return_estimators=True)
runs = [glm_run, gbm_run]
estimators = {**glm_estimators, **gbm_estimators}
train_idx = np.asarray(glm_run.train_indices, dtype=int)
test_idx = np.asarray(glm_run.test_indices, dtype=int)
Verify identical partitions and complete row coverage 
assert set(glm_run.models) == set(glm_config.models)
assert set(gbm_run.models) == set(gbm_config.models)
assert glm_run.data_fingerprint == gbm_run.data_fingerprint
assert glm_run.train_indices == gbm_run.train_indices
assert glm_run.test_indices == gbm_run.test_indices
pd.testing.assert_frame_equal(portfolio, pd.read_parquet(PORTFOLIO_PATH))

assert len(train_idx) == glm_run.n_train
assert len(test_idx) == glm_run.n_test
assert np.intersect1d(train_idx, test_idx).size == 0
np.testing.assert_array_equal(
    np.sort(np.concatenate([train_idx, test_idx])), np.arange(len(portfolio))
)

The split happens before pipeline fitting, so the held-out outcomes cannot influence the learned bins or groups. The returned positions identify the actual random holdout; no second split is made. Positions refer to the exact input frame in its original row order, not to index labels or policy IDs.

3.6 Inspect the fitted tariff structure

Grouping relativities are sum(claim_amount) / sum(exposure), and min_claims is an aggregate claim-count floor. Nominal levels are stably sorted by risk; ordered categories keep their declared order. Under-credible groups merge with their closest-risk neighbour, then the closest adjacent pairs merge until the group limit is met. Each merge recomputes aggregate risk, with distance ties going left.

These are training-fitted candidates for review. Exposure and claim-count floors do not establish statistical certainty. Tree cuts may change across samples. Similarity grouping uses loss rates before adjustment for other predictors, so membership is not an adjusted causal effect.

Summarize fitted groups and reconcile training totals 
train_frame = portfolio.iloc[train_idx].copy()
pd.testing.assert_index_equal(
    train_frame.index, portfolio.index.take(glm_run.train_indices)
)
y_train = train_frame["claim_amount"].to_numpy()
w_train = train_frame["exposure"].to_numpy()

raw_predict_columns = [*FEATURES, "exposure"]
special_predict_columns = [*FEATURES, "exposure", "claim_amount", "claim_count"]

glm_pipeline = glm_estimators["direct-tweedie-glm"]
fitted_preprocessing = glm_pipeline[:-1]
fitted_binner = glm_pipeline.named_steps["binner"]
fitted_grouper = glm_pipeline.named_steps["grouper"]
bin_snapshot = {name: edges.copy() for name, edges in fitted_binner.mapping_.items()}
group_snapshot = {name: mapping.copy() for name, mapping in fitted_grouper.mapping_.items()}
train_grouped = fitted_preprocessing.transform(train_frame[special_predict_columns])
train_grouped[["claim_count", "claim_amount"]] = train_frame[["claim_count", "claim_amount"]]

group_summaries = {}
for feature in FEATURES:
    grouped = train_grouped.groupby(feature, observed=True, sort=True, dropna=False)
    summary = grouped.agg(
        policies=("exposure", "size"), exposure=("exposure", "sum"),
        claim_count=("claim_count", "sum"), claim_amount=("claim_amount", "sum"),
    )
    summary["observed_pure_premium"] = summary.claim_amount / summary.exposure
    if feature in fitted_grouper.mapping_:
        mapping = fitted_grouper.mapping_[feature]
        summary["members"] = [
            ", ".join(str(level) for level, group in mapping.items() if group == label)
            for label in summary.index
        ]
    assert summary.policies.sum() == len(train_frame)
    np.testing.assert_allclose(
        summary[["exposure", "claim_count", "claim_amount"]].sum(),
        train_frame[["exposure", "claim_count", "claim_amount"]].sum(),
    )
    group_summaries[feature] = summary.drop(columns="claim_amount").reset_index().rename(columns={
        "claim_count": "claims", "observed_pure_premium": "observed rate",
    })

Start with one driver-age bin. Its label gives the age interval; policies inside that interval share one rating level. The observed rate is the bin’s training claim amount divided by its exposure, before adjustment for other features. The totals show how much data supports this level.

display(group_summaries["driver_age"].head(1))
driver_age policies exposure claims observed rate
0 (-inf, 21.5) 8960 3511.338776 741 633.274808

Now inspect one region group. The members column lists the original region codes pooled into this level. Similarity grouping uses their training loss rates; the fitted GLM will estimate a factor for this pooled level alongside its other predictors. Neither table reports a final tariff factor.

display(group_summaries["region"].head(1))
region policies exposure claims observed rate members
0 group_0 17654 7329.265747 397 80.125175 R73, R83

Review every interval and group membership, with the same reconciled training totals. This inspection runs even when the callout is collapsed.

Display every fitted rating level 
for feature, summary in group_summaries.items():
    print(feature)
    display(summary)
driver_age
driver_age policies exposure claims observed rate
0 (-inf, 21.5) 8960 3511.338776 741 633.274808
1 [21.5, 24.5) 14989 6441.658053 848 251.154309
2 [24.5, 27.5) 24285 10556.232154 982 183.211675
3 [27.5, 54.5) 350390 179760.345810 13421 132.841514
4 [54.5, 71.5) 108462 63128.585524 3979 105.832717
5 [71.5, inf) 27052 18620.614063 1183 152.744445 
vehicle_age
vehicle_age policies exposure claims observed rate
0 (-inf, 2.5) 146188 61482.401978 4589 146.869896
1 [2.5, 5.5) 104877 56901.041572 4236 134.411099
2 [5.5, 6.5) 28260 16391.701943 1344 149.202929
3 [6.5, 7.5) 26096 15230.582332 1331 174.700762
4 [7.5, 12.5) 127829 74084.745336 5996 142.445330
5 [12.5, 15.5) 59979 34114.147532 2343 131.598168
6 [15.5, 18.5) 27441 15801.411234 958 117.970870
7 [18.5, inf) 13468 8012.742455 357 61.077448 
bonus_malus
bonus_malus policies exposure claims observed rate
0 (-inf, 51.5) 314443 183382.422364 9603 93.239235
1 [51.5, 57.5) 41569 21181.018007 1536 139.789342
2 [57.5, 63.5) 30153 14939.112797 2131 242.679960
3 [63.5, 90.5) 112620 49147.943413 4973 186.068492
4 [90.5, 95.5) 13604 5515.677795 813 281.168394
5 [95.5, inf) 21749 7852.600007 2098 611.257498 
density
density policies exposure claims observed rate
0 (-inf, 18.5) 24431 14944.668795 720 83.189843
1 [18.5, 375.5) 238412 133502.937111 8581 117.141289
2 [375.5, 2351.0) 155916 80406.080225 6618 154.308531
3 [2351.0, 2676.5) 7639 3398.834366 362 269.451592
4 [2676.5, inf) 107740 49766.253884 4873 180.349177 
region
region policies exposure claims observed rate members
0 group_0 17654 7329.265747 397 80.125175 R73, R83
1 group_1 8699 3493.424806 250 108.617085 R42, R23
2 group_2 218926 136776.624395 9170 123.142018 R74, R41, R54, R52, R24, R53
3 group_3 69858 32093.789370 2331 135.427802 R72, R91, R25, R26
4 group_4 76176 32723.355817 2843 150.292611 R31, R11
5 group_5 68062 36308.019332 3358 163.951321 R43, R82
6 group_6 68880 30965.310914 2633 182.037678 R22, R93
7 group_7 5883 2328.984000 172 221.854002 R21, R94 
vehicle_brand
vehicle_brand policies exposure claims observed rate members
0 group_0 25783 14180.822090 1082 119.121614 B14, B6
1 group_1 255518 125274.843401 8815 131.632944 B2, B12
2 B1 128677 75039.561643 5558 134.354446 B1
3 B4 19988 10875.964280 879 140.083013 B4
4 B5 27443 15720.091851 1329 150.213047 B5
5 B13 9612 5355.869138 417 153.457723 B13
6 group_6 56422 30158.856187 2550 165.037567 B3, B10
7 B11 10695 5412.765792 524 228.134236 B11 
veh_power
veh_power policies exposure claims observed rate members
0 group_0 93053 48249.466357 3278 122.711444 15, 4
1 group_1 135275 71402.348888 5390 129.957604 5, 8
2 7 114640 61355.261341 4524 138.965893 7
3 group_3 141183 77018.537236 6026 149.643191 9, 6
4 11 14522 6688.204901 577 154.379367 11
5 10 24615 12119.088539 949 156.699463 10
6 12 6476 2984.137085 225 172.877393 12
7 group_7 4374 2201.730034 185 218.612456 13, 14 
veh_gas
veh_gas policies exposure claims observed rate members
0 'Regular' 271535 147418.719472 10350 131.622925 'Regular'
1 'Diesel' 262603 134600.054910 10804 146.929629 'Diesel' 
area
area policies exposure claims observed rate members
0 B 59500 33939.181419 2128 105.570435 B
1 A 81847 48684.808382 2702 112.346344 A
2 C 151118 81965.086651 5670 125.281468 C
3 D 119446 60773.218711 5111 156.796465 D
4 F 14087 6389.719780 644 157.534268 F
5 E 108140 50266.759439 4899 185.481476 E

Intervals are lower-inclusive and upper-exclusive at finite boundaries. Within each chapter, scoring and diagnostics reuse its training-fitted pipeline without refitting mappings. Other chapters recreate the same experiment in their own kernel.

Continue to essential diagnostics to assess these candidates.

3.7 Use your own portfolio

Start with one GLM and explicit columns. This standalone example needs Azoic, pandas, and a Parquet file named my_portfolio.parquet in your working directory; it does not use the book’s loader or shared helpers. Each row is a policy period, paid_claims is its non-negative aggregate cost, and insured_years is finite exposure in years, at least one day (1/366, with 0.1% tolerance). driver_age and vehicle_age are numeric. Clean missing/invalid values and choose a consistent loss layer before fitting.

import pandas as pd

from azoic.data import DatasetSpec
from azoic.workflow import ExperimentConfig, ModelSpec, run_experiment

my_features = ["driver_age", "vehicle_age"]
my_config = ExperimentConfig(
    name="my-first-tariff",
    data_path="my_portfolio.parquet",
    spec=DatasetSpec(target="paid_claims", exposure="insured_years"),
    features=my_features,
    split="random", test_size=0.2, random_state=42,
    models={
        "direct-glm": ModelSpec(
            kind="glm",
            params={"family": "tweedie", "link": "log", "tweedie_power": 1.5,
                    "alpha": 0.001},
        ),
    },
)
my_run, my_estimators = run_experiment(my_config, return_estimators=True)
my_portfolio = pd.read_parquet(my_config.data_path)
scoring_rows = my_portfolio.iloc[list(my_run.test_indices)][[*my_features, "insured_years"]]
scored = scoring_rows.copy()
scored["raw_rate"] = my_estimators["direct-glm"].predict(scoring_rows)
scored["expected_period_cost"] = scored["raw_rate"] * scored["insured_years"]
print(scored.head())

The scoring frame contains no claim outcomes. These are raw model rates: no training O/P adjustment has been applied. The simple GLM here uses numeric ages directly, so log rate is linear in each age; add the preprocessing shown above when discrete rating levels are needed. Do not copy freMTPL2 credibility floors without considering your portfolio’s exposure and claims volume.

Keep identifiers, outcomes, and protected attributes out of features; name any review-only protected columns in DatasetSpec.protected_cols. Exposure is a weight, carried into prediction as shown, not a rating feature. Keep the source row order unchanged when using the returned positions.

Python relative paths use the process working directory. With ExperimentConfig.from_yaml, relative data paths instead resolve from the YAML file’s directory; see the experiment guide. Use a random split when no real time field exists. If deployment means pricing future periods, use a genuine policy-time column and the temporal recipe; never invent time from row order. Choose revisions inside training data and reserve fresh evaluation data once you have used the displayed holdout to make decisions.

3.8 Optional: pool sparse risks and limit model flexibility

Sparse ages or vehicle categories can show large changes in predicted pure premium because a small amount of data has too much influence. Use the existing PreprocessingSpec and ModelSpec.params settings to compare a few deliberate alternatives:

Action in Azoic What it can achieve What it does not guarantee
Increase AutoBinner.min_exposure or min_claims, or reduce max_bins. Merge neighbouring age intervals so sparse tails share information with a larger group. Smooth transitions between bins, or enough evidence for every combination of rating factors.
Use the learned bins for a GBM as well as a GLM. Prevent age distinctions inside a bin, holding other inputs fixed. The best possible GBM: pooling can hide real differences, and other features can still split a bin into sparse leaves.
Increase AutoGrouper.min_exposure or min_claims, or reduce max_groups. Pool sparse categorical levels, such as vehicle brands, using the configured grouping strategy. Similar risk after accounting for other features: similarity grouping uses unadjusted loss rates.
Increase RiskGLM.alpha, keeping l1_ratio=0 for an L2 penalty. Pull fitted coefficients towards zero and temper extreme effects. Similar factors for adjacent ages: the penalty does not connect neighbouring bins.
Reduce RiskGBM.num_leaves, set a smaller positive max_depth, increase min_child_samples, or increase reg_lambda. Limit detailed interactions or shrink leaf estimates, making isolated fluctuations harder to fit. A minimum exposure per leaf: min_child_samples controls observation support, not an exposure floor.

If ages 80–95 share one bin, the model cannot use that age feature to charge an otherwise identical 93-year-old differently from an 82-year-old. Bins create flat steps with possible boundary jumps, not continuous curves. LightGBM’s internal histogram bins do not replace Azoic’s exposure and claim-count floors.

Use a direction constraint only when the risk relationship supports it. RiskGBM.monotone_constraints can require a numeric feature’s predicted effect to increase or decrease, holding other inputs fixed. Azoic rejects nonzero constraints on categorical features, including AutoBinner interval categories. Driver age need not have one direction across the whole age range. AutoBinner.monotonic influences how training bins are merged; it does not constrain the premiums subsequently fitted by the GLM or GBM.

3.9 Optional: check that stability has improved

If pooling is worth investigating, use the optional structured teacher to compare a binned GBM with the raw-feature GBM and direct GLM on the same validation rows. Review exposure-weighted Tweedie deviance, portfolio and segment O/P, and age profiles. Refit on repeated training samples to see whether tail premiums and learned boundaries move substantially. Relearn supervised bins and groups within each training fold, and choose settings using validation inside the outer training data. Keep the final test set for one evaluation.

Prefer the simpler candidate when predictive performance is comparable and tail premiums are materially more stable. The evidence comes from these comparisons, not from a smooth line or an exposure threshold alone.