Workflow and operations API
These APIs turn validated configuration into reproducible runs, reports, optional tracking, tuning results, and exportable tariffs. Read experiment configuration for the run path and reporting and operations for task-oriented examples.
| Module | Public API | Purpose |
|---|---|---|
azoic.workflow |
ExperimentConfig, ModelSpec, PreprocessingSpec, TuningSpec, FloatDistribution, IntDistribution, CategoricalDistribution, Run, ModelResult, run_experiment |
Validated experiment configuration and reproducible runs |
azoic.tariff |
export_tariff, distill_gbm, extract_tariff, apply_tariff, recalibrate_for_total |
Multiplicative xlsx tariff export, GBM-to-GLM distillation, and tariff application |
azoic.reporting |
model_card, comparison_table, comparison_dashboard |
Markdown model cards, comparison tables, standalone HTML dashboard strings |
azoic.mlops |
log_run |
MLflow run logging (lazy import) |
azoic.tune |
tune_experiment, TuneResult |
Optuna tuning on inner splits of outer training data |
Run an actuarial modelling experiment from a YAML config.
ExperimentConfig (pydantic) captures data + spec + split + models. run_experiment
loads the data, splits it (random or temporal), fits every named model, and
returns a Run carrying per-model diagnostics (Gini / O-P ratio / Tweedie
deviance / calibration table + the fitted estimator).
Pure-premium convention (PRD sec. 5 rule 1): estimator is fit on
y = claim_amount / exposure with sample_weight = exposure (the exposure
column is popped inside RiskGLM / RiskGBM); diagnostics use aggregate
claim_amount as y_true and the predicted pure premium (rate) as
y_pred with sample_weight = exposure.
ExperimentConfig
Bases: BaseModel
End-to-end modelling experiment: data + spec + split + named models.
Load with ExperimentConfig.from_yaml(path). The data path is stored as
a string so the config round-trips YAML cleanly; from_yaml anchors a
relative local path to the YAML file's directory (URI schemes such as
s3:// pass through untouched), and run_experiment resolves it
through azoic.data.load_data (local + s3 paths via fsspec).
feature_columns(df)
Resolve the feature-column list: explicit features if given,
else every column that is not a special (spec-required) column.
ModelResult
Bases: BaseModel
Diagnostics for one fitted model inside a Run.
Workflow evaluation snapshots parameter containers. Attributes cannot be reassigned, but public dictionaries and DataFrames remain mutable. Fitted estimators are returned separately when requested.
ModelSpec
Bases: BaseModel
One estimator spec inside an ExperimentConfig.
kind selects the wrapper class; params are forwarded verbatim to its
constructor (family/link/exposure_col for GLM,
objective/tweedie_variance_power/n_estimators for GBM, ...).
build(dataset_spec=None)
Construct the (unfitted) estimator from this spec.
effective_params(dataset_spec=None)
Parameters exactly as applied to the constructed estimator.
Single source for build and ModelResult.params so the recorded
configuration cannot drift from the fitted estimator.
Run
Bases: BaseModel
The full result of a single run_experiment invocation.
train_indices and test_indices are immutable positional integer
tuples in fit/evaluation order, relative to the fingerprinted input frame.
Preserve that frame's row order and use df.iloc[list(run.test_indices)]
to recover the holdout. Tuned runs record the final outer partition.
Workflow evaluation deep-copies configuration and recorded parameters; later caller mutations cannot change them. Public containers remain mutable.
run_experiment(config, *, return_estimators=False)
Load, validate, split, fit, and score an experiment.
Multiplicative tariff export for a fitted RiskGLM.
A log-link GLM (Tweedie / Poisson / Gamma) is a multiplicative model:
predict(X) = exp(intercept + sum_numeric coef * x + sum_cat coef[level])
= base * prod_numeric exp(coef * x) * prod_cat factor[level]
glum encodes categoricals as full one-hot (drop_first=False); the actuary
picks a reference level per categorical feature, folds its coefficient into
the base rate, and applies every other level as a relativity to that
reference. extract_tariff returns that structure; apply_tariff
applies it back to data; export_tariff writes a 3-sheet xlsx.
Three sheets per the PRD section 3 base / factors / mappings contract:
* base_rate -- one row with the multiplicative base (intercept
folded with reference-level factors) plus the GLM family / link and the
recalibration flag. When (X, y, exposure_col) and
recalibrate=True are given, the base is shifted so the tariff
reproduces the observed portfolio total claim amount.
* factors -- one row per (feature, level). Numeric features
carry a single _per_unit row (apply as factor ** value);
categorical features carry one row per level (apply as the matching
factor, reference level = 1.0).
* mappings -- self-documenting feature encoding the GLM saw
(feature / dtype / role / levels / reference_level).
apply_tariff(tariff, X)
Per-row pure-premium rate under the multiplicative tariff on X.
Equivalent to RiskGLM.predict(X) when tariff has not been
recalibrated. Numeric inputs must be finite and categorical levels must
have been seen when the tariff was extracted. The linear predictor is
summed before exponentiation. Invalid factors and unrepresentable rates
raise ValueError; a zero recalibrated base produces zero rates.
distill_gbm(teacher, X_fit, X_validation)
Fit an exportable log-link GLM to a fitted positive-objective GBM.
The student is a RiskGLM that inherits the teacher's objective as its
family (tweedie, poisson, or gamma) and, for tweedie, the teacher's
tweedie_variance_power; it fits with the RiskGLM default
alpha=0.001. The fit frame teaches the student; the separate
validation frame supplies only held-out fidelity metrics. Both use the
teacher's exposure column as sample weight, and both frames must yield
positive, finite exposure and teacher predictions. A pipeline result
contains copied, already-fitted upstream preprocessing (bins and group
mappings are not relearned) and exposes the metrics in
distillation_metrics_.
Callers must supply disjoint observations for fitting and validation. The object-identity guard only rejects reuse of the same frame object; distinct frames (including copies) do not prove row independence. The CLI uses the run's stored training and test positions.
export_tariff(glm, path, *, X=None, y=None, exposure_col=None, reference=None, recalibrate=True)
Write a multiplicative-tariff xlsx from a fitted RiskGLM or pipeline.
Three sheets per PRD section 3: base_rate / factors / mappings.
By default recalibrate=True and an (X, y, exposure_col) triple is
expected; the base is shifted so the tariff reproduces the observed
portfolio total claim amount. Pass recalibrate=False for the structural
tariff (intercept + reference levels only).
y must be a finite, non-negative one-dimensional vector of aggregate
claim amounts with one value per row of X. Library callers choose this
calibration frame explicitly; keep evaluation observations separate. The
CLI recalibrates only on the run's stored training positions.
Base and factors must be representable as finite floats (strictly positive except for a zero-total recalibrated base). User text is written as literal Excel strings; numeric and boolean level labels retain their types.
extract_tariff(glm, *, reference=None)
Decompose a fitted RiskGLM into a multiplicative tariff structure.
Returns a dict with keys
base_rate -- structural base (intercept folded with the
reference-level coefficients for each categorical feature).
reference -- {categorical_feature: chosen_reference_level}.
numeric -- {numeric_feature: coefficient}.
categorical -- {categorical_feature: {level: multiplicative_factor
relative to reference}} (reference level has factor 1.0).
mapping -- pd.DataFrame describing the feature encoding.
The returned base_rate is structural (intercept + ref levels only);
pass the returned dict through recalibrate_for_total (or call
export_tariff with the (X, y, exposure_col, recalibrate=True)
quadruple) to shift the base so the tariff reproduces an observed
portfolio total.
Coefficients must be finite; the structural base and categorical factors must be representable as positive, finite floats. Numeric coefficients remain in log space; workbook export additionally requires representable per-unit factors.
recalibrate_for_total(tariff, *, predicted_total, observed_total)
Shift the tariff base so it reproduces an observed portfolio total.
predicted_total is the sum of model-predicted pure premium times
exposure; observed_total is the sum of observed aggregate claim amount.
Returns the new base rate. A zero observed total returns zero; otherwise
an unrepresentable adjusted base raises ValueError.
Model card rendering for a azoic.workflow.Run.
model_card(run) returns a markdown summary of the experiment: config name,
dataset shape + features, and a per-model block with metrics + a calibration-
table preview.
ponytail: deliberately tiny -- md only.
comparison_dashboard(runs)
Return standalone Plotly HTML with the same context warning as the table.
comparison_table(runs)
Return all model metrics, warning once for mixed evaluation contexts.
Dataset fingerprints, target/exposure columns, and holdout membership must match for direct comparisons. Holdout order does not matter. Mixed contexts remain available for descriptive comparison; the warning names affected runs.
model_card(run)
Return the model card for run as markdown.
Calibration tables are truncated to their first 12 rows -- a preview, not the full table; write full tables separately when all segments are needed.
Thin mlflow run logger for a azoic.workflow.Run (lazy mlflow import).
mlflow is a heavy optional dep and lives in the mlops extra
(uv sync --extra mlops). log_run imports it lazily and raises with a
clear pointer when the extra is missing.
The whole M6 mlops surface is one function: a Run already carries
everything worth logging -- the config fingerprint (top-level), per-model
params, per-model metrics, and the calibration tables (CLI can persist those
as artifacts before calling log_run).
log_run(run, *, tracking_uri=None, experiment_name=None, run_name=None, artifacts=None, artifact_path='artifacts')
Log run to mlflow and return the mlflow run_id (string).
run_name defaults to the experiment config name. Records:
* Top-level params: config name, split, test_size, random_state,
n_rows / n_train / n_test (prefixed experiment.*).
* Per-model: <model>.kind + one <model>.params.<k> per param,
plus one <model>.<metric> metric per metric (NaN and infinite
metrics are skipped -- mlflow rejects non-finite values since 3.x).
* Artifacts: every path in artifacts (file or directory). A missing
path raises FileNotFoundError. Files upload flat under
artifact_path (default "artifacts"); a directory's contents
upload nested under <artifact_path>/<dir_name> so the directory
name is preserved in the artifact tree.
Optuna hyperparameter search for ExperimentConfig models (v0.2 part 1).
Optuna lives in the tune extra (uv sync --extra tune); tune_experiment
imports it lazily and raises with a clear pointer when the extra is missing.
For each named model, tune_experiment runs an optuna study whose per-trial
objective is the actuarial-aware numeric penalty decided in PRD section 9 (in
lieu of the cut TariffOptimizer constraint DSL):
deviance_test + calibration_penalty * |1 - op_ratio_test|
per trial → fit/predict/score on a fixed inner split of the outer training
partition. The outer test partition remains untouched until the selected models
are refit on all outer training rows and evaluated once for the returned
canonical Run.
The default search space touches only regularization / tree-structure hyperparams. A typed custom model space replaces that model's default space. Identity, routing, and seed params stay put, so PRD rules 1 and 4 are never violated by a search suggestion.
TuneResult
Bases: BaseModel
Full result of one tune_experiment invocation.
best_params carries only the sampled (tuned) hyperparams per model; the
identity params from the YAML live on Run.models[name].params.
estimators is populated only when return_estimators=True (mirror of
the same kwarg on run_experiment; needed by export_tariff after a
tuned run).
tune_experiment(config, *, n_trials=None, calibration_penalty=None, random_state=42, return_estimators=False)
Run an optuna study per named model; return a TuneResult whose
run refits every selected model on outer training data and scores outer test.
calibration_penalty scales |1 - op_ratio_test| to be commensurate
with deviance_test -- tune it to your portfolio's deviance magnitude
(a unitless ratio penalty is invisible against a six-figure Tweedie
deviance; raise the penalty until calibration loss shows up in the trial
ordering).
Explicit n_trials and calibration_penalty values override
config.tuning; omitted values use YAML, then 20 and 1.0.
return_estimators=True also populates TuneResult.estimators with
{name: fitted_estimator} from the final run_experiment.