Layer 1

Calibrated escalation

For uncertified actions, selective prediction and conformal risk control convert uncertainty into a review-rate SLA. The plots below are generated from real public datasets in this workspace.

Risk-coverage as product dashboard

Coverage is the auto-execution rate. Risk is the error rate on auto-executed cases. On the Wisconsin Diagnostic Breast Cancer dataset, a logistic-regression router auto-executes 195/200 held-out cases at 0.51% observed selective error and escalates the remaining 2.5%.

569real dataset cases
200held-out test cases
97.5%auto-execute at 1% target risk
0.51%observed selective error
Risk coverage plot from the Wisconsin Diagnostic Breast Cancer dataset
Generated by scripts/generate_real_plots.py. Data: data/breast_cancer_risk_coverage.csv and data/breast_cancer_operating_points.csv.

Conformal review budget

On sklearn Digits, split conformal prediction sets turn uncertainty into an explicit review rule: auto-execute singleton sets, review non-singleton or empty sets.

Conformal review budget plot from sklearn Digits
At alpha 1%, this split observed 100% coverage and 31.1% review; at alpha 10%, observed coverage was 88.2% on this one split.
Conformal coverage plot from sklearn Digits
Coverage is measured on a test split separate from the calibration split. See data/digits_conformal_sets.csv.