megathon_travellers - green-product propensity models (CatBoost)

Eight CatBoost binary classifiers (catboost_GP01.cbm ... catboost_GP08.cbm), one per green product, predicting the probability that an eligible customer takes the product up. Built for Megathon 26 on a synthetic customer dataset (not real people). Scores are Platt-calibrated (calibration.json).

Files

• catboost_GP0x.cbm - model per product
• manifest.json - features, categorical features and calibration per product
• calibration.json - Platt parameters: p = sigmoid(a * logit(raw) + b)
• train_catboost.py, predict_catboost.py - training and scoring code (need the project's data/ feature files, not included)
• cat_boostreport.md - full metrics report

Use

import json, numpy as np
from catboost import CatBoostClassifier
man = json.load(open("manifest.json"))["GP03"]
model = CatBoostClassifier().load_model("catboost_GP03.cbm")
# X: DataFrame with man["features"]; categorical columns (man["cat_features"]) as strings, NaN -> "NA"
raw = model.predict_proba(X)[:, 1]
c = man["calibration"]; r = np.clip(raw, 1e-6, 1-1e-6)
propensity = 1 / (1 + np.exp(-(c["a"] * np.log(r / (1 - r)) + c["b"])))

Results (held-out test customers)

Per-product CatBoost classifiers (GP01-GP08), reloaded from models/*.cbm by predict_catboost.py (no retraining) and scored on the test split: customers never used in training or hyper-parameter tuning. Training: train_catboost.py (Optuna, 20 trials x 3-fold CV maximising PR-AUC; Platt calibration on out-of-fold scores).

Ranking quality (test split)

ROC-AUC / PR-AUC in brackets: 95% bootstrap CI. oracle_auc is the ceiling from the hidden true probabilities. lift_top10 / capture_top10: lift and recall within the top 10% of scores.

F1, precision, recall (test split)

OOF-tuned best-F1 threshold: the calibrated-propensity cut-off that maximises F1 on dev out-of-fold scores, applied unchanged to the test split. top 10% flagged: flag the 10% highest-scoring customers. Take-up is rare, so F1 is inherently low; treat it alongside the ranking metrics.

Take-up is rare (1-4%) and some test sets have under 40 positives, so confidence intervals are wide and F1 is low; the ranking metrics (ROC-AUC, lift) are the better summary. No license has been set yet.