How well does the model match reality?
Every EDHbots number is a model output. This page compares the model's predictions against real games logged with the free tracker, and it is published whether or not the numbers are flattering. If predicted win shares drift more than 15 percentage points from logged reality, that is a bug in the model, not a marketing problem.
validation corpus
Below the 500-game threshold we consider honest to score against. Until the corpus is large enough, no accuracy claim is made here — log your games to build it.
Latest scored report
No scored report yet. The scoring pipeline runs once the corpus passes 500 games with commander-identified seats: for each logged pod we simulate the matchup with the production engine and score predicted win shares against outcomes (mean absolute error and Brier score, segmented by bracket). The pipeline and its thresholds are in the repository — nothing about this page is hand-tuned.
Method: predictions come from the same pod engine users run, with the adaptive interaction policy and default settings — no per-game tuning. Games whose decks the engine cannot identify are excluded and the exclusion count reported. Draws in simulation are excluded from predicted win shares exactly as they are in the product.