A frozen suite of 1,000 six-max, 100bb cash-game composite quizzes for evaluating model decisions against stored GTO answer data. Every listed model completed the same V1 quiz set.
All listed models completed the same 1,000 V1 quizzes. The ranking is sorted by the selected metric.
Evaluation Protocol: Quizzes are drawn deterministically from frozen composite manifests. The model follows its own action path across preflop, flop, turn, and river.
Ground-Truth Isolation: Model prompts strictly exclude action IDs, opponent hole cards, quality labels, EV loss values, or GTO score indicators.
Score Scope: Average GTO Score gives each of the 1,000 hands equal weight. EV loss is averaged per scored decision along each model’s action path. How GTO Score is calculated ↓
GTO stands for Game Theory Optimal. GTO Score is this benchmark’s 0–100% decision-quality metric, based on each answer’s stored GTO quality label and expected value (EV) loss. Higher scores indicate better decisions.
| Quality | GTO Score |
|---|---|
| Best | 80–100% |
| Correct | 60–79% |
| Inaccurate | 40–59% |
| Wrong | 20–39% |
| Blunder | 0–19% |
The quality label sets the range. Within that range, lower EV loss earns a higher score. The answer’s EV loss percentage is divided by the largest EV loss percentage among the available answers at the same decision point, then clamped to 0–1.
Score = lower bound + (1 − normalized loss) × (upper bound − lower bound)
For example, a Best answer with a normalized loss of 0.25 scores 95%: 80 + (1 − 0.25) × 20.
We first average the scored decisions within each hand along the model’s own action path, then average those scores across all 1,000 hands. Each hand has equal weight, regardless of how many decisions it contains. The result is displayed as a percentage. Street breakdowns average scored decisions on each street.
Frozen set of 1,000 quizzes
6 positions · 3 starting spots
Legal choices match the run format
No opponent cards or scores
Preflop → Flop → Turn → River
Each model chooses its own path
Stored EV loss for each answer
Quality-based GTO score
A deterministic seed fixes the quiz selection. The 1,000-quiz manifest balances six table positions (166–167 each) and three preflop starting spots: First Raise In (333), Facing Open (333), and Facing 3-Bet (334).
The model chooses its own path through each action tree. A preflop fold can end the hand; continuing can lead to flop, turn, and river decisions.
Each chosen answer carries a quality label and EV loss. The reported GTO score combines its quality tier with normalized EV loss; average EV loss is reported separately in big blinds per scored decision.
Each card opens one decision sampled from the frozen V1 quiz set. These cards illustrate situations; they do not report per-spot model performance.
Decide whether to open a hand from early position at a six-player table.
Choose an opening action from late position after earlier players fold.
Choose whether to complete, raise, or fold when action reaches the small blind.
Face a cutoff raise in the big blind and choose whether to fold, call, or 3-bet.
Face an early-position open with a suited wheel ace and choose a response.
An UTG opener faces a button 3-bet and must choose between folding, calling, and 4-betting.
After raising over a small-blind limp, choose whether to check or bet a paired flop.
After a flop bet is called, choose whether to check or bet again on a paired turn.
After both players check flop and turn, choose whether to check or value bet the river.
Simple mean across all listed models. Later-street scores cover only decisions each model reached.
We acknowledge the earlier work PokerBench: Training Large Language Models to Become Professional Poker Players by Zhuang et al. (2025) and its contribution to evaluating poker-playing language models. Our project was developed independently, and the shared name is coincidental. Other projects also use the same or similar names, including PokerBench.ai and AI Poker Bench. We recognize the potential for confusion and offer the following clarification of our benchmark’s scope and methodology.
Our benchmark differs from the 2025 work in several respects:
Every hand in our benchmark begins pre-flop, whereas the earlier benchmark comprises separate datasets of pre-flop and post-flop decision scenarios.
Our core evaluation uses a closed-loop environment: agents make successive decisions from pre-flop until the hand ends, with their actions shaping subsequent states. The earlier benchmark’s primary dataset evaluation is open-loop, asking for a single decision at a given poker spot. The paper also includes separate head-to-head gameplay experiments.
We introduce a GTO Score that accounts for both action frequencies and expected value (EV). The earlier benchmark evaluates individual decisions using Action Accuracy and Exact Match Accuracy.
We include human players as part of our benchmark, providing a direct reference for interpreting model performance.
We evaluate more recent generations of models, providing an updated view of current poker-playing capabilities.
We appreciate the earlier authors’ contribution and hope this clarification helps readers distinguish the two independently developed projects.