chrismaghuhn/argentum-p1

Argentum P1 — Magic: The Gathering policy models

Small policy/value networks that play Magic: The Gathering inside the Argentum engine rules engine. They were trained from scratch on game data: first by imitating the engine's built-in AI, then with PPO self-play. No language model plays the game. The ML code lives in the fork chrismaghuhn/argentum-engine (branch chris/p1-02-ppo-league-20261001, ml/src/argentum_ml/p1).

Contents

Path	Format	What it is
standard/p1-std-v2e50big-0001	Standard 60-card, 1v1, 20 life	Imitation-learning base model (48,592 engine-AI games)
standard/p1-std-ppo-0001 … p1-std-ppo-0016	Standard	PPO checkpoints, one per iteration, starting from the base model
commander/p1-ppo-0048	Commander, 1v1	Earlier model (v1 features) that beats the engine AI 55–45
decks/	JSON	Deck pools used for training (standard-train*.json) and the held-out test (standard-heldout.json)

Each checkpoint directory holds model.safetensors, config.json, vocab.json, metrics.json (training history) and, for Standard models, card_features.json.

How the model sees the game

• Observation: only the acting player's public view. The opponent's hand and library are counts.
• Card tokens: one per visible card (battlefield, own hand, graveyards, exile, stack) with name, zone, owner side, types/subtypes, colors, keywords, counters, P/T, tapped, summoning sickness, damage and attachment. Standard models add per-card rules text (bag of word/bigram embeddings), parsed mana cost, subtypes, and a frozen 384-dim text embedding of the oracle text (BAAI/bge-small-en-v1.5, computed once offline).
• Global token: life, hand/library/graveyard/exile sizes and floating mana for both players, turn, phase/step, stack size.
• Candidates: the legal top-level actions for the current decision (variable count, pass included), each with its kind (cast, ability, land, attack, …), source card, targets and mana cost.

Architecture

A set transformer (3 layers, d_model 128, 4 heads, ≈2.7M parameters for Standard) over the card tokens plus a global token. Each candidate gets one score from a small MLP over its kind embedding, the encoded source token, the pooled target tokens and its cost features; a softmax over the candidates is the policy, closer to a pointer network than to a classifier over a fixed action space. A value head predicts the game outcome. The engine AI still fills in sub-decisions of the chosen action (exact targets, mana payment, combat damage assignment).

Training (Standard)

1. Imitation: 48,592 games of the engine AI against itself on 65 Standard meta decks (10 + 55 MTGGoldfish archetypes, fetched 2026-10-02), 3 epochs, warm-started from a smaller run. Validation: 93.3% top-1 agreement with the engine AI, 72.8% on non-pass decisions.
2. PPO: 16 iterations × 4,000 games (≈64k games). League per iteration: self-play (weight 6), the base model (2), the engine AI (2) and the two previous checkpoints (1 each). GAE (γ 1.0, λ 0.95), clip 0.2, entropy 0.01, KL penalty to the base model 0.03, learning rate 3e-4, early stop at approx. KL 0.03. Rollouts on a 360-vCPU machine (≈8 games/s), updates on one V100.

Results (Standard)

Argmax play, ≈340 games per match; ±5 percentage points is within noise.

Checkpoint	vs. base model	vs. engine AI on the 4 held-out decks
p1-std-ppo-0004	57%	31%
p1-std-ppo-0008	59%	32%
p1-std-ppo-0012	63%	40%
p1-std-ppo-0016	60%	43%

The held-out decks were never used in training; 30 of their 77 distinct cards never occurred in a training game. In the sampled PPO training games on the training decks, the win rate against the engine AI rose from 38% (iteration 1) to about 45–47% (iterations 11–16). Training was stopped by budget, not by convergence.

Commander (p1-ppo-0048): 55–45 against the engine AI over 100 argmax games (Elo ≈1523 vs. 1500), after imitation, two DAgger rounds and 48 PPO iterations (≈34k games).

Usage

from pathlib import Path
from argentum_ml.p1.model import load_checkpoint  # from the fork's ml/src

model, vocab = load_checkpoint(Path("standard/p1-std-ppo-0016"), device="cpu")

Decision requests come from the Argentum engine; argentum_ml.p1.serve is the stdin/stdout process the engine-side policy worker talks to. CPU inference is fast enough to play.

Limitations

• Only top-level actions are chosen by the model; targets, payments and combat assignment come from the engine AI.
• No search or lookahead: one forward pass per decision.
• Trained and evaluated against the Argentum engine AI only, on the listed Standard decks.
• About 0.5% of rollout games ended in an engine exception and were skipped.

Card data

Card text is limited to the 477 cards of the listed decks (plus tokens) in card_features.json; for all other cards the release keeps only the name id and the frozen text-embedding vector. Argentum P1 is unofficial Fan Content permitted under the Fan Content Policy. Not approved/endorsed by Wizards. Portions of the materials used are property of Wizards of the Coast. ©Wizards of the Coast LLC. Decklists: MTGGoldfish metagame sample lists.

License

MIT for the weights and code. The bge-small-en-v1.5 encoder (used offline for the text vectors) is MIT-licensed by BAAI.