Action-Conditioned Transformers for
Decentralized Multi-Agent World Models

MACT across five SMAC maps, captured from the StarCraft II engine. The coloured rings are the world model's own prediction of where each enemy will be, one to five steps ahead, and the bars below show how close each of those predictions landed against a baseline that assumes nothing moves.

Multi-agent reinforcement learning (MARL) has achieved strong results on large-scale decision making, yet most methods are model-free, limiting sample efficiency and making coordination harder as teammates' policies evolve during training. Model-based reinforcement learning can reduce data usage, but planning and search scale poorly with joint action spaces. We introduce MACT, a decentralized action-conditioned Transformer world model for long-horizon cooperative control. Each agent processes discretized observation-action tokens with a shared Transformer, while a single cross-agent Perceiver step provides global context under centralized training and decentralized execution. MACT targets long-horizon coordination by coupling Perceiver-derived global context with an action-conditioned contrastive objective that predicts future latent representations over a short horizon conditioned on planned actions. On the StarCraft Multi-Agent Challenge, MACT achieves the strongest aggregate performance among 6 model-free and model-based baselines under matched low-data training budgets. Across 12 maps, including 2 SuperHard scenarios, MACT attains the best mean and median win rates, and ablations show that multi-step prediction and per-agent action conditioning are central to its gains.

A decentralized world model that predicts the future it plans for

Architecture

World-model training. Observations are tokenized by a VQ encoder, a shared per-agent transformer produces local latents and one-step heads, and the AC-CPC predictor is contrasted against projected Perceiver targets computed from augmented future observations.

Behaviour learning. The frozen world model rolls forward in latent space, and each agent's actor picks the next action from its own decoded observation.

SMAC under a tight data budget

Every method is re-run in one harness on a pinned SC2.4.1.2.60604 build, with identical per-map step budgets and the same 100-episode greedy evaluation: 50,000 environment frames on ten maps, 200,000 on 3s_vs_5z and the two SuperHard maps.

Quantitative comparison

Under one protocol the ranking changes. MATWM does not reproduce its published strength, and MAMBA is the strongest baseline. MACT still has the highest mean and median across the twelve maps and leads on five, with the largest margins on the coordination-heavy and long-horizon maps. The limits are equally clear: 3s_vs_4z stays low for everyone, 2s3z favours MBVD, and 6h_vs_8z is unsolved by all.

Aggregate win rate over the twelve maps, coloured by world-model family. MAMBA is the strongest baseline once every method is re-run in the same harness.

Table 1 of the paper. Mean±std win rate (%), smoothed over the last five evaluations. Bold marks the best on each map, grey the second. Every method was re-run in this harness. No number is carried over from another paper.

Because single SMAC maps are high variance, aggregate uncertainty is reported with the rliable protocol over all twelve maps. MACT reaches an IQM of 80.4, against 60.9 for MAMBA, 52.1 for MARIE, 14.1 for MATWM, 11.3 for MAPPO, 10.0 for MAT and 3.8 for MBVD. Probability of improvement is 0.63 over MAMBA and 0.68 over MARIE.

Evaluation win rate against environment steps for every method under the unified protocol (mean±std over seeds, Gaussian-smoothed).

Qualitative comparisons

Episodes rolled out from the trained checkpoints, captured from the StarCraft II engine itself. In every clip the learned policy plays red and the built-in AI plays blue. Both methods run through the same evaluation path (tokenizer plus decentralized actor, no world model at test time) on the same pinned SC2.4.1.2.60604 build. Every released checkpoint was re-run for 30 evaluation episodes, and Table 1 reports the mean over seeds. Where only one checkpoint survives for a method on a map the seed count is stated, because a single seed cannot show the spread the paper reports.

8m, focus fire at scale

Eight Marines a side. A kill only happens if several agents commit to the same target and hold that commitment for several steps, which is the behaviour the action-conditioned objective is meant to represent.

MACT on 8m. The learned policy plays red, the built-in AI plays blue.

MMM, the largest margin and the widest spread

Heterogeneous teams of Marines, Marauders and a healing Medivac differ in range and role, so the team must commit to a plan that only pays off several steps later. This is also the map with the widest seed spread, and one MAMBA wins outright.

MACT (left) and MARIE (right) on MMM. Across 30 episodes per seed, MACT's two available policies scored 46.7% and 0.0%, a mean of 23.4%, against the paper's 43.3 ± 7.2. MARIE scored 6.7% (paper: 23.5 ± 10.7). On this map a run either finds the coordinated behaviour or collapses outright. The clip shows the seed that found it.

so_many_baneling, spacing under splash damage

Banelings detonate on contact and damage everything nearby, so a clumped formation loses instantly. MACT reaches 94.2% against MARIE's 56.5%.

MACT (left) and MARIE (right) on so_many_baneling. Across 30 episodes per seed, MACT's three policies won 100%, 100% and 96.7%, a mean of 98.9%, against the paper's 94.2 ± 2.4. MARIE's single available policy won 36.7% (paper: 56.5 ± 26.6). This is the cleanest reproduction of the paper's ordering in our set.

3s_vs_3z, where the methods are comparable

Not every map favours MACT. On 3s_vs_3z MARIE is slightly ahead (85.0% vs 82.5%), and both are within each other's seed variance. We include it because the honest read of the results is that the advantage is concentrated in coordination-heavy settings rather than uniform across SMAC.

MACT (left) and MARIE (right) on 3s_vs_3z, both winning. MACT's three policies scored 86.7%, 70.0%, 90.0%, a mean of 82.2% (paper: 82.5 ± 4.7). MARIE scored 96.7% (paper: 85.0 ± 16.4). MARIE leads here, as it does in the paper. We include the case the method loses.

BibTeX

@article{kich2026mact,
  title   = {Action-Conditioned Transformers for Decentralized Multi-Agent World Models},
  author  = {Kich, Victor A. and Yamamori, Satoshi and de Jesus, Junior C. and Morimoto, Jun},
  journal = {Transactions on Machine Learning Research (TMLR)},
  year    = {2026},
  issn    = {2835-8856},
  url     = {https://openreview.net/forum?id=99nyrFfTJf}
}