MAD Studio is an operational thinking console for structured AI deliberation. Configure 2–100 reasoning agents, run them through peer-reviewed debate protocols, and watch them pressure-test ideas one turn at a time.
MAD Studio operationalizes the leading academic frameworks in multi-agent debate, turning them from research notebooks into a production-grade workbench. Every protocol links to its published source — on arXiv, ACL Anthology, or NeurIPS.
New to the field? Read our guide on when to use multi-agent debate vs self-consistency.
Du, Li, Torralba, Tenenbaum, Mordatch
Foundational result: agents critiquing each other across rounds converge on more factual, better-reasoned answers.
Liang, He, Jiao, Wang, Wang, Wang, Yang, Shi, Tu
Introduces the Degeneration-of-Thought problem and shows multi-agent debate unlocks divergent thinking where self-reflection stalls.
Feng, Zhao, Lyu, Li, Tu, Wang
Introduces the per-dimension arbiter sweep that powers MAD Studio's truth-seeking verdict scoring.
Chan, Chen, Yu, Lu, Sun, Liu
Demonstrates that multi-agent debate panels evaluate generated text more reliably than single-judge baselines.
Khan, Hughes, Valentine, Ruis, Sachan, Radhakrishnan, Bowman, Perez
Strong empirical evidence that debate makes weaker judges reliably select truthful answers from stronger debaters.
Hu, Tan, Wang, Qu, Chen
Formalizes debate among LLM judges and adds adaptive stability detection so debates stop when consensus stabilizes — improving accuracy over majority vote.
Wu, Bansal, Zhang, Wu, Li, Zhu, Wang, Saied, Awadallah, Awadalla, Wang
Shows that role-specialized agent groups orchestrated through structured conversation consistently outperform monolithic prompts on complex tasks.
Shinn, Cassano, Berman, Gopinath, Narasimhan, Yao
Verbal self-critique loops iteratively raise agent performance — the direct precedent for Saga's recursive optimization passes.
Madaan, Tandon, Gupta, Hallinan, Gao, Wiegreffe, Alon, et al.
Single-model iterative refinement via self-generated feedback. The minimal version of what multi-agent debate scales up across roles.
Wang, Bai, Liu, Chen, Cardie, Zhang, et al.
Layered multi-LLM collaboration where each layer's agents refine the previous layer's outputs. Open-source MoA reaches 65.1% on AlpacaEval 2.0, beating GPT-4 Omni.
Choi, Zhu, Li, et al.
Pinpoints when multi-agent debate actually beats majority vote: diversity-aware initialization plus calibrated confidence communication.
Swastik Raj
Practical walkthrough of orchestrating multi-agent debate in production — useful context for how debate systems are built and scaled in the wild.
Read on MediumPeer-reviewed methodology, explained for builders — free to read, no signup required.
A research-backed guide to choosing between self-consistency (CoT-SC) and multi-agent debate (MAD) — with benchmark numbers, cost analysis, failure modes, and a hybrid playbook for production.
Read guide →A practical playbook for adversarial AI deliberation — stress-test political messaging, product decisions, research claims, and legal arguments before they ship.
Read guide →Plain-English definitions for M-MAD, CoT-SC, Degeneration of Thought, sycophancy, and every term used across the research.
Browse terms →Real-time hallucination detection — every claim cross-checked against evidence, the web, and prior turns.
Learn how it works →Five built-in debate engines and a custom protocol library you can fork and save. Each mode is peer-reviewed or production-hardened — from 10-phase M-MAD truth-seeking to blind pairwise ping-pong.
Independent openings and rebuttals, then a five-dimension arbiter sweep — correctness, evidence, responsiveness, calibration, citations — before a binding verdict.
Rolling brainstorm where 2–100 agents reason in public. Models nominate the next speaker; human interventions steer mid-run.
Two saved Teams of 2–6 Workers each. Private huddles, public spokesperson turns — battle mode for adversarial critique, collaboration mode for joint synthesis.
Two participants alternate in a blind, human-style chat. Identities masked until external stop — ideal for unbiased pairwise reasoning.
Round-robin debate on a single structured question. Score-based decisions across rounds with anti-conformity prompts built in.
Fork any built-in engine, edit prompt sections, and save named variants. Your protocol — without forking the codebase.
Attach plain-text evidence to truth-seeking runs. Claims are tracked across phases so the arbiter scores against what was actually cited — not what agents wish they had said.
Compose participants from a reusable Worker library. Snapshotted into the conversation so editing a Worker never rewrites historical transcripts.
Five independent arbiter passes feed a structured verdict the model cannot retroactively rewrite — each dimension scored before the final call.
Bounded prompts that scale to long-running runs. Full transcripts stay persisted, but each turn is fed a compact, faithful context.
Hidden child runs iteratively refine the parent prompt. Stops on convergence, score threshold, cost cap, or generation limit.
Sweep temperature, repetition, frequency, and presence penalties across hidden child copies. Score transcripts against a validation prompt and expected outcome.
Inject guidance mid-run, optionally targeting the next responding agent. Interventions are first-class transcript inputs.
OpenRouter, LM Studio, and deterministic dummy providers. Configure per-agent fallbacks with server-side prompt caching.
Hard ceilings on every dimension that matters. Sessions self-terminate before they burn budget or time.
Supabase-backed job claims, runtime locks, and transactional mutations. One failed conversation never stalls the others.
Reusable prompt guidance, hidden discussion rules, and snapshotted 2–6 Worker teams. Configure once, run forever.
Get pinged when a run finishes or hits its cost cap. Resend integration when configured, in-app fallback when not.
Full REST API and a first-class Model Context Protocol server. Spin up sessions, inject human turns, stream transcripts, and run entire experiments — programmatically, from anywhere.
Every UI action is mirrored as a documented endpoint with stable contracts, idempotency keys, and webhook callbacks for long-running runs.
Drop MAD Studio into Claude Desktop, Cursor, or any MCP-compatible client. Orchestrate full debates as a callable tool — your agent of agents.
{
"mcpServers": {
"mad-studio": {
"command": "npx",
"args": ["-y", "@mad-studio/mcp"]
}
}
}Saga and Experiments run hidden recursive debates until the agents stop surprising each other. Surface insights no model — not GPT 5.5, not Opus 4.7 — would give you from a single shot.
Saga spawns zero-turn child sessions, scores each transcript against your rubric, applies the best optimizer suggestion, and re-runs. It stops only when the score curve flattens — or when the answer changes everything you thought you knew.
Experiments fan out hidden child runs across temperature, repetition, frequency, and presence sweeps. Score every transcript against a validation prompt. Promote the configuration that beats the field. The answer you would have hand-written, rewritten by the search.
Beyond what one prompt can reach. Both Saga and Experiments surface answers no human or single model would have arrived at unaided.
The MAD Studio Evaluation Matrix scores every agent, turn, and run across twelve rigorous dimensions — from logical consistency to steelman quality to calibration. The single number you ship is the single number you can prove.
| Agent | Logical | Evidence | Rebuttal | Steelman | Calibration | Novelty | Signal-to-noise | Citation | Position | Concession | Falsifiability | Decision-readiness |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Debater A | 92 | 78 | 88 | 71 | 84 | 66 | 81 | 90 | 58 | 73 | 87 | 79 |
| Debater B | 85 | 91 | 76 | 82 | 70 | 88 | 74 | 83 | 65 | 80 | 72 | 86 |
| Arbiter | 94 | 89 | 95 | 90 | 93 | 71 | 92 | 96 | 84 | 91 | 88 | 95 |
Every factual claim is extracted, cross-checked against your evidence pack, the public web, and the agent's own earlier turns. Hallucinated citations, drifted positions, false precision, and unsupported assertions all push the needle. Bullshit gets flagged before it becomes a quote.
Define your own dimensions, weights, and scoring guidance. The arbiter pipeline picks them up without re-architecting the protocol.
Pairwise debate outcomes update per-agent, per-topic ELO ratings. Find which Worker is actually best at adversarial law versus product strategy.
Compare two runs side by side. Spot the dimension that shifted when you tweaked the prompt. Export to CSV, JSON, or paste straight into a report.
Every score is traceable to the turn that earned it. Click any cell to jump to the supporting transcript span and the arbiter rationale.
MAD Studio is the only platform on the web that delivers peer-reviewed multi-agent debate as a daily-driver tool. Where AutoGen, CrewAI, and LangGraph give you a graph of agents, MAD Studio gives you a verdict — built on M-MAD scoring and the Bullshit Index. From war rooms to lab notebooks to weekend curiosity — the same engine, the same rigor.
Stage opposition agents that hammer your platform with the strongest rebuttals from across the spectrum. Find weak claims before journalists do.
Run hypotheses through a panel of skeptical agents with persona-specific priors. Capture structured rebuttals, citations needed, and open questions.
Run two competing campaign angles as Teams. Battle mode surfaces critique; collaboration mode synthesizes the best of both into a single brief.
Configure prosecutor and defense agents over a shared evidence pack. Get a structured claims ledger and a five-dimension verdict scorecard.
Pressure-test roadmap decisions, pricing models, and launch plans against agents seeded with competitor personas, customer archetypes, and risk lenses.
Make critical thinking visible. Students follow turn-by-turn argument structure, evidence handling, and the arbiter's dimension-level rationale.
Before publication, simulate the most aggressive defense your subject could mount. Identify the holes that will get raised — and patch them.
Pick a spicy topic. Pick six agents. Hit start. Watch them go. Export the transcript. No wrong answers — that's what the arbiter is for.
Quick answers to the questions teams ask before they run their first MAD Studio session.
Multi-agent debate is an AI reasoning technique where multiple language models (or multiple instances of the same model with different roles) argue about a question across structured rounds. Peer-reviewed research consistently shows debate produces more factual, better-calibrated answers than single-prompt baselines, especially on hard reasoning and evaluation tasks.
A single prompt gives you one model's first-pass answer. MAD Studio runs 2–100 reasoning agents through five built-in protocol engines — Truth-Seeking Debate (10-phase M-MAD), Open Discussion, Team Discussion (battle/collaboration), Blind Ping Pong, Scored Debate (FREE-MAD) — plus a custom Protocol Library you can fork and save. Claims get rebutted, evidence gets weighed, and verdicts come with auditable per-dimension scorecards.
MAD Studio supports any model on OpenRouter (including GPT 5.5, Claude Opus 4.7, Gemini, Llama, Mixtral, and dozens more), local models served by LM Studio, and deterministic dummy providers for testing. You can mix providers per agent and configure automatic fallbacks.
Yes. MAD Studio is built on peer-reviewed research from MIT, Google Brain, Anthropic, Tencent AI Lab, and others. Every protocol traces back to published methodology. Key papers:
We publish free, in-depth guides on multi-agent debate methodology — no signup required. Start here:
Political campaigns stress-test messaging against simulated opposition. Researchers run hypotheses through skeptical peer-review panels. Marketers debate competing campaign angles. Lawyers map adversarial arguments. Product teams institutionalize the devil's advocate. Educators make critical thinking visible. Anyone can run debates for fun — pick a topic, pick six agents, hit start.
The Bullshit Index is MAD Studio's real-time fact-checking layer. Every claim made by an agent is extracted, cross-referenced against your evidence pack, the public web, and the agent's earlier turns. Hallucinated citations, drifted positions, false precision, and contradicted statements all push the meter up. It's hallucination detection built directly into the debate loop.
Yes. MAD Studio offers a full REST API and a native Model Context Protocol (MCP) server. Spin up sessions, inject human turns, stream transcripts, and run experiments programmatically. The MCP server drops directly into Claude Desktop, Cursor, and any MCP-compatible client.
Saga is MAD Studio's recursive optimization engine. It spawns hidden child sessions from a source conversation, scores each transcript against your rubric, applies the best optimizer suggestion, and re-runs — generation after generation — until the score curve flattens or a stop condition fires. It's how you find answers that no single prompt would have produced.
MAD Studio is purpose-built for multi-agent debate specifically — verdict-grade protocols, the M-MAD arbiter pipeline, and the Bullshit Index — rather than general-purpose agent orchestration. If you need a graph of role-specialized agents executing arbitrary tasks, AutoGen, CrewAI, and LangGraph are excellent. If you need auditable structured disagreement with per-dimension scoring, MAD Studio is the right tool.
Token cost scales with agents × rounds, so a 6-agent, 5-round Truth-Seeking Debate is roughly 30× a single-prompt baseline before arbiter passes. MAD Studio mitigates this with rolling summaries, sparse communication topology (Li et al., EMNLP 2024), adaptive stopping (Hu et al., NeurIPS 2025), and hard cost caps that self-terminate sessions before they burn budget.
It depends on the task. For discrete math and multiple-choice with one correct answer, Self-Consistency (CoT-SC) is usually the better default. For factuality, open-ended strategy, and adversarial red-teaming, multi-agent debate consistently wins in peer-reviewed benchmarks (Du et al. ICML 2024, Khan et al. ICML 2024). MAD Studio supports both paradigms and a hybrid mode that combines them.
Yes. Every protocol — Open Discussion, Truth-Seeking Debate, Team Discussion, Saga, Lab Experiments — is configurable from the web UI. The REST API and MCP server are there if you want to drive debates programmatically from your own stack, but they are optional. The platform ships with reusable Personas, Playbooks, and Teams so a typical first session takes under two minutes to configure.
No. MAD Studio sends prompts to whichever provider you configure (OpenRouter, LM Studio, or your own endpoint) — we do not retrain models on your transcripts and do not share session data with third parties. Local-only deployments via LM Studio are fully private. Transcripts are stored in your Supabase workspace and you can purge them at any time.
Degeneration of Thought, formalized by Liang et al. (EMNLP 2024), is the failure mode where an LLM commits to an answer and then cannot produce genuinely novel reasoning during self-reflection — even when wrong. The critic and advocate inside one model share the same latent commitment. Multi-agent debate fixes this by separating roles into agents with distinct context.
Closed beta is rolling out now. Join 139 teams already on the waitlist — free early access, no card, no commitment, no spam.
By joining, you agree to receive infrequent product updates. Unsubscribe anytime. Questions? mad@multiagentdebates.com