multi-agent-debate
暂无简介
- License 许可
- Apache-2.0
- 最近更新
- 2026-11-14
- Stars
- ★ 13
- Forks
- 2
03 SKILL.md 正文
/multi-agent-debate
When to Use
- High-stakes reasoning where a single LLM gives confidently wrong answers
- Reducing correlated errors: one model family has systematic bias in a domain
- Factual tasks where self-consistency sampling has plateaued (debate breaks the saturation)
- Research synthesis, code review, or multi-perspective analysis
Do NOT use for
- Tasks with a single deterministic answer verifiable by tool (use CRITIC in [[self-refine-critic]])
- Latency-sensitive workloads (N × R LLM calls = N×R cost)
- Tasks where any model in the pool is clearly much weaker than others (weak models drag consensus)
Du et al. 2023 algorithm (ICML 2024)
Round 0 (initial): each agent i produces answer_i independently
Round r (debate): each agent i sees all agents' round r-1 answers
→ produces updated answer_i conditioned on the group
Final: majority-vote across agents' final answers
Two independent contributions to quality (from paper ablations):
- Multiple agents (1 round, majority vote): beats single-agent; plateaus
- Multiple rounds (1 agent, self-reflection): barely helps alone
- Both together: produces the big accuracy jumps
Debate loop
from typing import Callable
def debate(
question: str,
agents: list[Callable[[str, list[str] | None], str]],
rounds: int = 3,
) -> str:
"""
agents: list of callables (question, prior_answers) -> answer
Returns majority-voted final answer.
"""
# Round 0: independent proposals
answers: list[str] = [agent(question, None) for agent in agents]
# Rounds 1..R: each agent sees all others' prior answers
for r in range(1, rounds):
new_answers: list[str] = []
for i, agent in enumerate(agents):
others = [answers[j] for j in range(len(agents)) if j != i]
updated = agent(question, others)
new_answers.append(updated)
answers = new_answers
return majority_vote(answers)
def majority_vote(answers: list[str]) -> str:
from collections import Counter
# Normalize: extract final "Answer: X" from each response
import re
extracted = []
for a in answers:
m = re.search(r'(?:answer|final answer)[:\s]+(.+?)(?:\.|$)', a, re.IGNORECASE)
extracted.append(m.group(1).strip() if m else a.strip())
counts = Counter(extracted)
return counts.most_common(1)[0][0]
Agent prompt with debate context
def build_debate_prompt(question: str, prior_answers: list[str] | None) -> str:
if not prior_answers:
return f"""Answer the following question. Show your reasoning.
Question: {question}
Answer:"""
others_block = "\n\n".join(
f"Agent {i+1}'s answer:\n{ans}"
for i, ans in enumerate(prior_answers)
)
return f"""Answer the following question. You have seen other agents' answers below.
Consider their reasoning critically. You may update your answer or maintain it with justification.
Question: {question}
Other agents' answers:
{others_block}
Your updated answer (with reasoning):"""
Sycophancy cascade prevention
def build_adversarial_debate_prompt(question: str, prior_answers: list[str]) -> str:
"""Force at least one agent to argue the counter-position."""
others_block = "\n\n".join(f"Agent {i+1}: {a}" for i, a in enumerate(prior_answers))
return f"""Question: {question}
Other agents have proposed the following answers:
{others_block}
Your role: identify the weakest point in the most popular answer and argue for an alternative.
Be specific. If you ultimately agree after analysis, say so and explain why.
Counter-analysis:"""
# Pattern: rotate the adversarial role across agents each round
def assign_roles(n_agents: int, round_num: int) -> list[str]:
roles = ["standard"] * n_agents
adversarial_idx = round_num % n_agents
roles[adversarial_idx] = "adversarial"
return roles
Compute budget guide
N agents × R rounds = N×R LLM calls, each with growing context
5 agents × 5 rounds = 25 calls at increasing context size
Cost per question can exceed 10× single CoT call
Practical recommendations:
Low-stakes tasks: 3 agents × 2 rounds (6 calls) — diminishing returns past round 2
High-stakes tasks: 5 agents × 3 rounds (15 calls)
Research synthesis: 3 heterogeneous models × 3 rounds
Heterogeneous = different model families (Claude + GPT + Llama)
→ errors don't correlate → more diverse starting points → better convergence
Topic drift mitigation
def inject_question_every_round(question: str, prior_answers: list[str]) -> str:
"""Re-anchor agents to the original question every round to prevent drift."""
return f"""IMPORTANT: Keep your answer focused on the original question:
"{question}"
Do not let discussion drift to adjacent topics.
Other agents' latest answers:
{chr(10).join(f'- {a[:300]}' for a in prior_answers)}
Your focused answer to the original question:"""
Anti-Fake-Pass Checklist
❌ All agents from same model family → monoculture collapse; errors correlate; debate is just sampling
❌ No adversarial role → sycophancy cascade; all agents defer to most confident answer
❌ Round count > 3 without task-specific tuning → diminishing returns + context explosion
❌ Topic drift across rounds → re-inject the original question every round
❌ Weak model in the pool → drags consensus toward its wrong answer (Du et al. "MAD" follow-up)
❌ No majority vote normalization → agents phrase the same answer differently; vote splits
04 同仓库更多 Skills
openai--sentry--sentry
Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.
ito-basket-compare
Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.
implementing-network-traffic-baselining
Build network traffic baselines from NetFlow/IPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly/daily traffic pattern profiling
detecting-evasion-techniques-in-endpoint-logs
Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.
implementing-microsegmentation-with-guardicore
Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.
ramdisk-tmpfs-patterns
暂无简介