AGENT SKILL

simulation

Design Monte Carlo simulations for theoretical validation: sample size determination, convergence diagnostics, and result presentation. Specialized for statistical methodology papers.

License 许可
MIT
最近更新
2026-10-23
Stars
★ 0
Forks
0

01 前言 Front Matter

name
simulation
description
Design Monte Carlo simulations for theoretical validation: sample size determination, convergence diagnostics, and result presentation. Specialized for statistical methodology papers.

03 SKILL.md 正文

Monte Carlo Simulation Design

Help the researcher design rigorous Monte Carlo simulations to validate theoretical results. Simulations bridge the gap between theory and practice: they demonstrate that analytical formulas work as predicted and reveal finite-sample behavior.

Step 1: Read Context

Read .papermill/state.md (Read tool) for:

  • Thesis: What theoretical result needs validation.
  • Experiments: Any existing simulation experiments.

If .papermill/state.md does not exist, ask the user which theoretical result needs validation. Simulation design can proceed without the state file: suggest running /papermill:init afterward.

Scan the repository for existing simulation code and results (Glob/Read tools).

Step 2: Identify What to Validate

Ask: "Which theoretical result are you validating with this simulation?"

Common simulation targets in methodology papers:

  • Asymptotic distributions: Does the CLT approximation hold at finite n?
  • Estimator properties: Is the MLE unbiased? What is its MSE?
  • Covariance/information matrices: Does the empirical covariance match the theoretical Fisher information?
  • Confidence interval coverage: Do 95% CIs actually cover 95% of the time?
  • Power analysis: Can the test detect a given effect size?
  • Algorithm convergence: Does the optimizer find the right answer?

Step 3: Design the Simulation

Parameters

  • True parameter values: Choose several configurations spanning easy, moderate, and hard cases.
  • Sample sizes: Include small (n=20-50), moderate (n=100-500), and large (n=1000+) to show convergence.
  • Number of replications: Typically R=1,000-10,000. More replications reduce Monte Carlo error but cost time. The Monte Carlo standard error for a proportion p is sqrt(p(1-p)/R).

Data Generation

Specify exactly how to generate synthetic data:

  1. Set random seed for reproducibility.
  2. Generate from the known model with known true parameters.
  3. Apply the estimation/inference procedure to each replicate.
  4. Collect the quantities of interest.

Metrics

What to measure How to summarize
Bias Mean(estimate) - true value
Variance Var(estimates) across replicates
MSE Bias^2 + Variance
Coverage Fraction of CIs containing true value
Convergence rate Plot metric vs. n on log scale

Memory and Compute

  • Batch processing: If R * n is large, process in batches to control memory. Accumulate sufficient statistics rather than storing all raw data.
  • Parallelization: Independent replicates are embarrassingly parallel. Use parallel (R), multiprocessing (Python), or OpenMP (C++).
  • Progress reporting: For long simulations, report progress every ~10% of replicates.

Step 4: Convergence Diagnostics

Before presenting results, verify the simulation itself is reliable:

  1. Monte Carlo standard errors: Report them alongside point estimates. If MCSE is large relative to the quantity of interest, increase R.
  2. Trace plots: Plot running averages to verify convergence.
  3. Sensitivity to seed: Run with 2-3 different seeds. Results should be stable.

Step 5: Result Presentation

Design the output tables and figures:

Tables

  • Rows: parameter configurations or sample sizes
  • Columns: bias, SD, MSE, coverage, or whatever metrics are relevant
  • Include Monte Carlo standard errors in parentheses

Figures

  • Convergence plots: Metric vs. n (log scale) with theoretical prediction overlaid
  • QQ plots: Compare empirical distribution of estimator to theoretical asymptotic distribution
  • Box plots: Distribution of estimates across replicates for each configuration

Step 6: Common Pitfalls

Warn about:

  • Insufficient replicates: R=100 is rarely enough. For coverage studies, you need R >= 1,000.
  • Ignoring Monte Carlo error: Always report MCSE. A coverage of 0.94 with MCSE=0.007 is consistent with 0.95.
  • Conditioning bias: If you filter out "non-convergent" replicates, the remaining sample is biased. Report the rejection rate.
  • Single parameter configuration: Results at one parameter value do not generalize. Test across a range.
  • Floating-point issues: For extreme parameter values, numerical issues can corrupt results. Include sanity checks.

Step 7: Update State File

If .papermill/state.md exists, register the simulation (Edit tool) under experiments. If it does not exist, skip registration and suggest running /papermill:init.

The entry uses the standard experiment schema with an optional config block for simulation-specific parameters:

experiments:
  - name: "simulation-name"
    type: "simulation"
    hypothesis: "Empirical covariance matches theoretical FIM as n grows"
    status: "planned"
    script: "research/simulate_covariance.R"
    last_run: null
    config:  # optional experiment field (see docs/state-schema.md)
      replications: 5000
      sample_sizes: [50, 100, 200, 500, 1000]
      parameter_configs: 3

Append a timestamped note documenting the simulation design.

Step 8: Suggest Next Steps

Based on the simulation status, suggest the most relevant next step:

  • Implement: "Write the simulation script. Start with a small pilot (R=100) to debug before the full run."
  • If the simulation validates a proof → "Once results confirm the theory, integrate both the proof and simulation evidence into the paper. Use /papermill:proof if the proof itself needs work."
  • If simulation results are surprising or contradict theory → "The simulation suggests the theoretical result may need revisiting. Use /papermill:proof to re-examine the proof's assumptions."
  • If results are written up → "Use /papermill:review to get feedback on the presentation of simulation results."

04 同仓库更多 Skills

Abdulqudduz
Abdulqudduz/claude-anvil

worldsmith-methodology

Use when working in a project with a worldbuilding documentation ecosystem (docs/ or lore/ directory). Provides the documentation-first methodology: canonical hierarchy, doc roles (timeline, lore, characters, systems), propagation discipline, and the dual canonical and exploratory workflow.

★ 0
Abdulqudduz
Abdulqudduz/claude-anvil

prose-craft

Use when writing fiction, editing fiction prose, reviewing chapters, or drafting scenes. Covers prose craft, dialogue mechanics, scene structure, and common failure modes in AI-assisted fiction. Trigger phrases: "write fiction", "edit chapter", "prose review", "scene draft", "dialogue check", "fiction craft", "edit this scene", "review my prose".

★ 0
Abdulqudduz
Abdulqudduz/claude-anvil

vista

This skill should be used when the user asks about research directions, open problems, future work, or follow-up research from the academic literature. Trigger phrases include "open problems in X", "what's next for Y", "future work for paper Z", "research directions on T", "salient follow-up research", "what should I work on next in", "find me follow-up research unrelated to my prior work", "broad survey of W", "neglected directions in V". Routes the request to the right Vista MCP tool, reads back the structured paper sections, and synthesizes research directions in the conversation.

★ 0
Abdulqudduz
Abdulqudduz/claude-anvil

synthesize

Make a research-agent run conclude and write synthesis.md (outcome, key findings, failed approaches, open questions, recommendations). Signals a live researcher to wrap up, or launches one in synthesis mode when none is running. Use when results are good enough or the agent is stalling.

★ 0
Abdulqudduz
Abdulqudduz/claude-anvil

status

Show the current state of a research-agent run from .research/: liveness, cycles and budget, sub-problem and hypothesis statuses, eval trend, current focus, and branches. Read-only.

★ 0
Abdulqudduz
Abdulqudduz/claude-anvil

resume

Resume an interrupted research-agent run from .research/. Re-launches the researcher to reload its structured state and continue from the current focus, optionally on a named branch or with a new budget. Use after a session restart, crash, or explicit pause.

★ 0