simulation
Design Monte Carlo simulations for theoretical validation: sample size determination, convergence diagnostics, and result presentation. Specialized for statistical methodology papers.
- License 许可
- MIT
- 最近更新
- 2026-10-23
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- simulation
- description
- Design Monte Carlo simulations for theoretical validation: sample size determination, convergence diagnostics, and result presentation. Specialized for statistical methodology papers.
03 SKILL.md 正文
Monte Carlo Simulation Design
Help the researcher design rigorous Monte Carlo simulations to validate theoretical results. Simulations bridge the gap between theory and practice: they demonstrate that analytical formulas work as predicted and reveal finite-sample behavior.
Step 1: Read Context
Read .papermill/state.md (Read tool) for:
- Thesis: What theoretical result needs validation.
- Experiments: Any existing simulation experiments.
If .papermill/state.md does not exist, ask the user which theoretical result needs validation. Simulation design can proceed without the state file: suggest running /papermill:init afterward.
Scan the repository for existing simulation code and results (Glob/Read tools).
Step 2: Identify What to Validate
Ask: "Which theoretical result are you validating with this simulation?"
Common simulation targets in methodology papers:
- Asymptotic distributions: Does the CLT approximation hold at finite n?
- Estimator properties: Is the MLE unbiased? What is its MSE?
- Covariance/information matrices: Does the empirical covariance match the theoretical Fisher information?
- Confidence interval coverage: Do 95% CIs actually cover 95% of the time?
- Power analysis: Can the test detect a given effect size?
- Algorithm convergence: Does the optimizer find the right answer?
Step 3: Design the Simulation
Parameters
- True parameter values: Choose several configurations spanning easy, moderate, and hard cases.
- Sample sizes: Include small (n=20-50), moderate (n=100-500), and large (n=1000+) to show convergence.
- Number of replications: Typically R=1,000-10,000. More replications reduce Monte Carlo error but cost time. The Monte Carlo standard error for a proportion p is sqrt(p(1-p)/R).
Data Generation
Specify exactly how to generate synthetic data:
- Set random seed for reproducibility.
- Generate from the known model with known true parameters.
- Apply the estimation/inference procedure to each replicate.
- Collect the quantities of interest.
Metrics
| What to measure | How to summarize |
|---|---|
| Bias | Mean(estimate) - true value |
| Variance | Var(estimates) across replicates |
| MSE | Bias^2 + Variance |
| Coverage | Fraction of CIs containing true value |
| Convergence rate | Plot metric vs. n on log scale |
Memory and Compute
- Batch processing: If R * n is large, process in batches to control memory. Accumulate sufficient statistics rather than storing all raw data.
- Parallelization: Independent replicates are embarrassingly parallel. Use
parallel(R),multiprocessing(Python), or OpenMP (C++). - Progress reporting: For long simulations, report progress every ~10% of replicates.
Step 4: Convergence Diagnostics
Before presenting results, verify the simulation itself is reliable:
- Monte Carlo standard errors: Report them alongside point estimates. If MCSE is large relative to the quantity of interest, increase R.
- Trace plots: Plot running averages to verify convergence.
- Sensitivity to seed: Run with 2-3 different seeds. Results should be stable.
Step 5: Result Presentation
Design the output tables and figures:
Tables
- Rows: parameter configurations or sample sizes
- Columns: bias, SD, MSE, coverage, or whatever metrics are relevant
- Include Monte Carlo standard errors in parentheses
Figures
- Convergence plots: Metric vs. n (log scale) with theoretical prediction overlaid
- QQ plots: Compare empirical distribution of estimator to theoretical asymptotic distribution
- Box plots: Distribution of estimates across replicates for each configuration
Step 6: Common Pitfalls
Warn about:
- Insufficient replicates: R=100 is rarely enough. For coverage studies, you need R >= 1,000.
- Ignoring Monte Carlo error: Always report MCSE. A coverage of 0.94 with MCSE=0.007 is consistent with 0.95.
- Conditioning bias: If you filter out "non-convergent" replicates, the remaining sample is biased. Report the rejection rate.
- Single parameter configuration: Results at one parameter value do not generalize. Test across a range.
- Floating-point issues: For extreme parameter values, numerical issues can corrupt results. Include sanity checks.
Step 7: Update State File
If .papermill/state.md exists, register the simulation (Edit tool) under experiments. If it does not exist, skip registration and suggest running /papermill:init.
The entry uses the standard experiment schema with an optional config block for simulation-specific parameters:
experiments:
- name: "simulation-name"
type: "simulation"
hypothesis: "Empirical covariance matches theoretical FIM as n grows"
status: "planned"
script: "research/simulate_covariance.R"
last_run: null
config: # optional experiment field (see docs/state-schema.md)
replications: 5000
sample_sizes: [50, 100, 200, 500, 1000]
parameter_configs: 3
Append a timestamped note documenting the simulation design.
Step 8: Suggest Next Steps
Based on the simulation status, suggest the most relevant next step:
- Implement: "Write the simulation script. Start with a small pilot (R=100) to debug before the full run."
- If the simulation validates a proof → "Once results confirm the theory, integrate both the proof and simulation evidence into the paper. Use
/papermill:proofif the proof itself needs work." - If simulation results are surprising or contradict theory → "The simulation suggests the theoretical result may need revisiting. Use
/papermill:proofto re-examine the proof's assumptions." - If results are written up → "Use
/papermill:reviewto get feedback on the presentation of simulation results."
04 同仓库更多 Skills
worldsmith-methodology
Use when working in a project with a worldbuilding documentation ecosystem (docs/ or lore/ directory). Provides the documentation-first methodology: canonical hierarchy, doc roles (timeline, lore, characters, systems), propagation discipline, and the dual canonical and exploratory workflow.
prose-craft
Use when writing fiction, editing fiction prose, reviewing chapters, or drafting scenes. Covers prose craft, dialogue mechanics, scene structure, and common failure modes in AI-assisted fiction. Trigger phrases: "write fiction", "edit chapter", "prose review", "scene draft", "dialogue check", "fiction craft", "edit this scene", "review my prose".
vista
This skill should be used when the user asks about research directions, open problems, future work, or follow-up research from the academic literature. Trigger phrases include "open problems in X", "what's next for Y", "future work for paper Z", "research directions on T", "salient follow-up research", "what should I work on next in", "find me follow-up research unrelated to my prior work", "broad survey of W", "neglected directions in V". Routes the request to the right Vista MCP tool, reads back the structured paper sections, and synthesizes research directions in the conversation.
synthesize
Make a research-agent run conclude and write synthesis.md (outcome, key findings, failed approaches, open questions, recommendations). Signals a live researcher to wrap up, or launches one in synthesis mode when none is running. Use when results are good enough or the agent is stalling.
status
Show the current state of a research-agent run from .research/: liveness, cycles and budget, sub-problem and hypothesis statuses, eval trend, current focus, and branches. Read-only.
resume
Resume an interrupted research-agent run from .research/. Re-launches the researcher to reload its structured state and continue from the current focus, optionally on a named branch or with a new budget. Use after a session restart, crash, or explicit pause.