benchmarks
Load PhySO's shipped Feynman and Class benchmark problems, generate bounded samples, inspect symbolic metadata, and compare candidate SymPy expressions.
- License 许可
- MIT
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- benchmarks
- license
- MIT
- metadata
- { "disco-role": "operating" }
- read_when
- Use when a task needs benchmark-problem data or symbolic-equivalence checks; do not use it to launch the maintainer benchmark suites.
- description
- Load PhySO's shipped Feynman and Class benchmark problems, generate bounded samples, inspect symbolic metadata, and compare candidate SymPy expressions.
- disable-model-invocation
- true
02 资源文件 4
- 参考文档references/problem-loaders.md5.8 KB
- 参考文档references/symbolic-equivalence.md5.7 KB
- 参考文档references/troubleshooting.md6.3 KB
- 脚本scripts/smoke_benchmark_loaders.py2.9 KB
03 SKILL.md 正文
PhySO benchmark problems
Role
This sub-skill is the bounded, read-only benchmark-data route for the installed
physo package. It teaches a Researcher to select a Feynman or Class problem,
inspect its target and metadata, generate a small in-memory sample, obtain the
prefix form, and apply the package's symbolic comparison convention.
It is not a general benchmarking or model-training skill. Keep generated arrays small unless the caller explicitly owns a separate experiment workflow.
Boundaries
Included
FeynmanProblemfor the 120 Feynman entries (bulk indices 0--99 and bonus indices 100--119), including equation-name lookup.ClassProblemfor the 8 Class entries, including realization-specificKconstants and equation-name lookup.- Formula, variable, target, range, unit, SymPy-assumption, and prefix metadata.
generate_data_points,target_function,get_sympy, and non-blockingshow_sampleusage.FeynmanProblem.compare_expressionandphyso.benchmark.utils.symbolic_utils.compare_expression.- The small loader check in
scripts/smoke_benchmark_loaders.py.
Excluded
- Running SR or Class SR (
physo.SRandphyso.ClassSR); route those requests tosrorclass-sr. - Full benchmark reproduction, noisy benchmark campaigns, result analysis, cluster/HPC jobfiles, and long-running runners.
- Generic data-science benchmark design or numerical-only model scoring.
- Pareto-front and training-log handling; route those questions to
toolkitor the root skill.
The maintainer scripts benchmarking/FeynmanBenchmark/feynman_run.py and
benchmarking/ClassBenchmark/classbench_run.py are reference-only. The
benchmarking/*_make_run_file.py files and benchmarking/utils.py jobfile
helper are deliberately outside this runtime skill.
Verified entry points
The following imports and package facts are verified for PhySO 1.2.0:
import physo.benchmark.FeynmanDataset.FeynmanProblem as Feyn
import physo.benchmark.ClassDataset.ClassProblem as Cls
from physo.benchmark.utils import symbolic_utils as su
feyn_pb = Feyn.FeynmanProblem(i_eq=0)
class_pb = Cls.ClassProblem(i_eq=0)
The verified inspection/runtime baseline is CPU-only Python 3.12 with NumPy, SymPy, pandas, Matplotlib, scikit-learn, and a CPU PyTorch build. No CUDA claim is made here. Import-time warnings about unavailable system LaTeX are expected and do not by themselves invalidate loading or comparison.
Workflow
- Validate the selector before constructing the object. Use an integer in
0 <= i_eq < Feyn.N_EQS(120) or0 <= i_eq < Cls.N_EQS(8), or use the exacteq_namefrom the dataset. A name can be inspected withFeyn.EQS_FEYNMAN_DF["Name"]orCls.EQS_CLASS_DF["Name"]. If both selector arguments are supplied, the current constructors take the index branch. - Load and inspect metadata. Start with
print(pb), then inspectformula_sympy,formula_sympy_eval,formula_latex,X_names,X_lows,X_highs,X_units,y_name, andy_units. Class problems also exposeK_names,K_lows,K_highs, andK_units;Kmeans one row of realization-specific constants, not a Class SR class constant. - Generate a bounded sample. Set
np.random.seed(...)when reproducible values matter and pass an explicit smalln_samples. Feynman returnsX.shape == (pb.n_vars, n_samples)andy.shape == (n_samples,). Class returnsmulti_X.shape == (n_realizations, pb.n_vars, n_samples)andmulti_y.shape == (n_realizations, n_samples); withreturn_K=True, it also returnsK.shape == (n_realizations, pb.n_spe). - Check the target directly. For Feynman use
pb.target_function(X). For a Class realization usepb.target_function(multi_X[i], K[i]). Compare shapes before callingnp.allclose; do not transpose the package's(variables, samples)convention. - Inspect the expression representation.
pb.get_prefix_expression()returns a dictionary withtokens_str,arities, andtokens, traversed in SymPy pre-order. For a Class problem, the prefix form still contains symbolick*tokens. Usepb.get_sympy(K_vals=...)to substitute one or more concreteKrows. - Compare symbolically. Parse a candidate with the problem's assumption-
carrying symbol dictionary, then call the problem wrapper for Feynman or
su.compare_expressionfor Class. Follow the exact examples in symbolic-equivalence.md, and retain the returnedreportinstead of relying only on the Boolean. - Stop at inspection. If the request asks for all equations, noise sweeps, SR training, job submission, or result reproduction, hand off to the appropriate root/SR/Class SR workflow and keep this route as the loader and equivalence utility only.
Input/output contracts
| Operation | Inputs | Outputs and invariants |
|---|---|---|
FeynmanProblem(i_eq=...) or eq_name=... |
Valid index or exact name | One problem; X is variables-first; target y is one-dimensional. |
ClassProblem(i_eq=...) or eq_name=... |
Valid index or exact name | One problem; K has one row per realization and one column per K_names. |
generate_data_points |
Small positive n_samples; Class also n_realizations |
NumPy float arrays sampled within declared input and constant ranges. |
target_function |
Feynman X, or Class X plus one K row |
Target values for the supplied shape; it does not validate a transposed data layout for you. |
get_prefix_expression |
No arguments | Dict keys tokens_str, arities, tokens with equal lengths. |
compare_expression |
SymPy expressions with matching assumption-aware symbols | (is_equivalent: bool, report: dict); report includes symbolic error/fraction and solution flags. |
Safe helper and native checks
Run the bundled helper from an environment where physo is installed:
python skills/disco/physo/sub-skills/benchmarks/scripts/smoke_benchmark_loaders.py
It loads Feynman problem 0 and Class problem 0, generates only a few points, checks target evaluation and array shapes, and prints stable metadata. It never runs a benchmark trainer or creates result directories.
The corresponding native candidates are:
python -m unittest physo.benchmark.FeynmanDataset.tests.FeynmanProblem_UnitTest -qpython -m unittest physo.benchmark.ClassDataset.tests.ClassProblem_UnitTest -q
These tests iterate over all shipped equations with small samples; they are CPU-safe correctness checks, not full SR benchmark reproduction.
Troubleshooting
See problem-loaders.md for selector, shape, metadata, and plotting details; symbolic-equivalence.md for assumptions, rounding, and trigonometric comparison; and troubleshooting.md for recovery and the maintainer-runner boundary.
Common first responses:
- Normalize an index before construction. Positive out-of-range and unknown
names currently surface as pandas
IndexError; negative.ilocindices can accidentally select the last row, so reject them in caller code. - Use the exact shape contracts above. Feynman is
(n_vars, n_samples), not(n_samples, n_vars); Class adds a realization axis. - Parse candidates with
pb.sympy_X_symbols_dict(or the documented original- name dictionary), not fresh symbols with different assumptions. - Treat
compare_expressionas the benchmark's symbolic convention, not as a numerical error metric. Inspect its report, especially when rounding or trigonometric handling is enabled. - Use
show_sample(..., do_show=False)in headless work. Missing Matplotlib is an installation problem because these benchmark modules import it; missing system LaTeX only affects optional display and is normally warning-level.
Related routes
- Root routing:
../../SKILL.md - Single-dataset SR:
../sr/SKILL.md - Class SR data-shape and training guidance:
../class-sr/SKILL.md - Low-level expression and result inspection:
../toolkit/SKILL.md
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).