mlperf-inference-benchmarks
暂无简介
- License 许可
- Apache-2.0
- 最近更新
- 2026-11-14
- Stars
- ★ 13
- Forks
- 2
03 SKILL.md 正文
/mlperf-inference-benchmarks
When to Use
- Measure inference system performance with industry-standard reproducible metrics
- Compare GPU/CPU hardware choices for a given model/task
- Validate SLA targets: p99 latency < 100ms at 1000 QPS
- Pre-deployment stress test: flood the system to find breaking point
Do NOT use for
- Training benchmarks (use MLPerf Training)
- Quick sanity checks (use [[llm-eval-framework]])
MLPerf scenarios
SingleStream: one query at a time, measure latency distribution
Target: p90 latency (e.g., < 10ms for image classification)
Use: edge devices, latency-sensitive APIs
MultiStream: N queries in parallel, measure all-complete latency
Target: p99 latency across all N queries
Use: camera pipelines, fixed batch size hardware
Server: Poisson-distributed query arrival, measure sustained throughput
Target: max QPS while keeping p99 < SLA threshold
Use: production cloud inference
Offline: all queries available upfront, measure total throughput
Target: max queries per second (no latency constraint)
Use: batch processing, overnight jobs
Custom Loadgen harness (Python)
import mlperf_loadgen as lg
class YamtamSUT:
"""System Under Test — wraps your inference engine"""
def issue_queries(self, query_samples):
for qs in query_samples:
# Run inference
result = run_inference(qs.index)
# Report response
response = lg.QuerySampleResponse(
id = qs.id,
data = result.ctypes.data,
size = result.nbytes,
)
lg.QuerySamplesComplete([response])
def flush_queries(self): pass
class YamtamQSL:
"""Query Sample Library — dataset loader"""
def __init__(self, dataset, total_count, perf_count):
self.qsl = lg.ConstructQSL(
total_count, perf_count,
self.load_query_samples, self.unload_query_samples
)
self.dataset = dataset
def load_query_samples(self, sample_list):
for s in sample_list: self.dataset.load(s)
def unload_query_samples(self, sample_list):
for s in sample_list: self.dataset.unload(s)
# Run benchmark
settings = lg.TestSettings()
settings.scenario = lg.TestScenario.Server
settings.mode = lg.TestMode.PerformanceOnly
settings.server_target_qps = 500
sut = YamtamSUT()
qsl = YamtamQSL(dataset, total_count=5000, perf_count=1024)
lg.StartTest(sut, qsl.qsl, settings)
lg.DestroyQSL(qsl.qsl)
Key metrics to track
# After loadgen run, parse mlperf_log_summary.txt
grep -E "(Samples per second|Min latency|Max latency|Mean latency|p99.00 latency)" \
mlperf_log_summary.txt
# Target thresholds for yamtam tool calls:
# Mean latency: < 200ms (acceptable for interactive agent)
# p99 latency: < 500ms (hard SLA)
# Throughput: > 50 QPS (for 50 concurrent agents)
Quick server scenario stress test (without full MLPerf)
# Use wrk2 for HTTP inference endpoint load testing
wrk2 -t 4 -c 100 -d 60s -R 500 \
--script post.lua \
http://inference-server:8000/infer
# post.lua
wrk.method = "POST"
wrk.headers = {["Content-Type"] = "application/json"}
wrk.body = '{"input": "benchmark query"}'
Anti-Fake-Pass Checklist
❌ Using accuracy mode results for performance reporting → different sample count, not valid perf number
❌ Warmup not included → first queries hit cold cache; p99 inflated
❌ p50 latency meets SLA but p99 does not → tail latency the real constraint; always report p99
❌ QPS measured on a single sample → not representative; use Server scenario with realistic load
❌ Not isolating benchmark host → background processes inflate latency; dedicate hardware
❌ Comparing results across different MLPerf versions → LoadGen version affects timing methodology
04 同仓库更多 Skills
openai--sentry--sentry
Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.
ito-basket-compare
Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.
implementing-network-traffic-baselining
Build network traffic baselines from NetFlow/IPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly/daily traffic pattern profiling
detecting-evasion-techniques-in-endpoint-logs
Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.
implementing-microsegmentation-with-guardicore
Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.
ramdisk-tmpfs-patterns
暂无简介