anyscale-guide
Scale Ray workloads and serve models on Anyscale — managed Ray clusters for training and inference.
- License 许可
- MIT
- 最近更新
- 2026-11-07
- Stars
- ★ 0
- Forks
- 1
01 前言 Front Matter
- name
- anyscale-guide
- category
- ai-research
- description
- Scale Ray workloads and serve models on Anyscale — managed Ray clusters for training and inference.
03 SKILL.md 正文
Overview
Anyscale is the managed platform for Ray, the distributed computing framework — it runs your Ray workloads (training, batch processing, hyperparameter tuning, model serving with Ray Serve) on managed clusters without you operating the infrastructure. For AI teams, the practical shape is: develop Ray code locally, then run it at scale on Anyscale with managed cluster lifecycle, observability, and cost controls.
The distinctive value is Ray itself: a unified programming model for distributed Python that covers training, serving, and data processing in one framework. If your ML work spans "train a model," "tune hyperparameters," "run batch inference," and "serve the model," Ray does all four — and Anyscale removes the cluster-management burden. It's infrastructure for teams whose workloads are genuinely distributed, not just "a GPU in the cloud."
Anyscale's lens: if you're reaching for Ray, use the managed platform; if you're not using Ray, Anyscale probably isn't your starting point.
When to use
- Distributed training jobs that need multi-node/multi-GPU orchestration.
- Hyperparameter sweeps at scale (Ray Tune) without managing the cluster.
- Model serving with Ray Serve: composable deployments, multi-model pipelines.
- Batch inference over large datasets with distributed processing.
- Reinforcement learning workloads (Ray RLlib) at scale.
- Teams standardizing on Ray as their distributed compute layer.
Core concepts
- Ray core primitives: tasks (stateless functions), actors (stateful workers), and objects (distributed shared memory). Learn these three — every Ray program composes them.
- Managed clusters: Anyscale provisions and manages the cluster lifecycle (autoscaling node pools, spot instances, upgrades). You declare compute config; the platform handles the rest.
- Ray Tune: distributed hyperparameter optimization with schedulers (ASHA/PBT) that kill bad trials early. The cost savings from early stopping usually dwarf the platform cost.
- Ray Serve: model serving with deployment graphs — compose models, preprocessors, and business logic into one served application with autoscaling per deployment.
- Workspaces: interactive development environments on the cluster — develop against real distributed resources, not just your laptop.
- Jobs: the production unit — submit a Ray job (training run, batch pipeline) with pinned code, dependencies, and compute config. Jobs are auditable and reproducible.
- Autoscaling and spot: clusters scale node pools with demand; spot instances cut cost for fault-tolerant workloads. Configure per workload — serving on spot needs care.
- Observability: Ray dashboard plus Anyscale's monitoring — task timelines, actor states, GPU utilization, logs. Distributed debugging lives here.
Practical workflow
- Develop the Ray program locally. Write tasks/actors/serve deployments against a local Ray cluster. Get the logic right at small scale first.
- Define compute config. Node types, autoscaling bounds, GPU requirements per workload. Start modest; scale based on measured utilization.
- Run as a Job for anything repeatable. Notebooks and workspaces for exploration; Jobs for training runs and pipelines — reproducible, logged, reviewable.
- Tune with early stopping. For hyperparameter search, use ASHA or PBT schedulers — killing bad trials early is the highest-ROI optimization in distributed tuning.
- Serve with Ray Serve graphs. Compose the inference pipeline (preprocess → model → postprocess) as a deployment graph with per-deployment scaling. Load-test the graph, not just the model.
- Use spot where fault-tolerant. Training with checkpointing and batch jobs tolerate preemption — spot instances cut those costs substantially. Keep serving and stateful work on on-demand.
- Monitor and right-size. Watch GPU utilization, autoscaling behavior, and cost per job. Distributed systems drift toward waste — review regularly.
Checklist for Anyscale production work:
- Jobs pinned (code, deps, compute config) and reproducible.
- Autoscaling bounds set per workload; spot policy deliberate.
- Tuning uses early-stopping schedulers.
- Serve graphs load-tested end-to-end.
- GPU utilization and cost-per-job monitored.
Common pitfalls
- Ray for non-distributed work. A single-GPU training job doesn't need Ray's machinery. Use Ray when the workload is genuinely distributed.
- No checkpointing with spot. Spot preemptions without checkpoints waste the entire run. Checkpoint frequently; make training resumable.
- Autoscaling misconfiguration. Bounds too tight (jobs queue forever) or too loose (idle GPUs burning money). Tune from utilization data.
- Tuning without early stopping. Full-fidelity training of every hyperparameter combination. ASHA/PBT exist precisely to avoid this.
- Serving the model, not the graph. Load-testing the bare model while production runs a preprocess→model→postprocess graph with different scaling needs per stage.
- Workspace-as-production. Running production workloads from interactive workspaces instead of Jobs. Workspaces are for development; Jobs are for repeatability.
- Ignoring data locality. Distributed compute with centralized data access bottlenecks on I/O. Co-locate data access with compute; use Ray Data thoughtfully.
- Cost blindness. Multi-node GPU clusters without per-job cost tracking. Distributed waste scales as fast as distributed compute.
04 同仓库更多 Skills
web-scrape
Turn web pages into LLM-ready markdown via the Firecrawl API (renders JS, anti-bot handling). Use when browser_open/fetch fails on JS-heavy pages, or when you need clean markdown/structured extraction from a site. Requires FIRECRAWL_API_KEY.
design-taste-frontend
tasteskill v2, the anti-slop frontend skill. Art direction for websites, distilled from a study of 247 curated gallery sites (One Page Love, Awwwards Sites of the Day). Deals researched directions (idea, composition, type, palette, photographs, section blueprints) from a library, enforces a strict no-slop house style with a real-browser inspector, and runs a look-and-fix loop on screenshots. Use this whenever you build, design, redesign, restyle, polish or review a landing page, marketing site, portfolio, product page, one-pager, hero or any public-facing web page, in any stack (HTML/CSS, React, Next.js, Vue, Svelte, Astro, Tailwind) — also when the user says a page looks generic, templated, "AI-made", sloppy, or asks to make it look designed, premium, award-level or less like every other site. Not for dashboards or app screens.
test-driven-development
Use when implementing any feature or bugfix, before writing implementation code
systematic-debugging
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes
requesting-code-review
Use when completing tasks, implementing major features, or before merging to verify work meets requirements
brainstorming
You MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).