ai-ml-architect
Entry point for AI/ML system architecture — choosing between prompting, retrieval, fine-tuning and training; deciding where the model boundary sits; costing and gating inference; and the evaluation discipline that must exist before any of it ships. Use when planning an LLM product, a RAG system, an agent, a model serving stack, or an ML platform.
- License 许可
- MIT
- 最近更新
- 2026-11-13
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- ai-ml-architect
- license
- MIT
- metadata
- { "crusor-path": "ai-ml-architect", "crusor-domain": "ai-ml-architect", "crusor-origin": "original", "crusor-sources": "platform.openai.com/docs; docs.anthropic.com; huggingface.co/docs; arxiv.org/abs/2005.11401", "crusor-triggers": "llm, rag, fine tune, embedding, prompt, agent, inference, model serving, mlops, eval, vector database, training", "crusor-verified-on": "2026-10-30" }
- description
- Entry point for AI/ML system architecture — choosing between prompting, retrieval, fine-tuning and training; deciding where the model boundary sits; costing and gating inference; and the evaluation discipline that must exist before any of it ships. Use when planning an LLM product, a RAG system, an agent, a model serving stack, or an ML platform.
- compatibility
- Any provider or self-hosted stack
03 SKILL.md 正文
AI/ML Architect
The model is a dependency with non-determinism, latency variance, a price per token, and a quality contract you must measure. Everything in this domain follows from treating it that way.
The shelf
Eight shelves, 30 books.
| Shelf | Books |
|---|---|
providers-architect/ |
openai · anthropic · gemini · open-weights |
llm-architect/ |
llm-app · prompting · structured-output · model-selection · token-economics |
retrieval-architect/ |
rag · chunking · embeddings · vector-store · hybrid-search |
evaluation-architect/ |
evals · judge-calibration · red-teaming |
training-architect/ |
fine-tuning · classic-ml · recommender · vision |
serving-architect/ |
inference-serving · gpu-capacity · quantization · speech |
mlops-architect/ |
mlops-platform · feature-store · model-monitoring |
agent-architect/ |
agent · multi-agent |
Read the crown, then the one or two books the plan actually needs. Do not read the whole shelf to avoid a decision.
When to use
- Any plan that puts a model in the request path of a product.
- Choosing between API, self-host, fine-tune, or classic ML.
- An existing AI feature that is expensive, slow, or unmeasurable.
When NOT to use
- Deterministic logic solves it: rules, SQL, a regex, a lookup table.
- The task is "add a chatbot" with no defined job to be done.
- Nobody can state what a correct answer looks like — that is an eval problem
first (
evals-architect), not an architecture problem.
Decision 1 — Is a model the right tool
| Signal | Better answer |
|---|---|
| exact, auditable rule | deterministic code |
| classification over known labels | classic ML or a small model |
| free-form text transformation | LLM |
| answer must cite a fixed corpus | retrieval, not memory |
| high-volume, narrow format | fine-tune or smaller model |
| truly open-ended reasoning | large model, with evals |
Non-determinism is a product decision: anything user-visible must have a stated tolerance for variance and a human path when confidence is low.
Decision 2 — Where the boundary sits
- Prompt-only — fastest to ship, no training, weakest on format stability.
- Retrieval — grounds answers in your data; adds a retrieval system to own.
- Fine-tune — buys format and tone, not knowledge; needs a dataset and an eval set, plus retraining as a lifecycle.
- Train from scratch — rarely justified outside a domain with proprietary data and volume.
- Escalate in that order. Each step adds a system you must operate.
Decision 3 — The envelope
Every design doc states, numerically:
- quality: eval score on the golden set, and the floor below which it is a regression;
- latency: p50/p95 end to end, including every model call in a chain;
- cost: per request, per user, per month at projected volume;
- failure: what the user sees when the provider is down or rate-limits.
A chain of five model calls multiplies all four. Count the hops.
Decision 4 — Data and privacy
- What leaves your boundary, to whom, under which retention terms.
- PII in prompts is a data-flow decision, not a prompt-engineering one.
- Tenant isolation: do not mix documents across tenants in one index.
- Logging prompts is logging user content — apply the same retention rules.
Decision 5 — Build order
- Define the job and the golden set.
- Baseline: best prompt on the largest model. Measure.
- Add retrieval or a smaller model only where the baseline fails.
- Add evals to CI before adding features.
- Optimise cost and latency against a measured baseline, never by guessing.
Failure modes at scale
- Shipping without an eval set: quality debates settled by vibes.
- Fine-tuning to teach facts that retrieval should supply.
- Prompt chains whose cost and p95 were never summed.
- One provider, no fallback, no timeout, no budget cap.
- "It works" demonstrated with three cherry-picked examples.
Verification receipts
- Job statement plus a golden set with expected outputs.
- Baseline eval score, and the regression floor.
- Cost and latency model per request at projected volume.
- Data-flow diagram: what leaves the boundary and under what terms.
- Fallback behaviour verified by simulating a provider outage.
04 同仓库更多 Skills
interaction-patterns-architect
Architect-level interaction design in code — choosing established patterns over invented ones, feedback and latency behaviour, destructive-action safeguards, navigation and overlay conventions, and the state model behind every interactive surface. Use when a screen has to behave, not just look.
design-system-ops-architect
Architect-level operation of a design system in code — token pipelines and versioning, component API and variant policy, the catalogue as the contract, adoption metrics, and the governance that stops a system from forking into three. Use when a component library must survive several teams.
design-handoff-architect
Architect-level design handoff — what must be specified before implementation starts, tokens over raw values, responsive and content-limit contracts, state matrices, and the review loop that catches drift between design and code. Use when a design is about to become code.
content-microcopy-architect
Architect-level UI content — where strings live, voice and terminology consistency, error and empty-state copy as part of the interface, translation-ready authoring, and the content model a product needs before it multiplies screens. Use when the words in the interface matter as much as the pixels.
ui-ux-implementation-architect
Architect-level direction for turning a design into working UI — the handoff contract, component ownership versus screen-level layout, interaction and state conventions, design QA gates, and the review habits that keep an implementation faithful without frozen pixels. Use before any specific UI implementation skill.
test-strategy-architect
Architect-level test strategy — allocating assertions across unit, integration, contract and end-to-end layers, designing for testability through dependency boundaries, fixture and data strategy, and the flake and coverage policies that keep a suite trustworthy. Use when defining how a codebase is tested.