AGENT SKILL

models-and-serving

Configure DB-GPT 0.8.1 model providers and local backends, validate model TOML, operate model controller/worker/API-server roles, use model-management CLI commands, and diagnose serving failures without mistaking an import or registry record for a live deployment.

License 许可
MIT
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
models-and-serving
license
MIT
metadata
{ "disco-role": "operating" }
description
Configure DB-GPT 0.8.1 model providers and local backends, validate model TOML, operate model controller/worker/API-server roles, use model-management CLI commands, and diagnose serving failures without mistaking an import or registry record for a live deployment.
disable-model-invocation
true

02 资源文件 6

03 SKILL.md 正文

Models and serving

Use this route when a DB-GPT task mentions model/provider TOML, models.llms, embeddings or rerankers, OpenAI-compatible endpoints, Ollama, Hugging Face, vLLM, llama.cpp, MLX, model workers, controller, model API server, model CLI, model registration, health/heartbeat, GPU memory, quantization, or serving logs.

This route owns model selection and the model-service topology. Route general installation, profile creation, workspace paths, and generic start webserver setup to setup-and-cli; route agents/AWEL to agents-and-awel; route public client CRUD and sandbox calls to apis-client-and-sandbox; route document, embedding-pipeline, vector-store, and knowledge workflows to data-and-rag.

Operating contract

  • Work against the public DB-GPT 0.8.1 package interface, not a source checkout.
  • Treat a TOML parse, Python import, controller registry row, or healthy value observed before a real request as configuration evidence only. Deployment is successful only after the intended service is reachable and a suitable model operation succeeds.
  • Keep provider credentials in environment variables. Never print resolved secrets, copy them into generated files, or put them in shell history.
  • Choose one model role explicitly: llm, text2vec (embedding), or reranker. A chat model cannot silently substitute for an embedding model.
  • CPU is the verified baseline for this route. CUDA/local-model, vLLM, bitsandbytes, llama.cpp acceleration, and MLX are optional backend paths; do not claim they work from a CPU import or from host GPU visibility alone.

Read the focused contracts before making changes:

  • model-api-reference.md — configuration dataclasses, model/provider fields, worker types, and OpenAI-compatible API surface.
  • provider-and-backend-matrix.md — provider credentials, endpoint conventions, embedding pairing, optional dependencies, and backend boundaries.
  • serving-workflows.md — proxy/local TOML, unified webserver, split controller/worker/API-server deployment, and CLI operations.
  • backends.md — local-model, quantization, VRAM, and hardware-specific decision rules.
  • troubleshooting.md — symptom-to-check diagnosis and recovery boundaries.

Standard workflow

  1. Identify the topology and model roles. Decide whether the user needs a unified webserver process, a split cluster, a standalone model API server, or only a provider configuration. Record the LLM, embedding, and optional reranker names separately.

  2. Select the provider/backend. Prefer a remote proxy for a CPU-first smoke test. For a local model, require a real model path or an explicitly allowed model download and select hf, vllm, llama.cpp, llama.cpp.server, or mlx deliberately. Check the matrix for the provider-specific key and endpoint.

  3. Write a redacted TOML. Put model entries under [models] using repeated [[models.llms]], [[models.embeddings]], and [[models.rerankers]] tables. Use ${env:NAME} or ${env:NAME:-default} placeholders. Set default_llm, default_embedding, or default_reranker only when multiple entries make selection ambiguous. Keep web/database configuration owned by the setup route unless the serving topology requires the related model service section.

  4. Validate without network or model loading. Run the bundled checker:

    python scripts/model_config_check.py path/to/config.toml
    

    It parses TOML, checks model-table shape, provider/role requirements, duplicate names, placeholder syntax, endpoint URLs, and obvious service-port collisions. It never resolves secrets, calls a provider, contacts a controller, downloads a model, or starts a process. Use --json for a machine-readable report and --allow-missing-embeddings only when the configuration is intentionally chat-only; RAG/knowledge workflows must not use that exception.

  5. Run a safe package-level check. Use dbgpt --version, the relevant static --help, and import/config-parser checks. Do not use dbgpt model start --help or another dynamic model command as an offline gate: model command discovery can query a controller and return a timeout/502 when no controller is reachable.

  6. Start only the chosen topology. Use the exact commands in serving-workflows.md. Do not start both a unified model manager and an independent worker on the same ports. For split mode, start and verify the controller before workers, then verify workers before the API/web layer.

  7. Verify reachability and model capability. Check the controller health and registry, then issue a minimal request through the intended API or provider. dbgpt model list is a registry/heartbeat view, not proof that generation or embedding works. An unreachable controller, failed heartbeat, missing model, authentication failure, or failed embedding request is a deployment failure; report it as such with the exact next check.

  8. Diagnose from the right boundary. Inspect the command's resolved config path, service logs, bind address, port, controller address, provider base URL, model name, and optional-backend imports in that order. Redact API keys and authorization headers in reports.

CLI routing facts

The DB-GPT CLI registers model commands under dbgpt model and service commands under dbgpt start / dbgpt stop:

dbgpt model [--address CONTROLLER_URL] list|start|stop|restart|chat
dbgpt start controller --config FILE [--daemon]
dbgpt start worker --config FILE [--daemon]
dbgpt start apiserver --config FILE [--daemon]
dbgpt start webserver [web options]
dbgpt stop controller|worker|apiserver [--port PORT]

CONTROLLER_ADDRESS can supply the controller address used by model-management commands when --address is omitted. The dynamic model start command obtains supported-model metadata from the controller before constructing some options; therefore a failed remote request during help or invocation is not a successful start. For model-service launch commands, the configuration file is the source of worker/model parameters; do not rely on stale examples that show model flags which are not present in the installed command tree.

Validation and boundaries

A valid result for this route includes:

  • provider/model/role entries parse with no secret resolution or network call;
  • an embedding entry exists and is compatible with the intended RAG path;
  • each selected local backend's dependency, path, device, and VRAM assumptions are explicit;
  • service roles have distinct, reachable addresses and non-conflicting ports;
  • controller registry and heartbeat checks are separated from generation, completion, embedding, or reranking checks;
  • optional CUDA/GPU/provider/service dependencies are labeled unverified when they were not executed.

Do not claim that dbgpt model list proves deployment, that --daemon proves a healthy child process, that a provider key exists merely because a placeholder parsed, or that an A100-visible host proves CUDA/torch/vLLM/bitsandbytes support. Do not add API CRUD, agent orchestration, or RAG chunking instructions here.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351