agents-and-llm-orchestration
Build txtai LLM, RAG, and Agent workflows with backend selection, tools, templates, teams, and troubleshooting guidance.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- agents-and-llm-orchestration
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Build txtai LLM, RAG, and Agent workflows with backend selection, tools, templates, teams, and troubleshooting guidance.
- disable-model-invocation
- true
02 资源文件 4
- 参考文档references/backend-and-tooling.md7.8 KB
- 参考文档references/llm-rag-agent-recipes.md10.0 KB
- 参考文档references/troubleshooting.md8.4 KB
- 脚本scripts/rag_config_template.py5.3 KB
03 SKILL.md 正文
Agents and LLM Orchestration
Use this sub-skill when the task is to connect txtai generation, retrieval, and agentic tool use. It covers LLM, RAG, and Agent construction, YAML configuration, tool definitions, prompt templates, agent teams, backend selection, and optional dependency troubleshooting.
For low-level embeddings store creation and query tuning, route to embeddings-search. For deterministic pipeline/workflow task chains, route to pipelines-and-workflows. For serving agents/RAG over API, OpenAI-compatible endpoints, or hosted app deployment, route to api-and-deployment.
Quick Routing
| User goal | Use |
|---|---|
| Generate text directly | LLM(path=None, method=None, **kwargs) |
| Answer with retrieved context | RAG(similarity, path, template=..., context=..., output=...) |
| Let a model choose tools over multiple steps | Agent(model=..., tools=[...], max_steps=...) |
| Build multi-agent research/delegation | Agents as tool targets inside another Agent |
| Define via YAML/Application | top-level llm:, rag:, and agent: sections |
| Validate RAG config safely | python scripts/rag_config_template.py --write-template rag_template.py |
Minimal Import Check
from txtai import Agent, Embeddings, LLM, RAG
print(LLM)
print(RAG)
print(Agent)
If constructing Agent(...) raises ImportError: smolagents is not available - install "agent" extra to enable, install an agent-capable environment such as pip install "txtai[agent]". If LLM backends fail, see backend-and-tooling.
Direct LLM Pattern
from txtai import LLM
llm = LLM("ibm-granite/granite-4.0-350m")
answer = llm("Summarize retrieval augmented generation", maxlength=256)
LLM accepts a model path, optional method, and backend keyword arguments. It supports strings, lists of strings, chat-message dictionaries with role and content, streaming via stream=True, stop strings, defaultrole, and stripthink cleanup. Use method="transformers", method="llama.cpp", method="litellm", or a custom generation class path when inference cannot be inferred from the model path.
RAG Pattern
from txtai import Embeddings, RAG
embeddings = Embeddings(content=True)
embeddings.index([
("install", "txtai supports semantic search, RAG, agents, workflows and APIs"),
("agent", "txtai agents use tools such as Python functions, embeddings stores and web search"),
])
template = """
Answer using only the context.
Question: {question}
Context: {context}
"""
rag = RAG(
embeddings,
"ibm-granite/granite-4.0-350m",
template=template,
system="You answer from retrieved context.",
context=3,
output="flatten",
)
result = rag("What can txtai agents use?", maxlength=256)
RAG joins a similarity source with a generator or question-answering pipeline. The similarity source can be an Embeddings instance, a Similarity pipeline, a callable batch searcher, or an Application-provided embeddings index. Build and persist retrieval stores with embeddings-search before focusing on prompt generation here.
Agent Pattern
from datetime import datetime
from txtai import Agent
def today() -> str:
"""
Gets the current date.
Returns:
current date in ISO format
"""
return datetime.today().isoformat()
agent = Agent(
model="Qwen/Qwen3-4B-Instruct-2507",
tools=[today, "websearch"],
max_steps=5,
)
answer = agent("What is today's date and why might it matter for a news search?", maxlength=2048)
Use agents for multi-step questions where tool choice matters. Prefer plain RAG or deterministic workflows for simple, repeatable flows.
Tool Shapes
- Direct Python callable: add the function or callable instance to
tools; type hints and Google-style docstrings improve generated tool schemas. - Explicit function dictionary:
{"name": "lookup", "description": "...", "inputs": {...}, "target": callable}. - Embeddings dictionary:
{"name": "kb", "description": "Searches the knowledge base", "path": "index-path"}or{"target": embeddings}; results are lists of dicts withid,text, andscorewhen content is available. - Built-in shortcuts:
bash,defaults,edit,glob,grep,python,read,todowrite,websearch,webview, andwrite. - MCP tools: HTTP strings such as
http://localhost:8000/mcpload a Model Context Protocol tool collection. - Skill files: strings ending in
.mdload askill.md-style tool with frontmatternameanddescription.
Agent Teams
Agents can be tool targets for a supervisor agent.
from txtai import Agent, LLM
llm = LLM("Qwen/Qwen3-4B-Instruct-2507")
websearcher = Agent(model=llm, tools=["websearch"], max_steps=3)
supervisor = Agent(
model=llm,
tools=[{
"name": "websearcher",
"description": "Searches the web and returns concise findings",
"target": websearcher,
}],
max_steps=6,
)
Share a single LLM instance when team members should use the same backend configuration. Keep each agent tool description narrow so the supervisor can choose correctly.
YAML/Application Pattern
llm:
path: Qwen/Qwen3-4B-Instruct-2507
agent:
researcher:
max_steps: 5
tools:
- websearch
- name: today
description: Gets the current date
target: package.module.today
rag:
path: ibm-granite/granite-4.0-350m
template: |
Answer using only the context.
Question: {question}
Context: {context}
output: flatten
context: 3
Application(config) resolves configured pipelines first, then agents, then embeddings. A configured rag: pipeline can receive the Application embeddings index when its similarity is omitted. In agent: sections, target values are resolved as configured pipelines, workflows, or importable callables.
RAG Output Modes
output |
Shape |
|---|---|
default |
(name, answer) tuples, or dicts with answer for dict/string inputs |
flatten |
answer strings only |
reference |
(name, answer, reference) tuples or dicts with references |
RAG input can be a string, a list of strings, tuples shaped like (name, query, question, snippet), or dict rows with name, query, question, and snippet. Use output="flatten" for simple chat answers and output="reference" when the caller needs source ids from retrieved context.
References
- LLM, RAG, and Agent recipes for concrete Python and YAML patterns.
- Backend and tooling for extras, credentials, backends, and tool schemas.
- Troubleshooting for placeholder agents, template bugs, retrieval quality, output shapes, web/network failures, and CPU/GPU issues.
Validation Checklist
- Confirm imports and constructor signatures for
LLM,RAG, andAgentin the target environment. - For RAG, ensure the retrieval store is content-enabled and the prompt template includes both
{question}and{context}. - For agents, verify optional extras by instantiating a tiny
Agent(model=..., tools=[], max_steps=1)or by importingsmolagentsdirectly. - For hosted LLMs, verify credentials outside source files and run a one-prompt smoke test before wiring tools.
- Keep source repository docs, examples, and absolute checkout paths out of generated runtime content.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).