AGENT SKILL

langfuse

暂无简介

License 许可
MIT
最近更新
2026-11-14
Stars
★ 0
Forks
0

02 资源文件 1

03 SKILL.md 正文

Langfuse trace inspection

Read-only tool that queries the Langfuse observability API to fetch and summarise recent traces. Primarily used to diagnose implement-stage failures by inspecting the agent's own LLM-call traces linked to a specific ticket or run.

When to use it

  • When the mill's implement stage fails to produce a PR and the assistant needs to inspect the LLM traces to understand why.
  • When a ticket or run id is known and the assistant wants to see the most recent traces emitted during that run.
  • When debugging an anomalous LLM response — fetch the trace by its id for a full structured summary.
  • When analyzing fleet cost drivers — query traces from robotsix-mill, robotsix-ci-fix, or other components using the project parameter. To rank the most expensive traces across the whole fleet in a single call, pass a list of project names or project="all"; the combined result is sorted by totalCost descending.

Allowed operation

Tool Description
inspect_langfuse_trace Fetch one or more Langfuse traces and return a structured summary.

Tool signatures

inspect_langfuse_trace(
    trace_id: str = "",
    ticket_id: str = "",
    project: str | list[str] = "",
    limit: int = 5,
    from_timestamp: str = "",
    to_timestamp: str = "",
) -> str
  • At least one search criterion must be provided: trace_id, ticket_id, from_timestamp, or to_timestamp.

  • trace_id is mutually exclusive with the other criteria.

  • project: Langfuse project selector (optional). When empty, queries the main chat project. It accepts several forms:

    • a single project name — e.g. project="robotsix-mill", project="robotsix-ci-fix", project="robotsix-memory";
    • a list of project names, or a comma-separated string, to query several projects at once — e.g. project=["robotsix-mill", "robotsix-ci-fix"] or project="robotsix-mill,robotsix-chat";
    • the special value project="all" to query every configured project.

    When more than one project is queried, each trace summary carries an extra project field, the combined traces list is sorted by totalCost descending, and the response echoes the resolved projects list (plus a per-project errors map when some projects could not be queried). See the configuration schema for the full list of available projects.

  • When trace_id is given: fetch that single trace and return a detailed summary (name, timestamps, input/output tokens, total cost, top-level observations, scores).

  • When ticket_id is given: search for traces whose tags include ticket_id:<value> (most recent first) up to limit, and return a summary list (trace id, name, timestamp, duration, cost).

  • from_timestamp / to_timestamp: filter traces by time range (ISO 8601, e.g. 2026-08-01T00:00:00Z). Either or both may be provided, alone or combined with ticket_id.

  • limit caps the number of traces returned (default 5, max configured in settings).

Return value

A JSON string with a traces list. Each trace entry carries:

Field Description
id Langfuse trace id
name Trace name (e.g. "implement", "chat-turn")
timestamp ISO-8601 start time
userId User/agent id attached to the trace
latency End-to-end duration in seconds
totalCost Total cost in USD
usage {promptTokens, completionTokens, cachedPromptTokens, totalTokens}
observations Count of observations (LLM calls, spans, events)
scores List of {name, value} score pairs

cachedPromptTokens exposes the prompt tokens served from cache — compare it against promptTokens to spot wasteful token-ratio patterns (cache hits vs. primary tokens) and redundant retries.

When ticket_id is given, an additional ticket_id field echoes the search key and limit echoes the cap used. When multiple projects are queried, each trace carries a project field, and the response echoes the resolved projects list (and an errors map if any project failed).

Safety

  • Read-only — never mutates anything in Langfuse.
  • Only reaches the configured Langfuse host (default https://cloud.langfuse.com).
  • Authenticates with Basic auth using the specified project's configured credentials — no user credentials are ever exposed to the agent.
  • The tool is only available when langfuse_inspect.enabled is true in the server config.

Error handling — read before acting on a failure

  • Transient errors are retried automatically. The tool retries timeouts, 5xx responses, and network failures up to 2 times with exponential backoff before returning an error. If you receive an error, it has already survived multiple attempts — do NOT retry the tool repeatedly hoping for a different result.
  • Do NOT claim credentials are missing without evidence. The tool authenticates via Basic auth using the langfuse.projects config block — it checks credentials up front and returns an explicit "project X is not configured" error when they are absent. Any other error (HTTP status, timeout, network failure) means the credentials were sent and the Langfuse host rejected or did not receive the request. Blaming a missing credential when the tool did not return the credential-specific error message wastes time and erodes trust.
  • Do NOT suggest a restart. The environment-variable credential path (LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY) was decommissioned — the tool reads credentials from the server's JSON config file at startup. A restart will not change the credential state and will not fix a network, timeout, or API-level error.
  • Other components' projects come from deploy. A foreign project (e.g. robotsix-mill) is configured only when central-deploy dispatched its keys — the owning component tags them with a langfuse:<project> scope and chat consumes langfuse:*, which injects LANGFUSE_<PROJECT>_PUBLIC_KEY / _SECRET_KEY at deploy. If such a project is "not configured", the fix is the scope tag in central-deploy's env store plus a chat redeploy — never copying the keys into chat's config.
  • The self-check you already have: call the tool with any known trace id or ticket id. If it returns data, Langfuse access is working. If it returns an error, read the error message literally — it tells you exactly what failed (network, timeout, HTTP status, missing credentials). Do not layer your own theory on top of a clear error message.

Example calls

# Inspect a specific known trace
inspect_langfuse_trace(trace_id="01J...abc")

# Search for traces linked to a ticket (chat project)
inspect_langfuse_trace(ticket_id="20260727T001240Z-add-capability-5bd6", limit=5)

# Query the last 24 hours of traces from the robotsix-mill project to identify cost drivers
inspect_langfuse_trace(project="robotsix-mill", from_timestamp="2026-08-22T00:00:00Z",
                       to_timestamp="2026-08-23T00:00:00Z", limit=10)

# Find the most expensive robotsix-ci-fix traces from the last week
inspect_langfuse_trace(project="robotsix-ci-fix", from_timestamp="2026-08-11T00:00:00Z", limit=20)

# Rank the costliest traces across several projects in one call (sorted by totalCost desc)
inspect_langfuse_trace(project=["robotsix-mill", "robotsix-ci-fix", "robotsix-chat"],
                       from_timestamp="2026-08-22T00:00:00Z", limit=10)

# Rank the costliest traces across EVERY configured project (fleet-wide cost analysis)
inspect_langfuse_trace(project="all", from_timestamp="2026-08-22T00:00:00Z", limit=10)

Fast path for periodic-session speed complaints (moved from the system prompt, 2026-09-08)

– Periodic session speed complaints: when a user reports that a scheduled periodic session finishes too fast, stops too early, or produces short-lived sessions, and references specific Langfuse trace IDs or session IDs, use inspect_langfuse_trace to query those traces directly in your FIRST response. Query the provided trace IDs plus a wider time window (the 24–48 hours around the reported traces) via from_timestamp / to_timestamp to find other short-lived sessions. Report the outliers with their trace IDs, durations, and a one-sentence diagnosis in a single response — do NOT ask clarifying questions or offer to spawn a background subsession. The user has already told you what they need; act on it.

04 同仓库更多 Skills