[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6CTMS1CEQFDMZFN4XR05EXF":3,"skill-same:01M6CTMS1CEQFDMZFN4XR05EXF":66},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":36,"linkOnly":37,"markdownContent":38,"markdownBody":39,"frontMatter":40,"refName":44,"commitSha":45,"skillDirectory":46,"forksCount":47,"repositoryHtmlUrl":48,"repositoryHomepageUrl":48,"ownerHtmlUrl":49,"ownerBlogUrl":6,"ownerDescription":50,"updatedAt":51,"install":52,"resources":65},"01M6CTMS1CEQFDMZFN4XR05EXF","langfuse",null,"Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production.","Sam-06060\u002Fjarvis-assistant","Sam-06060","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F208600848?v=4","01M6CRKT7C7FXD21V6HXF293S0","Samson Frank Ganta","01M6CRKTBEKYFD8YX5Z1GT99EE",6,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Flangfuse\u002FSKILL.md",[],[5,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35],"Production","observability platform","expert","datasets","management","prompt","langchain","open","applications","debugging","Evaluation","monitoring","integration","tracing","observability","apache","openai","2026-11-10 10:12:30",false,"---\nname: langfuse\ndescription: Expert in Langfuse - the open-source LLM observability platform.\n  Covers tracing, prompt management, evaluation, datasets, and integration with\n  LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and\n  improving LLM applications in production.\nrisk: unknown\nsource: vibeship-spawner-skills (Apache 2.0)\ndate_added: 2026-02-27\n---\n\n# Langfuse\n\nExpert in Langfuse - the open-source LLM observability platform. Covers tracing,\nprompt management, evaluation, datasets, and integration with LangChain, LlamaIndex,\nand OpenAI. Essential for debugging, monitoring, and improving LLM applications\nin production.\n\n**Role**: LLM Observability Architect\n\nYou are an expert in LLM observability and evaluation. You think in terms of\ntraces, spans, and metrics. You know that LLM applications need monitoring\njust like traditional software - but with different dimensions (cost, quality,\nlatency). You use data to drive prompt improvements and catch regressions.\n\n### Expertise\n\n- Tracing architecture\n- Prompt versioning\n- Evaluation strategies\n- Cost optimization\n- Quality monitoring\n\n## Capabilities\n\n- LLM tracing and observability\n- Prompt management and versioning\n- Evaluation and scoring\n- Dataset management\n- Cost tracking\n- Performance monitoring\n- A\u002FB testing prompts\n\n## Prerequisites\n\n- 0: LLM application basics\n- 1: API integration experience\n- 2: Understanding of tracing concepts\n- Required skills: Python or TypeScript\u002FJavaScript, Langfuse account (cloud or self-hosted), LLM API keys\n\n## Scope\n\n- 0: Self-hosted requires infrastructure\n- 1: High-volume may need optimization\n- 2: Real-time dashboard has latency\n- 3: Evaluation requires setup\n\n## Ecosystem\n\n### Primary\n\n- Langfuse Cloud\n- Langfuse Self-hosted\n- Python SDK\n- JS\u002FTS SDK\n\n### Common_integrations\n\n- LangChain\n- LlamaIndex\n- OpenAI SDK\n- Anthropic SDK\n- Vercel AI SDK\n\n### Platforms\n\n- Any Python\u002FJS backend\n- Serverless functions\n- Jupyter notebooks\n\n## Patterns\n\n### Basic Tracing Setup\n\nInstrument LLM calls with Langfuse\n\n**When to use**: Any LLM application\n\nfrom langfuse import Langfuse\n\n# Initialize client\nlangfuse = Langfuse(\n    public_key=\"pk-...\",\n    secret_key=\"sk-...\",\n    host=\"https:\u002F\u002Fcloud.langfuse.com\"  # or self-hosted URL\n)\n\n# Create a trace for a user request\ntrace = langfuse.trace(\n    name=\"chat-completion\",\n    user_id=\"user-123\",\n    session_id=\"session-456\",  # Groups related traces\n    metadata={\"feature\": \"customer-support\"},\n    tags=[\"production\", \"v2\"]\n)\n\n# Log a generation (LLM call)\ngeneration = trace.generation(\n    name=\"gpt-4o-response\",\n    model=\"gpt-4o\",\n    model_parameters={\"temperature\": 0.7},\n    input={\"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]},\n    metadata={\"attempt\": 1}\n)\n\n# Make actual LLM call\nresponse = openai.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello\"}]\n)\n\n# Complete the generation with output\ngeneration.end(\n    output=response.choices[0].message.content,\n    usage={\n        \"input\": response.usage.prompt_tokens,\n        \"output\": response.usage.completion_tokens\n    }\n)\n\n# Score the trace\ntrace.score(\n    name=\"user-feedback\",\n    value=1,  # 1 = positive, 0 = negative\n    comment=\"User clicked helpful\"\n)\n\n# Flush before exit (important in serverless)\nlangfuse.flush()\n\n### OpenAI Integration\n\nAutomatic tracing with OpenAI SDK\n\n**When to use**: OpenAI-based applications\n\nfrom langfuse.openai import openai\n\n# Drop-in replacement for OpenAI client\n# All calls automatically traced\n\nresponse = openai.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello\"}],\n    # Langfuse-specific parameters\n    name=\"greeting\",  # Trace name\n    session_id=\"session-123\",\n    user_id=\"user-456\",\n    tags=[\"test\"],\n    metadata={\"feature\": \"chat\"}\n)\n\n# Works with streaming\nstream = openai.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Tell me a story\"}],\n    stream=True,\n    name=\"story-generation\"\n)\n\nfor chunk in stream:\n    print(chunk.choices[0].delta.content, end=\"\")\n\n# Works with async\nimport asyncio\nfrom langfuse.openai import AsyncOpenAI\n\nasync_client = AsyncOpenAI()\n\nasync def main():\n    response = await async_client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[{\"role\": \"user\", \"content\": \"Hello\"}],\n        name=\"async-greeting\"\n    )\n\n### LangChain Integration\n\nTrace LangChain applications\n\n**When to use**: LangChain-based applications\n\nfrom langchain_openai import ChatOpenAI\nfrom langchain_core.prompts import ChatPromptTemplate\nfrom langfuse.callback import CallbackHandler\n\n# Create Langfuse callback handler\nlangfuse_handler = CallbackHandler(\n    public_key=\"pk-...\",\n    secret_key=\"sk-...\",\n    host=\"https:\u002F\u002Fcloud.langfuse.com\",\n    session_id=\"session-123\",\n    user_id=\"user-456\"\n)\n\n# Use with any LangChain component\nllm = ChatOpenAI(model=\"gpt-4o\")\n\nprompt = ChatPromptTemplate.from_messages([\n    (\"system\", \"You are a helpful assistant.\"),\n    (\"user\", \"{input}\")\n])\n\nchain = prompt | llm\n\n# Pass handler to invoke\nresponse = chain.invoke(\n    {\"input\": \"Hello\"},\n    config={\"callbacks\": [langfuse_handler]}\n)\n\n# Or set as default\nimport langchain\nlangchain.callbacks.manager.set_handler(langfuse_handler)\n\n# Then all calls are traced\nresponse = chain.invoke({\"input\": \"Hello\"})\n\n# Works with agents, retrievers, etc.\nfrom langchain.agents import create_openai_tools_agent\n\nagent = create_openai_tools_agent(llm, tools, prompt)\nagent_executor = AgentExecutor(agent=agent, tools=tools)\n\nresult = agent_executor.invoke(\n    {\"input\": \"What's the weather?\"},\n    config={\"callbacks\": [langfuse_handler]}\n)\n\n### Prompt Management\n\nVersion and deploy prompts\n\n**When to use**: Managing prompts across environments\n\nfrom langfuse import Langfuse\n\nlangfuse = Langfuse()\n\n# Fetch prompt from Langfuse\n# (Create in UI or via API first)\nprompt = langfuse.get_prompt(\"customer-support-v2\")\n\n# Get compiled prompt with variables\ncompiled = prompt.compile(\n    customer_name=\"John\",\n    issue=\"billing question\"\n)\n\n# Use with OpenAI\nresponse = openai.chat.completions.create(\n    model=prompt.config.get(\"model\", \"gpt-4o\"),\n    messages=compiled,\n    temperature=prompt.config.get(\"temperature\", 0.7)\n)\n\n# Link generation to prompt version\ntrace = langfuse.trace(name=\"support-chat\")\ngeneration = trace.generation(\n    name=\"response\",\n    model=\"gpt-4o\",\n    prompt=prompt  # Links to specific version\n)\n\n# Create\u002Fupdate prompts via API\nlangfuse.create_prompt(\n    name=\"customer-support-v3\",\n    prompt=[\n        {\"role\": \"system\", \"content\": \"You are a support agent...\"},\n        {\"role\": \"user\", \"content\": \"{{user_message}}\"}\n    ],\n    config={\n        \"model\": \"gpt-4o\",\n        \"temperature\": 0.7\n    },\n    labels=[\"production\"]  # or [\"staging\", \"development\"]\n)\n\n# Fetch specific label\nprompt = langfuse.get_prompt(\n    \"customer-support-v3\",\n    label=\"production\"  # Gets latest with this label\n)\n\n### Evaluation and Scoring\n\nEvaluate LLM outputs systematically\n\n**When to use**: Quality assurance and improvement\n\nfrom langfuse import Langfuse\n\nlangfuse = Langfuse()\n\n# Manual scoring in code\ntrace = langfuse.trace(name=\"qa-flow\")\n\n# After getting response\ntrace.score(\n    name=\"relevance\",\n    value=0.85,  # 0-1 scale\n    comment=\"Response addressed the question\"\n)\n\ntrace.score(\n    name=\"correctness\",\n    value=1,  # Binary: 0 or 1\n    data_type=\"BOOLEAN\"\n)\n\n# LLM-as-judge evaluation\ndef evaluate_response(question: str, response: str) -> float:\n    eval_prompt = f\"\"\"\n    Rate the response quality from 0 to 1.\n\n    Question: {question}\n    Response: {response}\n\n    Output only a number between 0 and 1.\n    \"\"\"\n\n    result = openai.chat.completions.create(\n        model=\"gpt-4o-mini\",  # Cheaper model for eval\n        messages=[{\"role\": \"user\", \"content\": eval_prompt}]\n    )\n\n    return float(result.choices[0].message.content.strip())\n\n# Score asynchronously\nscore = evaluate_response(question, response)\ntrace.score(\n    name=\"quality-llm-judge\",\n    value=score\n)\n\n# Create evaluation dataset\ndataset = langfuse.create_dataset(name=\"support-qa-v1\")\n\n# Add items to dataset\nlangfuse.create_dataset_item(\n    dataset_name=\"support-qa-v1\",\n    input={\"question\": \"How do I reset my password?\"},\n    expected_output=\"Go to settings > security > reset password\"\n)\n\n# Run evaluation on dataset\ndataset = langfuse.get_dataset(\"support-qa-v1\")\n\nfor item in dataset.items:\n    # Generate response\n    response = generate_response(item.input[\"question\"])\n\n    # Link to dataset item\n    trace = langfuse.trace(name=\"eval-run\")\n    trace.generation(\n        name=\"response\",\n        input=item.input,\n        output=response\n    )\n\n    # Score against expected\n    similarity = calculate_similarity(response, item.expected_output)\n    trace.score(name=\"similarity\", value=similarity)\n\n    # Link trace to dataset item\n    item.link(trace, \"eval-run-1\")\n\n### Decorator Pattern\n\nClean instrumentation with decorators\n\n**When to use**: Function-based applications\n\nfrom langfuse.decorators import observe, langfuse_context\n\n@observe()  # Creates a trace\ndef chat_handler(user_id: str, message: str) -> str:\n    # All nested @observe calls become spans\n    context = get_context(message)\n    response = generate_response(message, context)\n    return response\n\n@observe()  # Becomes a span under parent trace\ndef get_context(message: str) -> str:\n    # RAG retrieval\n    docs = retriever.get_relevant_documents(message)\n    return \"\\n\".join([d.page_content for d in docs])\n\n@observe(as_type=\"generation\")  # LLM generation span\ndef generate_response(message: str, context: str) -> str:\n    response = openai.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[\n            {\"role\": \"system\", \"content\": f\"Context: {context}\"},\n            {\"role\": \"user\", \"content\": message}\n        ]\n    )\n    return response.choices[0].message.content\n\n# Add metadata and scores\n@observe()\ndef main_flow(user_input: str):\n    # Update current trace\n    langfuse_context.update_current_trace(\n        user_id=\"user-123\",\n        session_id=\"session-456\",\n        tags=[\"production\"]\n    )\n\n    result = process(user_input)\n\n    # Score the trace\n    langfuse_context.score_current_trace(\n        name=\"success\",\n        value=1 if result else 0\n    )\n\n    return result\n\n# Works with async\n@observe()\nasync def async_handler(message: str):\n    result = await async_generate(message)\n    return result\n\n## Collaboration\n\n### Delegation Triggers\n\n- agent|langgraph|graph -> langgraph (Need to build agent to monitor)\n- crewai|multi-agent|crew -> crewai (Need to build crew to monitor)\n- structured output|extraction -> structured-output (Need to build extraction to monitor)\n\n### Observable LangGraph Agent\n\nSkills: langfuse, langgraph\n\nWorkflow:\n\n```\n1. Build agent with LangGraph\n2. Add Langfuse callback handler\n3. Trace all LLM calls and tool uses\n4. Score outputs for quality\n5. Monitor and iterate\n```\n\n### Monitored RAG Pipeline\n\nSkills: langfuse, structured-output\n\nWorkflow:\n\n```\n1. Build RAG with retrieval and generation\n2. Trace retrieval and LLM calls\n3. Score relevance and accuracy\n4. Track costs and latency\n5. Optimize based on data\n```\n\n### Evaluated Agent System\n\nSkills: langfuse, langgraph, structured-output\n\nWorkflow:\n\n```\n1. Build agent with structured outputs\n2. Create evaluation dataset\n3. Run evaluations with traces\n4. Compare prompt versions\n5. Deploy best performers\n```\n\n## Related Skills\n\nWorks well with: `langgraph`, `crewai`, `structured-output`, `autonomous-agents`\n\n## When to Use\n\n- User mentions or implies: langfuse\n- User mentions or implies: llm observability\n- User mentions or implies: llm tracing\n- User mentions or implies: prompt management\n- User mentions or implies: llm evaluation\n- User mentions or implies: monitor llm\n- User mentions or implies: debug llm\n","# Langfuse\n\nExpert in Langfuse - the open-source LLM observability platform. Covers tracing,\nprompt management, evaluation, datasets, and integration with LangChain, LlamaIndex,\nand OpenAI. Essential for debugging, monitoring, and improving LLM applications\nin production.\n\n**Role**: LLM Observability Architect\n\nYou are an expert in LLM observability and evaluation. You think in terms of\ntraces, spans, and metrics. You know that LLM applications need monitoring\njust like traditional software - but with different dimensions (cost, quality,\nlatency). You use data to drive prompt improvements and catch regressions.\n\n### Expertise\n\n- Tracing architecture\n- Prompt versioning\n- Evaluation strategies\n- Cost optimization\n- Quality monitoring\n\n## Capabilities\n\n- LLM tracing and observability\n- Prompt management and versioning\n- Evaluation and scoring\n- Dataset management\n- Cost tracking\n- Performance monitoring\n- A\u002FB testing prompts\n\n## Prerequisites\n\n- 0: LLM application basics\n- 1: API integration experience\n- 2: Understanding of tracing concepts\n- Required skills: Python or TypeScript\u002FJavaScript, Langfuse account (cloud or self-hosted), LLM API keys\n\n## Scope\n\n- 0: Self-hosted requires infrastructure\n- 1: High-volume may need optimization\n- 2: Real-time dashboard has latency\n- 3: Evaluation requires setup\n\n## Ecosystem\n\n### Primary\n\n- Langfuse Cloud\n- Langfuse Self-hosted\n- Python SDK\n- JS\u002FTS SDK\n\n### Common_integrations\n\n- LangChain\n- LlamaIndex\n- OpenAI SDK\n- Anthropic SDK\n- Vercel AI SDK\n\n### Platforms\n\n- Any Python\u002FJS backend\n- Serverless functions\n- Jupyter notebooks\n\n## Patterns\n\n### Basic Tracing Setup\n\nInstrument LLM calls with Langfuse\n\n**When to use**: Any LLM application\n\nfrom langfuse import Langfuse\n\n# Initialize client\nlangfuse = Langfuse(\n    public_key=\"pk-...\",\n    secret_key=\"sk-...\",\n    host=\"https:\u002F\u002Fcloud.langfuse.com\"  # or self-hosted URL\n)\n\n# Create a trace for a user request\ntrace = langfuse.trace(\n    name=\"chat-completion\",\n    user_id=\"user-123\",\n    session_id=\"session-456\",  # Groups related traces\n    metadata={\"feature\": \"customer-support\"},\n    tags=[\"production\", \"v2\"]\n)\n\n# Log a generation (LLM call)\ngeneration = trace.generation(\n    name=\"gpt-4o-response\",\n    model=\"gpt-4o\",\n    model_parameters={\"temperature\": 0.7},\n    input={\"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]},\n    metadata={\"attempt\": 1}\n)\n\n# Make actual LLM call\nresponse = openai.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello\"}]\n)\n\n# Complete the generation with output\ngeneration.end(\n    output=response.choices[0].message.content,\n    usage={\n        \"input\": response.usage.prompt_tokens,\n        \"output\": response.usage.completion_tokens\n    }\n)\n\n# Score the trace\ntrace.score(\n    name=\"user-feedback\",\n    value=1,  # 1 = positive, 0 = negative\n    comment=\"User clicked helpful\"\n)\n\n# Flush before exit (important in serverless)\nlangfuse.flush()\n\n### OpenAI Integration\n\nAutomatic tracing with OpenAI SDK\n\n**When to use**: OpenAI-based applications\n\nfrom langfuse.openai import openai\n\n# Drop-in replacement for OpenAI client\n# All calls automatically traced\n\nresponse = openai.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello\"}],\n    # Langfuse-specific parameters\n    name=\"greeting\",  # Trace name\n    session_id=\"session-123\",\n    user_id=\"user-456\",\n    tags=[\"test\"],\n    metadata={\"feature\": \"chat\"}\n)\n\n# Works with streaming\nstream = openai.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Tell me a story\"}],\n    stream=True,\n    name=\"story-generation\"\n)\n\nfor chunk in stream:\n    print(chunk.choices[0].delta.content, end=\"\")\n\n# Works with async\nimport asyncio\nfrom langfuse.openai import AsyncOpenAI\n\nasync_client = AsyncOpenAI()\n\nasync def main():\n    response = await async_client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[{\"role\": \"user\", \"content\": \"Hello\"}],\n        name=\"async-greeting\"\n    )\n\n### LangChain Integration\n\nTrace LangChain applications\n\n**When to use**: LangChain-based applications\n\nfrom langchain_openai import ChatOpenAI\nfrom langchain_core.prompts import ChatPromptTemplate\nfrom langfuse.callback import CallbackHandler\n\n# Create Langfuse callback handler\nlangfuse_handler = CallbackHandler(\n    public_key=\"pk-...\",\n    secret_key=\"sk-...\",\n    host=\"https:\u002F\u002Fcloud.langfuse.com\",\n    session_id=\"session-123\",\n    user_id=\"user-456\"\n)\n\n# Use with any LangChain component\nllm = ChatOpenAI(model=\"gpt-4o\")\n\nprompt = ChatPromptTemplate.from_messages([\n    (\"system\", \"You are a helpful assistant.\"),\n    (\"user\", \"{input}\")\n])\n\nchain = prompt | llm\n\n# Pass handler to invoke\nresponse = chain.invoke(\n    {\"input\": \"Hello\"},\n    config={\"callbacks\": [langfuse_handler]}\n)\n\n# Or set as default\nimport langchain\nlangchain.callbacks.manager.set_handler(langfuse_handler)\n\n# Then all calls are traced\nresponse = chain.invoke({\"input\": \"Hello\"})\n\n# Works with agents, retrievers, etc.\nfrom langchain.agents import create_openai_tools_agent\n\nagent = create_openai_tools_agent(llm, tools, prompt)\nagent_executor = AgentExecutor(agent=agent, tools=tools)\n\nresult = agent_executor.invoke(\n    {\"input\": \"What's the weather?\"},\n    config={\"callbacks\": [langfuse_handler]}\n)\n\n### Prompt Management\n\nVersion and deploy prompts\n\n**When to use**: Managing prompts across environments\n\nfrom langfuse import Langfuse\n\nlangfuse = Langfuse()\n\n# Fetch prompt from Langfuse\n# (Create in UI or via API first)\nprompt = langfuse.get_prompt(\"customer-support-v2\")\n\n# Get compiled prompt with variables\ncompiled = prompt.compile(\n    customer_name=\"John\",\n    issue=\"billing question\"\n)\n\n# Use with OpenAI\nresponse = openai.chat.completions.create(\n    model=prompt.config.get(\"model\", \"gpt-4o\"),\n    messages=compiled,\n    temperature=prompt.config.get(\"temperature\", 0.7)\n)\n\n# Link generation to prompt version\ntrace = langfuse.trace(name=\"support-chat\")\ngeneration = trace.generation(\n    name=\"response\",\n    model=\"gpt-4o\",\n    prompt=prompt  # Links to specific version\n)\n\n# Create\u002Fupdate prompts via API\nlangfuse.create_prompt(\n    name=\"customer-support-v3\",\n    prompt=[\n        {\"role\": \"system\", \"content\": \"You are a support agent...\"},\n        {\"role\": \"user\", \"content\": \"{{user_message}}\"}\n    ],\n    config={\n        \"model\": \"gpt-4o\",\n        \"temperature\": 0.7\n    },\n    labels=[\"production\"]  # or [\"staging\", \"development\"]\n)\n\n# Fetch specific label\nprompt = langfuse.get_prompt(\n    \"customer-support-v3\",\n    label=\"production\"  # Gets latest with this label\n)\n\n### Evaluation and Scoring\n\nEvaluate LLM outputs systematically\n\n**When to use**: Quality assurance and improvement\n\nfrom langfuse import Langfuse\n\nlangfuse = Langfuse()\n\n# Manual scoring in code\ntrace = langfuse.trace(name=\"qa-flow\")\n\n# After getting response\ntrace.score(\n    name=\"relevance\",\n    value=0.85,  # 0-1 scale\n    comment=\"Response addressed the question\"\n)\n\ntrace.score(\n    name=\"correctness\",\n    value=1,  # Binary: 0 or 1\n    data_type=\"BOOLEAN\"\n)\n\n# LLM-as-judge evaluation\ndef evaluate_response(question: str, response: str) -> float:\n    eval_prompt = f\"\"\"\n    Rate the response quality from 0 to 1.\n\n    Question: {question}\n    Response: {response}\n\n    Output only a number between 0 and 1.\n    \"\"\"\n\n    result = openai.chat.completions.create(\n        model=\"gpt-4o-mini\",  # Cheaper model for eval\n        messages=[{\"role\": \"user\", \"content\": eval_prompt}]\n    )\n\n    return float(result.choices[0].message.content.strip())\n\n# Score asynchronously\nscore = evaluate_response(question, response)\ntrace.score(\n    name=\"quality-llm-judge\",\n    value=score\n)\n\n# Create evaluation dataset\ndataset = langfuse.create_dataset(name=\"support-qa-v1\")\n\n# Add items to dataset\nlangfuse.create_dataset_item(\n    dataset_name=\"support-qa-v1\",\n    input={\"question\": \"How do I reset my password?\"},\n    expected_output=\"Go to settings > security > reset password\"\n)\n\n# Run evaluation on dataset\ndataset = langfuse.get_dataset(\"support-qa-v1\")\n\nfor item in dataset.items:\n    # Generate response\n    response = generate_response(item.input[\"question\"])\n\n    # Link to dataset item\n    trace = langfuse.trace(name=\"eval-run\")\n    trace.generation(\n        name=\"response\",\n        input=item.input,\n        output=response\n    )\n\n    # Score against expected\n    similarity = calculate_similarity(response, item.expected_output)\n    trace.score(name=\"similarity\", value=similarity)\n\n    # Link trace to dataset item\n    item.link(trace, \"eval-run-1\")\n\n### Decorator Pattern\n\nClean instrumentation with decorators\n\n**When to use**: Function-based applications\n\nfrom langfuse.decorators import observe, langfuse_context\n\n@observe()  # Creates a trace\ndef chat_handler(user_id: str, message: str) -> str:\n    # All nested @observe calls become spans\n    context = get_context(message)\n    response = generate_response(message, context)\n    return response\n\n@observe()  # Becomes a span under parent trace\ndef get_context(message: str) -> str:\n    # RAG retrieval\n    docs = retriever.get_relevant_documents(message)\n    return \"\\n\".join([d.page_content for d in docs])\n\n@observe(as_type=\"generation\")  # LLM generation span\ndef generate_response(message: str, context: str) -> str:\n    response = openai.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[\n            {\"role\": \"system\", \"content\": f\"Context: {context}\"},\n            {\"role\": \"user\", \"content\": message}\n        ]\n    )\n    return response.choices[0].message.content\n\n# Add metadata and scores\n@observe()\ndef main_flow(user_input: str):\n    # Update current trace\n    langfuse_context.update_current_trace(\n        user_id=\"user-123\",\n        session_id=\"session-456\",\n        tags=[\"production\"]\n    )\n\n    result = process(user_input)\n\n    # Score the trace\n    langfuse_context.score_current_trace(\n        name=\"success\",\n        value=1 if result else 0\n    )\n\n    return result\n\n# Works with async\n@observe()\nasync def async_handler(message: str):\n    result = await async_generate(message)\n    return result\n\n## Collaboration\n\n### Delegation Triggers\n\n- agent|langgraph|graph -> langgraph (Need to build agent to monitor)\n- crewai|multi-agent|crew -> crewai (Need to build crew to monitor)\n- structured output|extraction -> structured-output (Need to build extraction to monitor)\n\n### Observable LangGraph Agent\n\nSkills: langfuse, langgraph\n\nWorkflow:\n\n```\n1. Build agent with LangGraph\n2. Add Langfuse callback handler\n3. Trace all LLM calls and tool uses\n4. Score outputs for quality\n5. Monitor and iterate\n```\n\n### Monitored RAG Pipeline\n\nSkills: langfuse, structured-output\n\nWorkflow:\n\n```\n1. Build RAG with retrieval and generation\n2. Trace retrieval and LLM calls\n3. Score relevance and accuracy\n4. Track costs and latency\n5. Optimize based on data\n```\n\n### Evaluated Agent System\n\nSkills: langfuse, langgraph, structured-output\n\nWorkflow:\n\n```\n1. Build agent with structured outputs\n2. Create evaluation dataset\n3. Run evaluations with traces\n4. Compare prompt versions\n5. Deploy best performers\n```\n\n## Related Skills\n\nWorks well with: `langgraph`, `crewai`, `structured-output`, `autonomous-agents`\n\n## When to Use\n\n- User mentions or implies: langfuse\n- User mentions or implies: llm observability\n- User mentions or implies: llm tracing\n- User mentions or implies: prompt management\n- User mentions or implies: llm evaluation\n- User mentions or implies: monitor llm\n- User mentions or implies: debug llm\n",{"name":5,"risk":41,"source":42,"date_added":43,"description":7},"unknown","vibeship-spawner-skills (Apache 2.0)",1772150400000,"main","a50a3aa6b39fb0e1b7db88bef1bfc9c93606e126","modules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Flangfuse",0,"https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant","https:\u002F\u002Fgithub.com\u002FSam-06060","Mobile Application Developer | Kotlin | Flutter, KMP, Swfit, Android,Swift UI | Building at Saar It Resources | Creator of Jarvis (AI Assistant)","2026-06-23 05:13:38",{"source":8,"skillName":5,"commands":53,"prompt":63,"zipAvailable":64},[54,57,60],{"tool":55,"command":56},"npx","npx skills add Sam-06060\u002Fjarvis-assistant --skill langfuse",{"tool":58,"command":59},"bunx","bunx skills add Sam-06060\u002Fjarvis-assistant --skill langfuse",{"tool":61,"command":62},"pnpm","pnpm dlx skills add Sam-06060\u002Fjarvis-assistant --skill langfuse","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Flangfuse\u002FSKILL.md",true,[],[67,80,97,123,142,168],{"skillNo":68,"name":69,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":70,"categories":71,"keywords":77,"publishedAt":36},"01M6GDRVS6FXX3ESH0C9DW01C0","test_skill","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fscratch\u002Ftest_skill\u002FSKILL.md",[72],{"categoryNo":73,"categoryName":74,"taxonomyType":75,"taxonomyTypeDesc":76},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",[78,79],"Test Skill","test",{"skillNo":81,"name":82,"nameZh":6,"description":83,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":36},"01M6GDRQ40HX22MA6XWC31RHHK","zustand-store-ts","Create Zustand stores following established patterns with proper TypeScript types and middleware.","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fzustand-store-ts\u002FSKILL.md",[],[87,88,89,90,91,92,93,94,95,96],"zustand store ts","zustand","store","with","types","middleware","patterns","community","create","typescript",{"skillNo":98,"name":99,"nameZh":6,"description":100,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":101,"categories":102,"keywords":106,"publishedAt":36},"01M6GDRKEA0WE670SPFWX5GKCN","zoom-automation","Automate Zoom meeting creation, management, recordings, webinars, and participant tracking via Rube MCP (Composio). Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fzoom-automation\u002FSKILL.md",[103],{"categoryNo":104,"categoryName":105,"taxonomyType":75,"taxonomyTypeDesc":76},"01JBSEEDCAT000000000000023","Skill与智能体开发",[107,108,109,110,111,112,113,114,115,116,117,118,119,120,94,121,23,122],"automation","zoom automation","zoom","first","Tools","search","MCP","via","meeting","automate","critical","creation","webinars","composio","recordings","tracking",{"skillNo":124,"name":125,"nameZh":6,"description":126,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":127,"categories":128,"keywords":130,"publishedAt":36},"01M6GDRFS1452PC71JQ38S24HR","zoho-crm-automation","Automate Zoho CRM tasks via Rube MCP (Composio): create\u002Fupdate records, search contacts, manage leads, and convert leads. Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fzoho-crm-automation\u002FSKILL.md",[129],{"categoryNo":104,"categoryName":105,"taxonomyType":75,"taxonomyTypeDesc":76},[131,107,132,133,134,135,136,94,112,137,120,138,117,116,111,139,95,140,113,141],"crm automation","zoho crm","crm","zoho crm automation","zoho","leads","contacts","records","update","tasks","manage",{"skillNo":143,"name":144,"nameZh":6,"description":145,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":146,"categories":147,"keywords":151,"publishedAt":36},"01M6GDRCKQBAVTGHX5XBH26KPZ","zod-validation-expert","Expert in Zod — TypeScript-first schema validation. Covers parsing, custom errors, refinements, type inference, and integration with React Hook Form, Next.js, and tRPC.","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fzod-validation-expert\u002FSKILL.md",[148],{"categoryNo":149,"categoryName":150,"taxonomyType":75,"taxonomyTypeDesc":76},"01JBSEEDCAT000000000000009","前端开发",[21,152,153,154,155,96,156,157,158,159,160,161,90,162,163,164,165,94,166,167],"zod","zod validation expert","validation","react","safe","parsing","errors","schema validation","react hook form","hook","trpc","form","type","next","schema","Next.js",{"skillNo":169,"name":170,"nameZh":6,"description":171,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":172,"categories":173,"keywords":175,"publishedAt":36},"01M6GDR658B08K4MY0RWD4A6EH","zeroize-audit","Detects missing zeroization of sensitive data in source code and identifies zeroization removed by compiler optimizations, with assembly-level analysis, and control-flow verification. Use for auditing C\u002FC++\u002FRust code handling secrets, keys, passwords, or other sensitive data.","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fzeroize-audit\u002FSKILL.md",[174],{"categoryNo":104,"categoryName":105,"taxonomyType":75,"taxonomyTypeDesc":76},[176,177,178,179,180,181,182,183,94,184,185,186,187,188,189,190,191,192,193,113],"zeroize audit","audit","write","passwords","get","project","c++","activate","assembly","read","Bash","secrets","analysis","data","flow","verification","code","rust"]