[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6QF00Q05P4ABB87TDP4TZY8":3,"skill-same:01M6QF00Q05P4ABB87TDP4TZY8":88},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":55,"commitSha":56,"skillDirectory":57,"forksCount":14,"repositoryHtmlUrl":58,"repositoryHomepageUrl":59,"ownerHtmlUrl":60,"ownerBlogUrl":61,"ownerDescription":62,"updatedAt":63,"install":64,"resources":77},"01M6QF00Q05P4ABB87TDP4TZY8","langsmith-observability",null,"LLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.","Tashima-Tarsh\u002FDisha6.6","Tashima-Tarsh","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F269906498?v=4","01M6PBH98X03XERH0HFT4PFB1P","NITISH KUMAR","01M6PBH9DA0M6KM3Z12P848F0K",0,"MIT","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fkal-langsmith-observability\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"observability","langsmith observability","LangSmith","datasets","outputs","for","LLM Applications","systematic","systems","platform","observability platform","testing","pipelines","use","model","applications","Evaluation","monitoring","tracing","inference","2026-11-10 10:12:30",false,"---\nname: langsmith-observability\ndescription: LLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  category: ml-inference-and-ops\n  maintainer: Kalaris Labs\n  tags: Observability, LangSmith, Tracing, Evaluation, Monitoring, Debugging, Testing, LLM Ops, Production\n  dependencies: langsmith>=0.2.0\n---\n\n# LangSmith - LLM Observability Platform\n\nDevelopment platform for debugging, evaluating, and monitoring language models and AI applications.\n\n## When to use LangSmith\n\n**Use LangSmith when:**\n- Debugging LLM application issues (prompts, chains, agents)\n- Evaluating model outputs systematically against datasets\n- Monitoring production LLM systems\n- Building regression testing for AI features\n- Analyzing latency, token usage, and costs\n- Collaborating on prompt engineering\n\n**Key features:**\n- **Tracing**: Capture inputs, outputs, latency for all LLM calls\n- **Evaluation**: Systematic testing with built-in and custom evaluators\n- **Datasets**: Create test sets from production traces or manually\n- **Monitoring**: Track metrics, errors, and costs in production\n- **Integrations**: Works with OpenAI, Anthropic, LangChain, LlamaIndex\n\n**Use alternatives instead:**\n- **Weights & Biases**: Deep learning experiment tracking, model training\n- **MLflow**: General ML lifecycle, model registry focus\n- **Arize\u002FWhyLabs**: ML monitoring, data drift detection\n\n## Quick start\n\n### Installation\n\n```bash\npip install langsmith\n\n# Set environment variables\nexport LANGSMITH_API_KEY=\"your-api-key\"\nexport LANGSMITH_TRACING=true\n```\n\n### Basic tracing with @traceable\n\n```python\nfrom langsmith import traceable\nfrom openai import OpenAI\n\nclient = OpenAI()\n\n@traceable\ndef generate_response(prompt: str) -> str:\n    response = client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[{\"role\": \"user\", \"content\": prompt}]\n    )\n    return response.choices[0].message.content\n\n# Automatically traced to LangSmith\nresult = generate_response(\"What is machine learning?\")\n```\n\n### OpenAI wrapper (automatic tracing)\n\n```python\nfrom langsmith.wrappers import wrap_openai\nfrom openai import OpenAI\n\n# Wrap client for automatic tracing\nclient = wrap_openai(OpenAI())\n\n# All calls automatically traced\nresponse = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n```\n\n## Core concepts\n\n### Runs and traces\n\nA **run** is a single execution unit (LLM call, chain, tool). Runs form hierarchical **traces** showing the full execution flow.\n\n```python\nfrom langsmith import traceable\n\n@traceable(run_type=\"chain\")\ndef process_query(query: str) -> str:\n    # Parent run\n    context = retrieve_context(query)  # Child run\n    response = generate_answer(query, context)  # Child run\n    return response\n\n@traceable(run_type=\"retriever\")\ndef retrieve_context(query: str) -> list:\n    return vector_store.search(query)\n\n@traceable(run_type=\"llm\")\ndef generate_answer(query: str, context: list) -> str:\n    return llm.invoke(f\"Context: {context}\\n\\nQuestion: {query}\")\n```\n\n### Projects\n\nProjects organize related runs. Set via environment or code:\n\n```python\nimport os\nos.environ[\"LANGSMITH_PROJECT\"] = \"my-project\"\n\n# Or per-function\n@traceable(project_name=\"my-project\")\ndef my_function():\n    pass\n```\n\n## Client API\n\n```python\nfrom langsmith import Client\n\nclient = Client()\n\n# List runs\nruns = list(client.list_runs(\n    project_name=\"my-project\",\n    filter='eq(status, \"success\")',\n    limit=100\n))\n\n# Get run details\nrun = client.read_run(run_id=\"...\")\n\n# Create feedback\nclient.create_feedback(\n    run_id=\"...\",\n    key=\"correctness\",\n    score=0.9,\n    comment=\"Good answer\"\n)\n```\n\n## Datasets and evaluation\n\n### Create dataset\n\n```python\nfrom langsmith import Client\n\nclient = Client()\n\n# Create dataset\ndataset = client.create_dataset(\"qa-test-set\", description=\"QA evaluation\")\n\n# Add examples\nclient.create_examples(\n    inputs=[\n        {\"question\": \"What is Python?\"},\n        {\"question\": \"What is ML?\"}\n    ],\n    outputs=[\n        {\"answer\": \"A programming language\"},\n        {\"answer\": \"Machine learning\"}\n    ],\n    dataset_id=dataset.id\n)\n```\n\n### Run evaluation\n\n```python\nfrom langsmith import evaluate\n\ndef my_model(inputs: dict) -> dict:\n    # Your model logic\n    return {\"answer\": generate_answer(inputs[\"question\"])}\n\ndef correctness_evaluator(run, example):\n    prediction = run.outputs[\"answer\"]\n    reference = example.outputs[\"answer\"]\n    score = 1.0 if reference.lower() in prediction.lower() else 0.0\n    return {\"key\": \"correctness\", \"score\": score}\n\nresults = evaluate(\n    my_model,\n    data=\"qa-test-set\",\n    evaluators=[correctness_evaluator],\n    experiment_prefix=\"v1\"\n)\n\nprint(f\"Average score: {results.aggregate_metrics['correctness']}\")\n```\n\n### Built-in evaluators\n\n```python\nfrom langsmith.evaluation import LangChainStringEvaluator\n\n# Use LangChain evaluators\nresults = evaluate(\n    my_model,\n    data=\"qa-test-set\",\n    evaluators=[\n        LangChainStringEvaluator(\"qa\"),\n        LangChainStringEvaluator(\"cot_qa\")\n    ]\n)\n```\n\n## Advanced tracing\n\n### Tracing context\n\n```python\nfrom langsmith import tracing_context\n\nwith tracing_context(\n    project_name=\"experiment-1\",\n    tags=[\"production\", \"v2\"],\n    metadata={\"version\": \"2.0\"}\n):\n    # All traceable calls inherit context\n    result = my_function()\n```\n\n### Manual runs\n\n```python\nfrom langsmith import trace\n\nwith trace(\n    name=\"custom_operation\",\n    run_type=\"tool\",\n    inputs={\"query\": \"test\"}\n) as run:\n    result = do_something()\n    run.end(outputs={\"result\": result})\n```\n\n### Process inputs\u002Foutputs\n\n```python\ndef sanitize_inputs(inputs: dict) -> dict:\n    if \"password\" in inputs:\n        inputs[\"password\"] = \"***\"\n    return inputs\n\n@traceable(process_inputs=sanitize_inputs)\ndef login(username: str, password: str):\n    return authenticate(username, password)\n```\n\n### Sampling\n\n```python\nimport os\nos.environ[\"LANGSMITH_TRACING_SAMPLING_RATE\"] = \"0.1\"  # 10% sampling\n```\n\n## LangChain integration\n\n```python\nfrom langchain_openai import ChatOpenAI\nfrom langchain_core.prompts import ChatPromptTemplate\n\n# Tracing enabled automatically with LANGSMITH_TRACING=true\nllm = ChatOpenAI(model=\"gpt-4o\")\nprompt = ChatPromptTemplate.from_messages([\n    (\"system\", \"You are a helpful assistant.\"),\n    (\"user\", \"{input}\")\n])\n\nchain = prompt | llm\n\n# All chain runs traced automatically\nresponse = chain.invoke({\"input\": \"Hello!\"})\n```\n\n## Production monitoring\n\n### Hub prompts\n\n```python\nfrom langsmith import Client\n\nclient = Client()\n\n# Pull prompt from hub\nprompt = client.pull_prompt(\"my-org\u002Fqa-prompt\")\n\n# Use in application\nresult = prompt.invoke({\"question\": \"What is AI?\"})\n```\n\n### Async client\n\n```python\nfrom langsmith import AsyncClient\n\nasync def main():\n    client = AsyncClient()\n\n    runs = []\n    async for run in client.list_runs(project_name=\"my-project\"):\n        runs.append(run)\n\n    return runs\n```\n\n### Feedback collection\n\n```python\nfrom langsmith import Client\n\nclient = Client()\n\n# Collect user feedback\ndef record_feedback(run_id: str, user_rating: int, comment: str = None):\n    client.create_feedback(\n        run_id=run_id,\n        key=\"user_rating\",\n        score=user_rating \u002F 5.0,  # Normalize to 0-1\n        comment=comment\n    )\n\n# In your application\nrecord_feedback(run_id=\"...\", user_rating=4, comment=\"Helpful response\")\n```\n\n## Testing integration\n\n### Pytest integration\n\n```python\nfrom langsmith import test\n\n@test\ndef test_qa_accuracy():\n    result = my_qa_function(\"What is Python?\")\n    assert \"programming\" in result.lower()\n```\n\n### Evaluation in CI\u002FCD\n\n```python\nfrom langsmith import evaluate\n\ndef run_evaluation():\n    results = evaluate(\n        my_model,\n        data=\"regression-test-set\",\n        evaluators=[accuracy_evaluator]\n    )\n\n    # Fail CI if accuracy drops\n    assert results.aggregate_metrics[\"accuracy\"] >= 0.9, \\\n        f\"Accuracy {results.aggregate_metrics['accuracy']} below threshold\"\n```\n\n## Best practices\n\n1. **Structured naming** - Use consistent project\u002Frun naming conventions\n2. **Add metadata** - Include version, environment, user info\n3. **Sample in production** - Use sampling rate to control volume\n4. **Create datasets** - Build test sets from interesting production cases\n5. **Automate evaluation** - Run evaluations in CI\u002FCD pipelines\n6. **Monitor costs** - Track token usage and latency trends\n\n## Common issues\n\n**Traces not appearing:**\n```python\nimport os\n# Ensure tracing is enabled\nos.environ[\"LANGSMITH_TRACING\"] = \"true\"\nos.environ[\"LANGSMITH_API_KEY\"] = \"your-key\"\n\n# Verify connection\nfrom langsmith import Client\nclient = Client()\nprint(client.list_projects())  # Should work\n```\n\n**High latency from tracing:**\n```python\n# Enable background batching (default)\nfrom langsmith import Client\nclient = Client(auto_batch_tracing=True)\n\n# Or use sampling\nos.environ[\"LANGSMITH_TRACING_SAMPLING_RATE\"] = \"0.1\"\n```\n\n**Large payloads:**\n```python\n# Hide sensitive\u002Flarge fields\n@traceable(\n    process_inputs=lambda x: {k: v for k, v in x.items() if k != \"large_field\"}\n)\ndef my_function(data):\n    pass\n```\n\n## References\n\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Custom evaluators, distributed tracing, hub prompts\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues, debugging, performance\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fdocs.smith.langchain.com\n- **Python SDK**: https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flangsmith-sdk\n- **Web App**: https:\u002F\u002Fsmith.langchain.com\n- **Version**: 0.2.0+\n- **License**: MIT\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm hardware, framework and server versions, model format, and expected load.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Serve or log a single request or run end to end before scaling.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Measure latency, throughput and output correctness against a reference; check resource usage and costs.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| The server fails to start or OOMs | Check model size versus memory, quantization and parallelism settings. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Do not expose services or credentials publicly; confirm cloud costs before provisioning.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `phoenix-observability`: Open-source AI observability platform for LLM tracing, evaluation, and monitoring.\n- `llamaindex`: Data framework for building LLM applications with RAG.\n- `langchain`: Framework for building LLM-powered applications with agents, chains, and RAG.\n","# LangSmith - LLM Observability Platform\n\nDevelopment platform for debugging, evaluating, and monitoring language models and AI applications.\n\n## When to use LangSmith\n\n**Use LangSmith when:**\n- Debugging LLM application issues (prompts, chains, agents)\n- Evaluating model outputs systematically against datasets\n- Monitoring production LLM systems\n- Building regression testing for AI features\n- Analyzing latency, token usage, and costs\n- Collaborating on prompt engineering\n\n**Key features:**\n- **Tracing**: Capture inputs, outputs, latency for all LLM calls\n- **Evaluation**: Systematic testing with built-in and custom evaluators\n- **Datasets**: Create test sets from production traces or manually\n- **Monitoring**: Track metrics, errors, and costs in production\n- **Integrations**: Works with OpenAI, Anthropic, LangChain, LlamaIndex\n\n**Use alternatives instead:**\n- **Weights & Biases**: Deep learning experiment tracking, model training\n- **MLflow**: General ML lifecycle, model registry focus\n- **Arize\u002FWhyLabs**: ML monitoring, data drift detection\n\n## Quick start\n\n### Installation\n\n```bash\npip install langsmith\n\n# Set environment variables\nexport LANGSMITH_API_KEY=\"your-api-key\"\nexport LANGSMITH_TRACING=true\n```\n\n### Basic tracing with @traceable\n\n```python\nfrom langsmith import traceable\nfrom openai import OpenAI\n\nclient = OpenAI()\n\n@traceable\ndef generate_response(prompt: str) -> str:\n    response = client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[{\"role\": \"user\", \"content\": prompt}]\n    )\n    return response.choices[0].message.content\n\n# Automatically traced to LangSmith\nresult = generate_response(\"What is machine learning?\")\n```\n\n### OpenAI wrapper (automatic tracing)\n\n```python\nfrom langsmith.wrappers import wrap_openai\nfrom openai import OpenAI\n\n# Wrap client for automatic tracing\nclient = wrap_openai(OpenAI())\n\n# All calls automatically traced\nresponse = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n```\n\n## Core concepts\n\n### Runs and traces\n\nA **run** is a single execution unit (LLM call, chain, tool). Runs form hierarchical **traces** showing the full execution flow.\n\n```python\nfrom langsmith import traceable\n\n@traceable(run_type=\"chain\")\ndef process_query(query: str) -> str:\n    # Parent run\n    context = retrieve_context(query)  # Child run\n    response = generate_answer(query, context)  # Child run\n    return response\n\n@traceable(run_type=\"retriever\")\ndef retrieve_context(query: str) -> list:\n    return vector_store.search(query)\n\n@traceable(run_type=\"llm\")\ndef generate_answer(query: str, context: list) -> str:\n    return llm.invoke(f\"Context: {context}\\n\\nQuestion: {query}\")\n```\n\n### Projects\n\nProjects organize related runs. Set via environment or code:\n\n```python\nimport os\nos.environ[\"LANGSMITH_PROJECT\"] = \"my-project\"\n\n# Or per-function\n@traceable(project_name=\"my-project\")\ndef my_function():\n    pass\n```\n\n## Client API\n\n```python\nfrom langsmith import Client\n\nclient = Client()\n\n# List runs\nruns = list(client.list_runs(\n    project_name=\"my-project\",\n    filter='eq(status, \"success\")',\n    limit=100\n))\n\n# Get run details\nrun = client.read_run(run_id=\"...\")\n\n# Create feedback\nclient.create_feedback(\n    run_id=\"...\",\n    key=\"correctness\",\n    score=0.9,\n    comment=\"Good answer\"\n)\n```\n\n## Datasets and evaluation\n\n### Create dataset\n\n```python\nfrom langsmith import Client\n\nclient = Client()\n\n# Create dataset\ndataset = client.create_dataset(\"qa-test-set\", description=\"QA evaluation\")\n\n# Add examples\nclient.create_examples(\n    inputs=[\n        {\"question\": \"What is Python?\"},\n        {\"question\": \"What is ML?\"}\n    ],\n    outputs=[\n        {\"answer\": \"A programming language\"},\n        {\"answer\": \"Machine learning\"}\n    ],\n    dataset_id=dataset.id\n)\n```\n\n### Run evaluation\n\n```python\nfrom langsmith import evaluate\n\ndef my_model(inputs: dict) -> dict:\n    # Your model logic\n    return {\"answer\": generate_answer(inputs[\"question\"])}\n\ndef correctness_evaluator(run, example):\n    prediction = run.outputs[\"answer\"]\n    reference = example.outputs[\"answer\"]\n    score = 1.0 if reference.lower() in prediction.lower() else 0.0\n    return {\"key\": \"correctness\", \"score\": score}\n\nresults = evaluate(\n    my_model,\n    data=\"qa-test-set\",\n    evaluators=[correctness_evaluator],\n    experiment_prefix=\"v1\"\n)\n\nprint(f\"Average score: {results.aggregate_metrics['correctness']}\")\n```\n\n### Built-in evaluators\n\n```python\nfrom langsmith.evaluation import LangChainStringEvaluator\n\n# Use LangChain evaluators\nresults = evaluate(\n    my_model,\n    data=\"qa-test-set\",\n    evaluators=[\n        LangChainStringEvaluator(\"qa\"),\n        LangChainStringEvaluator(\"cot_qa\")\n    ]\n)\n```\n\n## Advanced tracing\n\n### Tracing context\n\n```python\nfrom langsmith import tracing_context\n\nwith tracing_context(\n    project_name=\"experiment-1\",\n    tags=[\"production\", \"v2\"],\n    metadata={\"version\": \"2.0\"}\n):\n    # All traceable calls inherit context\n    result = my_function()\n```\n\n### Manual runs\n\n```python\nfrom langsmith import trace\n\nwith trace(\n    name=\"custom_operation\",\n    run_type=\"tool\",\n    inputs={\"query\": \"test\"}\n) as run:\n    result = do_something()\n    run.end(outputs={\"result\": result})\n```\n\n### Process inputs\u002Foutputs\n\n```python\ndef sanitize_inputs(inputs: dict) -> dict:\n    if \"password\" in inputs:\n        inputs[\"password\"] = \"***\"\n    return inputs\n\n@traceable(process_inputs=sanitize_inputs)\ndef login(username: str, password: str):\n    return authenticate(username, password)\n```\n\n### Sampling\n\n```python\nimport os\nos.environ[\"LANGSMITH_TRACING_SAMPLING_RATE\"] = \"0.1\"  # 10% sampling\n```\n\n## LangChain integration\n\n```python\nfrom langchain_openai import ChatOpenAI\nfrom langchain_core.prompts import ChatPromptTemplate\n\n# Tracing enabled automatically with LANGSMITH_TRACING=true\nllm = ChatOpenAI(model=\"gpt-4o\")\nprompt = ChatPromptTemplate.from_messages([\n    (\"system\", \"You are a helpful assistant.\"),\n    (\"user\", \"{input}\")\n])\n\nchain = prompt | llm\n\n# All chain runs traced automatically\nresponse = chain.invoke({\"input\": \"Hello!\"})\n```\n\n## Production monitoring\n\n### Hub prompts\n\n```python\nfrom langsmith import Client\n\nclient = Client()\n\n# Pull prompt from hub\nprompt = client.pull_prompt(\"my-org\u002Fqa-prompt\")\n\n# Use in application\nresult = prompt.invoke({\"question\": \"What is AI?\"})\n```\n\n### Async client\n\n```python\nfrom langsmith import AsyncClient\n\nasync def main():\n    client = AsyncClient()\n\n    runs = []\n    async for run in client.list_runs(project_name=\"my-project\"):\n        runs.append(run)\n\n    return runs\n```\n\n### Feedback collection\n\n```python\nfrom langsmith import Client\n\nclient = Client()\n\n# Collect user feedback\ndef record_feedback(run_id: str, user_rating: int, comment: str = None):\n    client.create_feedback(\n        run_id=run_id,\n        key=\"user_rating\",\n        score=user_rating \u002F 5.0,  # Normalize to 0-1\n        comment=comment\n    )\n\n# In your application\nrecord_feedback(run_id=\"...\", user_rating=4, comment=\"Helpful response\")\n```\n\n## Testing integration\n\n### Pytest integration\n\n```python\nfrom langsmith import test\n\n@test\ndef test_qa_accuracy():\n    result = my_qa_function(\"What is Python?\")\n    assert \"programming\" in result.lower()\n```\n\n### Evaluation in CI\u002FCD\n\n```python\nfrom langsmith import evaluate\n\ndef run_evaluation():\n    results = evaluate(\n        my_model,\n        data=\"regression-test-set\",\n        evaluators=[accuracy_evaluator]\n    )\n\n    # Fail CI if accuracy drops\n    assert results.aggregate_metrics[\"accuracy\"] >= 0.9, \\\n        f\"Accuracy {results.aggregate_metrics['accuracy']} below threshold\"\n```\n\n## Best practices\n\n1. **Structured naming** - Use consistent project\u002Frun naming conventions\n2. **Add metadata** - Include version, environment, user info\n3. **Sample in production** - Use sampling rate to control volume\n4. **Create datasets** - Build test sets from interesting production cases\n5. **Automate evaluation** - Run evaluations in CI\u002FCD pipelines\n6. **Monitor costs** - Track token usage and latency trends\n\n## Common issues\n\n**Traces not appearing:**\n```python\nimport os\n# Ensure tracing is enabled\nos.environ[\"LANGSMITH_TRACING\"] = \"true\"\nos.environ[\"LANGSMITH_API_KEY\"] = \"your-key\"\n\n# Verify connection\nfrom langsmith import Client\nclient = Client()\nprint(client.list_projects())  # Should work\n```\n\n**High latency from tracing:**\n```python\n# Enable background batching (default)\nfrom langsmith import Client\nclient = Client(auto_batch_tracing=True)\n\n# Or use sampling\nos.environ[\"LANGSMITH_TRACING_SAMPLING_RATE\"] = \"0.1\"\n```\n\n**Large payloads:**\n```python\n# Hide sensitive\u002Flarge fields\n@traceable(\n    process_inputs=lambda x: {k: v for k, v in x.items() if k != \"large_field\"}\n)\ndef my_function(data):\n    pass\n```\n\n## References\n\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Custom evaluators, distributed tracing, hub prompts\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues, debugging, performance\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fdocs.smith.langchain.com\n- **Python SDK**: https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flangsmith-sdk\n- **Web App**: https:\u002F\u002Fsmith.langchain.com\n- **Version**: 0.2.0+\n- **License**: MIT\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm hardware, framework and server versions, model format, and expected load.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Serve or log a single request or run end to end before scaling.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Measure latency, throughput and output correctness against a reference; check resource usage and costs.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| The server fails to start or OOMs | Check model size versus memory, quantization and parallelism settings. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Do not expose services or credentials publicly; confirm cloud costs before provisioning.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `phoenix-observability`: Open-source AI observability platform for LLM tracing, evaluation, and monitoring.\n- `llamaindex`: Data framework for building LLM applications with RAG.\n- `langchain`: Framework for building LLM-powered applications with agents, chains, and RAG.\n",{"name":5,"license":15,"metadata":49,"description":7},{"tags":50,"version":51,"category":52,"maintainer":53,"dependencies":54},"Observability, LangSmith, Tracing, Evaluation, Monitoring, Debugging, Testing, LLM Ops, Production","1.0.0","ml-inference-and-ops","Kalaris Labs","langsmith>=0.2.0","main","0da4485785ee599388900be50db1635489d53056",".agents\u002Fskills\u002Fkal-langsmith-observability","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6","https:\u002F\u002Fdisha6.6.thenitishkr.in\u002F","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh","www.thenitishkr","thenitishkr | AI • Cyber-security • Research\r\n","2026-11-08 03:33:25",{"source":8,"skillName":5,"commands":65,"prompt":75,"zipAvailable":76},[66,69,72],{"tool":67,"command":68},"npx","npx skills add Tashima-Tarsh\u002FDisha6.6 --skill langsmith-observability",{"tool":70,"command":71},"bunx","bunx skills add Tashima-Tarsh\u002FDisha6.6 --skill langsmith-observability",{"tool":73,"command":74},"pnpm","pnpm dlx skills add Tashima-Tarsh\u002FDisha6.6 --skill langsmith-observability","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fkal-langsmith-observability\u002FSKILL.md",true,[78,84],{"path":79,"resourceType":80,"resourceTypeDesc":81,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Fadvanced-usage.md","REFERENCE","参考文档",12006,"https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fkal-langsmith-observability\u002Freferences\u002Fadvanced-usage.md",{"path":85,"resourceType":80,"resourceTypeDesc":81,"sizeBytes":86,"sourceHtmlUrl":87},"references\u002Ftroubleshooting.md",10983,"https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fkal-langsmith-observability\u002Freferences\u002Ftroubleshooting.md",[89,117,140,167,192,216],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":93,"categories":94,"keywords":98,"publishedAt":44},"01M6QRD07D4TGNDK6VJ4EHF8BV","youtube-full","Use when the user needs YouTube transcripts, video search, channel browsing, playlist extraction, or content monitoring. Trigger phrases: 'get the transcript for', 'search YouTube for', 'what are the latest videos on', 'list this playlist', 'monitor this channel', or any request involving a YouTube URL, video ID, or @handle. Do NOT use for downloading video or audio files, YouTube engagement data (likes, comments), or private\u002Fage-restricted videos.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fyoutube-full\u002FSKILL.md",[95],{"categoryNo":96,"categoryName":97,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体",[99,100,101,102,103,41,104,105,106,107,108,109,110,111,112,113,29,114,115,116],"youtube full","youtube","full","transcript","search","data","Video","Audio","list","files","do","get","age","trigger","URL","browsing","request","youtube transcripts",{"skillNo":118,"name":119,"nameZh":6,"description":120,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":121,"categories":122,"keywords":123,"publishedAt":44},"01M6QRCJ3112VYPA97YG7KGMC2","x-twitter-growth","X\u002FTwitter growth engine for building audience, crafting viral content, and analyzing engagement. Use when the user wants to grow on X\u002FTwitter, write tweets or threads, analyze their X profile, research competitors on X, plan a posting strategy, or optimize engagement. Complements social-content (generic multi-platform) with X-specific depth: algorithm mechanics, thread engineering, reply strategy, profile optimization, and competitive intelligence via web search.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fx-twitter-growth\u002FSKILL.md",[],[124,125,126,127,103,128,129,130,131,132,133,134,135,136,37,33,137,138,139,29],"twitter","x twitter growth","growth","research","content","engineering","plan","web search","competitors","analyze","engine","write","strategy","grow","posting","competitive",{"skillNo":141,"name":142,"nameZh":6,"description":143,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":144,"categories":145,"keywords":149,"publishedAt":44},"01M6QRBN3EHQNJY42RERP7A149","write-a-skill","Create new agent skills with proper structure, progressive disclosure, and bundled resources. Use when user wants to create, write, build, or author a new skill.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fwrite-a-skill\u002FSKILL.md",[146],{"categoryNo":147,"categoryName":148,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[150,135,151,152,153,154,155,156,37,157,158,55,159,160,161,162,163,164,165,166],"write a skill","github","create","https","productivity","matt pocock","direct","example","build","new skill","structure","new agent","resources","with","tree","progressive disclosure","author",{"skillNo":168,"name":169,"nameZh":6,"description":170,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":171,"categories":172,"keywords":173,"publishedAt":44},"01M6QRB6G8P33PNGAF9Y69EJ9T","wrap-up","Close out a launched Claude Managed Agent — recap every primitive the founder now owns, regenerate the single-file overview page, and suggest the next 1-2 upgrades. Use when the user says \"wrap up\", \"close this out\", \"what do I own now\", \"give me the summary\", \"recap the agent\", or when the orchestrator routes phase=wrap-up. primitives_inventory.py tables everything owned (agent, environment, session, memory, outcome, deployment); overview_page.py regenerates a self-contained .\u002Fmy-agent\u002Fagent-overview.html; upgrade_suggester.py ranks the next moves from recorded deferrals plus standing hardening steps. Companion to run-without-you; the last stop before phase=done.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fwrap-up\u002FSKILL.md",[],[174,175,176,177,178,179,180,181,182,37,183,109,184,185,186,187,188,189,190,191],"wrap up","up","wrap","gemini","deployment","run","file","hardening","closeout","stop","upgrade","founder","close","companion","you","tables","directions","phase",{"skillNo":193,"name":194,"nameZh":6,"description":195,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":196,"categories":197,"keywords":199,"publishedAt":44},"01M6QRA5AYC1RBFZ39GTWGS3N8","workflow-builder","Design and write deterministic multi-agent workflow scripts (.js files in .claude\u002Fworkflows\u002F) for Claude Code's Workflow tool. Use when a user wants to build, create, author, scaffold, or run a custom Claude Code workflow, orchestrate sub-agents (fan-out, pipeline, loop, judge-panel), or automate a repeatable multi-step task across fresh-context agents.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fworkflow-builder\u002FSKILL.md",[198],{"categoryNo":147,"categoryName":148,"taxonomyType":21,"taxonomyTypeDesc":22},[200,201,202,203,204,152,179,205,206,207,208,209,135,210,108,211,212,213,214,215],"workflow","workflow builder","builder","design","workflows","code","orchestrate","pipeline","loop","multi-agent","context","across","fresh","scripts","judge","claude code workflow",{"skillNo":217,"name":218,"nameZh":6,"description":219,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":220,"categories":221,"keywords":222,"publishedAt":44},"01M6QR7QMCZESKQA0ERW1E657M","weekly-review","Use when someone wants to run a weekly review, close open loops, audit stalled projects and commitments, get their system back to trusted, restart a lapsed review habit, or says \"\u002Fcs:weekly-review\". Walks David Allen's three-phase loop — GET CLEAR, GET CURRENT, GET CREATIVE — with deterministic scripts that inventory open loops, gate the checklist with named gaps, and score commitment health 0-100.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fweekly-review\u002FSKILL.md",[],[223,224,225,226,227,228,229,230,231,208,232,233,234,235,236,237,110,238,239,191],"review","weekly review","weekly","deterministic scripts","reflect","feeds","inventory","open","team","projects","project-management","scanner","conversation","back","project","sprint","cadence"]