[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M73AAS18K0JZ2AT3R545ZJMN":3,"skill-same:01M73AAS18K0JZ2AT3R545ZJMN":66},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":42,"commitSha":43,"skillDirectory":44,"forksCount":45,"repositoryHtmlUrl":46,"repositoryHomepageUrl":47,"ownerHtmlUrl":48,"ownerBlogUrl":49,"ownerDescription":50,"updatedAt":51,"install":52,"resources":65},"01M73AAS18K0JZ2AT3R545ZJMN","deepeval",null,"DeepEval — LLM evaluation framework, RAG metrics, hallucination detection, red-teaming, CI\u002FCD integration","Horonomos-laps\u002FYana-AI","Horonomos-laps","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F355299122?v=4","01M76B3QPXE88ET2SQPDPVATZT"," Horonomos-laps","01M72ZQRJPCYPSSWT4D00VJR2A",13,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdeepeval\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: deepeval\ndescription: DeepEval — LLM evaluation framework, RAG metrics, hallucination detection, red-teaming, CI\u002FCD integration\ntriggers:\n  - deepeval\n  - deepeval metrics\n  - llm evaluation deepeval\n  - hallucination detection deepeval\n  - deepeval rag\n  - deepeval red team\n  - deepeval ci\n  - deepeval test\n  - llm unit test\n  - deepeval dataset\ndo_not_use_for:\n  - vector storage — use qdrant\n  - tracing\u002Fobservability — use langfuse\n  - generic benchmarks — use ragas for RAG-specific\nsee_also:\n  - ragas\n  - langfuse\n  - pydantic-ai\n---\n\n# DeepEval — LLM Evaluation\n\n## Core: Test Cases + Metrics\n\n```python\nfrom deepeval import evaluate\nfrom deepeval.metrics import (\n    AnswerRelevancyMetric,\n    FaithfulnessMetric,\n    ContextualPrecisionMetric,\n    ContextualRecallMetric,\n    HallucinationMetric,\n    BiasMetric,\n    ToxicityMetric,\n)\nfrom deepeval.test_case import LLMTestCase\n\ntest_case = LLMTestCase(\n    input=\"What is RAG?\",\n    actual_output=\"RAG stands for Retrieval-Augmented Generation...\",\n    expected_output=\"Retrieval-Augmented Generation is a technique...\",\n    retrieval_context=[\n        \"RAG is an AI framework that retrieves relevant documents...\",\n        \"The retrieval step fetches context from a knowledge base...\",\n    ],\n)\n\nmetrics = [\n    AnswerRelevancyMetric(threshold=0.7, model=\"gpt-4o\"),\n    FaithfulnessMetric(threshold=0.8, model=\"gpt-4o\"),\n    ContextualPrecisionMetric(threshold=0.7, model=\"gpt-4o\"),\n    ContextualRecallMetric(threshold=0.7, model=\"gpt-4o\"),\n]\n\nevaluate([test_case], metrics)\n```\n\n## Pytest Integration\n\n```python\n# test_rag.py\nimport pytest\nfrom deepeval import assert_test\nfrom deepeval.metrics import AnswerRelevancyMetric, FaithfulnessMetric\nfrom deepeval.test_case import LLMTestCase\n\n@pytest.mark.parametrize(\"test_case\", [\n    LLMTestCase(\n        input=\"What is the capital of France?\",\n        actual_output=rag_pipeline(\"What is the capital of France?\"),\n        retrieval_context=[\"France is a country in Europe. Its capital is Paris.\"],\n    ),\n])\ndef test_rag_pipeline(test_case):\n    answer_relevancy = AnswerRelevancyMetric(threshold=0.7, model=\"gpt-4o\")\n    faithfulness = FaithfulnessMetric(threshold=0.8, model=\"gpt-4o\")\n    assert_test(test_case, [answer_relevancy, faithfulness])\n\n# Run: deepeval test run test_rag.py\n```\n\n## Custom Metric\n\n```python\nfrom deepeval.metrics import BaseMetric\nfrom deepeval.test_case import LLMTestCase\n\nclass ResponseLengthMetric(BaseMetric):\n    def __init__(self, max_tokens: int = 200, threshold: float = 1.0):\n        self.max_tokens = max_tokens\n        self.threshold = threshold\n        self.name = \"Response Length\"\n\n    def measure(self, test_case: LLMTestCase) -> float:\n        tokens = len(test_case.actual_output.split())\n        self.score = 1.0 if tokens \u003C= self.max_tokens else 0.0\n        self.reason = f\"Response has {tokens} tokens (max {self.max_tokens})\"\n        self.success = self.score >= self.threshold\n        return self.score\n\n    async def a_measure(self, test_case: LLMTestCase) -> float:\n        return self.measure(test_case)\n\n    def is_successful(self) -> bool:\n        return self.success\n```\n\n## Hallucination Detection\n\n```python\nfrom deepeval.metrics import HallucinationMetric\nfrom deepeval.test_case import LLMTestCase\n\ntest_case = LLMTestCase(\n    input=\"Tell me about Einstein\",\n    actual_output=\"Einstein won the Nobel Prize in Physics in 1921 for the photoelectric effect.\",\n    context=[\n        \"Einstein received the Nobel Prize in Physics in 1921.\",\n        \"The prize was awarded for his discovery of the photoelectric effect.\",\n    ],\n)\n\nmetric = HallucinationMetric(threshold=0.5, model=\"gpt-4o\")\nmetric.measure(test_case)\nprint(metric.score)         # 0.0 = no hallucination, 1.0 = full hallucination\nprint(metric.reason)\nprint(metric.is_successful())\n```\n\n## Datasets\n\n```python\nfrom deepeval.dataset import EvaluationDataset\n\n# Create dataset from test cases\ndataset = EvaluationDataset(\n    test_cases=[test_case1, test_case2, test_case3],\n    alias=\"rag-eval-v1\",\n)\n\n# Push to Confident AI (deepeval cloud)\ndataset.push(overwrite=True)\n\n# Pull existing dataset\nfrom deepeval.dataset import EvaluationDataset\ndataset = EvaluationDataset()\ndataset.pull(alias=\"rag-eval-v1\")\n\n# Evaluate entire dataset\nevaluate(dataset, metrics=[AnswerRelevancyMetric(threshold=0.7)])\n```\n\n## Red Teaming\n\n```python\nfrom deepeval.red_teaming import RedTeamer, AttackEnhancer, Vulnerability\n\nred_teamer = RedTeamer(\n    target_purpose=\"Customer support chatbot for e-commerce\",\n    target_system_prompt=\"You are a helpful customer support agent...\",\n)\n\nresults = red_teamer.scan(\n    target_model=my_model,\n    attacks_per_vulnerability=5,\n    vulnerabilities=[\n        Vulnerability.PROMPT_INJECTION,\n        Vulnerability.PII_LEAKAGE,\n        Vulnerability.JAILBREAKING,\n        Vulnerability.HARMFUL_CONTENT,\n    ],\n    attack_enhancements={\n        AttackEnhancer.BASE64: 0.25,\n        AttackEnhancer.ROT13: 0.25,\n        AttackEnhancer.JAILBREAK: 0.5,\n    },\n)\nprint(red_teamer.vulnerability_scores_breakdown)\n```\n\n## Use Custom LLM Judge\n\n```python\nfrom deepeval.models import DeepEvalBaseLLM\nimport anthropic\n\nclass ClaudeJudge(DeepEvalBaseLLM):\n    def __init__(self):\n        self.client = anthropic.Anthropic()\n\n    def load_model(self):\n        return self.client\n\n    def generate(self, prompt: str) -> str:\n        response = self.client.messages.create(\n            model=\"claude-sonnet-4-6\",\n            max_tokens=1024,\n            messages=[{\"role\": \"user\", \"content\": prompt}],\n        )\n        return response.content[0].text\n\n    async def a_generate(self, prompt: str) -> str:\n        return self.generate(prompt)\n\n    def get_model_name(self) -> str:\n        return \"claude-sonnet-4-6\"\n\njudge = ClaudeJudge()\nmetric = AnswerRelevancyMetric(threshold=0.7, model=judge)\n```\n\n## Anti-Fake-Pass Checks\n\n- Metrics require an LLM judge (`model=`) — costs API credits per evaluation\n- `threshold` is minimum passing score — `measure()` returns float, `is_successful()` checks threshold\n- `retrieval_context` for RAG metrics must be the actual retrieved chunks, not all docs\n- `assert_test()` in pytest raises `AssertionError` on failure — fails the test\n- Red teaming requires `OPENAI_API_KEY` or custom model — scans are expensive (N attacks × M vulns)\n- `dataset.push()` requires Confident AI account — use local `evaluate()` without cloud\n","# DeepEval — LLM Evaluation\n\n## Core: Test Cases + Metrics\n\n```python\nfrom deepeval import evaluate\nfrom deepeval.metrics import (\n    AnswerRelevancyMetric,\n    FaithfulnessMetric,\n    ContextualPrecisionMetric,\n    ContextualRecallMetric,\n    HallucinationMetric,\n    BiasMetric,\n    ToxicityMetric,\n)\nfrom deepeval.test_case import LLMTestCase\n\ntest_case = LLMTestCase(\n    input=\"What is RAG?\",\n    actual_output=\"RAG stands for Retrieval-Augmented Generation...\",\n    expected_output=\"Retrieval-Augmented Generation is a technique...\",\n    retrieval_context=[\n        \"RAG is an AI framework that retrieves relevant documents...\",\n        \"The retrieval step fetches context from a knowledge base...\",\n    ],\n)\n\nmetrics = [\n    AnswerRelevancyMetric(threshold=0.7, model=\"gpt-4o\"),\n    FaithfulnessMetric(threshold=0.8, model=\"gpt-4o\"),\n    ContextualPrecisionMetric(threshold=0.7, model=\"gpt-4o\"),\n    ContextualRecallMetric(threshold=0.7, model=\"gpt-4o\"),\n]\n\nevaluate([test_case], metrics)\n```\n\n## Pytest Integration\n\n```python\n# test_rag.py\nimport pytest\nfrom deepeval import assert_test\nfrom deepeval.metrics import AnswerRelevancyMetric, FaithfulnessMetric\nfrom deepeval.test_case import LLMTestCase\n\n@pytest.mark.parametrize(\"test_case\", [\n    LLMTestCase(\n        input=\"What is the capital of France?\",\n        actual_output=rag_pipeline(\"What is the capital of France?\"),\n        retrieval_context=[\"France is a country in Europe. Its capital is Paris.\"],\n    ),\n])\ndef test_rag_pipeline(test_case):\n    answer_relevancy = AnswerRelevancyMetric(threshold=0.7, model=\"gpt-4o\")\n    faithfulness = FaithfulnessMetric(threshold=0.8, model=\"gpt-4o\")\n    assert_test(test_case, [answer_relevancy, faithfulness])\n\n# Run: deepeval test run test_rag.py\n```\n\n## Custom Metric\n\n```python\nfrom deepeval.metrics import BaseMetric\nfrom deepeval.test_case import LLMTestCase\n\nclass ResponseLengthMetric(BaseMetric):\n    def __init__(self, max_tokens: int = 200, threshold: float = 1.0):\n        self.max_tokens = max_tokens\n        self.threshold = threshold\n        self.name = \"Response Length\"\n\n    def measure(self, test_case: LLMTestCase) -> float:\n        tokens = len(test_case.actual_output.split())\n        self.score = 1.0 if tokens \u003C= self.max_tokens else 0.0\n        self.reason = f\"Response has {tokens} tokens (max {self.max_tokens})\"\n        self.success = self.score >= self.threshold\n        return self.score\n\n    async def a_measure(self, test_case: LLMTestCase) -> float:\n        return self.measure(test_case)\n\n    def is_successful(self) -> bool:\n        return self.success\n```\n\n## Hallucination Detection\n\n```python\nfrom deepeval.metrics import HallucinationMetric\nfrom deepeval.test_case import LLMTestCase\n\ntest_case = LLMTestCase(\n    input=\"Tell me about Einstein\",\n    actual_output=\"Einstein won the Nobel Prize in Physics in 1921 for the photoelectric effect.\",\n    context=[\n        \"Einstein received the Nobel Prize in Physics in 1921.\",\n        \"The prize was awarded for his discovery of the photoelectric effect.\",\n    ],\n)\n\nmetric = HallucinationMetric(threshold=0.5, model=\"gpt-4o\")\nmetric.measure(test_case)\nprint(metric.score)         # 0.0 = no hallucination, 1.0 = full hallucination\nprint(metric.reason)\nprint(metric.is_successful())\n```\n\n## Datasets\n\n```python\nfrom deepeval.dataset import EvaluationDataset\n\n# Create dataset from test cases\ndataset = EvaluationDataset(\n    test_cases=[test_case1, test_case2, test_case3],\n    alias=\"rag-eval-v1\",\n)\n\n# Push to Confident AI (deepeval cloud)\ndataset.push(overwrite=True)\n\n# Pull existing dataset\nfrom deepeval.dataset import EvaluationDataset\ndataset = EvaluationDataset()\ndataset.pull(alias=\"rag-eval-v1\")\n\n# Evaluate entire dataset\nevaluate(dataset, metrics=[AnswerRelevancyMetric(threshold=0.7)])\n```\n\n## Red Teaming\n\n```python\nfrom deepeval.red_teaming import RedTeamer, AttackEnhancer, Vulnerability\n\nred_teamer = RedTeamer(\n    target_purpose=\"Customer support chatbot for e-commerce\",\n    target_system_prompt=\"You are a helpful customer support agent...\",\n)\n\nresults = red_teamer.scan(\n    target_model=my_model,\n    attacks_per_vulnerability=5,\n    vulnerabilities=[\n        Vulnerability.PROMPT_INJECTION,\n        Vulnerability.PII_LEAKAGE,\n        Vulnerability.JAILBREAKING,\n        Vulnerability.HARMFUL_CONTENT,\n    ],\n    attack_enhancements={\n        AttackEnhancer.BASE64: 0.25,\n        AttackEnhancer.ROT13: 0.25,\n        AttackEnhancer.JAILBREAK: 0.5,\n    },\n)\nprint(red_teamer.vulnerability_scores_breakdown)\n```\n\n## Use Custom LLM Judge\n\n```python\nfrom deepeval.models import DeepEvalBaseLLM\nimport anthropic\n\nclass ClaudeJudge(DeepEvalBaseLLM):\n    def __init__(self):\n        self.client = anthropic.Anthropic()\n\n    def load_model(self):\n        return self.client\n\n    def generate(self, prompt: str) -> str:\n        response = self.client.messages.create(\n            model=\"claude-sonnet-4-6\",\n            max_tokens=1024,\n            messages=[{\"role\": \"user\", \"content\": prompt}],\n        )\n        return response.content[0].text\n\n    async def a_generate(self, prompt: str) -> str:\n        return self.generate(prompt)\n\n    def get_model_name(self) -> str:\n        return \"claude-sonnet-4-6\"\n\njudge = ClaudeJudge()\nmetric = AnswerRelevancyMetric(threshold=0.7, model=judge)\n```\n\n## Anti-Fake-Pass Checks\n\n- Metrics require an LLM judge (`model=`) — costs API credits per evaluation\n- `threshold` is minimum passing score — `measure()` returns float, `is_successful()` checks threshold\n- `retrieval_context` for RAG metrics must be the actual retrieved chunks, not all docs\n- `assert_test()` in pytest raises `AssertionError` on failure — fails the test\n- Red teaming requires `OPENAI_API_KEY` or custom model — scans are expensive (N attacks × M vulns)\n- `dataset.push()` requires Confident AI account — use local `evaluate()` without cloud\n",{"name":5,"see_also":24,"triggers":28,"description":7,"do_not_use_for":38},[25,26,27],"ragas","langfuse","pydantic-ai",[5,29,30,31,32,33,34,35,36,37],"deepeval metrics","llm evaluation deepeval","hallucination detection deepeval","deepeval rag","deepeval red team","deepeval ci","deepeval test","llm unit test","deepeval dataset",[39,40,41],"vector storage — use qdrant","tracing\u002Fobservability — use langfuse","generic benchmarks — use ragas for RAG-specific","main","481dcd7dd2e288b9d627a2bd017db70e64e6c6fa","core\u002Fskills\u002Fdeepeval",2,"https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI","https:\u002F\u002Fyana.vutam.link\u002F","https:\u002F\u002Fgithub.com\u002FHoronomos-laps","https:\u002F\u002Fyana.vutam.link","An open-source governed runtime & execution environment for multi-agent AI systems.","2026-11-14 13:24:09",{"source":8,"skillName":5,"commands":53,"prompt":63,"zipAvailable":64},[54,57,60],{"tool":55,"command":56},"npx","npx skills add Horonomos-laps\u002FYana-AI --skill deepeval",{"tool":58,"command":59},"bunx","bunx skills add Horonomos-laps\u002FYana-AI --skill deepeval",{"tool":61,"command":62},"pnpm","pnpm dlx skills add Horonomos-laps\u002FYana-AI --skill deepeval","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdeepeval\u002FSKILL.md",true,[],[67,75,82,90,98,106],{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":71,"licenseFileUrl":6,"sourceHtmlUrl":72,"categories":73,"keywords":74,"publishedAt":19},"01M735GKET5YVYKAY62W2P5YXB","openai--sentry--sentry","Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.","MIT","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fopenai--sentry--sentry\u002FSKILL.md",[],[],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":19},"01M73DCAP83T17HY7SMGK7MGFE","ito-basket-compare","Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fito-basket-compare\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":86,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":19},"01M734CYEMZFDR3KH6SW0JP8TD","implementing-network-traffic-baselining","Build network traffic baselines from NetFlow\u002FIPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly\u002Fdaily traffic pattern profiling","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-network-traffic-baselining\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-network-traffic-baselining\u002FSKILL.md",[],[],{"skillNo":91,"name":92,"nameZh":6,"description":93,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":94,"sourceHtmlUrl":95,"categories":96,"keywords":97,"publishedAt":19},"01M73ASZ6NJRJM44Z2P84NGQ3Q","detecting-evasion-techniques-in-endpoint-logs","Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdetecting-evasion-techniques-in-endpoint-logs\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdetecting-evasion-techniques-in-endpoint-logs\u002FSKILL.md",[],[],{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":102,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":19},"01M734AFNXD06Z78QRK0VKGT7Q","implementing-microsegmentation-with-guardicore","Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-microsegmentation-with-guardicore\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-microsegmentation-with-guardicore\u002FSKILL.md",[],[],{"skillNo":107,"name":108,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":19},"01M736TK27QMCXK0DVCH25SJN5","ramdisk-tmpfs-patterns","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Framdisk-tmpfs-patterns\u002FSKILL.md",[],[]]