[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7BWH5MZ14V95YP8HN7Z6B1M":3,"skill-same:01M7BWH5MZ14V95YP8HN7Z6B1M":47},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":29,"ownerHtmlUrl":30,"ownerBlogUrl":6,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M7BWH5MZ14V95YP8HN7Z6B1M","llm-evaluation",null,"Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.","YYC-Cube\u002FYYC3-Knowledge-Base","YYC-Cube","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F235006543?v=4","01M79J2PH4SWD2X0825X6PN748","yyc3","01M79J2PHVK3F6ZBYT8K2SCZCW",1,"MIT","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-Plugin-插件协议\u002FYYC3-Plugin-插件协议-plugins-claude\u002Fllm-application-dev\u002Fskills\u002Fllm-evaluation\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: llm-evaluation\ndescription: Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.\n---\n\n# LLM Evaluation\n\nMaster comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A\u002FB testing.\n\n## When to Use This Skill\n\n- Measuring LLM application performance systematically\n- Comparing different models or prompts\n- Detecting performance regressions before deployment\n- Validating improvements from prompt changes\n- Building confidence in production systems\n- Establishing baselines and tracking progress over time\n- Debugging unexpected model behavior\n\n## Core Evaluation Types\n\n### 1. Automated Metrics\n\nFast, repeatable, scalable evaluation using computed scores.\n\n**Text Generation:**\n\n- **BLEU**: N-gram overlap (translation)\n- **ROUGE**: Recall-oriented (summarization)\n- **METEOR**: Semantic similarity\n- **BERTScore**: Embedding-based similarity\n- **Perplexity**: Language model confidence\n\n**Classification:**\n\n- **Accuracy**: Percentage correct\n- **Precision\u002FRecall\u002FF1**: Class-specific performance\n- **Confusion Matrix**: Error patterns\n- **AUC-ROC**: Ranking quality\n\n**Retrieval (RAG):**\n\n- **MRR**: Mean Reciprocal Rank\n- **NDCG**: Normalized Discounted Cumulative Gain\n- **Precision@K**: Relevant in top K\n- **Recall@K**: Coverage in top K\n\n### 2. Human Evaluation\n\nManual assessment for quality aspects difficult to automate.\n\n**Dimensions:**\n\n- **Accuracy**: Factual correctness\n- **Coherence**: Logical flow\n- **Relevance**: Answers the question\n- **Fluency**: Natural language quality\n- **Safety**: No harmful content\n- **Helpfulness**: Useful to the user\n\n### 3. LLM-as-Judge\n\nUse stronger LLMs to evaluate weaker model outputs.\n\n**Approaches:**\n\n- **Pointwise**: Score individual responses\n- **Pairwise**: Compare two responses\n- **Reference-based**: Compare to gold standard\n- **Reference-free**: Judge without ground truth\n\n## Quick Start\n\n```python\nfrom dataclasses import dataclass\nfrom typing import Callable\nimport numpy as np\n\n@dataclass\nclass Metric:\n    name: str\n    fn: Callable\n\n    @staticmethod\n    def accuracy():\n        return Metric(\"accuracy\", calculate_accuracy)\n\n    @staticmethod\n    def bleu():\n        return Metric(\"bleu\", calculate_bleu)\n\n    @staticmethod\n    def bertscore():\n        return Metric(\"bertscore\", calculate_bertscore)\n\n    @staticmethod\n    def custom(name: str, fn: Callable):\n        return Metric(name, fn)\n\nclass EvaluationSuite:\n    def __init__(self, metrics: list[Metric]):\n        self.metrics = metrics\n\n    async def evaluate(self, model, test_cases: list[dict]) -> dict:\n        results = {m.name: [] for m in self.metrics}\n\n        for test in test_cases:\n            prediction = await model.predict(test[\"input\"])\n\n            for metric in self.metrics:\n                score = metric.fn(\n                    prediction=prediction,\n                    reference=test.get(\"expected\"),\n                    context=test.get(\"context\")\n                )\n                results[metric.name].append(score)\n\n        return {\n            \"metrics\": {k: np.mean(v) for k, v in results.items()},\n            \"raw_scores\": results\n        }\n\n# Usage\nsuite = EvaluationSuite([\n    Metric.accuracy(),\n    Metric.bleu(),\n    Metric.bertscore(),\n    Metric.custom(\"groundedness\", check_groundedness)\n])\n\ntest_cases = [\n    {\n        \"input\": \"What is the capital of France?\",\n        \"expected\": \"Paris\",\n        \"context\": \"France is a country in Europe. Paris is its capital.\"\n    },\n]\n\nresults = await suite.evaluate(model=your_model, test_cases=test_cases)\n```\n\n## Automated Metrics Implementation\n\n### BLEU Score\n\n```python\nfrom nltk.translate.bleu_score import sentence_bleu, SmoothingFunction\n\ndef calculate_bleu(reference: str, hypothesis: str, **kwargs) -> float:\n    \"\"\"Calculate BLEU score between reference and hypothesis.\"\"\"\n    smoothie = SmoothingFunction().method4\n\n    return sentence_bleu(\n        [reference.split()],\n        hypothesis.split(),\n        smoothing_function=smoothie\n    )\n```\n\n### ROUGE Score\n\n```python\nfrom rouge_score import rouge_scorer\n\ndef calculate_rouge(reference: str, hypothesis: str, **kwargs) -> dict:\n    \"\"\"Calculate ROUGE scores.\"\"\"\n    scorer = rouge_scorer.RougeScorer(\n        ['rouge1', 'rouge2', 'rougeL'],\n        use_stemmer=True\n    )\n    scores = scorer.score(reference, hypothesis)\n\n    return {\n        'rouge1': scores['rouge1'].fmeasure,\n        'rouge2': scores['rouge2'].fmeasure,\n        'rougeL': scores['rougeL'].fmeasure\n    }\n```\n\n### BERTScore\n\n```python\nfrom bert_score import score\n\ndef calculate_bertscore(\n    references: list[str],\n    hypotheses: list[str],\n    **kwargs\n) -> dict:\n    \"\"\"Calculate BERTScore using pre-trained model.\"\"\"\n    P, R, F1 = score(\n        hypotheses,\n        references,\n        lang='en',\n        model_type='microsoft\u002Fdeberta-xlarge-mnli'\n    )\n\n    return {\n        'precision': P.mean().item(),\n        'recall': R.mean().item(),\n        'f1': F1.mean().item()\n    }\n```\n\n### Custom Metrics\n\n```python\ndef calculate_groundedness(response: str, context: str, **kwargs) -> float:\n    \"\"\"Check if response is grounded in provided context.\"\"\"\n    from transformers import pipeline\n\n    nli = pipeline(\n        \"text-classification\",\n        model=\"microsoft\u002Fdeberta-large-mnli\"\n    )\n\n    result = nli(f\"{context} [SEP] {response}\")[0]\n\n    # Return confidence that response is entailed by context\n    return result['score'] if result['label'] == 'ENTAILMENT' else 0.0\n\ndef calculate_toxicity(text: str, **kwargs) -> float:\n    \"\"\"Measure toxicity in generated text.\"\"\"\n    from detoxify import Detoxify\n\n    results = Detoxify('original').predict(text)\n    return max(results.values())  # Return highest toxicity score\n\ndef calculate_factuality(claim: str, sources: list[str], **kwargs) -> float:\n    \"\"\"Verify factual claims against sources.\"\"\"\n    from transformers import pipeline\n\n    nli = pipeline(\"text-classification\", model=\"facebook\u002Fbart-large-mnli\")\n\n    scores = []\n    for source in sources:\n        result = nli(f\"{source}\u003C\u002Fs>\u003C\u002Fs>{claim}\")[0]\n        if result['label'] == 'entailment':\n            scores.append(result['score'])\n\n    return max(scores) if scores else 0.0\n```\n\n## LLM-as-Judge Patterns\n\n### Single Output Evaluation\n\n```python\nfrom anthropic import Anthropic\nfrom pydantic import BaseModel, Field\nimport json\n\nclass QualityRating(BaseModel):\n    accuracy: int = Field(ge=1, le=10, description=\"Factual correctness\")\n    helpfulness: int = Field(ge=1, le=10, description=\"Answers the question\")\n    clarity: int = Field(ge=1, le=10, description=\"Well-written and understandable\")\n    reasoning: str = Field(description=\"Brief explanation\")\n\nasync def llm_judge_quality(\n    response: str,\n    question: str,\n    context: str = None\n) -> QualityRating:\n    \"\"\"Use Claude to judge response quality.\"\"\"\n    client = Anthropic()\n\n    system = \"\"\"You are an expert evaluator of AI responses.\n    Rate responses on accuracy, helpfulness, and clarity (1-10 scale).\n    Provide brief reasoning for your ratings.\"\"\"\n\n    prompt = f\"\"\"Rate the following response:\n\nQuestion: {question}\n{f'Context: {context}' if context else ''}\nResponse: {response}\n\nProvide ratings in JSON format:\n{{\n  \"accuracy\": \u003C1-10>,\n  \"helpfulness\": \u003C1-10>,\n  \"clarity\": \u003C1-10>,\n  \"reasoning\": \"\u003Cbrief explanation>\"\n}}\"\"\"\n\n    message = client.messages.create(\n        model=\"claude-sonnet-4-6\",\n        max_tokens=500,\n        system=system,\n        messages=[{\"role\": \"user\", \"content\": prompt}]\n    )\n\n    return QualityRating(**json.loads(message.content[0].text))\n```\n\n### Pairwise Comparison\n\n```python\nfrom pydantic import BaseModel, Field\nfrom typing import Literal\n\nclass ComparisonResult(BaseModel):\n    winner: Literal[\"A\", \"B\", \"tie\"]\n    reasoning: str\n    confidence: int = Field(ge=1, le=10)\n\nasync def compare_responses(\n    question: str,\n    response_a: str,\n    response_b: str\n) -> ComparisonResult:\n    \"\"\"Compare two responses using LLM judge.\"\"\"\n    client = Anthropic()\n\n    prompt = f\"\"\"Compare these two responses and determine which is better.\n\nQuestion: {question}\n\nResponse A: {response_a}\n\nResponse B: {response_b}\n\nConsider accuracy, helpfulness, and clarity.\n\nAnswer with JSON:\n{{\n  \"winner\": \"A\" or \"B\" or \"tie\",\n  \"reasoning\": \"\u003Cexplanation>\",\n  \"confidence\": \u003C1-10>\n}}\"\"\"\n\n    message = client.messages.create(\n        model=\"claude-sonnet-4-6\",\n        max_tokens=500,\n        messages=[{\"role\": \"user\", \"content\": prompt}]\n    )\n\n    return ComparisonResult(**json.loads(message.content[0].text))\n```\n\n### Reference-Based Evaluation\n\n```python\nclass ReferenceEvaluation(BaseModel):\n    semantic_similarity: float = Field(ge=0, le=1)\n    factual_accuracy: float = Field(ge=0, le=1)\n    completeness: float = Field(ge=0, le=1)\n    issues: list[str]\n\nasync def evaluate_against_reference(\n    response: str,\n    reference: str,\n    question: str\n) -> ReferenceEvaluation:\n    \"\"\"Evaluate response against gold standard reference.\"\"\"\n    client = Anthropic()\n\n    prompt = f\"\"\"Compare the response to the reference answer.\n\nQuestion: {question}\nReference Answer: {reference}\nResponse to Evaluate: {response}\n\nEvaluate:\n1. Semantic similarity (0-1): How similar is the meaning?\n2. Factual accuracy (0-1): Are all facts correct?\n3. Completeness (0-1): Does it cover all key points?\n4. List any specific issues or errors.\n\nRespond in JSON:\n{{\n  \"semantic_similarity\": \u003C0-1>,\n  \"factual_accuracy\": \u003C0-1>,\n  \"completeness\": \u003C0-1>,\n  \"issues\": [\"issue1\", \"issue2\"]\n}}\"\"\"\n\n    message = client.messages.create(\n        model=\"claude-sonnet-4-6\",\n        max_tokens=500,\n        messages=[{\"role\": \"user\", \"content\": prompt}]\n    )\n\n    return ReferenceEvaluation(**json.loads(message.content[0].text))\n```\n\n## Human Evaluation Frameworks\n\n### Annotation Guidelines\n\n```python\nfrom dataclasses import dataclass, field\nfrom typing import Optional\n\n@dataclass\nclass AnnotationTask:\n    \"\"\"Structure for human annotation task.\"\"\"\n    response: str\n    question: str\n    context: Optional[str] = None\n\n    def get_annotation_form(self) -> dict:\n        return {\n            \"question\": self.question,\n            \"context\": self.context,\n            \"response\": self.response,\n            \"ratings\": {\n                \"accuracy\": {\n                    \"scale\": \"1-5\",\n                    \"description\": \"Is the response factually correct?\"\n                },\n                \"relevance\": {\n                    \"scale\": \"1-5\",\n                    \"description\": \"Does it answer the question?\"\n                },\n                \"coherence\": {\n                    \"scale\": \"1-5\",\n                    \"description\": \"Is it logically consistent?\"\n                }\n            },\n            \"issues\": {\n                \"factual_error\": False,\n                \"hallucination\": False,\n                \"off_topic\": False,\n                \"unsafe_content\": False\n            },\n            \"feedback\": \"\"\n        }\n```\n\n### Inter-Rater Agreement\n\n```python\nfrom sklearn.metrics import cohen_kappa_score\n\ndef calculate_agreement(\n    rater1_scores: list[int],\n    rater2_scores: list[int]\n) -> dict:\n    \"\"\"Calculate inter-rater agreement.\"\"\"\n    kappa = cohen_kappa_score(rater1_scores, rater2_scores)\n\n    if kappa \u003C 0:\n        interpretation = \"Poor\"\n    elif kappa \u003C 0.2:\n        interpretation = \"Slight\"\n    elif kappa \u003C 0.4:\n        interpretation = \"Fair\"\n    elif kappa \u003C 0.6:\n        interpretation = \"Moderate\"\n    elif kappa \u003C 0.8:\n        interpretation = \"Substantial\"\n    else:\n        interpretation = \"Almost Perfect\"\n\n    return {\n        \"kappa\": kappa,\n        \"interpretation\": interpretation\n    }\n```\n\n## A\u002FB Testing\n\n### Statistical Testing Framework\n\n```python\nfrom scipy import stats\nimport numpy as np\nfrom dataclasses import dataclass, field\n\n@dataclass\nclass ABTest:\n    variant_a_name: str = \"A\"\n    variant_b_name: str = \"B\"\n    variant_a_scores: list[float] = field(default_factory=list)\n    variant_b_scores: list[float] = field(default_factory=list)\n\n    def add_result(self, variant: str, score: float):\n        \"\"\"Add evaluation result for a variant.\"\"\"\n        if variant == \"A\":\n            self.variant_a_scores.append(score)\n        else:\n            self.variant_b_scores.append(score)\n\n    def analyze(self, alpha: float = 0.05) -> dict:\n        \"\"\"Perform statistical analysis.\"\"\"\n        a_scores = np.array(self.variant_a_scores)\n        b_scores = np.array(self.variant_b_scores)\n\n        # T-test\n        t_stat, p_value = stats.ttest_ind(a_scores, b_scores)\n\n        # Effect size (Cohen's d)\n        pooled_std = np.sqrt((np.std(a_scores)**2 + np.std(b_scores)**2) \u002F 2)\n        cohens_d = (np.mean(b_scores) - np.mean(a_scores)) \u002F pooled_std\n\n        return {\n            \"variant_a_mean\": np.mean(a_scores),\n            \"variant_b_mean\": np.mean(b_scores),\n            \"difference\": np.mean(b_scores) - np.mean(a_scores),\n            \"relative_improvement\": (np.mean(b_scores) - np.mean(a_scores)) \u002F np.mean(a_scores),\n            \"p_value\": p_value,\n            \"statistically_significant\": p_value \u003C alpha,\n            \"cohens_d\": cohens_d,\n            \"effect_size\": self._interpret_cohens_d(cohens_d),\n            \"winner\": self.variant_b_name if np.mean(b_scores) > np.mean(a_scores) else self.variant_a_name\n        }\n\n    @staticmethod\n    def _interpret_cohens_d(d: float) -> str:\n        \"\"\"Interpret Cohen's d effect size.\"\"\"\n        abs_d = abs(d)\n        if abs_d \u003C 0.2:\n            return \"negligible\"\n        elif abs_d \u003C 0.5:\n            return \"small\"\n        elif abs_d \u003C 0.8:\n            return \"medium\"\n        else:\n            return \"large\"\n```\n\n## Regression Testing\n\n### Regression Detection\n\n```python\nfrom dataclasses import dataclass\n\n@dataclass\nclass RegressionResult:\n    metric: str\n    baseline: float\n    current: float\n    change: float\n    is_regression: bool\n\nclass RegressionDetector:\n    def __init__(self, baseline_results: dict, threshold: float = 0.05):\n        self.baseline = baseline_results\n        self.threshold = threshold\n\n    def check_for_regression(self, new_results: dict) -> dict:\n        \"\"\"Detect if new results show regression.\"\"\"\n        regressions = []\n\n        for metric in self.baseline.keys():\n            baseline_score = self.baseline[metric]\n            new_score = new_results.get(metric)\n\n            if new_score is None:\n                continue\n\n            # Calculate relative change\n            relative_change = (new_score - baseline_score) \u002F baseline_score\n\n            # Flag if significant decrease\n            is_regression = relative_change \u003C -self.threshold\n            if is_regression:\n                regressions.append(RegressionResult(\n                    metric=metric,\n                    baseline=baseline_score,\n                    current=new_score,\n                    change=relative_change,\n                    is_regression=True\n                ))\n\n        return {\n            \"has_regression\": len(regressions) > 0,\n            \"regressions\": regressions,\n            \"summary\": f\"{len(regressions)} metric(s) regressed\"\n        }\n```\n\n## LangSmith Evaluation Integration\n\n```python\nfrom langsmith import Client\nfrom langsmith.evaluation import evaluate, LangChainStringEvaluator\n\n# Initialize LangSmith client\nclient = Client()\n\n# Create dataset\ndataset = client.create_dataset(\"qa_test_cases\")\nclient.create_examples(\n    inputs=[{\"question\": q} for q in questions],\n    outputs=[{\"answer\": a} for a in expected_answers],\n    dataset_id=dataset.id\n)\n\n# Define evaluators\nevaluators = [\n    LangChainStringEvaluator(\"qa\"),           # QA correctness\n    LangChainStringEvaluator(\"context_qa\"),   # Context-grounded QA\n    LangChainStringEvaluator(\"cot_qa\"),       # Chain-of-thought QA\n]\n\n# Run evaluation\nasync def target_function(inputs: dict) -> dict:\n    result = await your_chain.ainvoke(inputs)\n    return {\"answer\": result}\n\nexperiment_results = await evaluate(\n    target_function,\n    data=dataset.name,\n    evaluators=evaluators,\n    experiment_prefix=\"v1.0.0\",\n    metadata={\"model\": \"claude-sonnet-4-6\", \"version\": \"1.0.0\"}\n)\n\nprint(f\"Mean score: {experiment_results.aggregate_metrics['qa']['mean']}\")\n```\n\n## Benchmarking\n\n### Running Benchmarks\n\n```python\nfrom dataclasses import dataclass\nimport numpy as np\n\n@dataclass\nclass BenchmarkResult:\n    metric: str\n    mean: float\n    std: float\n    min: float\n    max: float\n\nclass BenchmarkRunner:\n    def __init__(self, benchmark_dataset: list[dict]):\n        self.dataset = benchmark_dataset\n\n    async def run_benchmark(\n        self,\n        model,\n        metrics: list[Metric]\n    ) -> dict[str, BenchmarkResult]:\n        \"\"\"Run model on benchmark and calculate metrics.\"\"\"\n        results = {metric.name: [] for metric in metrics}\n\n        for example in self.dataset:\n            # Generate prediction\n            prediction = await model.predict(example[\"input\"])\n\n            # Calculate each metric\n            for metric in metrics:\n                score = metric.fn(\n                    prediction=prediction,\n                    reference=example[\"reference\"],\n                    context=example.get(\"context\")\n                )\n                results[metric.name].append(score)\n\n        # Aggregate results\n        return {\n            metric: BenchmarkResult(\n                metric=metric,\n                mean=np.mean(scores),\n                std=np.std(scores),\n                min=min(scores),\n                max=max(scores)\n            )\n            for metric, scores in results.items()\n        }\n```\n","# LLM Evaluation\n\nMaster comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A\u002FB testing.\n\n## When to Use This Skill\n\n- Measuring LLM application performance systematically\n- Comparing different models or prompts\n- Detecting performance regressions before deployment\n- Validating improvements from prompt changes\n- Building confidence in production systems\n- Establishing baselines and tracking progress over time\n- Debugging unexpected model behavior\n\n## Core Evaluation Types\n\n### 1. Automated Metrics\n\nFast, repeatable, scalable evaluation using computed scores.\n\n**Text Generation:**\n\n- **BLEU**: N-gram overlap (translation)\n- **ROUGE**: Recall-oriented (summarization)\n- **METEOR**: Semantic similarity\n- **BERTScore**: Embedding-based similarity\n- **Perplexity**: Language model confidence\n\n**Classification:**\n\n- **Accuracy**: Percentage correct\n- **Precision\u002FRecall\u002FF1**: Class-specific performance\n- **Confusion Matrix**: Error patterns\n- **AUC-ROC**: Ranking quality\n\n**Retrieval (RAG):**\n\n- **MRR**: Mean Reciprocal Rank\n- **NDCG**: Normalized Discounted Cumulative Gain\n- **Precision@K**: Relevant in top K\n- **Recall@K**: Coverage in top K\n\n### 2. Human Evaluation\n\nManual assessment for quality aspects difficult to automate.\n\n**Dimensions:**\n\n- **Accuracy**: Factual correctness\n- **Coherence**: Logical flow\n- **Relevance**: Answers the question\n- **Fluency**: Natural language quality\n- **Safety**: No harmful content\n- **Helpfulness**: Useful to the user\n\n### 3. LLM-as-Judge\n\nUse stronger LLMs to evaluate weaker model outputs.\n\n**Approaches:**\n\n- **Pointwise**: Score individual responses\n- **Pairwise**: Compare two responses\n- **Reference-based**: Compare to gold standard\n- **Reference-free**: Judge without ground truth\n\n## Quick Start\n\n```python\nfrom dataclasses import dataclass\nfrom typing import Callable\nimport numpy as np\n\n@dataclass\nclass Metric:\n    name: str\n    fn: Callable\n\n    @staticmethod\n    def accuracy():\n        return Metric(\"accuracy\", calculate_accuracy)\n\n    @staticmethod\n    def bleu():\n        return Metric(\"bleu\", calculate_bleu)\n\n    @staticmethod\n    def bertscore():\n        return Metric(\"bertscore\", calculate_bertscore)\n\n    @staticmethod\n    def custom(name: str, fn: Callable):\n        return Metric(name, fn)\n\nclass EvaluationSuite:\n    def __init__(self, metrics: list[Metric]):\n        self.metrics = metrics\n\n    async def evaluate(self, model, test_cases: list[dict]) -> dict:\n        results = {m.name: [] for m in self.metrics}\n\n        for test in test_cases:\n            prediction = await model.predict(test[\"input\"])\n\n            for metric in self.metrics:\n                score = metric.fn(\n                    prediction=prediction,\n                    reference=test.get(\"expected\"),\n                    context=test.get(\"context\")\n                )\n                results[metric.name].append(score)\n\n        return {\n            \"metrics\": {k: np.mean(v) for k, v in results.items()},\n            \"raw_scores\": results\n        }\n\n# Usage\nsuite = EvaluationSuite([\n    Metric.accuracy(),\n    Metric.bleu(),\n    Metric.bertscore(),\n    Metric.custom(\"groundedness\", check_groundedness)\n])\n\ntest_cases = [\n    {\n        \"input\": \"What is the capital of France?\",\n        \"expected\": \"Paris\",\n        \"context\": \"France is a country in Europe. Paris is its capital.\"\n    },\n]\n\nresults = await suite.evaluate(model=your_model, test_cases=test_cases)\n```\n\n## Automated Metrics Implementation\n\n### BLEU Score\n\n```python\nfrom nltk.translate.bleu_score import sentence_bleu, SmoothingFunction\n\ndef calculate_bleu(reference: str, hypothesis: str, **kwargs) -> float:\n    \"\"\"Calculate BLEU score between reference and hypothesis.\"\"\"\n    smoothie = SmoothingFunction().method4\n\n    return sentence_bleu(\n        [reference.split()],\n        hypothesis.split(),\n        smoothing_function=smoothie\n    )\n```\n\n### ROUGE Score\n\n```python\nfrom rouge_score import rouge_scorer\n\ndef calculate_rouge(reference: str, hypothesis: str, **kwargs) -> dict:\n    \"\"\"Calculate ROUGE scores.\"\"\"\n    scorer = rouge_scorer.RougeScorer(\n        ['rouge1', 'rouge2', 'rougeL'],\n        use_stemmer=True\n    )\n    scores = scorer.score(reference, hypothesis)\n\n    return {\n        'rouge1': scores['rouge1'].fmeasure,\n        'rouge2': scores['rouge2'].fmeasure,\n        'rougeL': scores['rougeL'].fmeasure\n    }\n```\n\n### BERTScore\n\n```python\nfrom bert_score import score\n\ndef calculate_bertscore(\n    references: list[str],\n    hypotheses: list[str],\n    **kwargs\n) -> dict:\n    \"\"\"Calculate BERTScore using pre-trained model.\"\"\"\n    P, R, F1 = score(\n        hypotheses,\n        references,\n        lang='en',\n        model_type='microsoft\u002Fdeberta-xlarge-mnli'\n    )\n\n    return {\n        'precision': P.mean().item(),\n        'recall': R.mean().item(),\n        'f1': F1.mean().item()\n    }\n```\n\n### Custom Metrics\n\n```python\ndef calculate_groundedness(response: str, context: str, **kwargs) -> float:\n    \"\"\"Check if response is grounded in provided context.\"\"\"\n    from transformers import pipeline\n\n    nli = pipeline(\n        \"text-classification\",\n        model=\"microsoft\u002Fdeberta-large-mnli\"\n    )\n\n    result = nli(f\"{context} [SEP] {response}\")[0]\n\n    # Return confidence that response is entailed by context\n    return result['score'] if result['label'] == 'ENTAILMENT' else 0.0\n\ndef calculate_toxicity(text: str, **kwargs) -> float:\n    \"\"\"Measure toxicity in generated text.\"\"\"\n    from detoxify import Detoxify\n\n    results = Detoxify('original').predict(text)\n    return max(results.values())  # Return highest toxicity score\n\ndef calculate_factuality(claim: str, sources: list[str], **kwargs) -> float:\n    \"\"\"Verify factual claims against sources.\"\"\"\n    from transformers import pipeline\n\n    nli = pipeline(\"text-classification\", model=\"facebook\u002Fbart-large-mnli\")\n\n    scores = []\n    for source in sources:\n        result = nli(f\"{source}\u003C\u002Fs>\u003C\u002Fs>{claim}\")[0]\n        if result['label'] == 'entailment':\n            scores.append(result['score'])\n\n    return max(scores) if scores else 0.0\n```\n\n## LLM-as-Judge Patterns\n\n### Single Output Evaluation\n\n```python\nfrom anthropic import Anthropic\nfrom pydantic import BaseModel, Field\nimport json\n\nclass QualityRating(BaseModel):\n    accuracy: int = Field(ge=1, le=10, description=\"Factual correctness\")\n    helpfulness: int = Field(ge=1, le=10, description=\"Answers the question\")\n    clarity: int = Field(ge=1, le=10, description=\"Well-written and understandable\")\n    reasoning: str = Field(description=\"Brief explanation\")\n\nasync def llm_judge_quality(\n    response: str,\n    question: str,\n    context: str = None\n) -> QualityRating:\n    \"\"\"Use Claude to judge response quality.\"\"\"\n    client = Anthropic()\n\n    system = \"\"\"You are an expert evaluator of AI responses.\n    Rate responses on accuracy, helpfulness, and clarity (1-10 scale).\n    Provide brief reasoning for your ratings.\"\"\"\n\n    prompt = f\"\"\"Rate the following response:\n\nQuestion: {question}\n{f'Context: {context}' if context else ''}\nResponse: {response}\n\nProvide ratings in JSON format:\n{{\n  \"accuracy\": \u003C1-10>,\n  \"helpfulness\": \u003C1-10>,\n  \"clarity\": \u003C1-10>,\n  \"reasoning\": \"\u003Cbrief explanation>\"\n}}\"\"\"\n\n    message = client.messages.create(\n        model=\"claude-sonnet-4-6\",\n        max_tokens=500,\n        system=system,\n        messages=[{\"role\": \"user\", \"content\": prompt}]\n    )\n\n    return QualityRating(**json.loads(message.content[0].text))\n```\n\n### Pairwise Comparison\n\n```python\nfrom pydantic import BaseModel, Field\nfrom typing import Literal\n\nclass ComparisonResult(BaseModel):\n    winner: Literal[\"A\", \"B\", \"tie\"]\n    reasoning: str\n    confidence: int = Field(ge=1, le=10)\n\nasync def compare_responses(\n    question: str,\n    response_a: str,\n    response_b: str\n) -> ComparisonResult:\n    \"\"\"Compare two responses using LLM judge.\"\"\"\n    client = Anthropic()\n\n    prompt = f\"\"\"Compare these two responses and determine which is better.\n\nQuestion: {question}\n\nResponse A: {response_a}\n\nResponse B: {response_b}\n\nConsider accuracy, helpfulness, and clarity.\n\nAnswer with JSON:\n{{\n  \"winner\": \"A\" or \"B\" or \"tie\",\n  \"reasoning\": \"\u003Cexplanation>\",\n  \"confidence\": \u003C1-10>\n}}\"\"\"\n\n    message = client.messages.create(\n        model=\"claude-sonnet-4-6\",\n        max_tokens=500,\n        messages=[{\"role\": \"user\", \"content\": prompt}]\n    )\n\n    return ComparisonResult(**json.loads(message.content[0].text))\n```\n\n### Reference-Based Evaluation\n\n```python\nclass ReferenceEvaluation(BaseModel):\n    semantic_similarity: float = Field(ge=0, le=1)\n    factual_accuracy: float = Field(ge=0, le=1)\n    completeness: float = Field(ge=0, le=1)\n    issues: list[str]\n\nasync def evaluate_against_reference(\n    response: str,\n    reference: str,\n    question: str\n) -> ReferenceEvaluation:\n    \"\"\"Evaluate response against gold standard reference.\"\"\"\n    client = Anthropic()\n\n    prompt = f\"\"\"Compare the response to the reference answer.\n\nQuestion: {question}\nReference Answer: {reference}\nResponse to Evaluate: {response}\n\nEvaluate:\n1. Semantic similarity (0-1): How similar is the meaning?\n2. Factual accuracy (0-1): Are all facts correct?\n3. Completeness (0-1): Does it cover all key points?\n4. List any specific issues or errors.\n\nRespond in JSON:\n{{\n  \"semantic_similarity\": \u003C0-1>,\n  \"factual_accuracy\": \u003C0-1>,\n  \"completeness\": \u003C0-1>,\n  \"issues\": [\"issue1\", \"issue2\"]\n}}\"\"\"\n\n    message = client.messages.create(\n        model=\"claude-sonnet-4-6\",\n        max_tokens=500,\n        messages=[{\"role\": \"user\", \"content\": prompt}]\n    )\n\n    return ReferenceEvaluation(**json.loads(message.content[0].text))\n```\n\n## Human Evaluation Frameworks\n\n### Annotation Guidelines\n\n```python\nfrom dataclasses import dataclass, field\nfrom typing import Optional\n\n@dataclass\nclass AnnotationTask:\n    \"\"\"Structure for human annotation task.\"\"\"\n    response: str\n    question: str\n    context: Optional[str] = None\n\n    def get_annotation_form(self) -> dict:\n        return {\n            \"question\": self.question,\n            \"context\": self.context,\n            \"response\": self.response,\n            \"ratings\": {\n                \"accuracy\": {\n                    \"scale\": \"1-5\",\n                    \"description\": \"Is the response factually correct?\"\n                },\n                \"relevance\": {\n                    \"scale\": \"1-5\",\n                    \"description\": \"Does it answer the question?\"\n                },\n                \"coherence\": {\n                    \"scale\": \"1-5\",\n                    \"description\": \"Is it logically consistent?\"\n                }\n            },\n            \"issues\": {\n                \"factual_error\": False,\n                \"hallucination\": False,\n                \"off_topic\": False,\n                \"unsafe_content\": False\n            },\n            \"feedback\": \"\"\n        }\n```\n\n### Inter-Rater Agreement\n\n```python\nfrom sklearn.metrics import cohen_kappa_score\n\ndef calculate_agreement(\n    rater1_scores: list[int],\n    rater2_scores: list[int]\n) -> dict:\n    \"\"\"Calculate inter-rater agreement.\"\"\"\n    kappa = cohen_kappa_score(rater1_scores, rater2_scores)\n\n    if kappa \u003C 0:\n        interpretation = \"Poor\"\n    elif kappa \u003C 0.2:\n        interpretation = \"Slight\"\n    elif kappa \u003C 0.4:\n        interpretation = \"Fair\"\n    elif kappa \u003C 0.6:\n        interpretation = \"Moderate\"\n    elif kappa \u003C 0.8:\n        interpretation = \"Substantial\"\n    else:\n        interpretation = \"Almost Perfect\"\n\n    return {\n        \"kappa\": kappa,\n        \"interpretation\": interpretation\n    }\n```\n\n## A\u002FB Testing\n\n### Statistical Testing Framework\n\n```python\nfrom scipy import stats\nimport numpy as np\nfrom dataclasses import dataclass, field\n\n@dataclass\nclass ABTest:\n    variant_a_name: str = \"A\"\n    variant_b_name: str = \"B\"\n    variant_a_scores: list[float] = field(default_factory=list)\n    variant_b_scores: list[float] = field(default_factory=list)\n\n    def add_result(self, variant: str, score: float):\n        \"\"\"Add evaluation result for a variant.\"\"\"\n        if variant == \"A\":\n            self.variant_a_scores.append(score)\n        else:\n            self.variant_b_scores.append(score)\n\n    def analyze(self, alpha: float = 0.05) -> dict:\n        \"\"\"Perform statistical analysis.\"\"\"\n        a_scores = np.array(self.variant_a_scores)\n        b_scores = np.array(self.variant_b_scores)\n\n        # T-test\n        t_stat, p_value = stats.ttest_ind(a_scores, b_scores)\n\n        # Effect size (Cohen's d)\n        pooled_std = np.sqrt((np.std(a_scores)**2 + np.std(b_scores)**2) \u002F 2)\n        cohens_d = (np.mean(b_scores) - np.mean(a_scores)) \u002F pooled_std\n\n        return {\n            \"variant_a_mean\": np.mean(a_scores),\n            \"variant_b_mean\": np.mean(b_scores),\n            \"difference\": np.mean(b_scores) - np.mean(a_scores),\n            \"relative_improvement\": (np.mean(b_scores) - np.mean(a_scores)) \u002F np.mean(a_scores),\n            \"p_value\": p_value,\n            \"statistically_significant\": p_value \u003C alpha,\n            \"cohens_d\": cohens_d,\n            \"effect_size\": self._interpret_cohens_d(cohens_d),\n            \"winner\": self.variant_b_name if np.mean(b_scores) > np.mean(a_scores) else self.variant_a_name\n        }\n\n    @staticmethod\n    def _interpret_cohens_d(d: float) -> str:\n        \"\"\"Interpret Cohen's d effect size.\"\"\"\n        abs_d = abs(d)\n        if abs_d \u003C 0.2:\n            return \"negligible\"\n        elif abs_d \u003C 0.5:\n            return \"small\"\n        elif abs_d \u003C 0.8:\n            return \"medium\"\n        else:\n            return \"large\"\n```\n\n## Regression Testing\n\n### Regression Detection\n\n```python\nfrom dataclasses import dataclass\n\n@dataclass\nclass RegressionResult:\n    metric: str\n    baseline: float\n    current: float\n    change: float\n    is_regression: bool\n\nclass RegressionDetector:\n    def __init__(self, baseline_results: dict, threshold: float = 0.05):\n        self.baseline = baseline_results\n        self.threshold = threshold\n\n    def check_for_regression(self, new_results: dict) -> dict:\n        \"\"\"Detect if new results show regression.\"\"\"\n        regressions = []\n\n        for metric in self.baseline.keys():\n            baseline_score = self.baseline[metric]\n            new_score = new_results.get(metric)\n\n            if new_score is None:\n                continue\n\n            # Calculate relative change\n            relative_change = (new_score - baseline_score) \u002F baseline_score\n\n            # Flag if significant decrease\n            is_regression = relative_change \u003C -self.threshold\n            if is_regression:\n                regressions.append(RegressionResult(\n                    metric=metric,\n                    baseline=baseline_score,\n                    current=new_score,\n                    change=relative_change,\n                    is_regression=True\n                ))\n\n        return {\n            \"has_regression\": len(regressions) > 0,\n            \"regressions\": regressions,\n            \"summary\": f\"{len(regressions)} metric(s) regressed\"\n        }\n```\n\n## LangSmith Evaluation Integration\n\n```python\nfrom langsmith import Client\nfrom langsmith.evaluation import evaluate, LangChainStringEvaluator\n\n# Initialize LangSmith client\nclient = Client()\n\n# Create dataset\ndataset = client.create_dataset(\"qa_test_cases\")\nclient.create_examples(\n    inputs=[{\"question\": q} for q in questions],\n    outputs=[{\"answer\": a} for a in expected_answers],\n    dataset_id=dataset.id\n)\n\n# Define evaluators\nevaluators = [\n    LangChainStringEvaluator(\"qa\"),           # QA correctness\n    LangChainStringEvaluator(\"context_qa\"),   # Context-grounded QA\n    LangChainStringEvaluator(\"cot_qa\"),       # Chain-of-thought QA\n]\n\n# Run evaluation\nasync def target_function(inputs: dict) -> dict:\n    result = await your_chain.ainvoke(inputs)\n    return {\"answer\": result}\n\nexperiment_results = await evaluate(\n    target_function,\n    data=dataset.name,\n    evaluators=evaluators,\n    experiment_prefix=\"v1.0.0\",\n    metadata={\"model\": \"claude-sonnet-4-6\", \"version\": \"1.0.0\"}\n)\n\nprint(f\"Mean score: {experiment_results.aggregate_metrics['qa']['mean']}\")\n```\n\n## Benchmarking\n\n### Running Benchmarks\n\n```python\nfrom dataclasses import dataclass\nimport numpy as np\n\n@dataclass\nclass BenchmarkResult:\n    metric: str\n    mean: float\n    std: float\n    min: float\n    max: float\n\nclass BenchmarkRunner:\n    def __init__(self, benchmark_dataset: list[dict]):\n        self.dataset = benchmark_dataset\n\n    async def run_benchmark(\n        self,\n        model,\n        metrics: list[Metric]\n    ) -> dict[str, BenchmarkResult]:\n        \"\"\"Run model on benchmark and calculate metrics.\"\"\"\n        results = {metric.name: [] for metric in metrics}\n\n        for example in self.dataset:\n            # Generate prediction\n            prediction = await model.predict(example[\"input\"])\n\n            # Calculate each metric\n            for metric in metrics:\n                score = metric.fn(\n                    prediction=prediction,\n                    reference=example[\"reference\"],\n                    context=example.get(\"context\")\n                )\n                results[metric.name].append(score)\n\n        # Aggregate results\n        return {\n            metric: BenchmarkResult(\n                metric=metric,\n                mean=np.mean(scores),\n                std=np.std(scores),\n                min=min(scores),\n                max=max(scores)\n            )\n            for metric, scores in results.items()\n        }\n```\n",{"name":5,"description":7},"main","87afe6ce96220accf50a345dea46784c8c9f210e","YYC3-Plugin-插件协议\u002FYYC3-Plugin-插件协议-plugins-claude\u002Fllm-application-dev\u002Fskills\u002Fllm-evaluation",0,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base","https:\u002F\u002Fkb.yyc3.top\u002F","https:\u002F\u002Fgithub.com\u002FYYC-Cube","万象归元于云枢 | 深栈智启新纪元","2026-11-10 10:37:32",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add YYC-Cube\u002FYYC3-Knowledge-Base --skill llm-evaluation",{"tool":39,"command":40},"bunx","bunx skills add YYC-Cube\u002FYYC3-Knowledge-Base --skill llm-evaluation",{"tool":42,"command":43},"pnpm","pnpm dlx skills add YYC-Cube\u002FYYC3-Knowledge-Base --skill llm-evaluation","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-Plugin-插件协议\u002FYYC3-Plugin-插件协议-plugins-claude\u002Fllm-application-dev\u002Fskills\u002Fllm-evaluation\u002FSKILL.md",true,[],[48,55,62,69,76,83],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":52,"categories":53,"keywords":54,"publishedAt":19},"01M7CTDWHERCXPQCP0K03A86AS","vercel-sandbox","Run agent-browser + Chrome inside Vercel Sandbox microVMs for browser automation from any Vercel-deployed app. Use when the user needs browser automation in a Vercel app (Next.js, SvelteKit, Nuxt, Remix, Astro, etc.), wants to run headless Chrome without binary size limits, needs persistent browser sessions across commands, or wants ephemeral isolated browser environments. Triggers include \"Vercel Sandbox browser\", \"microVM Chrome\", \"agent-browser in sandbox\", \"browser automation on Vercel\", or any task requiring Chrome in a Vercel Sandbox.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-WebSpec-Web标准\u002FYYC3-WebSpec-Web标准-standards\u002Fbrowser-agent\u002Fskills\u002Fvercel-sandbox\u002FSKILL.md",[],[],{"skillNo":56,"name":57,"nameZh":6,"description":58,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":59,"categories":60,"keywords":61,"publishedAt":19},"01M7CTD8D2BNJY54S65M47HEJA","slack","Interact with Slack workspaces using browser automation. Use when the user needs to check unread channels, navigate Slack, send messages, extract data, find information, search conversations, or automate any Slack task. Triggers include \"check my Slack\", \"what channels have unreads\", \"send a message to\", \"search Slack for\", \"extract from Slack\", \"find who said\", or any task requiring programmatic Slack interaction.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-WebSpec-Web标准\u002FYYC3-WebSpec-Web标准-standards\u002Fbrowser-agent\u002Fskills\u002Fslack\u002FSKILL.md",[],[],{"skillNo":63,"name":64,"nameZh":6,"description":65,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":66,"categories":67,"keywords":68,"publishedAt":19},"01M7CTCSSB8ZP3Q58C3YZDZK1W","electron","Automate Electron desktop apps (VS Code, Slack, Discord, Figma, Notion, Spotify, etc.) using agent-browser via Chrome DevTools Protocol. Use when the user needs to interact with an Electron app, automate a desktop app, connect to a running app, control a native app, or test an Electron application. Triggers include \"automate Slack app\", \"control VS Code\", \"interact with Discord app\", \"test this Electron app\", \"connect to desktop app\", or any task requiring automation of a native Electron application.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-WebSpec-Web标准\u002FYYC3-WebSpec-Web标准-standards\u002Fbrowser-agent\u002Fskills\u002Felectron\u002FSKILL.md",[],[],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":75,"publishedAt":19},"01M7CTCBJY76688V4DCZGWFHGQ","dogfood","Systematically explore and test a web application to find bugs, UX issues, and other problems. Use when asked to \"dogfood\", \"QA\", \"exploratory test\", \"find issues\", \"bug hunt\", \"test this app\u002Fsite\u002Fplatform\", or review the quality of a web application. Produces a structured report with full reproduction evidence -- step-by-step screenshots, repro videos, and detailed repro steps for every issue -- so findings can be handed directly to the responsible teams.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-WebSpec-Web标准\u002FYYC3-WebSpec-Web标准-standards\u002Fbrowser-agent\u002Fskills\u002Fdogfood\u002FSKILL.md",[],[],{"skillNo":77,"name":78,"nameZh":6,"description":79,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":80,"categories":81,"keywords":82,"publishedAt":19},"01M7CTC4H0FFGVEAQR9PA4GA1A","agent-browser","Browser automation CLI for AI agents. Use when the user needs to interact with websites, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task. Triggers include requests to \"open a website\", \"fill out a form\", \"click a button\", \"take a screenshot\", \"scrape data from a page\", \"test this web app\", \"login to a site\", \"automate browser actions\", or any task requiring programmatic web interaction.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-WebSpec-Web标准\u002FYYC3-WebSpec-Web标准-standards\u002Fbrowser-agent\u002Fskills\u002Fagent-browser\u002FSKILL.md",[],[],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":19},"01M7CTBKVFYBG78FDSABSDJTTQ","shadcn","Manages shadcn components and projects — adding, searching, fixing, debugging, styling, and composing UI, including chat interfaces. Provides project context, component docs, and usage examples. Applies when working with shadcn\u002Fui, component registries, presets, --preset codes, or any project with a components.json file. Also triggers for \"shadcn init\", \"create an app with --preset\", \"switch to --preset\", and for building a design system or applying a DESIGN.md or brand spec to shadcn\u002Fui.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-UI-界面组件\u002FYYC3-UI-界面组件-ui\u002Fskills\u002Fshadcn\u002FSKILL.md",[],[]]