[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M787T0FDYYMP7SJX3K2EGJ96":3,"skill-same:01M787T0FDYYMP7SJX3K2EGJ96":109},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":26,"commitSha":27,"skillDirectory":28,"forksCount":14,"repositoryHtmlUrl":29,"repositoryHomepageUrl":30,"ownerHtmlUrl":31,"ownerBlogUrl":32,"ownerDescription":33,"updatedAt":34,"install":35,"resources":48},"01M787T0FDYYMP7SJX3K2EGJ96","agent-platform-eval-flywheel",null,"Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology. Use when evaluating an agent or model, building an eval dataset, picking or writing evaluation metrics, analyzing failures, comparing results before and after a fix, or when guidance is needed on Agent Platform eval methodology — including dataset schema, LLM-as-judge scoring, and common failure causes. For fine-tuning, use agent-platform-tuning. For general production deployment, use agent-platform-deploy.","Solizardking\u002Fskillhub-main","Solizardking","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F108096810?v=4","01M786KDB10AYV042N0F9BD5B5","Clawd","01M786NYYXSZK1D05WM8JQPC48",0,"MIT","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: agent-platform-eval-flywheel\nmetadata:\n  category: AiAndMachineLearning\ndescription: >-\n  Measures and improves the quality of AI models and agents on Google Cloud\n  using the Eval Quality Flywheel methodology. Use when evaluating an agent or\n  model, building an eval dataset, picking or writing evaluation metrics,\n  analyzing failures, comparing results before and after a fix, or when\n  guidance is needed on Agent Platform eval methodology — including\n  dataset schema, LLM-as-judge scoring, and common failure causes. For\n  fine-tuning, use agent-platform-tuning. For general production deployment,\n  use agent-platform-deploy.\n---\n\n# Agent Platform Eval Flywheel Skill\n\nHelp users evaluate and iteratively improve GenAI models and agents using\nthe Agent Platform GenAI Evaluation SDK (`google.genai` \u002F `agentplatform`).\n\n## When to use this skill\n\n-   Evaluating GenAI agents or models with the Agent Platform GenAI\n    Evaluation SDK (`client.evals.evaluate()`).\n-   Creating evaluation datasets from session traces, pandas DataFrames, or\n    synthetic generation.\n-   Selecting, configuring, or writing custom evaluation metrics.\n-   Analyzing rubric verdicts, loss patterns, and clustering failures.\n-   Suggesting concrete code\u002Fprompt improvements based on eval results.\n-   Evaluating a model served on an Agent Platform **endpoint** (BYOM) or a\n    **Model-as-a-Service (MaaS)** model by ID — including deploying the model\n    first if needed. For this case, follow\n    [references\u002Fdeployment.md](references\u002Fdeployment.md) and use the\n    `endpoint_evaluation.py` \u002F `maas_evaluation.py` scripts.\n\n## Safety & Confirmation Tiers (CRITICAL)\n\nBefore executing any commands or scripts on behalf of the user, you MUST adhere\nto the following safety tiers based on the action requested:\n\n1.  **Tier R**: Read-only (`inspect_results.py`, `compare_results.py`, `validate_dataset.py`, `parse_adk_traces.py`, `render_html_report.py`)\n    *   **Rule**: No confirmation needed. You may execute these helper scripts immediately to inspect data, validate schemas, parse traces, or compare evaluation results.\n2.  **Tier M: Read-only with Compute Costs (`client.evals.run_inference`, `client.evals.evaluate`, `client.evals.generate_user_scenarios`, `client.evals.generate_loss_clusters`)**\n    *   **Rule**: These operations invoke LLMs or remote evaluation services\n    that consume compute resources and incur costs. This requires\n    **interactive confirmation** with 'Yes'\u002F'No' options. Once granted once,\n    you do not have to prompt for future evaluation.\n\n## Setup\n\nInstall the SDK:\n\n```bash\npip install google-cloud-aiplatform[evaluation]>=1.154.0 google-genai>=1.0.0\n```\n\nNeed `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION`. Check env vars\nfirst; if missing, ask the user. Newer Gemini models often need\n`location=\"global\"`.\n\n## The Quality Flywheel\n\nFive stages, run in order on the first pass, then loop 2 → 5 until quality\ntargets are met.\n\n### Shortcuts that waste time\n\n| Shortcut                                                 | Why it fails                                                                                              |\n| -------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |\n| \"I'll tune the metric threshold down so it passes.\"      | Hides real failures. Fix the agent, not the bar.                                                          |\n| \"This case is flaky, I'll skip it.\"                      | Flakiness reveals non-determinism in the agent. Fix with `temperature=0` or stricter instructions.        |\n| \"I just need to fix the eval dataset, not the agent.\"    | If expected outputs keep moving, the agent has a behavior problem.                                        |\n| \"I can tell from the trace it works — skip Stage 3.\"     | Self-grading doesn't generalize. Always run `evaluate()` and read scores.                                 |\n| \"One iteration is enough.\"                               | Expect 5–10+ iterations. Stopping early leaves regressions on other metrics undetected.                   |\n\n### 1. Prepare Data\n\nProduce an `EvaluationDataset`. There are three input shapes, pick the one\nthat matches the data the user already has:\n\n-   **`EvalCase` list (single-turn or multi-turn):**\n\n    ```python\n    from agentplatform import types\n    dataset = types.EvaluationDataset(eval_cases=[\n        types.EvalCase(prompt=\"What is 2+2?\", response=\"4\", reference=\"4\"),\n        # For multi-turn agent traces, set agent_data instead of prompt\u002Fresponse.\n    ])\n    ```\n\n    Multi-turn agent traces wrap each conversation in `AgentData` →\n    `ConversationTurn` → `AgentEvent`. See\n    [references\u002Fdataset_schema.md](references\u002Fdataset_schema.md) for the\n    full type hierarchy.\n\n-   **Pandas DataFrame (tabular sources — CSV, BigQuery, Sheets):**\n\n    ```python\n    import pandas as pd\n    from agentplatform import types\n\n    df = pd.DataFrame({\n        \"prompt\":    [\"What is 2+2?\", \"Capital of France?\"],\n        \"response\":  [\"4\",            \"Paris\"],\n        \"reference\": [\"4\",            \"Paris\"],\n    })\n    dataset = types.EvaluationDataset(eval_dataset_df=df)\n    ```\n\n    Column names must match the fields the chosen metrics expect (see\n    [references\u002Fdataset_schema.md](references\u002Fdataset_schema.md) for the\n    per-metric requirements table).\n\n-   **Cold start (no data at all):** synthesize scenarios server-side with\n    `client.evals.generate_user_scenarios(...)` and a\n    `UserScenarioGenerationConfig` (`user_scenario_count`,\n    `simulation_instruction`, `environment_data`). Stage 2 plays them out.\n\nFor ADK session dumps, use `scripts\u002Fparse_adk_traces.py` instead of writing\nthe conversion by hand.\n\n### 2. Run Inference\n\nPopulate responses\u002Ftraces on the dataset. **Skip this stage** if traces are\nalready complete (e.g., production logs or replay).\n\n```python\n# Agent eval — pass a callable wrapping the user's ADK Agent\u002FApp.\nclient.evals.run_inference(model=agent_callable, src=dataset)\n\n# Model eval — pass a model ID directly.\nclient.evals.run_inference(model=\"gemini-2.5-flash\", src=dataset)\n\n# Synthesized scenarios — let the simulator drive.\nclient.evals.run_inference(\n    model=agent_callable,\n    src=dataset,\n    user_simulator_config=UserSimulatorConfig(max_turn=10),\n)\n\n# DataFrame also works as src= — no EvalCase wrapping needed.\nclient.evals.run_inference(model=\"gemini-2.5-flash\", src=df)\n```\n\n### 3. Grade (always run)\n\n```python\nresult = client.evals.evaluate(dataset=dataset, metrics=[...])\n```\n\n**Pick metrics by what you want to measure.** Full catalog in\n[references\u002Fmetric_registry.md](references\u002Fmetric_registry.md).\n\n**Agent metrics (multi-turn, adaptive rubrics)** — start here for agent eval.\n\n| Goal                                          | Metric                          |\n| --------------------------------------------- | ------------------------------- |\n| Did the agent achieve the user's goal?        | `multi_turn_task_success`       |\n| Was the reasoning path logical and efficient? | `multi_turn_trajectory_quality` |\n| Tool\u002Ffunction calling quality across turns    | `multi_turn_tool_use_quality`   |\n| Overall conversational quality                | `multi_turn_general_quality`    |\n| Final response quality (no reference needed)  | `final_response_quality`        |\n| Final response vs. a golden reference         | `final_response_match`          |\n| Single-turn tool use                          | `tool_use_quality`              |\n\n**General quality metrics (single-turn, adaptive rubrics)** — for model eval.\n\n| Goal                                                  | Metric                  |\n| ----------------------------------------------------- | ----------------------- |\n| Overall response quality (recommended starting point) | `general_quality`       |\n| Linguistic quality (fluency, coherence, grammar)      | `text_quality`          |\n| Adherence to specific constraints \u002F instructions      | `instruction_following` |\n\n**Static rubric metrics (fixed criteria)** — apply alongside the above.\n\n| Goal                                              | Metric          |\n| ------------------------------------------------- | --------------- |\n| Catch hallucinated claims (RAG, factual answers)  | `hallucination` |\n| Factuality \u002F consistency against provided context | `grounding`     |\n| Safety policy compliance                          | `safety`        |\n\n**Domain-specific check no built-in covers:** write a custom metric.\n\n-   **Predefined:** `types.RubricMetric.\u003CNAME>` — server-side AutoRater, no\n    judge model needed.\n-   **Custom LLM-as-a-judge:** `types.LLMMetric` with `prompt_template` or\n    `types.MetricPromptBuilder` for structured rubrics.\n-   **Custom code:** `types.CodeExecutionMetric` with a `custom_function`\n    string containing `def evaluate(instance: dict)` for remote sandboxed\n    execution; or `types.Metric` with `custom_function=\u003Ccallable>` for\n    local execution.\n\n**Always persist the result** so Stage 4 and 5 can read it. Save both JSON\n(machine-readable, diffable) and HTML (human-readable, linkable):\n\n```python\nimport datetime\nfrom pathlib import Path\n\nfrom agentplatform._genai import _evals_visualization\n\nout_dir = Path(\"artifacts\u002Fgrade_results\")\nout_dir.mkdir(parents=True, exist_ok=True)\nts = datetime.datetime.now().strftime(\"%Y%m%d_%H%M%S\")\n\nresult_json = result.model_dump_json()\n(out_dir \u002F f\"results_{ts}.json\").write_text(result_json)\n\nhtml = _evals_visualization.get_evaluation_html(result_json)\n(out_dir \u002F f\"results_{ts}.html\").write_text(str(html))\n```\n\nOr after the fact: `scripts\u002Frender_html_report.py --type evaluation` or\n`scripts\u002Finspect_results.py --save-html`.\n\n### 4. Analyze Failures\n\nRead `summary_metrics` and `eval_case_results` — never fabricate scores.\nUse `scripts\u002Finspect_results.py --failing-only` to filter to failures.\n\nFor each failed metric, see\n[references\u002Ffailure_patterns.md](references\u002Ffailure_patterns.md) for deeper\ndiagnoses. The compact mapping:\n\n| Failing metric                       | What to change                                                                                                            |\n| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------- |\n| `multi_turn_task_success` low        | The agent isn't completing the goal — fix orchestration, missing tool calls, premature termination, wrong tool selection. |\n| `multi_turn_trajectory_quality` low  | The agent reaches the goal inefficiently — refine planning prompts, remove redundant tool calls.                          |\n| `multi_turn_tool_use_quality` low    | Fix tool descriptions, parameter docstrings, or agent instructions for tool selection.                                    |\n| `final_response_quality` low         | Read auto-generated rubric verdicts; refine instructions to address the worst-scoring criterion.                          |\n| `final_response_match` low           | The agent's final answer doesn't match the golden reference — adjust response format or update the reference.             |\n| `hallucination` low                  | Tighten instructions to stay grounded in tool output; verify the tool actually returned the claimed data.                 |\n| `grounding` low                      | The response contradicts the provided context — add explicit \"cite only from context\" instructions.                       |\n| `safety` low                         | Add safety guardrails; review the violating content category in the rubric verdict.                                       |\n| `general_quality` \u002F `text_quality` low | Adjust system instruction wording; the model's default phrasing is too generic for the task.                            |\n| `instruction_following` low          | The agent is ignoring constraints — restate them in the system instruction or use stricter wording.                       |\n| Agent calls wrong tools              | Fix tool descriptions, agent instructions, or `tool_config`.                                                              |\n| Agent calls extra tools              | Add explicit stop instructions, or switch to `multi_turn_tool_use_quality` to surface the extra calls in the rubric.      |\n\n**For 10+ failures on the same metric**, use the **Error Analysis service**\nto cluster failures into themes (L1\u002FL2 taxonomy categories) instead of\nreading every trace:\n\n```python\n# Only supports multi_turn_task_success and multi_turn_tool_use_quality.\n# Service runs in the global region.\nanalysis_client = agentplatform.Client(project=\"PROJECT_ID\", location=\"global\")\nresponse = analysis_client.evals.generate_loss_clusters(\n    eval_result=result,\n    metric=\"multi_turn_task_success\",\n    config={\"max_top_cluster_count\": 5},\n)\nfor r in response.results:\n    for cluster in r.clusters:\n        print(\n            f\"[{cluster.taxonomy_entry.l1_category}\u002F\"\n            f\"{cluster.taxonomy_entry.l2_category}] \"\n            f\"{cluster.item_count} cases — {cluster.taxonomy_entry.description}\"\n        )\n```\n\nSave `response.model_dump_json()` and render with\n`scripts\u002Frender_html_report.py --type loss-analysis`.\n\n### 5. Optimize & Iterate\n\nApply a fix targeting the failing metric. Re-run Stage 3. Compare with\n`scripts\u002Fcompare_results.py --baseline \u003Cprev> --candidate \u003Cnew>` to confirm\nthe target improved AND no other metric regressed.\n\nTrack progress across iterations:\n\n| Iteration | Metric A | Metric B | Change made             |\n| --------- | -------- | -------- | ----------------------- |\n| Baseline  | 0.62     | 0.55     | —                       |\n| v2        | 0.78     | 0.68     | Added grounding prompt  |\n| v3        | 0.81     | 0.72     | Fixed tool selection    |\n\nExpect 5–10+ iterations per failing case. Only after a case passes should\nyou expand coverage with more eval cases.\n\n## Proving your work\n\nNever claim eval results you didn't read from an actual `result` object.\n\n-   After running eval, print the `summary_metrics` table\n    (`scripts\u002Finspect_results.py`).\n-   After a fix, show before\u002Fafter via `scripts\u002Fcompare_results.py`.\n-   Before declaring success, confirm ALL cases pass — not just the one you\n    were working on.\n\nIf you can't produce the evidence (SDK call failed, result truncated,\nmetric unsupported), say so explicitly. Don't paper over gaps.\n\n## Rules of Engagement\n\n1.  **Always Plan First:** Before writing a script, output a `\u003Cplan>` block\n    detailing the steps you are about to take.\n2.  **Step-by-Step Execution:** Write the script, execute it, wait for\n    output, then analyze. Don't do everything in one response.\n3.  **Standard Python:** Use standard Python imports (`import\n    agentplatform`, `from google.genai import types`). Don't use internal\n    import paths.\n4.  **Verify Before Guessing:** When unsure about SDK types or metrics,\n    check the SDK source code rather than guessing or hallucinating.\n\n## SDK Quick Reference\n\n```python\nimport agentplatform\nfrom agentplatform import types\nfrom google.genai import types as genai_types\nimport pandas as pd\n\n# Initialize client\nclient = agentplatform.Client(project=\"PROJECT_ID\", location=\"LOCATION\")\n\n# --- SINGLE-TURN EVAL (EvalCase list) ---\ndataset = types.EvaluationDataset(eval_cases=[\n    types.EvalCase(prompt=\"Query here\", response=\"Model response here\"),\n])\n\n# --- SINGLE-TURN EVAL (pandas DataFrame) ---\ndf = pd.DataFrame({\n    \"prompt\":   [\"Q1\", \"Q2\"],\n    \"response\": [\"A1\", \"A2\"],\n})\ndataset = types.EvaluationDataset(eval_dataset_df=df)\n\n# --- MULTI-TURN AGENT EVAL ---\nagent_data = types.evals.AgentData(\n    agents={\"my_agent\": types.evals.AgentConfig(\n        agent_id=\"my_agent\", instruction=\"You are helpful.\")},\n    turns=[types.evals.ConversationTurn(turn_index=0, events=[\n        types.evals.AgentEvent(author=\"user\",\n            content=genai_types.Content(role=\"user\",\n                parts=[genai_types.Part(text=\"Hello\")])),\n        types.evals.AgentEvent(author=\"my_agent\",\n            content=genai_types.Content(role=\"model\",\n                parts=[genai_types.Part(text=\"Hi! How can I help?\")])),\n    ])],\n)\ndataset = types.EvaluationDataset(\n    eval_cases=[types.EvalCase(agent_data=agent_data)])\n\n# --- METRICS ---\npredefined = types.RubricMetric.MULTI_TURN_TRAJECTORY_QUALITY\ncustom_llm = types.LLMMetric(name=\"tone\",\n    prompt_template=\"Is this polite? Response: {response}\")\ncustom_code = types.CodeExecutionMetric(name=\"check\",\n    custom_function='def evaluate(instance): return {\"score\": 1.0}')\n\n# --- EVALUATE ---\nresult = client.evals.evaluate(dataset=dataset, metrics=[predefined])\n\n# --- RESULTS ---\nfor s in result.summary_metrics:\n    print(f\"{s.metric_name}: mean={s.mean_score}, pass_rate={s.pass_rate}\")\nfor case in result.eval_case_results:\n    for cand in case.response_candidate_results:\n        for name, r in cand.metric_results.items():\n            print(f\"  {name}: score={r.score}, explanation={r.explanation}\")\n```\n\nSee [references\u002Fsdk_patterns.md](references\u002Fsdk_patterns.md) for advanced\npatterns: synthetic data generation, pairwise comparison,\n`MetricPromptBuilder`, multi-agent evaluation.\n\n## Bundled scripts\n\nScript                   | When to use\n------------------------ | -----------\n`validate_dataset.py`    | Before Stage 3 — catch malformed `EvaluationDataset` JSON.\n`parse_adk_traces.py`    | Stage 1 — convert ADK session dumps to the canonical dataset shape.\n`inspect_results.py`     | Stages 3\u002F4 — render summary + per-case scores. `--save-html` for a browsable report.\n`compare_results.py`     | Stage 5 — diff baseline vs. candidate, detect regressions.\n`render_html_report.py`  | Render HTML from a saved result JSON or loss-clusters JSON.\n`endpoint_evaluation.py` | Stages 2\u002F3 against a deployed Agent Platform endpoint (BYOM). See [references\u002Fdeployment.md](references\u002Fdeployment.md).\n`maas_evaluation.py`     | Stages 2\u002F3 against a Model-as-a-Service model by ID. See [references\u002Fdeployment.md](references\u002Fdeployment.md).\n","# Agent Platform Eval Flywheel Skill\n\nHelp users evaluate and iteratively improve GenAI models and agents using\nthe Agent Platform GenAI Evaluation SDK (`google.genai` \u002F `agentplatform`).\n\n## When to use this skill\n\n-   Evaluating GenAI agents or models with the Agent Platform GenAI\n    Evaluation SDK (`client.evals.evaluate()`).\n-   Creating evaluation datasets from session traces, pandas DataFrames, or\n    synthetic generation.\n-   Selecting, configuring, or writing custom evaluation metrics.\n-   Analyzing rubric verdicts, loss patterns, and clustering failures.\n-   Suggesting concrete code\u002Fprompt improvements based on eval results.\n-   Evaluating a model served on an Agent Platform **endpoint** (BYOM) or a\n    **Model-as-a-Service (MaaS)** model by ID — including deploying the model\n    first if needed. For this case, follow\n    [references\u002Fdeployment.md](references\u002Fdeployment.md) and use the\n    `endpoint_evaluation.py` \u002F `maas_evaluation.py` scripts.\n\n## Safety & Confirmation Tiers (CRITICAL)\n\nBefore executing any commands or scripts on behalf of the user, you MUST adhere\nto the following safety tiers based on the action requested:\n\n1.  **Tier R**: Read-only (`inspect_results.py`, `compare_results.py`, `validate_dataset.py`, `parse_adk_traces.py`, `render_html_report.py`)\n    *   **Rule**: No confirmation needed. You may execute these helper scripts immediately to inspect data, validate schemas, parse traces, or compare evaluation results.\n2.  **Tier M: Read-only with Compute Costs (`client.evals.run_inference`, `client.evals.evaluate`, `client.evals.generate_user_scenarios`, `client.evals.generate_loss_clusters`)**\n    *   **Rule**: These operations invoke LLMs or remote evaluation services\n    that consume compute resources and incur costs. This requires\n    **interactive confirmation** with 'Yes'\u002F'No' options. Once granted once,\n    you do not have to prompt for future evaluation.\n\n## Setup\n\nInstall the SDK:\n\n```bash\npip install google-cloud-aiplatform[evaluation]>=1.154.0 google-genai>=1.0.0\n```\n\nNeed `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION`. Check env vars\nfirst; if missing, ask the user. Newer Gemini models often need\n`location=\"global\"`.\n\n## The Quality Flywheel\n\nFive stages, run in order on the first pass, then loop 2 → 5 until quality\ntargets are met.\n\n### Shortcuts that waste time\n\n| Shortcut                                                 | Why it fails                                                                                              |\n| -------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |\n| \"I'll tune the metric threshold down so it passes.\"      | Hides real failures. Fix the agent, not the bar.                                                          |\n| \"This case is flaky, I'll skip it.\"                      | Flakiness reveals non-determinism in the agent. Fix with `temperature=0` or stricter instructions.        |\n| \"I just need to fix the eval dataset, not the agent.\"    | If expected outputs keep moving, the agent has a behavior problem.                                        |\n| \"I can tell from the trace it works — skip Stage 3.\"     | Self-grading doesn't generalize. Always run `evaluate()` and read scores.                                 |\n| \"One iteration is enough.\"                               | Expect 5–10+ iterations. Stopping early leaves regressions on other metrics undetected.                   |\n\n### 1. Prepare Data\n\nProduce an `EvaluationDataset`. There are three input shapes, pick the one\nthat matches the data the user already has:\n\n-   **`EvalCase` list (single-turn or multi-turn):**\n\n    ```python\n    from agentplatform import types\n    dataset = types.EvaluationDataset(eval_cases=[\n        types.EvalCase(prompt=\"What is 2+2?\", response=\"4\", reference=\"4\"),\n        # For multi-turn agent traces, set agent_data instead of prompt\u002Fresponse.\n    ])\n    ```\n\n    Multi-turn agent traces wrap each conversation in `AgentData` →\n    `ConversationTurn` → `AgentEvent`. See\n    [references\u002Fdataset_schema.md](references\u002Fdataset_schema.md) for the\n    full type hierarchy.\n\n-   **Pandas DataFrame (tabular sources — CSV, BigQuery, Sheets):**\n\n    ```python\n    import pandas as pd\n    from agentplatform import types\n\n    df = pd.DataFrame({\n        \"prompt\":    [\"What is 2+2?\", \"Capital of France?\"],\n        \"response\":  [\"4\",            \"Paris\"],\n        \"reference\": [\"4\",            \"Paris\"],\n    })\n    dataset = types.EvaluationDataset(eval_dataset_df=df)\n    ```\n\n    Column names must match the fields the chosen metrics expect (see\n    [references\u002Fdataset_schema.md](references\u002Fdataset_schema.md) for the\n    per-metric requirements table).\n\n-   **Cold start (no data at all):** synthesize scenarios server-side with\n    `client.evals.generate_user_scenarios(...)` and a\n    `UserScenarioGenerationConfig` (`user_scenario_count`,\n    `simulation_instruction`, `environment_data`). Stage 2 plays them out.\n\nFor ADK session dumps, use `scripts\u002Fparse_adk_traces.py` instead of writing\nthe conversion by hand.\n\n### 2. Run Inference\n\nPopulate responses\u002Ftraces on the dataset. **Skip this stage** if traces are\nalready complete (e.g., production logs or replay).\n\n```python\n# Agent eval — pass a callable wrapping the user's ADK Agent\u002FApp.\nclient.evals.run_inference(model=agent_callable, src=dataset)\n\n# Model eval — pass a model ID directly.\nclient.evals.run_inference(model=\"gemini-2.5-flash\", src=dataset)\n\n# Synthesized scenarios — let the simulator drive.\nclient.evals.run_inference(\n    model=agent_callable,\n    src=dataset,\n    user_simulator_config=UserSimulatorConfig(max_turn=10),\n)\n\n# DataFrame also works as src= — no EvalCase wrapping needed.\nclient.evals.run_inference(model=\"gemini-2.5-flash\", src=df)\n```\n\n### 3. Grade (always run)\n\n```python\nresult = client.evals.evaluate(dataset=dataset, metrics=[...])\n```\n\n**Pick metrics by what you want to measure.** Full catalog in\n[references\u002Fmetric_registry.md](references\u002Fmetric_registry.md).\n\n**Agent metrics (multi-turn, adaptive rubrics)** — start here for agent eval.\n\n| Goal                                          | Metric                          |\n| --------------------------------------------- | ------------------------------- |\n| Did the agent achieve the user's goal?        | `multi_turn_task_success`       |\n| Was the reasoning path logical and efficient? | `multi_turn_trajectory_quality` |\n| Tool\u002Ffunction calling quality across turns    | `multi_turn_tool_use_quality`   |\n| Overall conversational quality                | `multi_turn_general_quality`    |\n| Final response quality (no reference needed)  | `final_response_quality`        |\n| Final response vs. a golden reference         | `final_response_match`          |\n| Single-turn tool use                          | `tool_use_quality`              |\n\n**General quality metrics (single-turn, adaptive rubrics)** — for model eval.\n\n| Goal                                                  | Metric                  |\n| ----------------------------------------------------- | ----------------------- |\n| Overall response quality (recommended starting point) | `general_quality`       |\n| Linguistic quality (fluency, coherence, grammar)      | `text_quality`          |\n| Adherence to specific constraints \u002F instructions      | `instruction_following` |\n\n**Static rubric metrics (fixed criteria)** — apply alongside the above.\n\n| Goal                                              | Metric          |\n| ------------------------------------------------- | --------------- |\n| Catch hallucinated claims (RAG, factual answers)  | `hallucination` |\n| Factuality \u002F consistency against provided context | `grounding`     |\n| Safety policy compliance                          | `safety`        |\n\n**Domain-specific check no built-in covers:** write a custom metric.\n\n-   **Predefined:** `types.RubricMetric.\u003CNAME>` — server-side AutoRater, no\n    judge model needed.\n-   **Custom LLM-as-a-judge:** `types.LLMMetric` with `prompt_template` or\n    `types.MetricPromptBuilder` for structured rubrics.\n-   **Custom code:** `types.CodeExecutionMetric` with a `custom_function`\n    string containing `def evaluate(instance: dict)` for remote sandboxed\n    execution; or `types.Metric` with `custom_function=\u003Ccallable>` for\n    local execution.\n\n**Always persist the result** so Stage 4 and 5 can read it. Save both JSON\n(machine-readable, diffable) and HTML (human-readable, linkable):\n\n```python\nimport datetime\nfrom pathlib import Path\n\nfrom agentplatform._genai import _evals_visualization\n\nout_dir = Path(\"artifacts\u002Fgrade_results\")\nout_dir.mkdir(parents=True, exist_ok=True)\nts = datetime.datetime.now().strftime(\"%Y%m%d_%H%M%S\")\n\nresult_json = result.model_dump_json()\n(out_dir \u002F f\"results_{ts}.json\").write_text(result_json)\n\nhtml = _evals_visualization.get_evaluation_html(result_json)\n(out_dir \u002F f\"results_{ts}.html\").write_text(str(html))\n```\n\nOr after the fact: `scripts\u002Frender_html_report.py --type evaluation` or\n`scripts\u002Finspect_results.py --save-html`.\n\n### 4. Analyze Failures\n\nRead `summary_metrics` and `eval_case_results` — never fabricate scores.\nUse `scripts\u002Finspect_results.py --failing-only` to filter to failures.\n\nFor each failed metric, see\n[references\u002Ffailure_patterns.md](references\u002Ffailure_patterns.md) for deeper\ndiagnoses. The compact mapping:\n\n| Failing metric                       | What to change                                                                                                            |\n| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------- |\n| `multi_turn_task_success` low        | The agent isn't completing the goal — fix orchestration, missing tool calls, premature termination, wrong tool selection. |\n| `multi_turn_trajectory_quality` low  | The agent reaches the goal inefficiently — refine planning prompts, remove redundant tool calls.                          |\n| `multi_turn_tool_use_quality` low    | Fix tool descriptions, parameter docstrings, or agent instructions for tool selection.                                    |\n| `final_response_quality` low         | Read auto-generated rubric verdicts; refine instructions to address the worst-scoring criterion.                          |\n| `final_response_match` low           | The agent's final answer doesn't match the golden reference — adjust response format or update the reference.             |\n| `hallucination` low                  | Tighten instructions to stay grounded in tool output; verify the tool actually returned the claimed data.                 |\n| `grounding` low                      | The response contradicts the provided context — add explicit \"cite only from context\" instructions.                       |\n| `safety` low                         | Add safety guardrails; review the violating content category in the rubric verdict.                                       |\n| `general_quality` \u002F `text_quality` low | Adjust system instruction wording; the model's default phrasing is too generic for the task.                            |\n| `instruction_following` low          | The agent is ignoring constraints — restate them in the system instruction or use stricter wording.                       |\n| Agent calls wrong tools              | Fix tool descriptions, agent instructions, or `tool_config`.                                                              |\n| Agent calls extra tools              | Add explicit stop instructions, or switch to `multi_turn_tool_use_quality` to surface the extra calls in the rubric.      |\n\n**For 10+ failures on the same metric**, use the **Error Analysis service**\nto cluster failures into themes (L1\u002FL2 taxonomy categories) instead of\nreading every trace:\n\n```python\n# Only supports multi_turn_task_success and multi_turn_tool_use_quality.\n# Service runs in the global region.\nanalysis_client = agentplatform.Client(project=\"PROJECT_ID\", location=\"global\")\nresponse = analysis_client.evals.generate_loss_clusters(\n    eval_result=result,\n    metric=\"multi_turn_task_success\",\n    config={\"max_top_cluster_count\": 5},\n)\nfor r in response.results:\n    for cluster in r.clusters:\n        print(\n            f\"[{cluster.taxonomy_entry.l1_category}\u002F\"\n            f\"{cluster.taxonomy_entry.l2_category}] \"\n            f\"{cluster.item_count} cases — {cluster.taxonomy_entry.description}\"\n        )\n```\n\nSave `response.model_dump_json()` and render with\n`scripts\u002Frender_html_report.py --type loss-analysis`.\n\n### 5. Optimize & Iterate\n\nApply a fix targeting the failing metric. Re-run Stage 3. Compare with\n`scripts\u002Fcompare_results.py --baseline \u003Cprev> --candidate \u003Cnew>` to confirm\nthe target improved AND no other metric regressed.\n\nTrack progress across iterations:\n\n| Iteration | Metric A | Metric B | Change made             |\n| --------- | -------- | -------- | ----------------------- |\n| Baseline  | 0.62     | 0.55     | —                       |\n| v2        | 0.78     | 0.68     | Added grounding prompt  |\n| v3        | 0.81     | 0.72     | Fixed tool selection    |\n\nExpect 5–10+ iterations per failing case. Only after a case passes should\nyou expand coverage with more eval cases.\n\n## Proving your work\n\nNever claim eval results you didn't read from an actual `result` object.\n\n-   After running eval, print the `summary_metrics` table\n    (`scripts\u002Finspect_results.py`).\n-   After a fix, show before\u002Fafter via `scripts\u002Fcompare_results.py`.\n-   Before declaring success, confirm ALL cases pass — not just the one you\n    were working on.\n\nIf you can't produce the evidence (SDK call failed, result truncated,\nmetric unsupported), say so explicitly. Don't paper over gaps.\n\n## Rules of Engagement\n\n1.  **Always Plan First:** Before writing a script, output a `\u003Cplan>` block\n    detailing the steps you are about to take.\n2.  **Step-by-Step Execution:** Write the script, execute it, wait for\n    output, then analyze. Don't do everything in one response.\n3.  **Standard Python:** Use standard Python imports (`import\n    agentplatform`, `from google.genai import types`). Don't use internal\n    import paths.\n4.  **Verify Before Guessing:** When unsure about SDK types or metrics,\n    check the SDK source code rather than guessing or hallucinating.\n\n## SDK Quick Reference\n\n```python\nimport agentplatform\nfrom agentplatform import types\nfrom google.genai import types as genai_types\nimport pandas as pd\n\n# Initialize client\nclient = agentplatform.Client(project=\"PROJECT_ID\", location=\"LOCATION\")\n\n# --- SINGLE-TURN EVAL (EvalCase list) ---\ndataset = types.EvaluationDataset(eval_cases=[\n    types.EvalCase(prompt=\"Query here\", response=\"Model response here\"),\n])\n\n# --- SINGLE-TURN EVAL (pandas DataFrame) ---\ndf = pd.DataFrame({\n    \"prompt\":   [\"Q1\", \"Q2\"],\n    \"response\": [\"A1\", \"A2\"],\n})\ndataset = types.EvaluationDataset(eval_dataset_df=df)\n\n# --- MULTI-TURN AGENT EVAL ---\nagent_data = types.evals.AgentData(\n    agents={\"my_agent\": types.evals.AgentConfig(\n        agent_id=\"my_agent\", instruction=\"You are helpful.\")},\n    turns=[types.evals.ConversationTurn(turn_index=0, events=[\n        types.evals.AgentEvent(author=\"user\",\n            content=genai_types.Content(role=\"user\",\n                parts=[genai_types.Part(text=\"Hello\")])),\n        types.evals.AgentEvent(author=\"my_agent\",\n            content=genai_types.Content(role=\"model\",\n                parts=[genai_types.Part(text=\"Hi! How can I help?\")])),\n    ])],\n)\ndataset = types.EvaluationDataset(\n    eval_cases=[types.EvalCase(agent_data=agent_data)])\n\n# --- METRICS ---\npredefined = types.RubricMetric.MULTI_TURN_TRAJECTORY_QUALITY\ncustom_llm = types.LLMMetric(name=\"tone\",\n    prompt_template=\"Is this polite? Response: {response}\")\ncustom_code = types.CodeExecutionMetric(name=\"check\",\n    custom_function='def evaluate(instance): return {\"score\": 1.0}')\n\n# --- EVALUATE ---\nresult = client.evals.evaluate(dataset=dataset, metrics=[predefined])\n\n# --- RESULTS ---\nfor s in result.summary_metrics:\n    print(f\"{s.metric_name}: mean={s.mean_score}, pass_rate={s.pass_rate}\")\nfor case in result.eval_case_results:\n    for cand in case.response_candidate_results:\n        for name, r in cand.metric_results.items():\n            print(f\"  {name}: score={r.score}, explanation={r.explanation}\")\n```\n\nSee [references\u002Fsdk_patterns.md](references\u002Fsdk_patterns.md) for advanced\npatterns: synthetic data generation, pairwise comparison,\n`MetricPromptBuilder`, multi-agent evaluation.\n\n## Bundled scripts\n\nScript                   | When to use\n------------------------ | -----------\n`validate_dataset.py`    | Before Stage 3 — catch malformed `EvaluationDataset` JSON.\n`parse_adk_traces.py`    | Stage 1 — convert ADK session dumps to the canonical dataset shape.\n`inspect_results.py`     | Stages 3\u002F4 — render summary + per-case scores. `--save-html` for a browsable report.\n`compare_results.py`     | Stage 5 — diff baseline vs. candidate, detect regressions.\n`render_html_report.py`  | Render HTML from a saved result JSON or loss-clusters JSON.\n`endpoint_evaluation.py` | Stages 2\u002F3 against a deployed Agent Platform endpoint (BYOM). See [references\u002Fdeployment.md](references\u002Fdeployment.md).\n`maas_evaluation.py`     | Stages 2\u002F3 against a Model-as-a-Service model by ID. See [references\u002Fdeployment.md](references\u002Fdeployment.md).\n",{"name":5,"metadata":24,"description":7},{"category":25},"AiAndMachineLearning","main","be3de0244dc1a57e323d8160787ebb40fb64930e","public\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main","https:\u002F\u002Fskills.x402.wtf","https:\u002F\u002Fgithub.com\u002FSolizardking","https:\u002F\u002Fmusebook.trade","Got a mile high and never came down","2026-10-08 23:20:13",{"source":8,"skillName":5,"commands":36,"prompt":46,"zipAvailable":47},[37,40,43],{"tool":38,"command":39},"npx","npx skills add Solizardking\u002Fskillhub-main --skill agent-platform-eval-flywheel",{"tool":41,"command":42},"bunx","bunx skills add Solizardking\u002Fskillhub-main --skill agent-platform-eval-flywheel",{"tool":44,"command":45},"pnpm","pnpm dlx skills add Solizardking\u002Fskillhub-main --skill agent-platform-eval-flywheel","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002FSKILL.md",true,[49,55,59,63,67,71,75,81,85,89,93,97,101,105],{"path":50,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":53,"sourceHtmlUrl":54},"metadata.json","REFERENCE","参考文档",761,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Fmetadata.json",{"path":56,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":57,"sourceHtmlUrl":58},"references\u002Fdataset_schema.md",9358,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Freferences\u002Fdataset_schema.md",{"path":60,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":61,"sourceHtmlUrl":62},"references\u002Fdeployment.md",6948,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Freferences\u002Fdeployment.md",{"path":64,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":65,"sourceHtmlUrl":66},"references\u002Ffailure_patterns.md",6356,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Freferences\u002Ffailure_patterns.md",{"path":68,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":69,"sourceHtmlUrl":70},"references\u002Fmetric_registry.md",12479,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Freferences\u002Fmetric_registry.md",{"path":72,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":73,"sourceHtmlUrl":74},"references\u002Fsdk_patterns.md",8229,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Freferences\u002Fsdk_patterns.md",{"path":76,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":79,"sourceHtmlUrl":80},"scripts\u002Fcompare_results.py","SCRIPT","脚本",6328,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Fscripts\u002Fcompare_results.py",{"path":82,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":83,"sourceHtmlUrl":84},"scripts\u002Fendpoint_evaluation.py",5410,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Fscripts\u002Fendpoint_evaluation.py",{"path":86,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":87,"sourceHtmlUrl":88},"scripts\u002Finspect_results.py",7068,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Fscripts\u002Finspect_results.py",{"path":90,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":91,"sourceHtmlUrl":92},"scripts\u002Fmaas_evaluation.py",2344,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Fscripts\u002Fmaas_evaluation.py",{"path":94,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":95,"sourceHtmlUrl":96},"scripts\u002Fparse_adk_traces.py",8639,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Fscripts\u002Fparse_adk_traces.py",{"path":98,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":99,"sourceHtmlUrl":100},"scripts\u002Frender_html_report.py",3600,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Fscripts\u002Frender_html_report.py",{"path":102,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":103,"sourceHtmlUrl":104},"scripts\u002Fvalidate_dataset.py",10911,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Fscripts\u002Fvalidate_dataset.py",{"path":106,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":107,"sourceHtmlUrl":108},"verification.json",3505,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fpublic\u002Fapi\u002Fskills\u002Fgoogle\u002Fcloud\u002Fagent-platform-eval-flywheel\u002Fverification.json",[110,117,124,131,138,146],{"skillNo":111,"name":112,"nameZh":6,"description":113,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":114,"categories":115,"keywords":116,"publishedAt":19},"01M78E0TGCAMNPS207BA3G07S7","zkrouter","Self-hosted OpenAI-compatible LLM router — smart tier-based routing, ZK-stamped routing decisions, mode overrides, free OpenRouter at install via the Birth bot. Save 60-80% on inference costs by routing to the cheapest capable model across 12+ OpenRouter models.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzkrouter\u002FSKILL.md",[],[],{"skillNo":118,"name":119,"nameZh":6,"description":120,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":121,"categories":122,"keywords":123,"publishedAt":19},"01M78E0R1Z8BDT6EDTHY17MTNB","zk","For custom ZK Solana programs and privacy-preserving applications to prevent double spending. Guide to integrate rent-free nullifier PDAs for double-spend prevention.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk\u002FSKILL.md",[],[],{"skillNo":125,"name":126,"nameZh":6,"description":127,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":128,"categories":129,"keywords":130,"publishedAt":19},"01M78E0P3ZTZ1YWF8MD5MM23QE","zk-omni-messaging","Zero-knowledge omnichain messaging between Robinhood Chain and Solana using CheshireZkOmniMessenger (msgType 4) with nullifier anti-replay and the zk-omni-relayer service.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk-omni-messaging\u002FSKILL.md",[],[],{"skillNo":132,"name":133,"nameZh":6,"description":134,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":135,"categories":136,"keywords":137,"publishedAt":19},"01M78E0JXZ2PNB1896VXAGDBWD","zerion-wallet-data","Use when the user asks to check a wallet's portfolio, token balances, DeFi positions, PnL, transaction history, or NFTs. Provides interpreted on-chain data across 40+ EVM chains and Solana via x402 pay-per-request or API key — no raw RPC decoding needed.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzerion-wallet-data\u002FSKILL.md",[],[],{"skillNo":139,"name":140,"nameZh":6,"description":141,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":142,"sourceHtmlUrl":143,"categories":144,"keywords":145,"publishedAt":19},"01M78E0G14FH9GRRTQYMKD5E59","youtube-clipper","YouTube 视频智能剪辑工具。下载视频和字幕，AI 分析生成精细章节（几分钟级别）， 用户选择片段后自动剪辑、翻译字幕为中英双语、烧录字幕到视频，并生成总结文案。 使用场景：当用户需要剪辑 YouTube 视频、生成短视频片段、制作双语字幕版本时。 关键词：视频剪辑、YouTube、字幕翻译、双语字幕、视频下载、clip video","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FSKILL.md",[],[],{"skillNo":147,"name":148,"nameZh":6,"description":149,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":150,"categories":151,"keywords":152,"publishedAt":19},"01M78E0C2ZYA99CFFDNB7ZMK5S","yield-optimization","Multi-chain yield optimization via yield.xyz (StakeKit). Discover 2,988+ yield opportunities across 75+ blockchains, build deposit\u002Fwithdrawal transactions with shell scripts, and sign them with a MoonPay wallet. Use when the user wants to earn yield, find the best lending or staking rate, enter\u002Fexit a position, or check portfolio balances.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyield-optimization\u002FSKILL.md",[],[]]