[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7F9P15K9BHHTK2J14JZF8CX":3,"skill-same:01M7F9P15K9BHHTK2J14JZF8CX":63},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":28,"commitSha":29,"skillDirectory":30,"forksCount":31,"repositoryHtmlUrl":32,"repositoryHomepageUrl":6,"ownerHtmlUrl":33,"ownerBlogUrl":6,"ownerDescription":34,"updatedAt":35,"install":36,"resources":48},"01M7F9P15K9BHHTK2J14JZF8CX","modeling-and-evaluation",null,"Use for H2O LLM Studio model wrappers, losses, metrics, evaluation outputs, inference routing, plots, and AI-judge metric behavior.","lza6\u002FArmor-piercing-system-prompts","lza6","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F128880206?v=4","01M7EB3K7WPX5T28EJ2NVHE6EG","Chinese-tingfeng","01M7EB3K97E1KQ16QWKJMFNGX8",11,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F06-前端与设计\u002Fh2o-llmstudio\u002Fsub-skills\u002Fmodeling-and-evaluation\u002FSKILL.md",[],[],"2026-11-16 10:05:34",false,"---\nname: modeling-and-evaluation\ndescription: \"Use for H2O LLM Studio model wrappers, losses, metrics, evaluation\n  outputs, inference routing, plots, and AI-judge metric behavior.\"\ndisable-model-invocation: true\nmetadata:\n  disco-role: operating\nlicense: Apache 2.0\n---\n\n# H2O LLM Studio modeling and evaluation\n\nUse this sub-skill when the task is about how H2O LLM Studio maps a problem type to model wrappers, losses, metrics, inference behavior, generated prediction outputs, validation plots, or AI-judge metrics.\n\nDo **not** use this sub-skill for:\n\n- creating or repairing experiment YAML\u002Fdata schemas: use `configuration-and-data`;\n- launching, scheduling, or debugging training runs: use `training-and-experiments`;\n- interactive chat, model-card export, or Hugging Face publishing: use `export-and-prompt`.\n\n## Operating workflow\n\n1. Identify the `problem_type` first. Valid modeling\u002Fevaluation problem types are:\n   `text_causal_language_modeling`, `text_sequence_to_sequence_modeling`,\n   `text_dpo_modeling`, `text_causal_classification_modeling`, and\n   `text_causal_regression_modeling`.\n2. Load [references\u002Fmodel-and-metric-reference.md](references\u002Fmodel-and-metric-reference.md) for class mappings, forward\u002Fgenerate contracts, losses, metric direction, and expected result keys.\n3. Load [references\u002Fevaluation-workflows.md](references\u002Fevaluation-workflows.md) for `run_inference`, `run_eval`, prediction files, plot files, generation-vs-forward routing, and GPT\u002FMT-Bench judge behavior.\n4. For unsupported metrics, NaNs, empty outputs, OpenAI\u002FAzure endpoint issues, missing `prompts\u002F`, or classification\u002Fregression shape errors, load [references\u002Ftroubleshooting.md](references\u002Ftroubleshooting.md).\n5. Use [scripts\u002Finspect_problem_type.py](scripts\u002Finspect_problem_type.py) for a safe local inspection that lists the configured model\u002Floss\u002Fmetric\u002Fplot classes without instantiating or downloading a model.\n\n## Quick commands\n\n```bash\npython scripts\u002Finspect_problem_type.py --problem-type text_causal_classification_modeling --list-metrics\npython scripts\u002Finspect_problem_type.py --problem-type all --json\npython scripts\u002Finspect_problem_type.py --problem-type text_dpo_modeling --verify-imports\n```\n\nThe inspection script is safe by default: it prints static and import-level metadata only. It does not start training, call an AI judge, load Hugging Face weights, or create model instances.\n\n## Decision rules\n\n- For generative validation metrics other than `Perplexity`, causal LM, sequence-to-sequence, and DPO use generation; `Perplexity` uses a forward pass.\n- Classification and regression are non-generation tasks. They always use a forward pass and postprocess logits or regression head outputs into predictions before metrics are computed.\n- `GPT` metrics call an OpenAI-compatible Chat Completions endpoint and can incur network cost. Require endpoint, credential, and budget confirmation before running; use static reasoning or mocked tests when possible.\n- Validation artifacts are written by the training\u002Fevaluation workflow, not by the model classes themselves. Expect raw prediction pickle, prediction CSV, and parquet plot data when evaluation reaches rank 0 successfully.\n","# H2O LLM Studio modeling and evaluation\n\nUse this sub-skill when the task is about how H2O LLM Studio maps a problem type to model wrappers, losses, metrics, inference behavior, generated prediction outputs, validation plots, or AI-judge metrics.\n\nDo **not** use this sub-skill for:\n\n- creating or repairing experiment YAML\u002Fdata schemas: use `configuration-and-data`;\n- launching, scheduling, or debugging training runs: use `training-and-experiments`;\n- interactive chat, model-card export, or Hugging Face publishing: use `export-and-prompt`.\n\n## Operating workflow\n\n1. Identify the `problem_type` first. Valid modeling\u002Fevaluation problem types are:\n   `text_causal_language_modeling`, `text_sequence_to_sequence_modeling`,\n   `text_dpo_modeling`, `text_causal_classification_modeling`, and\n   `text_causal_regression_modeling`.\n2. Load [references\u002Fmodel-and-metric-reference.md](references\u002Fmodel-and-metric-reference.md) for class mappings, forward\u002Fgenerate contracts, losses, metric direction, and expected result keys.\n3. Load [references\u002Fevaluation-workflows.md](references\u002Fevaluation-workflows.md) for `run_inference`, `run_eval`, prediction files, plot files, generation-vs-forward routing, and GPT\u002FMT-Bench judge behavior.\n4. For unsupported metrics, NaNs, empty outputs, OpenAI\u002FAzure endpoint issues, missing `prompts\u002F`, or classification\u002Fregression shape errors, load [references\u002Ftroubleshooting.md](references\u002Ftroubleshooting.md).\n5. Use [scripts\u002Finspect_problem_type.py](scripts\u002Finspect_problem_type.py) for a safe local inspection that lists the configured model\u002Floss\u002Fmetric\u002Fplot classes without instantiating or downloading a model.\n\n## Quick commands\n\n```bash\npython scripts\u002Finspect_problem_type.py --problem-type text_causal_classification_modeling --list-metrics\npython scripts\u002Finspect_problem_type.py --problem-type all --json\npython scripts\u002Finspect_problem_type.py --problem-type text_dpo_modeling --verify-imports\n```\n\nThe inspection script is safe by default: it prints static and import-level metadata only. It does not start training, call an AI judge, load Hugging Face weights, or create model instances.\n\n## Decision rules\n\n- For generative validation metrics other than `Perplexity`, causal LM, sequence-to-sequence, and DPO use generation; `Perplexity` uses a forward pass.\n- Classification and regression are non-generation tasks. They always use a forward pass and postprocess logits or regression head outputs into predictions before metrics are computed.\n- `GPT` metrics call an OpenAI-compatible Chat Completions endpoint and can incur network cost. Require endpoint, credential, and budget confirmation before running; use static reasoning or mocked tests when possible.\n- Validation artifacts are written by the training\u002Fevaluation workflow, not by the model classes themselves. Expect raw prediction pickle, prediction CSV, and parquet plot data when evaluation reaches rank 0 successfully.\n",{"name":5,"license":24,"metadata":25,"description":7,"disable-model-invocation":27},"Apache 2.0",{"disco-role":26},"operating",true,"main","d69096e239f2a89aabc2cd078c47664a9d3b1e11","06-前端与设计\u002Fh2o-llmstudio\u002Fsub-skills\u002Fmodeling-and-evaluation",3,"https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts","https:\u002F\u002Fgithub.com\u002Flza6","个人vx：Tf00798\r\n个人✈：@TFGY999","2026-10-08 14:29:22",{"source":8,"skillName":5,"commands":37,"prompt":47,"zipAvailable":27},[38,41,44],{"tool":39,"command":40},"npx","npx skills add lza6\u002FArmor-piercing-system-prompts --skill modeling-and-evaluation",{"tool":42,"command":43},"bunx","bunx skills add lza6\u002FArmor-piercing-system-prompts --skill modeling-and-evaluation",{"tool":45,"command":46},"pnpm","pnpm dlx skills add lza6\u002FArmor-piercing-system-prompts --skill modeling-and-evaluation","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F06-前端与设计\u002Fh2o-llmstudio\u002Fsub-skills\u002Fmodeling-and-evaluation\u002FSKILL.md",[49,55,59],{"path":50,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":53,"sourceHtmlUrl":54},"references\u002Fevaluation-workflows.md","REFERENCE","参考文档",8438,"https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F06-前端与设计\u002Fh2o-llmstudio\u002Fsub-skills\u002Fmodeling-and-evaluation\u002Freferences\u002Fevaluation-workflows.md",{"path":56,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":57,"sourceHtmlUrl":58},"references\u002Fmodel-and-metric-reference.md",9983,"https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F06-前端与设计\u002Fh2o-llmstudio\u002Fsub-skills\u002Fmodeling-and-evaluation\u002Freferences\u002Fmodel-and-metric-reference.md",{"path":60,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":61,"sourceHtmlUrl":62},"references\u002Ftroubleshooting.md",7928,"https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F06-前端与设计\u002Fh2o-llmstudio\u002Fsub-skills\u002Fmodeling-and-evaluation\u002Freferences\u002Ftroubleshooting.md",[64,72,77,84,91,98],{"skillNo":65,"name":66,"nameZh":6,"description":67,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":68,"licenseFileUrl":6,"sourceHtmlUrl":69,"categories":70,"keywords":71,"publishedAt":19},"01M7FFG485KY83TD6HSC4YPCMC","obsidian-clipper-template-creator","Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.","MIT","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobsidian-clipper-template-creator\u002FSKILL.md",[],[],{"skillNo":73,"name":66,"nameZh":6,"description":67,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":68,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":76,"publishedAt":19},"01M7FFG0T6AZ0STEJPSB3CX84K","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobsidian-clipper-template-creator-antigravity-awesome-skills-main\u002FSKILL.md",[],[],{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":68,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":19},"01M7FFFY0QEM14SKWPDYGMXM2P","python-api","Programmatic Object-Detection-Metrics API guidance for BoundingBox, BoundingBoxes, Evaluator, enums, coordinate conversion, AP\u002FmAP semantics, and source-style import diagnostics.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobject-detection-metrics\u002Fsub-skills\u002Fpython-api\u002FSKILL.md",[],[],{"skillNo":85,"name":86,"nameZh":6,"description":87,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":68,"licenseFileUrl":6,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":19},"01M7FFFVZQZ2GJ0C0TGP7JHD20","file-evaluation","Folder-based PASCAL VOC AP\u002FmAP evaluation from ground-truth and detection text files.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobject-detection-metrics\u002Fsub-skills\u002Ffile-evaluation\u002FSKILL.md",[],[],{"skillNo":92,"name":93,"nameZh":6,"description":94,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":68,"licenseFileUrl":6,"sourceHtmlUrl":95,"categories":96,"keywords":97,"publishedAt":19},"01M7FFFSFNYSE77SJ6PE1CK2DA","object-detection-metrics","Guides legacy Object-Detection-Metrics PASCAL VOC AP\u002FmAP evaluation from detection text folders and direct Python API classes.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobject-detection-metrics\u002FSKILL.md",[],[],{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":68,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":19},"01M7FFFP0GCK1RVXBCPQ8APBMT","nx-workspace-patterns","Configure and optimize Nx monorepo workspaces. Use when setting up Nx, configuring project boundaries, optimizing build caching, or implementing affected commands.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fnx-workspace-patterns\u002FSKILL.md",[],[]]