[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M71FN248A0H4NM4VNA2T8ASV":3,"skill-same:01M71FN248A0H4NM4VNA2T8ASV":99},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":25,"publishedAt":46,"linkOnly":47,"markdownContent":48,"markdownBody":49,"frontMatter":50,"refName":65,"commitSha":66,"skillDirectory":67,"forksCount":13,"repositoryHtmlUrl":68,"repositoryHomepageUrl":6,"ownerHtmlUrl":69,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":70,"install":71,"resources":84},"01M71FN248A0H4NM4VNA2T8ASV","hermes-atropos-environments",null,"Build, test, and debug YOUSEF SHTIWE RL environments for Atropos training. Covers the YousefShtiweBaseEnv interface, reward functions, agent loop integration, evaluation with tools, wandb logging, and the three CLI modes (serve\u002Fprocess\u002Fevaluate). Use when creating, reviewing, or fixing RL environments in the yousef-sovereign-core repo.","gjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6","gjbvsjjvgm-sketch","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F278511911?v=4","01M71F3JXKZQ70H2KCHHTXKZDA","01M71F3JY8A79Z3NZBW08TF4JD",0,"MIT","https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fhermes-atropos-environments\u002FSKILL.md",[17,22],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",{"categoryNo":23,"categoryName":24,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000023","Skill与智能体开发",[26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45],"environments","hermes","hermes atropos environments","repo","when","with tools","for","interface","WandB","TRL","three","cli","loop","tuning","Tools","Evaluation","logging","harness","core","training","2026-11-10 10:12:30",false,"---\nname: hermes-atropos-environments\ndescription: Build, test, and debug YOUSEF SHTIWE RL environments for Atropos training. Covers the YousefShtiweBaseEnv interface, reward functions, agent loop integration, evaluation with tools, wandb logging, and the three CLI modes (serve\u002Fprocess\u002Fevaluate). Use when creating, reviewing, or fixing RL environments in the yousef-sovereign-core repo.\nversion: 1.1.0\nauthor: YOUSEF SHTIWE\nlicense: MIT\nmetadata:\n  hermes:\n    tags: [atropos, rl, environments, training, reinforcement-learning, reward-functions]\n    related_skills: [axolotl, grpo-rl-training, trl-fine-tuning, lm-evaluation-harness]\n---\n\n# YOUSEF SHTIWE Atropos Environments\n\nGuide for building RL environments in the yousef-sovereign-core repo that integrate with the Atropos training framework.\n\n## Architecture Overview\n\n```\nAtropos BaseEnv (atroposlib\u002Fenvs\u002Fbase.py)\n    └── YousefShtiweBaseEnv (environments\u002Fhermes_base_env.py)\n            ├── Handles agent loop orchestration\n            ├── Handles tool resolution per group\n            ├── Handles ToolContext for reward verification\n            └── YOUR ENVIRONMENT (environments\u002Fyour_env.py)\n                    Only implements: setup, get_next_item, format_prompt,\n                                    compute_reward, evaluate, wandb_log\n```\n\nHermes environments are special because they run a **multi-turn agent loop with tool calling** — not just single-turn completions. The base env handles the loop; you implement the task and scoring.\n\n## File Locations\n\n| File | Purpose |\n|------|---------|\n| `environments\u002Fhermes_base_env.py` | Base class with agent loop + tool resolution |\n| `environments\u002Fagent_loop.py` | `YousefShtiweLoop` + `AgentResult` dataclass |\n| `environments\u002Ftool_context.py` | `ToolContext` for reward verification |\n| `environments\u002Ftool_call_parsers.py` | Phase 2 tool call parsers (hermes, mistral, etc.) |\n| `environments\u002Fyour_env.py` | Your environment implementation |\n\n## Inference Setup — Ask the User First\n\n**IMPORTANT:** Before running any test, evaluation, or data generation command, always ask the user how they want to handle inference. Do NOT assume OpenRouter or any specific endpoint. Present these options:\n\n1. **OpenRouter** — Ask which model they want to use (e.g., `anthropic\u002Fclaude-sonnet-4.5`, `google\u002Fgemini-2.5-pro`, `meta-llama\u002Fllama-3.3-70b-instruct`, etc.). Requires `OPENROUTER_API_KEY` in environment.\n2. **Self-hosted VLLM endpoint** — Ask for their base URL (e.g., `http:\u002F\u002Flocalhost:8000\u002Fv1`) and model name. Set `--openai.server_type vllm`.\n3. **Other OpenAI-compatible API** — Ask for the base URL, model name, and any required API key. Set `--openai.server_type openai` and `--openai.health_check false`.\n4. **Local Atropos training server** — For `serve` mode with a live training loop. Default `http:\u002F\u002Flocalhost:8000\u002Fv1`.\n\nOnce the user tells you their setup, use those values in all CLI commands for that session. Example prompts:\n\n> \"Before I run this, how would you like to handle inference?\n> 1. OpenRouter (I'll need your preferred model, e.g. claude-sonnet-4.5)\n> 2. A self-hosted VLLM endpoint (give me the URL and model name)\n> 3. Another OpenAI-compatible API (give me the URL, model, and any auth details)\n> 4. Local Atropos training server (serve mode)\"\n\n### Key flags by provider:\n\n| Provider | `--openai.server_type` | `--openai.health_check` | `--openai.api_key` |\n|----------|----------------------|------------------------|-------------------|\n| OpenRouter | `openai` | `false` | `$OPENROUTER_API_KEY` |\n| VLLM (self-hosted) | `vllm` | (default) | (not needed) |\n| Other OpenAI-compatible | `openai` | `false` | As needed |\n| Local Atropos | (default) | (default) | (not needed) |\n\n## Required Methods\n\n### 1. `setup()` — Load dataset and initialize state\n\n```python\nasync def setup(self) -> None:\n    \"\"\"Called once at startup. Load datasets, initialize state.\"\"\"\n    # Try HuggingFace first, fallback to built-in samples\n    try:\n        from datasets import load_dataset\n        ds = load_dataset(\"your\u002Fdataset\", split=\"test\")\n        self._items = [...]\n    except Exception:\n        self._items = BUILTIN_SAMPLES\n\n    # Always split into train\u002Feval\n    random.shuffle(self._items)\n    eval_size = max(20, int(len(self._items) * 0.1))\n    self._eval_items = self._items[:eval_size]\n    self._items = self._items[eval_size:]\n```\n\n### 2. `get_next_item()` — Return next training item\n\n```python\nasync def get_next_item(self) -> dict:\n    \"\"\"Return next item, cycling through dataset.\"\"\"\n    item = self._items[self._index % len(self._items)]\n    self._index += 1\n    return item\n```\n\n### 3. `format_prompt(item)` — Convert item to user message\n\n```python\ndef format_prompt(self, item: dict) -> str:\n    \"\"\"Convert a dataset item into the user-facing prompt.\"\"\"\n    return f\"Research this question: {item['question']}\"\n```\n\n### 4. `compute_reward(item, result, ctx)` — Score the rollout\n\n**CRITICAL**: `result` is an `AgentResult`, NOT a dict. It has these attributes:\n- `result.messages` — List of message dicts (OpenAI format)\n- `result.turns_used` — Number of LLM calls made\n- `result.finished_naturally` — True if model stopped voluntarily\n- `result.tool_errors` — List of ToolError objects\n\n**AgentResult does NOT have**: `final_response`, `tool_calls`, `tools_used`.\nYou must extract these from `result.messages`:\n\n```python\nasync def compute_reward(self, item, result: AgentResult, ctx: ToolContext) -> float:\n    # Extract final response (last assistant message with content)\n    final_response = \"\"\n    tools_used = []\n    for msg in reversed(result.messages):\n        if msg.get(\"role\") == \"assistant\" and msg.get(\"content\") and not final_response:\n            final_response = msg[\"content\"]\n        if msg.get(\"role\") == \"assistant\" and msg.get(\"tool_calls\"):\n            for tc in msg[\"tool_calls\"]:\n                fn = tc.get(\"function\", {}) if isinstance(tc, dict) else {}\n                name = fn.get(\"name\", \"\")\n                if name:\n                    tools_used.append(name)\n\n    # Score using LLM judge, heuristic, or ToolContext verification\n    correctness = await self._llm_judge(item, final_response)\n    return correctness\n```\n\n`ctx` (ToolContext) gives you terminal\u002Ffile access to the agent's sandbox for verification:\n```python\n# Run tests in the agent's sandbox\nresult = ctx.terminal(\"pytest \u002Fworkspace\u002Ftest.py\")\nreturn 1.0 if result[\"exit_code\"] == 0 else 0.0\n```\n\n### 5. `evaluate()` — Periodic evaluation with full agent loop\n\n**MUST use the full agent loop with tools**, not single-turn chat_completion.\nThe whole point of yousef-sovereign-core environments is agentic evaluation:\n\n```python\nasync def evaluate(self, *args, **kwargs) -> None:\n    import time, uuid\n    from environments.agent_loop import YousefShtiweLoop\n    from environments.tool_context import ToolContext\n\n    start_time = time.time()\n    tools, valid_names = self._resolve_tools_for_group()\n    samples = []\n\n    for item in self._eval_items[:self.config.eval_size]:\n        task_id = str(uuid.uuid4())\n        messages = []\n        if self.config.system_prompt:\n            messages.append({\"role\": \"system\", \"content\": self.config.system_prompt})\n        messages.append({\"role\": \"user\", \"content\": self.format_prompt(item)})\n\n        agent = YousefShtiweLoop(\n            server=self.server,\n            tool_schemas=tools,\n            valid_tool_names=valid_names,\n            max_turns=self.config.max_agent_turns,\n            task_id=task_id,\n            temperature=0.0,  # Deterministic for eval\n            max_tokens=self.config.max_token_length,\n            extra_body=self.config.extra_body,\n        )\n        result = await agent.run(messages)\n\n        ctx = ToolContext(task_id)\n        try:\n            reward = await self.compute_reward(item, result, ctx)\n        finally:\n            ctx.cleanup()\n\n        samples.append({\"prompt\": ..., \"response\": ..., \"reward\": reward})\n\n    eval_metrics = {\"eval\u002Fmean_reward\": ...}\n    await self.evaluate_log(metrics=eval_metrics, samples=samples,\n                            start_time=start_time, end_time=time.time())\n```\n\n### 6. `wandb_log()` — Custom metrics logging\n\nAlways call `super().wandb_log()` at the end:\n\n```python\nasync def wandb_log(self, wandb_metrics=None):\n    if wandb_metrics is None:\n        wandb_metrics = {}\n    if self._reward_buffer:\n        n = len(self._reward_buffer)\n        wandb_metrics[\"train\u002Fmean_reward\"] = sum(self._reward_buffer) \u002F n\n        self._reward_buffer.clear()\n    await super().wandb_log(wandb_metrics)  # MUST call super\n```\n\n**Pitfall**: `compute_reward` appends to metric buffers. During eval, this pollutes training metrics. Roll back buffer entries added during eval.\n\n## Config Class\n\nAlways create a custom config subclass with Pydantic Field descriptors. Key inherited fields you can tune: `enabled_toolsets`, `max_agent_turns`, `agent_temperature`, `system_prompt`, `terminal_backend`, `group_size`, `steps_per_eval`, `total_steps`.\n\n## config_init() — Default Configuration\n\nClassmethod returning `(YourEnvConfig, [APIServerConfig(...)])`. Set server_type to \"openai\" for OpenRouter\u002Fexternal APIs. Load API key from environment variable.\n\n## Three CLI Modes\n\n```bash\n# SERVE — Full training loop (connects to Atropos API server)\npython environments\u002Fmy_env.py serve --openai.base_url http:\u002F\u002Flocalhost:8000\u002Fv1\n\n# PROCESS — Offline data generation (saves JSONL)\npython environments\u002Fmy_env.py process --env.total_steps 10 --env.group_size 1 \\\n    --env.use_wandb false --env.data_path_to_save_groups output.jsonl \\\n    --openai.base_url \"\u003CUSER_BASE_URL>\" \\\n    --openai.model_name \"\u003CUSER_MODEL>\" \\\n    --openai.server_type \u003CUSER_SERVER_TYPE> --openai.health_check false\n\n# EVALUATE — Standalone eval (runs setup + evaluate only)\npython environments\u002Fmy_env.py evaluate --env.eval_size 20 \\\n    --env.data_dir_to_save_evals \u002Ftmp\u002Feval_results \\\n    --openai.base_url \"\u003CUSER_BASE_URL>\" \\\n    --openai.model_name \"\u003CUSER_MODEL>\" \\\n    --openai.server_type \u003CUSER_SERVER_TYPE> --openai.health_check false\n```\n\nConfig priority: CLI args > YAML file > config_init() defaults.\n\n## Common Pitfalls\n\n1. **AgentResult has .messages, not .final_response** — Extract the final response by iterating reversed(result.messages) looking for the last assistant message with content.\n\n2. **evaluate() must use YousefShtiweLoop, not chat_completion** — Single-turn chat_completion has no tools. The whole point of yousef-sovereign-core benchmarks is agentic evaluation with tool use.\n\n3. **Don't call _llm_judge twice** — If compute_reward already calls it, extract the score from the buffer instead of calling judge separately in evaluate().\n\n4. **Eval pollutes training buffers** — compute_reward appends to metric buffers. During eval, roll back buffer entries to keep training metrics clean.\n\n5. **Always set health_check=false for OpenRouter** — OpenRouter has no \u002Fhealth endpoint.\n\n6. **Set data_dir_to_save_evals in evaluate mode** — Without it, results aren't saved.\n\n7. **default_toolsets class variable vs enabled_toolsets config** — The class variable is a hint; the config field is what actually controls tool resolution.\n\n8. **Tool call parsing in messages** — Tool calls are dicts with `{\"function\": {\"name\": ..., \"arguments\": ...}}`. Always check `isinstance(tc, dict)`.\n\n9. **ToolContext.cleanup()** — Always call in a finally block to release sandbox resources.\n\n10. **server_type must be \"openai\" for external APIs** — Without it, Atropos assumes a local VLLM server.\n\n11. **Always ask the user for their inference setup** — Never hardcode or assume a specific provider\u002Fmodel. See the \"Inference Setup\" section above.\n\n## Reward Function Patterns\n\n### LLM Judge (for open-ended tasks)\nUse `self.server.chat_completion()` with a scoring prompt. Parse JSON response for score float. Always include a heuristic fallback (keyword overlap) for when the judge call fails.\n\n### Binary Verification (for code\u002Fterminal tasks)\nUse `ctx.terminal(\"pytest test.py -q\")` to run tests in the agent's sandbox. Return 1.0 for pass, 0.0 for fail.\n\n### Multi-Signal (combine multiple indicators)\nWeight correctness (0.6) + tool usage (0.2) + efficiency (0.2) + optional bonuses. Clamp to [0, 1].\n\n## Testing Your Environment\n\n1. **Import test**: `python -c \"from environments.my_env import MyEnv; print('OK')\"`\n2. **Ask the user for inference setup** (see \"Inference Setup\" section above)\n3. **Process mode** (1 item): Verify JSONL output has valid tokens, masks, scores\n4. **Evaluate mode**: Verify full agent loop runs with tools, metrics logged correctly\n5. **Check reward range**: Scores should be in [0, 1], not all identical\n\n## Minimum Implementation Checklist\n\n```python\nclass MyEnv(YousefShtiweBaseEnv):\n    name = \"my-env\"\n    env_config_cls = MyEnvConfig\n\n    @classmethod\n    def config_init(cls): ...          # Default server + env config\n    async def setup(self): ...         # Load dataset + train\u002Feval split\n    async def get_next_item(self): ... # Cycle through training items\n    def format_prompt(self, item): ... # Item → user message string\n    async def compute_reward(self, item, result, ctx): ...  # Score rollout\n    async def evaluate(self, *args, **kwargs): ...  # Full agent loop eval\n    async def wandb_log(self, metrics=None): ...    # Custom metrics + super()\n\nif __name__ == \"__main__\":\n    MyEnv.cli()\n```\n","# YOUSEF SHTIWE Atropos Environments\n\nGuide for building RL environments in the yousef-sovereign-core repo that integrate with the Atropos training framework.\n\n## Architecture Overview\n\n```\nAtropos BaseEnv (atroposlib\u002Fenvs\u002Fbase.py)\n    └── YousefShtiweBaseEnv (environments\u002Fhermes_base_env.py)\n            ├── Handles agent loop orchestration\n            ├── Handles tool resolution per group\n            ├── Handles ToolContext for reward verification\n            └── YOUR ENVIRONMENT (environments\u002Fyour_env.py)\n                    Only implements: setup, get_next_item, format_prompt,\n                                    compute_reward, evaluate, wandb_log\n```\n\nHermes environments are special because they run a **multi-turn agent loop with tool calling** — not just single-turn completions. The base env handles the loop; you implement the task and scoring.\n\n## File Locations\n\n| File | Purpose |\n|------|---------|\n| `environments\u002Fhermes_base_env.py` | Base class with agent loop + tool resolution |\n| `environments\u002Fagent_loop.py` | `YousefShtiweLoop` + `AgentResult` dataclass |\n| `environments\u002Ftool_context.py` | `ToolContext` for reward verification |\n| `environments\u002Ftool_call_parsers.py` | Phase 2 tool call parsers (hermes, mistral, etc.) |\n| `environments\u002Fyour_env.py` | Your environment implementation |\n\n## Inference Setup — Ask the User First\n\n**IMPORTANT:** Before running any test, evaluation, or data generation command, always ask the user how they want to handle inference. Do NOT assume OpenRouter or any specific endpoint. Present these options:\n\n1. **OpenRouter** — Ask which model they want to use (e.g., `anthropic\u002Fclaude-sonnet-4.5`, `google\u002Fgemini-2.5-pro`, `meta-llama\u002Fllama-3.3-70b-instruct`, etc.). Requires `OPENROUTER_API_KEY` in environment.\n2. **Self-hosted VLLM endpoint** — Ask for their base URL (e.g., `http:\u002F\u002Flocalhost:8000\u002Fv1`) and model name. Set `--openai.server_type vllm`.\n3. **Other OpenAI-compatible API** — Ask for the base URL, model name, and any required API key. Set `--openai.server_type openai` and `--openai.health_check false`.\n4. **Local Atropos training server** — For `serve` mode with a live training loop. Default `http:\u002F\u002Flocalhost:8000\u002Fv1`.\n\nOnce the user tells you their setup, use those values in all CLI commands for that session. Example prompts:\n\n> \"Before I run this, how would you like to handle inference?\n> 1. OpenRouter (I'll need your preferred model, e.g. claude-sonnet-4.5)\n> 2. A self-hosted VLLM endpoint (give me the URL and model name)\n> 3. Another OpenAI-compatible API (give me the URL, model, and any auth details)\n> 4. Local Atropos training server (serve mode)\"\n\n### Key flags by provider:\n\n| Provider | `--openai.server_type` | `--openai.health_check` | `--openai.api_key` |\n|----------|----------------------|------------------------|-------------------|\n| OpenRouter | `openai` | `false` | `$OPENROUTER_API_KEY` |\n| VLLM (self-hosted) | `vllm` | (default) | (not needed) |\n| Other OpenAI-compatible | `openai` | `false` | As needed |\n| Local Atropos | (default) | (default) | (not needed) |\n\n## Required Methods\n\n### 1. `setup()` — Load dataset and initialize state\n\n```python\nasync def setup(self) -> None:\n    \"\"\"Called once at startup. Load datasets, initialize state.\"\"\"\n    # Try HuggingFace first, fallback to built-in samples\n    try:\n        from datasets import load_dataset\n        ds = load_dataset(\"your\u002Fdataset\", split=\"test\")\n        self._items = [...]\n    except Exception:\n        self._items = BUILTIN_SAMPLES\n\n    # Always split into train\u002Feval\n    random.shuffle(self._items)\n    eval_size = max(20, int(len(self._items) * 0.1))\n    self._eval_items = self._items[:eval_size]\n    self._items = self._items[eval_size:]\n```\n\n### 2. `get_next_item()` — Return next training item\n\n```python\nasync def get_next_item(self) -> dict:\n    \"\"\"Return next item, cycling through dataset.\"\"\"\n    item = self._items[self._index % len(self._items)]\n    self._index += 1\n    return item\n```\n\n### 3. `format_prompt(item)` — Convert item to user message\n\n```python\ndef format_prompt(self, item: dict) -> str:\n    \"\"\"Convert a dataset item into the user-facing prompt.\"\"\"\n    return f\"Research this question: {item['question']}\"\n```\n\n### 4. `compute_reward(item, result, ctx)` — Score the rollout\n\n**CRITICAL**: `result` is an `AgentResult`, NOT a dict. It has these attributes:\n- `result.messages` — List of message dicts (OpenAI format)\n- `result.turns_used` — Number of LLM calls made\n- `result.finished_naturally` — True if model stopped voluntarily\n- `result.tool_errors` — List of ToolError objects\n\n**AgentResult does NOT have**: `final_response`, `tool_calls`, `tools_used`.\nYou must extract these from `result.messages`:\n\n```python\nasync def compute_reward(self, item, result: AgentResult, ctx: ToolContext) -> float:\n    # Extract final response (last assistant message with content)\n    final_response = \"\"\n    tools_used = []\n    for msg in reversed(result.messages):\n        if msg.get(\"role\") == \"assistant\" and msg.get(\"content\") and not final_response:\n            final_response = msg[\"content\"]\n        if msg.get(\"role\") == \"assistant\" and msg.get(\"tool_calls\"):\n            for tc in msg[\"tool_calls\"]:\n                fn = tc.get(\"function\", {}) if isinstance(tc, dict) else {}\n                name = fn.get(\"name\", \"\")\n                if name:\n                    tools_used.append(name)\n\n    # Score using LLM judge, heuristic, or ToolContext verification\n    correctness = await self._llm_judge(item, final_response)\n    return correctness\n```\n\n`ctx` (ToolContext) gives you terminal\u002Ffile access to the agent's sandbox for verification:\n```python\n# Run tests in the agent's sandbox\nresult = ctx.terminal(\"pytest \u002Fworkspace\u002Ftest.py\")\nreturn 1.0 if result[\"exit_code\"] == 0 else 0.0\n```\n\n### 5. `evaluate()` — Periodic evaluation with full agent loop\n\n**MUST use the full agent loop with tools**, not single-turn chat_completion.\nThe whole point of yousef-sovereign-core environments is agentic evaluation:\n\n```python\nasync def evaluate(self, *args, **kwargs) -> None:\n    import time, uuid\n    from environments.agent_loop import YousefShtiweLoop\n    from environments.tool_context import ToolContext\n\n    start_time = time.time()\n    tools, valid_names = self._resolve_tools_for_group()\n    samples = []\n\n    for item in self._eval_items[:self.config.eval_size]:\n        task_id = str(uuid.uuid4())\n        messages = []\n        if self.config.system_prompt:\n            messages.append({\"role\": \"system\", \"content\": self.config.system_prompt})\n        messages.append({\"role\": \"user\", \"content\": self.format_prompt(item)})\n\n        agent = YousefShtiweLoop(\n            server=self.server,\n            tool_schemas=tools,\n            valid_tool_names=valid_names,\n            max_turns=self.config.max_agent_turns,\n            task_id=task_id,\n            temperature=0.0,  # Deterministic for eval\n            max_tokens=self.config.max_token_length,\n            extra_body=self.config.extra_body,\n        )\n        result = await agent.run(messages)\n\n        ctx = ToolContext(task_id)\n        try:\n            reward = await self.compute_reward(item, result, ctx)\n        finally:\n            ctx.cleanup()\n\n        samples.append({\"prompt\": ..., \"response\": ..., \"reward\": reward})\n\n    eval_metrics = {\"eval\u002Fmean_reward\": ...}\n    await self.evaluate_log(metrics=eval_metrics, samples=samples,\n                            start_time=start_time, end_time=time.time())\n```\n\n### 6. `wandb_log()` — Custom metrics logging\n\nAlways call `super().wandb_log()` at the end:\n\n```python\nasync def wandb_log(self, wandb_metrics=None):\n    if wandb_metrics is None:\n        wandb_metrics = {}\n    if self._reward_buffer:\n        n = len(self._reward_buffer)\n        wandb_metrics[\"train\u002Fmean_reward\"] = sum(self._reward_buffer) \u002F n\n        self._reward_buffer.clear()\n    await super().wandb_log(wandb_metrics)  # MUST call super\n```\n\n**Pitfall**: `compute_reward` appends to metric buffers. During eval, this pollutes training metrics. Roll back buffer entries added during eval.\n\n## Config Class\n\nAlways create a custom config subclass with Pydantic Field descriptors. Key inherited fields you can tune: `enabled_toolsets`, `max_agent_turns`, `agent_temperature`, `system_prompt`, `terminal_backend`, `group_size`, `steps_per_eval`, `total_steps`.\n\n## config_init() — Default Configuration\n\nClassmethod returning `(YourEnvConfig, [APIServerConfig(...)])`. Set server_type to \"openai\" for OpenRouter\u002Fexternal APIs. Load API key from environment variable.\n\n## Three CLI Modes\n\n```bash\n# SERVE — Full training loop (connects to Atropos API server)\npython environments\u002Fmy_env.py serve --openai.base_url http:\u002F\u002Flocalhost:8000\u002Fv1\n\n# PROCESS — Offline data generation (saves JSONL)\npython environments\u002Fmy_env.py process --env.total_steps 10 --env.group_size 1 \\\n    --env.use_wandb false --env.data_path_to_save_groups output.jsonl \\\n    --openai.base_url \"\u003CUSER_BASE_URL>\" \\\n    --openai.model_name \"\u003CUSER_MODEL>\" \\\n    --openai.server_type \u003CUSER_SERVER_TYPE> --openai.health_check false\n\n# EVALUATE — Standalone eval (runs setup + evaluate only)\npython environments\u002Fmy_env.py evaluate --env.eval_size 20 \\\n    --env.data_dir_to_save_evals \u002Ftmp\u002Feval_results \\\n    --openai.base_url \"\u003CUSER_BASE_URL>\" \\\n    --openai.model_name \"\u003CUSER_MODEL>\" \\\n    --openai.server_type \u003CUSER_SERVER_TYPE> --openai.health_check false\n```\n\nConfig priority: CLI args > YAML file > config_init() defaults.\n\n## Common Pitfalls\n\n1. **AgentResult has .messages, not .final_response** — Extract the final response by iterating reversed(result.messages) looking for the last assistant message with content.\n\n2. **evaluate() must use YousefShtiweLoop, not chat_completion** — Single-turn chat_completion has no tools. The whole point of yousef-sovereign-core benchmarks is agentic evaluation with tool use.\n\n3. **Don't call _llm_judge twice** — If compute_reward already calls it, extract the score from the buffer instead of calling judge separately in evaluate().\n\n4. **Eval pollutes training buffers** — compute_reward appends to metric buffers. During eval, roll back buffer entries to keep training metrics clean.\n\n5. **Always set health_check=false for OpenRouter** — OpenRouter has no \u002Fhealth endpoint.\n\n6. **Set data_dir_to_save_evals in evaluate mode** — Without it, results aren't saved.\n\n7. **default_toolsets class variable vs enabled_toolsets config** — The class variable is a hint; the config field is what actually controls tool resolution.\n\n8. **Tool call parsing in messages** — Tool calls are dicts with `{\"function\": {\"name\": ..., \"arguments\": ...}}`. Always check `isinstance(tc, dict)`.\n\n9. **ToolContext.cleanup()** — Always call in a finally block to release sandbox resources.\n\n10. **server_type must be \"openai\" for external APIs** — Without it, Atropos assumes a local VLLM server.\n\n11. **Always ask the user for their inference setup** — Never hardcode or assume a specific provider\u002Fmodel. See the \"Inference Setup\" section above.\n\n## Reward Function Patterns\n\n### LLM Judge (for open-ended tasks)\nUse `self.server.chat_completion()` with a scoring prompt. Parse JSON response for score float. Always include a heuristic fallback (keyword overlap) for when the judge call fails.\n\n### Binary Verification (for code\u002Fterminal tasks)\nUse `ctx.terminal(\"pytest test.py -q\")` to run tests in the agent's sandbox. Return 1.0 for pass, 0.0 for fail.\n\n### Multi-Signal (combine multiple indicators)\nWeight correctness (0.6) + tool usage (0.2) + efficiency (0.2) + optional bonuses. Clamp to [0, 1].\n\n## Testing Your Environment\n\n1. **Import test**: `python -c \"from environments.my_env import MyEnv; print('OK')\"`\n2. **Ask the user for inference setup** (see \"Inference Setup\" section above)\n3. **Process mode** (1 item): Verify JSONL output has valid tokens, masks, scores\n4. **Evaluate mode**: Verify full agent loop runs with tools, metrics logged correctly\n5. **Check reward range**: Scores should be in [0, 1], not all identical\n\n## Minimum Implementation Checklist\n\n```python\nclass MyEnv(YousefShtiweBaseEnv):\n    name = \"my-env\"\n    env_config_cls = MyEnvConfig\n\n    @classmethod\n    def config_init(cls): ...          # Default server + env config\n    async def setup(self): ...         # Load dataset + train\u002Feval split\n    async def get_next_item(self): ... # Cycle through training items\n    def format_prompt(self, item): ... # Item → user message string\n    async def compute_reward(self, item, result, ctx): ...  # Score rollout\n    async def evaluate(self, *args, **kwargs): ...  # Full agent loop eval\n    async def wandb_log(self, metrics=None): ...    # Custom metrics + super()\n\nif __name__ == \"__main__\":\n    MyEnv.cli()\n```\n",{"name":5,"author":51,"license":14,"version":52,"metadata":53,"description":7},"YOUSEF SHTIWE","1.1.0",{"hermes":54},{"tags":55,"related_skills":60},[56,57,26,45,58,59],"atropos","rl","reinforcement-learning","reward-functions",[61,62,63,64],"axolotl","grpo-rl-training","trl-fine-tuning","lm-evaluation-harness","main","93bbc269671fbb340afa15b36a8766f15dc30df2","hermes-agent\u002Foptional-skills\u002Fmlops\u002Fhermes-atropos-environments","https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6","https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch","2026-04-27 02:51:28",{"source":8,"skillName":5,"commands":72,"prompt":82,"zipAvailable":83},[73,76,79],{"tool":74,"command":75},"npx","npx skills add gjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6 --skill hermes-atropos-environments",{"tool":77,"command":78},"bunx","bunx skills add gjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6 --skill hermes-atropos-environments",{"tool":80,"command":81},"pnpm","pnpm dlx skills add gjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6 --skill hermes-atropos-environments","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fhermes-atropos-environments\u002FSKILL.md",true,[85,91,95],{"path":86,"resourceType":87,"resourceTypeDesc":88,"sizeBytes":89,"sourceHtmlUrl":90},"references\u002Fagentresult-fields.md","REFERENCE","参考文档",2055,"https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fhermes-atropos-environments\u002Freferences\u002Fagentresult-fields.md",{"path":92,"resourceType":87,"resourceTypeDesc":88,"sizeBytes":93,"sourceHtmlUrl":94},"references\u002Fatropos-base-env.md",3033,"https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fhermes-atropos-environments\u002Freferences\u002Fatropos-base-env.md",{"path":96,"resourceType":87,"resourceTypeDesc":88,"sizeBytes":97,"sourceHtmlUrl":98},"references\u002Fusage-patterns.md",5860,"https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fhermes-atropos-environments\u002Freferences\u002Fusage-patterns.md",[100,139,165,181,200,220],{"skillNo":101,"name":102,"nameZh":6,"description":103,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":104,"categories":105,"keywords":120,"publishedAt":46},"01M71G6TJ2RAQ1J0PV3K6D9GQA","writing-plans","Use when you have a spec or requirements for a multi-step task. Creates comprehensive implementation plans with bite-sized tasks, exact file paths, and complete code examples.","https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes_sovereign_data\u002Fskills\u002Fsoftware-development\u002Fwriting-plans\u002FSKILL.md",[106,109,112,117],{"categoryNo":107,"categoryName":108,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000015","视觉设计",{"categoryNo":110,"categoryName":111,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":113,"categoryName":114,"taxonomyType":115,"taxonomyTypeDesc":116},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",{"categoryNo":118,"categoryName":119,"taxonomyType":115,"taxonomyTypeDesc":116},"01JBSEEDCAT000000000000036","内容创作者",[121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,32,138,30],"writing","writing plans","plans","design","review","workflow","documentation","tasks","development","code","planning","code-review","subagent","superpowers","paths","test-driven-development","you","test-driven",{"skillNo":140,"name":136,"nameZh":6,"description":141,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":142,"categories":143,"keywords":150,"publishedAt":46},"01M71G6R81013R3T7W5VP56GCB","Use when implementing any feature or bugfix, before writing implementation code. Enforces RED-GREEN-REFACTOR cycle with test-first approach.","https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes_sovereign_data\u002Fskills\u002Fsoftware-development\u002Ftest-driven-development\u002FSKILL.md",[144,147,148,149],{"categoryNo":145,"categoryName":146,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000007","代码审查",{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":110,"categoryName":111,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":118,"categoryName":119,"taxonomyType":115,"taxonomyTypeDesc":116},[151,152,129,153,154,121,155,130,156,133,134,157,158,159,160,161,162,163,164,30],"test","test driven development","driven","testing","debugging","refactor","red-green-refactor","cycle","green","systematic","bugfix","implementing","systematic-debugging","test-first",{"skillNo":166,"name":163,"nameZh":6,"description":167,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":168,"categories":169,"keywords":171,"publishedAt":46},"01M71G6N0Y546JZEDF25E2PMCE","Use when encountering any bug, test failure, or unexpected behavior. 4-phase root cause investigation — NO fixes without understanding the problem first.","https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes_sovereign_data\u002Fskills\u002Fsoftware-development\u002Fsystematic-debugging\u002FSKILL.md",[170],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},[172,155,160,121,151,129,133,173,134,174,175,176,123,177,136,178,138,179,180,30],"systematic debugging","use","bug","root-cause","problem-solving","problem","fixes","phase","root cause",{"skillNo":182,"name":183,"nameZh":6,"description":184,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":185,"categories":186,"keywords":193,"publishedAt":46},"01M71G6J0DXEGY48GH4KVW2SJH","subagent-driven-development","Use when executing implementation plans with independent tasks. Dispatches fresh delegate_task per task with two-stage review (spec compliance then code quality).","https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes_sovereign_data\u002Fskills\u002Fsoftware-development\u002Fsubagent-driven-development\u002FSKILL.md",[187,188,189,190],{"categoryNo":110,"categoryName":111,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":113,"categoryName":114,"taxonomyType":115,"taxonomyTypeDesc":116},{"categoryNo":118,"categoryName":119,"taxonomyType":115,"taxonomyTypeDesc":116},{"categoryNo":191,"categoryName":192,"taxonomyType":115,"taxonomyTypeDesc":116},"01JBSEEDCAT000000000000037","律师与法务",[129,194,133,153,121,125,126,195,128,130,196,132,197,198,134,123,199,136,138,30],"subagent driven development","code quality","delegation","parallel","delegate","fresh",{"skillNo":201,"name":202,"nameZh":6,"description":203,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":204,"categories":205,"keywords":208,"publishedAt":46},"01M71G6EFFBQX6634E6XHB09Y1","requesting-code-review","Pre-commit verification pipeline — static security scan, baseline-aware quality gates, independent reviewer subagent, and auto-fix loop. Use after code changes and before committing, pushing, or opening a PR.","https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes_sovereign_data\u002Fskills\u002Fsoftware-development\u002Frequesting-code-review\u002FSKILL.md",[206,207],{"categoryNo":145,"categoryName":146,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":113,"categoryName":114,"taxonomyType":115,"taxonomyTypeDesc":116},[209,125,130,210,121,211,132,212,213,214,38,133,215,216,217,134,136,218,219,138],"code review","requesting code review","commit","verification","security scan","pipeline","reviewer","fix","auto","static","auto-fix",{"skillNo":221,"name":222,"nameZh":6,"description":223,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":224,"categories":225,"keywords":226,"publishedAt":46},"01M71G6CM8ANSWVFXCN9AAD58Y","plan","Plan mode for Hermes — inspect context, write a markdown plan into the active workspace's `.hermes\u002Fplans\u002F` directory, and do not execute the work.","https:\u002F\u002Fgithub.com\u002Fgjbvsjjvgm-sketch\u002Fhermes-sovereign-unified-v13.6\u002Fblob\u002Fmain\u002Fhermes_sovereign_data\u002Fskills\u002Fsoftware-development\u002Fplan\u002FSKILL.md",[],[222,121,227,126,129,131,228,133,229,230,231,232,233,234,123,235,236,237,32,238],"directory","markdown","workspace","work","write","implementation","do","active","plan mode","inspect","mode","plan-mode"]