temporary-tool-auditing
Use when a scenario test needs bounded evidence of an agent's tool attempts, outcomes, or absence of tool use from Codex or the demonstrated Devin Desktop hook contract.
- License 许可
- MIT
- 最近更新
- 2026-10-08
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- temporary-tool-auditing
- license
- MIT
- metadata
- { "owner": "Harley Bartles", "scope": "A stable user-wide Codex dispatcher with temporary session-family activation, or temporary Devin Desktop hooks, for sanitized, bounded tool-use evidence.", "status": "active", "use_when": [ "a scenario needs evidence that a selected agent did or did not attempt tools during a bounded run.", "session-scoped hook activation, expiry, evidence, and engagement cleanup must be tracked." ], "source-id": "temporary-tool-auditing", "source-path": "skills/temporary-tool-auditing/SKILL.md", "do_not_use_when": [ "general telemetry, permanent logging, policy enforcement, or tool authorization is needed.", "the scenario requires proof about hosted tools that the runtime hooks do not observe.", "the active runtime is Devin CLI; this skill currently supports only the separately observed Devin Desktop contract." ], "provenance-name": "Temporary Tool Auditing first-party skill", "source-category": "first_party" }
- description
- Use when a scenario test needs bounded evidence of an agent's tool attempts, outcomes, or absence of tool use from Codex or the demonstrated Devin Desktop hook contract.
02 资源文件 31
- 参考文档agents/openai.yaml944 B
- 资产文件assets/hooks/codex.json1.1 KB
- 资产文件assets/hooks/devin-desktop.json463 B
- 参考文档references/codex.md3.7 KB
- 参考文档references/devin.md2.3 KB
- 参考文档references/evidence.md4.3 KB
- 脚本scripts/activation.py9.1 KB
- 脚本scripts/assessment.py13.0 KB
- 脚本scripts/auditctl.py34.8 KB
- 脚本scripts/codex_trust.py23.7 KB
- 脚本scripts/global_registration.py8.3 KB
- 脚本scripts/record.py6.4 KB
- 脚本scripts/registration.py15.9 KB
- 脚本scripts/runtime.py7.4 KB
- 脚本scripts/sanitize.py9.0 KB
- 脚本scripts/store.py10.0 KB
- 参考文档tests/behavior/evidence-and-cleanup.md4.3 KB
- 参考文档tests/evaluator-only/audit-recovery-rubric.md3.9 KB
- 参考文档tests/pressure/audit-recovery.md2.0 KB
- 脚本tests/scripts/conftest.py128 B
- 脚本tests/scripts/test_activation.py8.7 KB
- 脚本tests/scripts/test_assessment.py10.8 KB
- 脚本tests/scripts/test_cli.py4.9 KB
- 脚本tests/scripts/test_codex_trust.py17.1 KB
- 脚本tests/scripts/test_global_registration.py3.8 KB
- 脚本tests/scripts/test_lifecycle.py36.9 KB
- 脚本tests/scripts/test_record.py3.8 KB
- 脚本tests/scripts/test_registration.py17.5 KB
- 脚本tests/scripts/test_runtime.py7.1 KB
- 脚本tests/scripts/test_sanitize.py15.3 KB
- 脚本tests/scripts/test_store.py5.6 KB
03 SKILL.md 正文
Temporary Tool Auditing
Use this skill when a scenario needs tool-use evidence. The stable Codex global hook runs across projects, but records only after its activation registry matches the declared parent session ID and exact worktree. Codex subagent events carry the parent session ID and retain child identifiers where available. Another session, including one in the same worktree, stays out of the run from the start. Devin Desktop selectors include session activity, while an individual child requires serialized dispatch attribution. Devin CLI remains unsupported. Evidence supports a bounded claim for verified coverage, not universal proof.
Workflow
- Identify the active runtime, runtime version, exact worktree, and intended subject. For Codex, prepare from the target parent session so the helper reads
CODEX_SESSION_ID; verify it is present and binds to subsequent hooksession_idvalues. Use the current isolated worktree when available, otherwise the project checkout. Put run data off-repository. Choose metadata-onlystatusdetail by default. It stores identifiers, tool name, and observed outcome, whilefull-resultsadditionally stores sanitized arguments and result bodies. Each run caps events at 25 MiB, health records at 256 KiB, controls at 1 MiB, and the manifest at 2 MiB, with space reserved for teardown metadata. Persisted run evidence stays below 32 MiB even during atomic manifest replacement, excluding fixed-size copied helpers. If any limit rejects a write, stop the round, report capture as incomplete, and do not renew that run expecting missing evidence to return. Even sanitized full results may contain credentials, personal/payment data, source code, internal paths, or confidential content, and pattern filters cannot identify every sensitive passage. - Resolve this skill's installed directory and use
scripts/auditctl.py. Run--help, thenprepare --checkandprepare --applywith a unique run directory, runtime, project, scenario question, assessment subject, and detail. Codex capture session comes fromCODEX_SESSION_ID; do not substitute an unrelated chat/thread identifier. If unavailable, stop and report the missing runtime capability. Do not use Devin Desktop with Devin CLI. The default lease is 30 minutes; choose a shorter or longer positive duration only as needed. - Run
install --check. For Codex this previews one stable hook dispatcher in$CODEX_HOME/hooks.jsonand an owned script under$CODEX_HOME/tool-auditing/; it does not change project trust. Runinstall --applyonce if the dispatcher is absent, inspect its commands, and have your human partner review and approve this definition. Restart Codex once only if required to load the global definition. Keep this installation for later engagements. If the installed helper hashes differ from the source, inspect the helper diff and obtain human approval beforeinstall --apply --refresh-global-helpers; this replaces only owned helper files and leaves hook definitions unchanged. Refresh is refused while any engagement remains in the activation registry because changing helpers during capture could create an evidence gap; finish, disable, and purge all active engagements before retrying. Runenable --applyto add this run's parent session ID and exact worktree to the activation registry; Windows updatesCODEX_TOOL_AUDIT_ACTIVATION_FILEin the current-user environment, which the hook rereads from the registry on every invocation. On other platforms, the dispatcher reads the activation file directly. Runverify --apply, make one harmless tool call in this parent session, and finish verification with its captured call ID. Confirm the event session ID matches the prepared parent ID and includes the expected worktree before starting the scenario. - Run
start --apply, conduct a bounded round, then runstop --applyto close that round. Assess and report; if the human requests more scenarios or follow-up, keep the same activation and installation, then start another round. Do not reinstall between rounds. An explicitrenew --duration-minutes N --applyextends the lease and refreshes the activation entry. If renewal fails, the previous lease remains in force; do not assume the requested extension took effect. Expiry creates a gap and requires a fresh verified interval before further capture. For Devin child subjects, the matching dispatch post is the completion boundary and the orchestrator/other agents must remain idle during that dispatch. - Run
assess --subject <selector>and inspect the selected subject, detail level, covered intervals, unresolved count, coverage gaps, health records, limitations, and redaction paths. For Devin child attribution, only if the orchestrator and other agents remained idle during the serialized dispatch, runassess --subject child:<dispatch-call-id> --confirm-parent-idle --apply; this persists the orchestrator's positional-attribution self-attestation. Missing controls, incomplete coverage, unmatched calls, health failures, redactions, unconfirmed orchestrator idleness, or ambiguous subject attribution prevent a no-tools claim. For an absence claim, capture a fresh positive control after the subject has completed (or a matching Codex session-end event) to confirm the hook path remains available through the end of observation. State counts and outcome statuses as observed, withobserved-unknownwhen the runtime does not establish a result. Use the assessment result to communicate the bounded claim, but do not persist a separate assessment file in the run directory. - When the engagement has fulfilled its purpose and no follow-up is pending, run
disable --apply. For Codex this removes only this run's activation entry and clears the environment switch when no other activation remains; it does not uninstall the stable hook. Runverify-teardown --apply --phase begin, make a harmless tool call in the same session, then finish withverify-teardown --apply --phase finish --canary-performed. This checks recorder health, global registration, and that the post-disable call added no event. No restart is required for disablement. For Devin or a legacy project-local Codex run, use the runtime-specific removal and unload handoff in its reference. After verified disablement, runpurge --applyand confirmlogs_purged: trueandcleanup_required: false. This removes event, health, and control logs, run-local helper copies, when present, and subject/question details from the manifest. Unknown files keep cleanup pending. If purge fails, report cleanup as unresolved.
This is operational evidence for an honest local agent in a trusted user account. It does not resist someone with access to change the hooks, activation registry, or evidence files. Boundary controls verify the hook at their own points only; a silent outage between them cannot be ruled out without runtime evidence, so report uncertain coverage rather than claim no tools.
The stable Codex dispatcher may remain installed and inert between engagements. One engagement may include several record/assess/report/resume rounds; cleanup is due when the purpose is fulfilled, abandoned, or cancelled and no follow-up is pending. If interrupted, inspect the manifest and activation registry, stop writes safely, preserve evidence needed for the claim, then disable and verify before purging. Expiry never completes cleanup. Start/end controls bound the hook check, but cannot detect a silent runtime outage between them; if activation state or hook availability is uncertain during the interval, report that interval as unverified. A failure, conflict, missing runtime, uncertain activation, or failed purge means report cleanup as pending.
Runtime details
- Read Codex hook behavior before using Codex hooks.
- Read Devin Desktop hook behavior before using Devin Desktop hooks.
- Use evidence limits when interpreting or reporting results.
The recorder sanitizes every persisted payload. It covers common credentials and structured personal/payment fields, plus selected high-confidence free-text patterns. It cannot reliably recognize arbitrary source code, names, confidential business content, every locale's identifiers, or every secret format. Sanitization is a defensive filter, not a guarantee; purge the logs after they have proved the claim. Never deliberately send credentials through a test tool call. Keep full results off unless they are necessary.
04 同仓库更多 Skills
writing
Use when drafting, revising, or reviewing prose for human readers and clarity, an authorised voice, or reader-fatigue concerns may interact.
writing-with-clarity
Use when drafting, revising, or reviewing prose intended for human readers, including documentation, README files, reports, explanations, UI copy, errors, comments, commit messages, and pull request text.
writing-style
Use when revising or reviewing human-facing prose against an authorised voice card or a contextual reader-fatigue profile.
skill
暂无简介
writing-skills
Use when creating new skills, editing existing skills, or verifying skills work before deployment.
writing-roadmaps
Use when a goal is too large for one writing-plans plan and requires a sequenced roadmap of consecutive plans.