AGENT SKILL

hamster-e2b-live-validation

Prove a claim about sandbox-side behavior by running it in a real E2B sandbox instead of reading comments. Use when a delivery or environment boot fails inside the sandbox, when a probe/health-check times out or hangs, when you need to know what is actually installed in a snapshot and at what version, when a comment asserts how a third-party CLI behaves, or when someone proposes raising a timeout. Covers the throw-shapes E2B uses for exit-vs-timeout, cold snapshot-resume timing, deliberate break-state testing, hang bisection via stdin, and offline npm lineage checks.

License 许可
MIT
所属仓库
gethamster/skills
最近更新
2026-11-05
Stars
★ 4
Forks
0

01 前言 Front Matter

name
hamster-e2b-live-validation
version
1.0.0
description
Prove a claim about sandbox-side behavior by running it in a real E2B sandbox instead of reading comments. Use when a delivery or environment boot fails inside the sandbox, when a probe/health-check times out or hangs, when you need to know what is actually installed in a snapshot and at what version, when a comment asserts how a third-party CLI behaves, or when someone proposes raising a timeout. Covers the throw-shapes E2B uses for exit-vs-timeout, cold snapshot-resume timing, deliberate break-state testing, hang bisection via stdin, and offline npm lineage checks.

03 SKILL.md 正文

Live E2B Validation

When to Apply

Reach for this the moment a question is about what actually happens inside the sandbox:

  • A boot/delivery step fails with deadline_exceeded, CommandExitError, or a hang.
  • A comment claims a third-party CLI behaves some way (--version exits, --help hangs, the binary lives at path X).
  • Someone proposes raising a timeout. Measure first — the timeout is usually not the bug.
  • You need the installed version / package identity / file layout of a baked toolchain.
  • You changed a sandbox-side command string and want the shipped string proven, not the one you retyped.

The governing rule: sandbox-side installs are unpinned (sandbox-agent install-agent <agent> takes npm-latest at bake time), and environment snapshots are baked from user sandboxes months earlier. So comments about third-party behavior rot silently and invisibly. Anything load-bearing gets measured.

Related: hamster-e2b-studio for the template/snapshot architecture itself, sandbox-agent for the agent runtime.

Credentials and Entry Point

apps/horde-v2/.env already carries a working key and the template deliveries launch from:

E2B_API_KEY=...
E2B_TEMPLATE_ID=hamster-pre-production/hamster-base-snapshot

Deployed envs share one E2B team, so this snapshot name is what most deliveries actually boot. Run probes with an absolute env-file path (the shell's cwd moves between calls):

npx tsx --env-file=/abs/path/to/apps/horde-v2/.env /abs/path/to/scratchpad/probe.ts

Write probe scripts into the scratchpad, never the repo. They are throwaway evidence, not deliverables.

The Harness

Copy this. The run() wrapper is not boilerplate — it is required, for the reason in the next section.

import { Sandbox } from 'e2b';

async function main() {
  const sbx = await Sandbox.create(process.env.E2B_TEMPLATE_ID!, {
    apiKey: process.env.E2B_API_KEY!,
    timeoutMs: 300_000,
  });
  console.log('Sandbox:', sbx.sandboxId);

  const run = async (label: string, cmd: string, timeoutMs = 45_000) => {
    const t = Date.now();
    try {
      const r = await sbx.commands.run(cmd, { timeoutMs });
      console.log(
        `### ${label}: exit=${r.exitCode} ms=${Date.now() - t} ` +
          `stdout=${JSON.stringify(r.stdout.slice(0, 300))} ` +
          `stderr=${JSON.stringify(r.stderr.slice(0, 300))}`,
      );
    } catch (e: any) {
      // Both a nonzero exit AND a timeout arrive here, as different shapes.
      console.log(
        `### ${label}: THREW ms=${Date.now() - t} ${e?.name} ` +
          `exitCode=${e?.exitCode} ` +
          `stdout=${JSON.stringify(String(e?.stdout ?? ''))} ` +
          `stderr=${JSON.stringify(String(e?.stderr ?? '').slice(0, 300))}`,
      );
    }
  };

  try {
    await run('presence', 'test -x /path/to/thing');
    // ...
  } finally {
    await sbx.kill(); // bills by wall clock — always kill
  }
}

main().catch((e) => {
  console.error(e);
  process.exit(1);
});

Always time every command. ms= is the whole point: it separates "slow" from "hung" and gives you the headroom number to argue about a timeout with.

Gotcha: commands.run throws on both failure modes, differently

You cannot read result.exitCode — there is often no result. Two distinct throws:

Failure e.name e.exitCode Where the diagnostic lives
Nonzero exit CommandExitError the exit code e.stdout / e.stderr
Timeout (timeoutMs expired) TimeoutError undefined e.message, containing [deadline_exceeded]

Consequences that bite:

  • A probe that lands in an "exit unknown" branch was a timeout, not an exit — the process never returned.
  • A TimeoutError carries no stdout/stderr at all. If you need output from a hung command, you cannot get it this way; restructure the probe.
  • Production code wrapping these needs toCommandErrorContext() (e2b/command-error.ts); toErrorContext() only reads .message/.code and silently drops the command output.

Import the real constants — do not retype the command

A probe that validates a string you typed by hand proves nothing about what ships. Import from the source module:

import {
  CODEX_ACP_PRESENCE_PROBE,
  CODEX_ACP_VERSION_PROBE,
} from '/abs/path/to/apps/horde-v2/src/delivery/sandbox/e2b/agent-versions';

tsx resolves the workspace deps fine from an absolute path. If the module pulls Nest/webpack asset imports it may not load — that constraint is why agent-versions.ts documents that it must stay free of *.sh asset imports.

Cold Snapshot-Resume Timing

The claim "the timeout is too short on a cold boot" is testable, and usually false. Memory snapshots resume warm.

const sbx = await Sandbox.create(template, { apiKey, timeoutMs: 300_000 });
// ... warm baseline ...
await sbx.betaPause();

// Resume, and make the command under test the FIRST command after resume.
const resumed = await Sandbox.connect(sbx.sandboxId, {
  apiKey,
  timeoutMs: 300_000,
  autoResume: true,
} as any);
await run('COLD first exec', commandUnderTest);
await resumed.betaPause();

Rules:

  • One candidate per resume. The first command pays the page-in cost; everything after it is warm and tells you nothing.
  • Sandbox.resume does not exist in e2b 2.38.3. The statics are list / create / connect / fork. Resuming is connect(id, { autoResume: true }).
  • Reference numbers measured on hamster-base-snapshot: cold first exec of a Node one-liner ≈ 515 ms, of a bundled Node CLI ≈ 795 ms. If your probe's budget is 45 s, cold-start is not your problem.

Break-State Testing

The happy path is the easy half. A health check earns trust by proving its failure paths, so corrupt the install on purpose and re-probe:

// Dangling symlink: bin survives, target package removed.
await run('break', `rm -rf $(dirname $(dirname $(readlink -f ${BIN})))`);
await run('presence after break', PRESENCE_PROBE);
await run('version after break', VERSION_PROBE);

// Fully absent.
await run('break', `rm -f ${BIN}`);
await run('presence when absent', PRESENCE_PROBE);

Record the exit code and stderr of each state and put the table in the PR. That is what distinguishes "fails loud" from "we hope it fails loud".

Bisecting a Hang

When a command hangs, find out why before changing anything:

  1. Bound it tightly (timeoutMs: 15_000) so iteration is fast.
  2. Re-run with stdin closed: cmd < /dev/null. This is the sharp discriminator — a process that exits instantly with /dev/null but hangs without it is a stdio server waiting on input, not a slow process. That is a fall-through into a JSON-RPC / ACP bridge, and no timeout will ever fix it.
  3. Compare sibling flags. --version vs --help vs no args. If one exits and another hangs, the tool has an unbounded default branch and every unrecognised flag reaches it.
  4. Conclude: a probe whose safety depends on a third party's argv table is a hang waiting for a release. Prefer reading static metadata (a package.json on disk) over launching the binary at all.

Offline Checks (no sandbox needed)

Cheaper than a sandbox, and the right tool for "did upstream change behavior":

npm view <pkg> version bin                 # entry point + current version
npm view <pkg> versions --json             # full release list
npm view <pkg> time --json                 # release dates — map a snapshot to its era
npm pack <pkg>@<version>                   # then untar and read dist/ or bin/

For a bundled JS CLI, grep the shipped bundle for its arg handling rather than trusting the README:

grep -o 'process.argv[^;]\{0,200\}' package/dist/index.js

For platform-specific native binaries, pack the variant matching your machine (e.g. -darwin-arm64) and run it locally. macOS has no timeout(1) — bound it with:

perl -e 'alarm 15; exec @ARGV' ./binary --version < /dev/null; echo "exit=$?"

Run the same flags across several versions in a loop to find exactly where behavior changed. This is how you catch a package lineage switch (upstream moving a tool to a different scope with different arg parsing), which is invisible from the install command alone.

Turn the Finding Into a Test

Live validation proves it once. A unit test keeps it proven.

Parameterise the shipped command by its variable input so a spec can execute the real string:

export function codexAcpVersionProbe(binPath: string): string { /* ... */ }
export const CODEX_ACP_VERSION_PROBE = codexAcpVersionProbe(CODEX_ACP_BIN);
// Build a fixture install in a tmpdir, then run it through a shell, as E2B does —
// this covers the quoting too.
const result = spawnSync(codexAcpVersionProbe(fixtureBin), {
  shell: true,
  encoding: 'utf8',
});

Then confirm the new tests fail against the old implementation before accepting them. String-matching a command (expect(cmd).not.toContain('--version')) checks syntax; executing it checks behavior. Keep a structural assertion only where the shape is the invariant (e.g. "must never exec the binary").

Hygiene

  • sbx.kill() in a finally. Sandboxes bill by wall clock.

  • Do not rely on the wall-clock timeout to clean up. The production helper createE2BSandbox deliberately sets lifecycle.onTimeout: 'pause' so a timed-out delivery stays recoverable — an abandoned sandbox there pauses rather than dies and lingers.

  • A sandbox you paused yourself (for cold-resume timing) is still alive, and a script that throws between betaPause() and its cleanup leaves it that way. Kill it explicitly: await (await Sandbox.connect(id, { apiKey, autoResume: true } as any)).kill().

  • Sweep for strays afterwards. Sandbox.list() returns a paginator, not an array — it has no .data and is not iterable:

    const p: any = await Sandbox.list({ apiKey } as any);
    const all: any[] = [];
    while (p.hasNext) all.push(...(await p.nextItems()));
    

    The account holds hundreds of legitimately paused sandboxes (user environments), so filter to the ids your run printed rather than judging by age, and never kill one you did not create.

  • Probe scripts live in the scratchpad. Never commit them.

  • Put the measured numbers in the commit/PR body. "Refuted by measurement: cold resume was 795 ms against a 45 s bound" is reviewable; "should be fine" is not.

04 同仓库更多 Skills