[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7ECJC2BAK18438KXDDS2KG0":3,"skill-same:01M7ECJC2BAK18438KXDDS2KG0":49},{"skillNo":4,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":20,"frontMatter":21,"refName":22,"commitSha":23,"skillDirectory":24,"forksCount":13,"repositoryHtmlUrl":25,"repositoryHomepageUrl":6,"ownerHtmlUrl":26,"ownerBlogUrl":6,"ownerDescription":27,"updatedAt":28,"install":29,"resources":42},"01M7ECJC2BAK18438KXDDS2KG0","local-voice-agent",null,"jonahthan433\u002FCortexOS","jonahthan433","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F211465495?v=4","01M7ECH9HPHGA89DCXHHJBTS3C","jonah.ai","01M7ECH9JCEM45DZMWPT7148S1",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fimported\u002F.archive\u002Flocal-voice-agent\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"# local-voice-agent\n\n# Local Voice Agent\n\nBuild a voice conversation system: mic → local STT → agent brain → local TTS → speakers.\nHalf-duplex (mic gated while agent speaks). Private by default — run the models locally.\n\n## Architecture (proven)\n```\nmic → Ears (whisper.cpp :2022 \u002Fv1\u002Faudio\u002Ftranscriptions)\n    → Brain (warm persistent agent session, streaming)\n    → Sentence chunker\n    → Mouth (Kokoro :8880 \u002Fv1\u002Faudio\u002Fspeech → PCM 24k int16)\n    → speakers\n```\nState machine: keydown→listening → keyup→thinking → first audio→speaking → done→idle.\nWhile speaking: mic closed, transcription off, no self-hear. Press talk while speaking →\ninterrupt + reopen mic. Typed input races PTT via asyncio.wait(FIRST_COMPLETED).\n\n## Gotchas (real, cost time — read references\u002F)\n- **whisper.cpp server** uses `--port` (NOT `-p`); default 8080; serves ONLY `\u002Finference`,\n  not `\u002Fv1\u002Faudio\u002Ftranscriptions`. Put a thin adapter in front to expose the OpenAI route.\n  See `references\u002Fwhisper_kokoro.md`.\n- **Kokoro** PyPI `kokoro[fastapi]` extra does NOT ship a server module in current versions.\n  Write a small FastAPI wrapping `KPipeline` (returns raw PCM). See `references\u002Fwhisper_kokoro.md`.\n- **Brain = Hermes locally**: `hermes serve` WS gateway (:9119) needs dashboard cookie auth\n  (401 headless). The proven headless path is `hermes chat -q \"\u003Ctext>\" -Q --resume \u003Cid>`;\n  session_id prints to STDERR. Capture it, reuse it → warm persistent session.\n  See `references\u002Fhermes_brain.md`.\n- **Audio I\u002FO needs a real machine**: sounddevice needs PortAudio; pynput needs an X display.\n  Headless servers can't capture\u002Fplay audio — run the loop on the desk machine.\n\n## Workflow rule (user correction, 2026-07-15)\nBefore building a visual\u002Fconversational system from a spec, ASK FIRST. Propose the\nconcept (or state→visual mapping) and wait for approval before writing code. The user\nexplicitly required this twice: \"Proceed, and always ask like you just did before any\nbuild.\" For the visualizer he picked a concept, then I proposed how each of the 5 states\nmaps to the scene and waited. Embed this in any creative\u002Fbuild task for this user.\n\n## Companion: browser visualizer (reads the same voice bus)\nA fullscreen Canvas visualizer can react to the voice agent via the `~\u002Fvoice-line\u002F`\nstatus files. Server is stdlib-only, read-only, binds 127.0.0.1:8777; `\u002Fstate` returns\n`{state, level, alert}`. Critical **stomp-tolerance**: if `.voice_waveform` is \u003C2s fresh,\nreport `speaking` regardless of `.voice_state`. See `references\u002Fvoice_bus_visualizer.md`.\n\n## Interruptibility & signal bus (root-cause fixes)\n- **Blocking playback kills interrupts.** `sd.OutputStream.write(whole_sentence)` blocks the\n  event loop, so `stop_playback()` from another task can't run mid-sentence. Fix: play in\n  `loop.run_in_executor`, write in small chunks (e.g. 2048 samples), check a\n  `threading.Event` stop flag between chunks (~40ms cancel latency). Root cause = single\n  blocking call, not the interrupt logic.\n- **Feed the visualizer REAL level.** Don't write `set_waveform([0.0]*64)`. Downsample the\n  actually-played PCM to 64 abs-normalized floats and push it to `.voice_waveform` so the\n  visualizer's speaking animation tracks the voice.\n\n## Remote \u002F thin-client deployment (audio ≠ brain machine)\nWhen the agent CLI (`hermes`) lives on a server but mic\u002Fspeakers are on a laptop, run the\naudio loop on the laptop and reach the rest over LAN. Make endpoints env-driven:\n- `HERMES_HOST` (default 127.0.0.1) → Whisper\u002FKokoro URLs `http:\u002F\u002F$HERMES_HOST:2022|8880`.\n- `HERMES_BIN` (default \"hermes\") → `.split()` it so `HERMES_BIN=\"ssh user@host hermes\"`\n  runs the brain remotely. **SSH argv trap:** `create_subprocess_exec([\"ssh\",\"host\",\"hermes\",\"chat\",\"-q\",text,\"-Q\"])`\n  passes each token separately; the remote sshd re-joins them and the remote shell re-splits\n  on spaces, so the quotes around `text` are LOST → `hermes: error: unrecognized arguments`\n  (a multi-word query splits into stray args). Fix: when prefix[0]==\"ssh\", build ONE remote\n  string with the query shell-quoted: `remote = \"hermes chat -q \" + shlex.quote(text) + \" -Q ...\"`\n  then `cmd = [*ssh_prefix, remote]`. Local (non-ssh) path keeps the argv list unchanged.\n- **BIND 0.0.0.0, NOT 127.0.0.1.** Whisper adapter (:2022), Kokoro (:8880), and the\n  visualizer server (:8777) all default to `127.0.0.1` — unreachable from the laptop.\n  For thin-client, bind `0.0.0.0` on every service the laptop hits (uvicorn `host=`,\n  `ThreadingHTTPServer((\"0.0.0.0\", PORT))`). Restart the actual PROCESS after the edit;\n  a running daemon keeps its old bind. Identify by port, kill that PID, relaunch.\n- run.sh guard: don't gate on `if [ \"$HERMES_HOST\" = 127.0.0.1 ]` — the env var often\n  fails to propagate into the script and it wrongly boots local torch\u002Fuvicorn. PROBE the\n  target instead: `if curl -s --max-time 2 http:\u002F\u002F$HERMES_HOST:2022\u002Fhealth; then skip\n  local servers; else start them; fi`. Probing the real endpoint is robust to env drift.\n- Needs keyless SSH (ed25519 → server `authorized_keys`). If server user has NO password,\n  key auth is the ONLY way in; `ssh-copy-id` can't run, so append the laptop's\n  `id_ed25519.pub` to the server's `~\u002F.ssh\u002Fauthorized_keys` directly. See `references\u002Fremote_client.md`.\n\n## Pitfalls that crashed a real session\n- **Duck\u002Fcontext-manager `__exit__` reads an attr `__init__` never set.** `Duck.__exit__`\n  read `self._vol`, but `__enter__` only set it when Spotify was running → `AttributeError`\n  killed the turn when Spotify was absent. Rule: init EVERY attribute referenced in\n  `__exit__`\u002Fcleanup in `__init__` (`self._vol = None`), so partial-enter never crashes exit.\n- **Scope: \"voice agent\" ≠ \"browser app\".** User expected to open the voice agent in a\n  browser like a dashboard. The talk loop is a LOCAL python app (mic\u002Fspeakers), not a web\n  UI — only the visualizer is browser-served. State this split upfront: visualizer = browser\n  (`http:\u002F\u002F\u003Chost>:8777`); talking = local app. Browser-mic voice is a separate, unbuilt feature.\n\n## Remote brain via OpenLive + ACP over SSH\nWhen the user runs OpenLive (a desktop voice app) on their laptop and wants Hermes on a\nheadless host to be the brain, OpenLive drives Hermes via ACP over keyless SSH. See\n`references\u002Fopenlive_acp_ssh.md` for the full setup: install `hermes-agent[acp]` in the\nHermes venv, generate an ed25519 key, add it to the laptop's `authorized_keys`, and point\nOpenLive's Hermes agent at a project path on the host.\n\n## Files shape\nmain.py (orchestrator) · ears.py (STT) · brain.py (agent) · mouth.py (TTS) ·\nptt.py (hotkey + held-state) · ducking.py (optional Spotify) · signals.py (status bus) ·\nrun.sh (start servers). Keep it to the files the spec needs; no extra layers.\n\n## Verify before claiming done\nWhisper health 200 · Kokoro returns PCM · brain returns a reply + reuses session_id ·\nfull text loop (transcript→brain→PCM) works. Real mic\u002Fspeaker only testable on desk machine.\n",{},"main","38b32c99e7ed0b8e215d638e92cfa452c1ec3079","skills\u002Fimported\u002F.archive\u002Flocal-voice-agent","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS","https:\u002F\u002Fgithub.com\u002Fjonahthan433","AI Consultant","2026-10-24 05:13:39",{"source":7,"skillName":5,"commands":30,"prompt":40,"zipAvailable":41},[31,34,37],{"tool":32,"command":33},"npx","npx skills add jonahthan433\u002FCortexOS --skill local-voice-agent",{"tool":35,"command":36},"bunx","bunx skills add jonahthan433\u002FCortexOS --skill local-voice-agent",{"tool":38,"command":39},"pnpm","pnpm dlx skills add jonahthan433\u002FCortexOS --skill local-voice-agent","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fimported\u002F.archive\u002Flocal-voice-agent\u002FSKILL.md",true,[43],{"path":44,"resourceType":45,"resourceTypeDesc":46,"sizeBytes":47,"sourceHtmlUrl":48},"skill.json","REFERENCE","参考文档",617,"https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fimported\u002F.archive\u002Flocal-voice-agent\u002Fskill.json",[50,56,62,68,74,80],{"skillNo":51,"name":52,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":53,"categories":54,"keywords":55,"publishedAt":18},"01M7EDQZDQX3Q8Y7GCQX6KN9V7","opportunity-brief","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fsales\u002Fopportunity-brief\u002FSKILL.md",[],[],{"skillNo":57,"name":58,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":59,"categories":60,"keywords":61,"publishedAt":18},"01M7EDQYA0V9Y99VKBREJJ3MRT","morning-intel","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fresearch\u002Fmorning-intel\u002FSKILL.md",[],[],{"skillNo":63,"name":64,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":65,"categories":66,"keywords":67,"publishedAt":18},"01M7EDQVX1JVY0B6PDHKKGM19G","decision-brief","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fresearch\u002Fdecision-brief\u002FSKILL.md",[],[],{"skillNo":69,"name":70,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":18},"01M7EDQV1GENZAC2SZSD8KZ3GM","weekly-review","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fproductivity\u002Fweekly-review\u002FSKILL.md",[],[],{"skillNo":75,"name":76,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":77,"categories":78,"keywords":79,"publishedAt":18},"01M7EDQT2KETJ1Y3CGVN9EKW2J","system-health","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fops\u002Fsystem-health\u002FSKILL.md",[],[],{"skillNo":81,"name":82,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":18},"01M7EDQQN2JK3XPWMS1M4T0F3K","backup-restore","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fops\u002Fbackup-restore\u002FSKILL.md",[],[]]