[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M713KE2W1WMPDVP7F25T5K7Y":3,"skill-same:01M713KE2W1WMPDVP7F25T5K7Y":102},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":52,"commitSha":53,"skillDirectory":54,"forksCount":55,"repositoryHtmlUrl":56,"repositoryHomepageUrl":57,"ownerHtmlUrl":58,"ownerBlogUrl":59,"ownerDescription":60,"updatedAt":61,"install":62,"resources":75},"01M713KE2W1WMPDVP7F25T5K7Y","smart-glasses-live-captions",null,"Live captions, translation and real-time hints on smart glasses - capture speech on the phone or the glasses, stream it to speech-to-text, turn partial and final results into lines that fit each glasses display, keep end-to-end latency low, and fall back to a short hint or voice in the ear. Covers Even Realities G2, Meta Ray-Ban Display (Web Apps and DAT), Rokid Glasses, Vuzix Z100, USB-C display glasses, phone notifications and audio-only glasses, with a runnable example (Node caption server, glasses web page, Even Hub plugin). Use when building subtitles for deaf or hard-of-hearing users, a live translator, a teleprompter, meeting captions or an AI assistant that shows or speaks text on glasses, or when captions on glasses lag, flicker, wrap badly or drop characters. Not for offline video subtitling or meeting transcription that never reaches glasses.","hashgraph-online\u002Fawesome-codex-plugins","hashgraph-online","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160371962?v=4","01M6CNB6NRJVNNGN14TR01Q6D3","HOL (Hashgraph Online)","01M6JT1HSKKG9V68KQAQGBA7XP",1566,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fhinty-eu\u002Fsmart-glasses-skills\u002Fskills\u002Fsmart-glasses-live-captions\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"smart glasses live captions","live","Captions","speech","assistant","offline","2026","translator","server","usb","text","notifications","display","apps","Meta","time","Audio","translation","Video","teleprompter","2026-11-10 10:12:30",false,"---\nname: smart-glasses-live-captions\ndescription: Live captions, translation and real-time hints on smart glasses - capture speech on the phone or the glasses, stream it to speech-to-text, turn partial and final results into lines that fit each glasses display, keep end-to-end latency low, and fall back to a short hint or voice in the ear. Covers Even Realities G2, Meta Ray-Ban Display (Web Apps and DAT), Rokid Glasses, Vuzix Z100, USB-C display glasses, phone notifications and audio-only glasses, with a runnable example (Node caption server, glasses web page, Even Hub plugin). Use when building subtitles for deaf or hard-of-hearing users, a live translator, a teleprompter, meeting captions or an AI assistant that shows or speaks text on glasses, or when captions on glasses lag, flicker, wrap badly or drop characters. Not for offline video subtitling or meeting transcription that never reaches glasses.\nlicense: Apache-2.0\nmetadata:\n  version: \"1.0.1\"\n  last-updated: \"2026-11-09\"\n---\n\n# Live captions on smart glasses\n\nLast checked: 2026-11-08. Verification: the example server and glasses web page ran locally, the Even G2 plugin ran in\nthe official Even Hub simulator; nothing in this skill ran on glasses yet. Vendor facts are `docs-only` unless marked.\n\nThis skill assumes the target glasses are known. If they are not, start with the `smart-glasses` skill (which\nglasses, which rung of the display-text ladder, vendor SDK pointers). The rungs used below: 1 vendor SDK, 2 web app on\nthe glasses, 3 USB-C display glasses as an external display, 4 phone notification mirrored by the glasses' app,\n5 voice in the ear.\n\n## The pipeline\n\n```\nmicrophone ─► VAD ─► streaming STT ─► caption events ─► transport ─► line fitter ─► glasses display\n (phone or     (drop     (partial and      seq, text,      WebSocket,   per display     (SDK, web page,\n  glasses)     silence)   final results)   final           HTTP poll    budget          USB-C, notification)\n                                                └──────────► short hint ─► text-to-speech ─► ear\n```\n\nMake these decisions in order. Each has a reference file with the details and sources.\n\n| # | Decision | Default | Reference |\n|---|---|---|---|\n| 1 | Where to capture audio | Phone microphone | `references\u002Fpipeline.md` |\n| 2 | Which speech-to-text | Cloud streaming STT with partial results | `references\u002Fstt-options.md` |\n| 3 | How partial and final results become lines | Replace the last line until final | `references\u002Fline-fitting.md` |\n| 4 | How many lines and how often to update | Per display budget, coalesced | `references\u002Fline-fitting.md` |\n| 5 | Latency budget | First words under 1 s, finals under 2 s | `references\u002Flatency.md` |\n| 6 | Non-Latin scripts | Check the font first, render to image if missing | `references\u002Fscripts-and-fonts.md` |\n| 7 | Tiny screen or no screen | A few-word hint, spoken finals only | `references\u002Fshort-hints.md` |\n\n## 1. Capture audio on the phone unless you need the wearer's own voice\n\nGlasses microphones are tuned for the wearer's mouth. Captions are usually about the other person, so the phone\n(on the table or in a hand, not in a pocket) hears them better. The phone path also keeps Bluetooth free for the\ndisplay and avoids the call-audio mode that some glasses switch into when their microphone opens.\n\n| Glasses (as of 2026-11-08) | Microphone available to your app | Use it for |\n|---|---|---|\n| Even Realities G2 | Yes, from the plugin itself: `AudioInputSource.Phone` (phone mic) or `.Glasses`, 16 kHz s16le mono PCM | phone mic: the other person; glasses mic: the wearer |\n| Meta Ray-Ban Display, Web App | No (Meta: only the Device Access Toolkit can capture audio) | nothing; capture on the phone |\n| Meta glasses via DAT | Bluetooth headset (HFP) microphone; on-glasses speech recognition is beta | wearer's speech, with known issues |\n| Rokid Glasses (CXR-L) | Yes: `startAudioStream`, 16 kHz 16-bit mono PCM (Rokid docs) | wearer's speech |\n| Vuzix Z100 | No microphone | nothing; capture on the phone |\n| USB-C display glasses | Usually a USB audio device on the phone | either |\n| Audio-only glasses (Ray-Ban Meta and others) | Bluetooth headset microphone only | wearer's speech; costs audio quality |\n\n**Even G2 needs no second app for the phone microphone.** The Even Hub plugin opens it itself with\n`bridge.audioControl(true, AudioInputSource.Phone)` (permission `phone-microphone`) to caption the other person, or\n`AudioInputSource.Glasses` (permission `g2-microphone`, only after `createStartUpPageContainer`) for the wearer. Frames\narrive in `onEvenHubEvent` as `event.audioEvent.audioPcm` (`Uint8Array`, PCM 16 kHz, signed 16-bit little-endian,\nmono). The SDK has no speech-to-text: send the PCM as binary WebSocket frames to your STT relay, which holds the\nprovider key and sends caption text back over the same socket. Use `audioControl`, not `getUserMedia`, which fails in\nthe plugin WebView on iOS ([device APIs](https:\u002F\u002Fhub.evenrealities.com\u002Fdocs\u002Fbuild\u002Fdevice-apis), 2026-08-29;\n[#11](https:\u002F\u002Fgithub.com\u002Feven-realities\u002Feverything-evenhub\u002Fissues\u002F11)). The example plugin has this as an optional\nphone-microphone mode.\n\nDetails, VAD and streaming: `references\u002Fpipeline.md`.\n\n### Even Hub plugin checklist (as of 2026-11-09)\n\n- `app.json` `permissions`: `{\"name\": \"phone-microphone\", \"desc\": \"...\"}` for `AudioInputSource.Phone`,\n  `g2-microphone` for `AudioInputSource.Glasses`, and `network` with a `whitelist` of full origins, both\n  `https:\u002F\u002Fcaptions.example.com` and `wss:\u002F\u002Fcaptions.example.com` (no wildcards, no bare hosts). The whitelist does\n  not bypass CORS: the relay must still send `Access-Control-Allow-Origin`. Declare only what the code uses\n  ([packaging](https:\u002F\u002Fhub.evenrealities.com\u002Fdocs\u002Fship\u002Fpackaging), [networking](https:\u002F\u002Fhub.evenrealities.com\u002Fdocs\u002Fbuild\u002Fnetworking)).\n- On the root page a double tap calls `bridge.shutDownPageContainer(1)` (system exit dialog); mode 0 is rejected.\n- No provider keys in the package: \"Never bundle secrets or API keys into the `.ehpk`\"\n  ([app submission](https:\u002F\u002Fhub.evenrealities.com\u002Fdocs\u002Fship\u002Fapp-submission), 2026-10-09). The official `asr`\n  template reads `VITE_STT_API_KEY`, which ends up in the bundle; replace it with your relay.\n- The privacy policy covers every permission, and the glasses show a message when setup is missing.\n- The simulator \"is not a hardware emulator\" (BLE timing, real font rendering and WebView suspension are not\n  reproduced): measure latency and microphone behaviour on glasses before you promise \"about a second\".\n\n## 2. Pick a streaming STT that returns partial results\n\nCaptions need partial (interim) results; a batch API that answers after the sentence ends feels broken on glasses.\n\n- **Cloud streaming** (Deepgram, AssemblyAI, Speechmatics, ElevenLabs Scribe Realtime, Google, Azure, OpenAI,\n  Amazon, Gladia): best accuracy and language coverage, about USD 0.15 to 1.00 per audio hour at list price\n  (2026-11-08), needs a network. Keep the key on a server; browsers and glasses get short-lived tokens or go through\n  your relay.\n- **On device** (Android `SpeechRecognizer` on-device, iOS `SpeechAnalyzer` or `SFSpeechRecognizer`, whisper.cpp,\n  WhisperKit, Moonshine): offline and private, fewer languages, session limits to work around.\n- **Browser Web Speech API**: free and quick for a prototype in Chrome or Safari; in Chrome the audio goes to a cloud\n  service unless an on-device language pack is installed (Chrome 139+).\n\nComparison with prices, latency claims, endpoints and limits: `references\u002Fstt-options.md`.\n\n## 3. Caption events: partial results replace, finals commit\n\nNormalise every provider to one event shape before anything reaches a display:\n\n```ts\ntype Caption = { seq: number; text: string; final: boolean } \u002F\u002F text = the whole sentence so far, not a delta\ntype CaptionState = { finals: string[]; partial: string; seq: number }\n\nfunction apply(st: CaptionState, m: Caption | { type: 'clear' }): boolean {\n  if ('type' in m) { st.finals = []; st.partial = ''; return true }\n  if (m.seq \u003C= st.seq) return false            \u002F\u002F replay after a reconnect, already shown\n  st.seq = m.seq\n  if (!m.final) { st.partial = m.text; return true }\n  st.partial = ''\n  if (m.text) st.finals = [...st.finals.slice(-29), m.text]\n  return true\n}\n```\n\nRules that keep text stable on a small display:\n\n- Committed lines never move or re-wrap. Only the lines of the current partial change.\n- A partial starts on a fresh line, so replacing it never shifts older text.\n- Providers revise earlier words in a partial (\"for\" becomes \"four\"). Show the revision; do not append.\n- Some providers finalise a sentence in several segments (Deepgram `is_final` segments until `speech_final`). Join\n  them in the adapter and emit one final per sentence.\n\n## 4. Fit lines per display and coalesce updates\n\nWrap with the real font metrics of the target and keep the newest N lines, newest at the bottom. Caption-friendly\nbudgets (as of 2026-11-08, verify on the device):\n\n| Display | Lines for captions | Width per line | Update path and rate |\n|---|---|---|---|\n| Even G2 (576x288, one firmware font, 27 px lines) | 3 to 8 of 10 | about 55 Latin characters | `textContainerUpgrade`, one call at a time, 120 ms debounce |\n| Meta Ray-Ban Display Web App (600x600) | 3 to 4 at 32 to 40 px | about 26 to 32 characters | DOM, 30 Hz panel |\n| Meta DAT display | 3 to 4 | preset text styles | every send replaces the whole screen |\n| Rokid Glasses (CXR-L custom view) | 3 at 24 sp | about 22 Latin characters (estimate) | `customViewUpdate`, at most one every 250 ms (estimate) |\n| Vuzix Z100 (640x480) | 3 (up to 7 fit) | about 36 in the text layout, which wraps near 42 (MentraOS measurement, community) | `sendText` or canvas `updateText` + `commit()` |\n| USB-C display glasses (1920x1080 per eye) | 2 to 4, large, centre of view | your choice | normal phone UI, 60 Hz |\n| Phone notification on glasses | 1 to 2 | about 30 to 40 characters | a new notification per final, at most one every 2-3 s |\n\nCoalesce partials to what the link can carry: queue only the newest frame, send it when the previous write finished.\nNever fire concurrent writes at a BLE display. Full rules, wrapping code, CJK and paging: `references\u002Fline-fitting.md`.\n\n## 5. Keep a latency budget\n\nAim for the first words of a sentence on the glasses within about 1 s of being spoken and the final sentence within\n2 s. The biggest items are usually the STT partial delay (vendor claims range from about 150 to 500 ms), endpointing\nbefore finals (300 ms to 1 s of silence) and the display path (Even G2: 120 ms debounce plus about 80 ms per container\nwrite, community measurement). Measure each stage with timestamps before tuning: `references\u002Flatency.md`.\n\n## 6. Check the font before promising a language\n\nThe Even G2 firmware font has Latin, Cyrillic, Greek and CJK, and silently drops Arabic, Hebrew, Thai and Indic\nscripts. Meta Web Apps render what the font stack covers, so ship a subset web font for your scripts. Vuzix uploads\nmissing glyphs from the phone. Rokid Arabic support is an open question. Strip characters the target cannot draw\nand tell the wearer, or render the line to an image (slow, finals only). Per brand: `references\u002Fscripts-and-fonts.md`.\n\n## 7. On tiny screens and in the ear, show a hint, not a transcript\n\nA two-line display or a voice in the ear cannot carry a running transcript. Turn each final sentence (or the other\nperson's question) into a 3 to 8 word hint, speak only finals, never talk over the wearer, and drop stale hints.\nPrompt patterns and per display budgets: `references\u002Fshort-hints.md`.\n\n## Architecture: where the code runs\n\n- **One phone app does everything** (capture, STT, fit, send to the glasses SDK): lowest latency, needed for rungs 1,\n  3, 4 and 5 on Android and iOS.\n- **Phone or browser captures, a server relays, the glasses page or plugin displays**: needed for rung 2 and for\n  plugin platforms such as Even Hub, where the display code runs in a WebView. On Even Hub the plugin can also be the\n  capture side: it streams PCM from `audioControl` to the relay and shows the captions that come back. Pair screens\n  with a short room code typed on the phone; keep STT keys on the server.\n- **WebSocket first, HTTP polling as the fallback.** Some glasses WebViews fail the WebSocket handshake while `fetch`\n  works (reported for the Even app on iOS, community). After two failed attempts, poll a `\u002Flatest` endpoint every\n  0.5 to 1 s and keep retrying the socket slowly.\n- **Reconnect on resume.** Glasses WebViews and phone apps get suspended in the background. Re-open the connection\n  when the page or plugin comes back to the foreground, and replay the last lines from the server.\n\n## Example app\n\n`examples\u002Flive-captions\u002F` in this repository: a Node server (`--demo` script, `--browser-mic` with the Web Speech\nAPI, `--stt deepgram` adapter), a web page for glasses browsers and the Meta Ray-Ban Display, an Even Hub plugin, and\nnative Android and iOS targets written against the same protocol.\n\n```sh\ncd examples\u002Flive-captions\u002Fserver && npm install && node server.mjs --demo   # prints a room code\n# open http:\u002F\u002Flocalhost:8787\u002F?room=CODE, or run the Even plugin in the simulator:\ncd ..\u002Feven-hub && npm install && npm run test:sim\n```\n\nThe README there has the protocol (`caption`, `clear`, `hello`, `action`, `GET \u002Flatest`), the verification level of\neach target with its date, and how to run each one on hardware.\n\n## Pitfalls\n\n- **Partial text treated as a delta.** Appending partials duplicates words. Every partial replaces the previous one.\n- **Re-wrapping everything on each partial.** Older lines jump and the wearer loses their place. Wrap committed\n  lines once; re-wrap only the partial.\n- **Concurrent display writes over BLE.** On Even G2, parallel `textContainerUpgrade` calls are reported to break the\n  link (community). Serialize all writes through one queue.\n- **An overflowing event-capture container on Even G2.** The firmware scrolls it and swallows swipes until a\n  boundary. Keep the capture container one line, or fit its text exactly.\n- **Long silent sessions.** Deepgram closes a stream after about 10 s without audio unless you send `KeepAlive`;\n  AssemblyAI bills the time the socket is open; Google STT streams end after 5 minutes; `SFSpeechRecognizer` plans\n  for a one-minute limit; Meta DAT speech stops reporting after one minute (known issue, restart the capability).\n  Rotate or restart sessions on a sentence boundary.\n- **Chrome on Android ignores `continuous`.** Restart Web Speech recognition in `onend`.\n- **Meta DAT microphone over HFP ends the display session** on Ray-Ban Display (meta-wearables-dat-android issue\n  #210, open as of 2026-11-08; Meta staff: the Display fix comes \"at a later date\"). Use the phone microphone with a\n  display.\n- **Notifications as a caption channel.** Android sheds updates to an existing notification beyond 5 per second per\n  app, and no brand documents how it forwards updates; companion apps show one or two lines and may delay. Post a new\n  notification per final, at most one every 2-3 s.\n- **Smart formatting changes what you show.** Deepgram `smart_format` turned \"four o'clock\" into \"04:00\" in our test.\n  Decide on number and time formatting for the display, not just for the transcript.\n- **Missing glyphs vanish silently** on Even G2: half a sentence disappears with no error. Strip and warn.\n\n## Privacy and consent\n\nCaptions transcribe people who did not install your app. Tell them (a visible indicator, a spoken notice for long\nsessions), check recording-consent law where the app is used, do not store audio, and keep transcripts on the device\nor delete them after the session. Platform rules apply too: Meta's developer policies prohibit surreptitiously\ncollecting recognisable audio of the user or others and identifying people by their voice; Android XR guidance limits\ncontinuous transcription when the wearer is not engaged. Prefer on-device STT when the content is sensitive.\n\n## Related skills\n\n- `smart-glasses`: choosing glasses, the display-text ladder, input, publishing, and the install commands of the\n  official vendor skills (Even Hub, Meta DAT, Meta Web Apps, Android XR Glimmer).\n- `smart-glasses-rokid` and `smart-glasses-vuzix`: the native display calls used by the Android and iOS targets.\n","# Live captions on smart glasses\n\nLast checked: 2026-11-08. Verification: the example server and glasses web page ran locally, the Even G2 plugin ran in\nthe official Even Hub simulator; nothing in this skill ran on glasses yet. Vendor facts are `docs-only` unless marked.\n\nThis skill assumes the target glasses are known. If they are not, start with the `smart-glasses` skill (which\nglasses, which rung of the display-text ladder, vendor SDK pointers). The rungs used below: 1 vendor SDK, 2 web app on\nthe glasses, 3 USB-C display glasses as an external display, 4 phone notification mirrored by the glasses' app,\n5 voice in the ear.\n\n## The pipeline\n\n```\nmicrophone ─► VAD ─► streaming STT ─► caption events ─► transport ─► line fitter ─► glasses display\n (phone or     (drop     (partial and      seq, text,      WebSocket,   per display     (SDK, web page,\n  glasses)     silence)   final results)   final           HTTP poll    budget          USB-C, notification)\n                                                └──────────► short hint ─► text-to-speech ─► ear\n```\n\nMake these decisions in order. Each has a reference file with the details and sources.\n\n| # | Decision | Default | Reference |\n|---|---|---|---|\n| 1 | Where to capture audio | Phone microphone | `references\u002Fpipeline.md` |\n| 2 | Which speech-to-text | Cloud streaming STT with partial results | `references\u002Fstt-options.md` |\n| 3 | How partial and final results become lines | Replace the last line until final | `references\u002Fline-fitting.md` |\n| 4 | How many lines and how often to update | Per display budget, coalesced | `references\u002Fline-fitting.md` |\n| 5 | Latency budget | First words under 1 s, finals under 2 s | `references\u002Flatency.md` |\n| 6 | Non-Latin scripts | Check the font first, render to image if missing | `references\u002Fscripts-and-fonts.md` |\n| 7 | Tiny screen or no screen | A few-word hint, spoken finals only | `references\u002Fshort-hints.md` |\n\n## 1. Capture audio on the phone unless you need the wearer's own voice\n\nGlasses microphones are tuned for the wearer's mouth. Captions are usually about the other person, so the phone\n(on the table or in a hand, not in a pocket) hears them better. The phone path also keeps Bluetooth free for the\ndisplay and avoids the call-audio mode that some glasses switch into when their microphone opens.\n\n| Glasses (as of 2026-11-08) | Microphone available to your app | Use it for |\n|---|---|---|\n| Even Realities G2 | Yes, from the plugin itself: `AudioInputSource.Phone` (phone mic) or `.Glasses`, 16 kHz s16le mono PCM | phone mic: the other person; glasses mic: the wearer |\n| Meta Ray-Ban Display, Web App | No (Meta: only the Device Access Toolkit can capture audio) | nothing; capture on the phone |\n| Meta glasses via DAT | Bluetooth headset (HFP) microphone; on-glasses speech recognition is beta | wearer's speech, with known issues |\n| Rokid Glasses (CXR-L) | Yes: `startAudioStream`, 16 kHz 16-bit mono PCM (Rokid docs) | wearer's speech |\n| Vuzix Z100 | No microphone | nothing; capture on the phone |\n| USB-C display glasses | Usually a USB audio device on the phone | either |\n| Audio-only glasses (Ray-Ban Meta and others) | Bluetooth headset microphone only | wearer's speech; costs audio quality |\n\n**Even G2 needs no second app for the phone microphone.** The Even Hub plugin opens it itself with\n`bridge.audioControl(true, AudioInputSource.Phone)` (permission `phone-microphone`) to caption the other person, or\n`AudioInputSource.Glasses` (permission `g2-microphone`, only after `createStartUpPageContainer`) for the wearer. Frames\narrive in `onEvenHubEvent` as `event.audioEvent.audioPcm` (`Uint8Array`, PCM 16 kHz, signed 16-bit little-endian,\nmono). The SDK has no speech-to-text: send the PCM as binary WebSocket frames to your STT relay, which holds the\nprovider key and sends caption text back over the same socket. Use `audioControl`, not `getUserMedia`, which fails in\nthe plugin WebView on iOS ([device APIs](https:\u002F\u002Fhub.evenrealities.com\u002Fdocs\u002Fbuild\u002Fdevice-apis), 2026-08-29;\n[#11](https:\u002F\u002Fgithub.com\u002Feven-realities\u002Feverything-evenhub\u002Fissues\u002F11)). The example plugin has this as an optional\nphone-microphone mode.\n\nDetails, VAD and streaming: `references\u002Fpipeline.md`.\n\n### Even Hub plugin checklist (as of 2026-11-09)\n\n- `app.json` `permissions`: `{\"name\": \"phone-microphone\", \"desc\": \"...\"}` for `AudioInputSource.Phone`,\n  `g2-microphone` for `AudioInputSource.Glasses`, and `network` with a `whitelist` of full origins, both\n  `https:\u002F\u002Fcaptions.example.com` and `wss:\u002F\u002Fcaptions.example.com` (no wildcards, no bare hosts). The whitelist does\n  not bypass CORS: the relay must still send `Access-Control-Allow-Origin`. Declare only what the code uses\n  ([packaging](https:\u002F\u002Fhub.evenrealities.com\u002Fdocs\u002Fship\u002Fpackaging), [networking](https:\u002F\u002Fhub.evenrealities.com\u002Fdocs\u002Fbuild\u002Fnetworking)).\n- On the root page a double tap calls `bridge.shutDownPageContainer(1)` (system exit dialog); mode 0 is rejected.\n- No provider keys in the package: \"Never bundle secrets or API keys into the `.ehpk`\"\n  ([app submission](https:\u002F\u002Fhub.evenrealities.com\u002Fdocs\u002Fship\u002Fapp-submission), 2026-10-09). The official `asr`\n  template reads `VITE_STT_API_KEY`, which ends up in the bundle; replace it with your relay.\n- The privacy policy covers every permission, and the glasses show a message when setup is missing.\n- The simulator \"is not a hardware emulator\" (BLE timing, real font rendering and WebView suspension are not\n  reproduced): measure latency and microphone behaviour on glasses before you promise \"about a second\".\n\n## 2. Pick a streaming STT that returns partial results\n\nCaptions need partial (interim) results; a batch API that answers after the sentence ends feels broken on glasses.\n\n- **Cloud streaming** (Deepgram, AssemblyAI, Speechmatics, ElevenLabs Scribe Realtime, Google, Azure, OpenAI,\n  Amazon, Gladia): best accuracy and language coverage, about USD 0.15 to 1.00 per audio hour at list price\n  (2026-11-08), needs a network. Keep the key on a server; browsers and glasses get short-lived tokens or go through\n  your relay.\n- **On device** (Android `SpeechRecognizer` on-device, iOS `SpeechAnalyzer` or `SFSpeechRecognizer`, whisper.cpp,\n  WhisperKit, Moonshine): offline and private, fewer languages, session limits to work around.\n- **Browser Web Speech API**: free and quick for a prototype in Chrome or Safari; in Chrome the audio goes to a cloud\n  service unless an on-device language pack is installed (Chrome 139+).\n\nComparison with prices, latency claims, endpoints and limits: `references\u002Fstt-options.md`.\n\n## 3. Caption events: partial results replace, finals commit\n\nNormalise every provider to one event shape before anything reaches a display:\n\n```ts\ntype Caption = { seq: number; text: string; final: boolean } \u002F\u002F text = the whole sentence so far, not a delta\ntype CaptionState = { finals: string[]; partial: string; seq: number }\n\nfunction apply(st: CaptionState, m: Caption | { type: 'clear' }): boolean {\n  if ('type' in m) { st.finals = []; st.partial = ''; return true }\n  if (m.seq \u003C= st.seq) return false            \u002F\u002F replay after a reconnect, already shown\n  st.seq = m.seq\n  if (!m.final) { st.partial = m.text; return true }\n  st.partial = ''\n  if (m.text) st.finals = [...st.finals.slice(-29), m.text]\n  return true\n}\n```\n\nRules that keep text stable on a small display:\n\n- Committed lines never move or re-wrap. Only the lines of the current partial change.\n- A partial starts on a fresh line, so replacing it never shifts older text.\n- Providers revise earlier words in a partial (\"for\" becomes \"four\"). Show the revision; do not append.\n- Some providers finalise a sentence in several segments (Deepgram `is_final` segments until `speech_final`). Join\n  them in the adapter and emit one final per sentence.\n\n## 4. Fit lines per display and coalesce updates\n\nWrap with the real font metrics of the target and keep the newest N lines, newest at the bottom. Caption-friendly\nbudgets (as of 2026-11-08, verify on the device):\n\n| Display | Lines for captions | Width per line | Update path and rate |\n|---|---|---|---|\n| Even G2 (576x288, one firmware font, 27 px lines) | 3 to 8 of 10 | about 55 Latin characters | `textContainerUpgrade`, one call at a time, 120 ms debounce |\n| Meta Ray-Ban Display Web App (600x600) | 3 to 4 at 32 to 40 px | about 26 to 32 characters | DOM, 30 Hz panel |\n| Meta DAT display | 3 to 4 | preset text styles | every send replaces the whole screen |\n| Rokid Glasses (CXR-L custom view) | 3 at 24 sp | about 22 Latin characters (estimate) | `customViewUpdate`, at most one every 250 ms (estimate) |\n| Vuzix Z100 (640x480) | 3 (up to 7 fit) | about 36 in the text layout, which wraps near 42 (MentraOS measurement, community) | `sendText` or canvas `updateText` + `commit()` |\n| USB-C display glasses (1920x1080 per eye) | 2 to 4, large, centre of view | your choice | normal phone UI, 60 Hz |\n| Phone notification on glasses | 1 to 2 | about 30 to 40 characters | a new notification per final, at most one every 2-3 s |\n\nCoalesce partials to what the link can carry: queue only the newest frame, send it when the previous write finished.\nNever fire concurrent writes at a BLE display. Full rules, wrapping code, CJK and paging: `references\u002Fline-fitting.md`.\n\n## 5. Keep a latency budget\n\nAim for the first words of a sentence on the glasses within about 1 s of being spoken and the final sentence within\n2 s. The biggest items are usually the STT partial delay (vendor claims range from about 150 to 500 ms), endpointing\nbefore finals (300 ms to 1 s of silence) and the display path (Even G2: 120 ms debounce plus about 80 ms per container\nwrite, community measurement). Measure each stage with timestamps before tuning: `references\u002Flatency.md`.\n\n## 6. Check the font before promising a language\n\nThe Even G2 firmware font has Latin, Cyrillic, Greek and CJK, and silently drops Arabic, Hebrew, Thai and Indic\nscripts. Meta Web Apps render what the font stack covers, so ship a subset web font for your scripts. Vuzix uploads\nmissing glyphs from the phone. Rokid Arabic support is an open question. Strip characters the target cannot draw\nand tell the wearer, or render the line to an image (slow, finals only). Per brand: `references\u002Fscripts-and-fonts.md`.\n\n## 7. On tiny screens and in the ear, show a hint, not a transcript\n\nA two-line display or a voice in the ear cannot carry a running transcript. Turn each final sentence (or the other\nperson's question) into a 3 to 8 word hint, speak only finals, never talk over the wearer, and drop stale hints.\nPrompt patterns and per display budgets: `references\u002Fshort-hints.md`.\n\n## Architecture: where the code runs\n\n- **One phone app does everything** (capture, STT, fit, send to the glasses SDK): lowest latency, needed for rungs 1,\n  3, 4 and 5 on Android and iOS.\n- **Phone or browser captures, a server relays, the glasses page or plugin displays**: needed for rung 2 and for\n  plugin platforms such as Even Hub, where the display code runs in a WebView. On Even Hub the plugin can also be the\n  capture side: it streams PCM from `audioControl` to the relay and shows the captions that come back. Pair screens\n  with a short room code typed on the phone; keep STT keys on the server.\n- **WebSocket first, HTTP polling as the fallback.** Some glasses WebViews fail the WebSocket handshake while `fetch`\n  works (reported for the Even app on iOS, community). After two failed attempts, poll a `\u002Flatest` endpoint every\n  0.5 to 1 s and keep retrying the socket slowly.\n- **Reconnect on resume.** Glasses WebViews and phone apps get suspended in the background. Re-open the connection\n  when the page or plugin comes back to the foreground, and replay the last lines from the server.\n\n## Example app\n\n`examples\u002Flive-captions\u002F` in this repository: a Node server (`--demo` script, `--browser-mic` with the Web Speech\nAPI, `--stt deepgram` adapter), a web page for glasses browsers and the Meta Ray-Ban Display, an Even Hub plugin, and\nnative Android and iOS targets written against the same protocol.\n\n```sh\ncd examples\u002Flive-captions\u002Fserver && npm install && node server.mjs --demo   # prints a room code\n# open http:\u002F\u002Flocalhost:8787\u002F?room=CODE, or run the Even plugin in the simulator:\ncd ..\u002Feven-hub && npm install && npm run test:sim\n```\n\nThe README there has the protocol (`caption`, `clear`, `hello`, `action`, `GET \u002Flatest`), the verification level of\neach target with its date, and how to run each one on hardware.\n\n## Pitfalls\n\n- **Partial text treated as a delta.** Appending partials duplicates words. Every partial replaces the previous one.\n- **Re-wrapping everything on each partial.** Older lines jump and the wearer loses their place. Wrap committed\n  lines once; re-wrap only the partial.\n- **Concurrent display writes over BLE.** On Even G2, parallel `textContainerUpgrade` calls are reported to break the\n  link (community). Serialize all writes through one queue.\n- **An overflowing event-capture container on Even G2.** The firmware scrolls it and swallows swipes until a\n  boundary. Keep the capture container one line, or fit its text exactly.\n- **Long silent sessions.** Deepgram closes a stream after about 10 s without audio unless you send `KeepAlive`;\n  AssemblyAI bills the time the socket is open; Google STT streams end after 5 minutes; `SFSpeechRecognizer` plans\n  for a one-minute limit; Meta DAT speech stops reporting after one minute (known issue, restart the capability).\n  Rotate or restart sessions on a sentence boundary.\n- **Chrome on Android ignores `continuous`.** Restart Web Speech recognition in `onend`.\n- **Meta DAT microphone over HFP ends the display session** on Ray-Ban Display (meta-wearables-dat-android issue\n  #210, open as of 2026-11-08; Meta staff: the Display fix comes \"at a later date\"). Use the phone microphone with a\n  display.\n- **Notifications as a caption channel.** Android sheds updates to an existing notification beyond 5 per second per\n  app, and no brand documents how it forwards updates; companion apps show one or two lines and may delay. Post a new\n  notification per final, at most one every 2-3 s.\n- **Smart formatting changes what you show.** Deepgram `smart_format` turned \"four o'clock\" into \"04:00\" in our test.\n  Decide on number and time formatting for the display, not just for the transcript.\n- **Missing glyphs vanish silently** on Even G2: half a sentence disappears with no error. Strip and warn.\n\n## Privacy and consent\n\nCaptions transcribe people who did not install your app. Tell them (a visible indicator, a spoken notice for long\nsessions), check recording-consent law where the app is used, do not store audio, and keep transcripts on the device\nor delete them after the session. Platform rules apply too: Meta's developer policies prohibit surreptitiously\ncollecting recognisable audio of the user or others and identifying people by their voice; Android XR guidance limits\ncontinuous transcription when the wearer is not engaged. Prefer on-device STT when the content is sensitive.\n\n## Related skills\n\n- `smart-glasses`: choosing glasses, the display-text ladder, input, publishing, and the install commands of the\n  official vendor skills (Even Hub, Meta DAT, Meta Web Apps, Android XR Glimmer).\n- `smart-glasses-rokid` and `smart-glasses-vuzix`: the native display calls used by the Android and iOS targets.\n",{"name":5,"license":15,"metadata":49,"description":7},{"version":50,"last-updated":51},"1.0.1","2026-11-09","main","f620abaff09061f8c8b1f1559dca3fdd031a32e5","plugins\u002Fhinty-eu\u002Fsmart-glasses-skills\u002Fskills\u002Fsmart-glasses-live-captions",483,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins","https:\u002F\u002Fhol.org\u002Fplugins\u002Fbest-codex-plugins","https:\u002F\u002Fgithub.com\u002Fhashgraph-online","https:\u002F\u002Fhol.org","Open-source standards and developer tools for interoperable AI agents: discovery, identity, communication, payments, privacy and security.","2026-11-13 02:16:32",{"source":8,"skillName":5,"commands":63,"prompt":73,"zipAvailable":74},[64,67,70],{"tool":65,"command":66},"npx","npx skills add hashgraph-online\u002Fawesome-codex-plugins --skill smart-glasses-live-captions",{"tool":68,"command":69},"bunx","bunx skills add hashgraph-online\u002Fawesome-codex-plugins --skill smart-glasses-live-captions",{"tool":71,"command":72},"pnpm","pnpm dlx skills add hashgraph-online\u002Fawesome-codex-plugins --skill smart-glasses-live-captions","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fhinty-eu\u002Fsmart-glasses-skills\u002Fskills\u002Fsmart-glasses-live-captions\u002FSKILL.md",true,[76,82,86,90,94,98],{"path":77,"resourceType":78,"resourceTypeDesc":79,"sizeBytes":80,"sourceHtmlUrl":81},"references\u002Flatency.md","REFERENCE","参考文档",5387,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fhinty-eu\u002Fsmart-glasses-skills\u002Fskills\u002Fsmart-glasses-live-captions\u002Freferences\u002Flatency.md",{"path":83,"resourceType":78,"resourceTypeDesc":79,"sizeBytes":84,"sourceHtmlUrl":85},"references\u002Fline-fitting.md",12623,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fhinty-eu\u002Fsmart-glasses-skills\u002Fskills\u002Fsmart-glasses-live-captions\u002Freferences\u002Fline-fitting.md",{"path":87,"resourceType":78,"resourceTypeDesc":79,"sizeBytes":88,"sourceHtmlUrl":89},"references\u002Fpipeline.md",8850,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fhinty-eu\u002Fsmart-glasses-skills\u002Fskills\u002Fsmart-glasses-live-captions\u002Freferences\u002Fpipeline.md",{"path":91,"resourceType":78,"resourceTypeDesc":79,"sizeBytes":92,"sourceHtmlUrl":93},"references\u002Fscripts-and-fonts.md",8277,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fhinty-eu\u002Fsmart-glasses-skills\u002Fskills\u002Fsmart-glasses-live-captions\u002Freferences\u002Fscripts-and-fonts.md",{"path":95,"resourceType":78,"resourceTypeDesc":79,"sizeBytes":96,"sourceHtmlUrl":97},"references\u002Fshort-hints.md",5493,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fhinty-eu\u002Fsmart-glasses-skills\u002Fskills\u002Fsmart-glasses-live-captions\u002Freferences\u002Fshort-hints.md",{"path":99,"resourceType":78,"resourceTypeDesc":79,"sizeBytes":100,"sourceHtmlUrl":101},"references\u002Fstt-options.md",10972,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fhinty-eu\u002Fsmart-glasses-skills\u002Fskills\u002Fsmart-glasses-live-captions\u002Freferences\u002Fstt-options.md",[103,111,118,125,152,177],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":110},"01M746ZWACVKMZ1BJXJZHYJSSX","migrate-to-e2e","Moves an existing Playwright, Cypress, Selenium, Appium, Detox, Maestro, or TesterArmy suite to e2e one slice at a time, with the old suite running beside the new one until each ported test is proven. Keeps a MIGRATION.md progress file so any later session resumes where the last stopped. Optionally plants a defect to show both suites catch it. Use for \u002Fmigrate-to-e2e, \"migrate our tests to e2e\", \"port these Maestro flows\", \"move our TesterArmy tests\", or when a project has an e2e config and an older suite still running.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fmigrate-to-e2e\u002FSKILL.md",[],[],"2026-11-11 15:11:14",{"skillNo":112,"name":113,"nameZh":6,"description":114,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":115,"categories":116,"keywords":117,"publishedAt":110},"01M746ZPTAX1Q0FBKAB1CYND7E","e2e","Writes, runs, and debugs end-to-end tests for web and mobile apps with e2e, the agentic test runner, which mixes agent.act goals with exact locator assertions on a Playwright browser or an iOS or Android device. Also drives a project's running app live over the e2e MCP server to look at a screen, check a UI change, take a screenshot or record a video, and try a locator; and runs bug bashes, parallel e2e explore runs with each bug proven by a repro test. Use when a project depends on e2e or has an e2e.config.ts, when asked for end-to-end, browser, mobile, or agentic UI tests, to check a change in the running app, to QA or bug bash an app or branch, or when an e2e run fails.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fe2e\u002FSKILL.md",[],[],{"skillNo":119,"name":120,"nameZh":6,"description":121,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":122,"categories":123,"keywords":124,"publishedAt":110},"01M746ZJN784HVHMDP9MQGCSGG","create-verification-skill","Generate a project-local verify-\u003Capp> skill on top of e2e, so any coding agent can launch the app, check it is worth driving, exercise a feature the way a user does, keep evidence, and bug bash it. Writes the skill, a feature map with one explore charter per feature, and proves it once. Use for \u002Fcreate-verification-skill, \"make a verification skill for this repo\", or when a project has no scripted way to prove UI behavior.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fcreate-verification-skill\u002FSKILL.md",[],[],{"skillNo":126,"name":127,"nameZh":6,"description":128,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":129,"categories":130,"keywords":131,"publishedAt":44},"01M716A6ETNB9B1SQ3GXZC75VR","deja-search","deja-vu memory — search the user's past AI coding sessions with the deja CLI. Holds what they already settled and never wrote into the repo (naming rules, banned or already-tried tools, required checks before merge, preferred values, who owns what). Use before your first edit in a task, before you call a change done or ready to merge, before you recommend a tool, library, design or reviewer, before re-debugging an error that may already be solved, when they say 'didn't we fix this before', 'what did we decide about X' or 'I already have that', and before telling them something on this machine does not exist.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fvshulcz\u002Fdeja-vu\u002Fskills\u002Fdeja-search\u002FSKILL.md",[],[132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151],"search","deja search","never","error","ready","memory","re","decide","fix","design","debugging","when","you","done","reviewer","io","change","cli","merge","past",{"skillNo":153,"name":154,"nameZh":6,"description":155,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":156,"categories":157,"keywords":158,"publishedAt":44},"01M716A3J2W5SENSE9VGMXB375","deja-reflect","End-of-session reflection and memory curation with deja-vu. Review what was decided, what commands broke and what repaired them, and record durable project rules or decisions with deja remember so the next session starts with them.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fvshulcz\u002Fdeja-vu\u002Fskills\u002Fdeja-reflect\u002FSKILL.md",[],[159,160,161,147,162,163,164,165,166,167,168,169,170,171,137,172,173,174,175,176],"deja reflect","reflect","review","with","record","github","next","https","node","Rules","decisions","brew","remember","project rules","project","session","end","reflection",{"skillNo":178,"name":179,"nameZh":6,"description":180,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":181,"categories":182,"keywords":190,"publishedAt":44},"01M715T3RR4BAYSR57HPPTJ0PF","revid","Make, edit, export and post AI videos with Revid (revid.ai) through the Revid MCP server. Use whenever the user wants a short-form or long-form video made (from an idea, a script, a URL, an article, a PDF, a tweet, a product page, their own clips, or a song), wants to find what is going viral on TikTok, YouTube, Instagram or X before making something, wants to edit or re-export an existing Revid project, generate a standalone AI image or clip, make a series episode, or publish and schedule a finished video to connected social accounts. Covers setup (hosted MCP with OAuth, or an API key), the 36 tools, credits, polling and the rules for publishing. Routes to the more specific revid-* skills when one fits.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftmakersas\u002Frevid-skills\u002Fskills\u002Frevid\u002FSKILL.md",[183,184,187],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":185,"categoryName":186,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":188,"categoryName":189,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口",[179,191,42,192,193,194,195,173,196,138,197,198,199,200,143,201,202,203,204,205],"URL","short-form","standalone","PDF","viral","tiktok","short","social","MCP","song","image","mcp server","schedule","api","publish"]