build-with-ipa

Build with IPA

IPA is an experimental browser API. The specification defines its contract; the ipa-tools README documents current helpers. Start with the chat demo for streaming, the translate demo for a final-only result, or the charts demo for page-executed tools. If this repository is available locally, read those files there instead. Check the relevant spec sections before implementing a feature; do not treat an example or a particular injector's extra methods as part of the standard.

Integrate into the app

1. Fit IPA into the app's existing architecture and requested UX. IPA runs in a top-level browser page on HTTPS or loopback HTTP. It may be absent until a compatible injector is installed. Check window.inference?.request before enabling inference; explain how to install a compatible provider when it is missing. If late injection matters, ipa-tools offers waitForInference, but do not await it before first paint.
2. Use window.inference.request for streaming. For method: "chat", iterate its AsyncIterable: accepted means the request is cleared to reach a provider; delta carries display text; reasoning_delta is optional; done contains the final message, model, and optional usage. A nonstreaming chat provider may yield only accepted and done. Iteration throws on failure, so handle error.code (permission_denied, invalid_request, unavailable, provider_error, aborted) rather than relying on instanceof across extension worlds. Pass an AbortSignal when the UI offers cancellation.
3. For apps needing only the final result, use complete or createInference().complete from ipa-tools. For page-executed function tools, prefer its runTools helper or implement the spec's multi-round loop. ipa-tools supplies types and helpers; it does not inject window.inference, select a provider or model, or expose API keys. Keep provider credentials out of page code.
4. Gate optional request features using window.inference.getFeatures?.() ?? {}. toolCalling, webSearch, imageInput, and imageOutput are independent. Do not send an unadvertised function tool, hosted search tool, image part, or output.images: true. Feature flags describe the injector's API surface, not a promise that the selected provider or model can fulfill a request. web_search is hosted, so do not put it in page-side tool handlers. Request options are preferences, not model selection; check features.options when deciding whether an option can have an effect.
5. Show a waiting state while permission resolves; the injector owns the permission prompt and the user chooses the provider and model there. An accepted chunk means permission has been resolved, including for an existing grant; it does not prove a prompt appeared. A tool result can be sent in a later round without another permission preview, so disclose what page tools may send before the first request. Treat image input and output and hosted search as distinct capabilities when presenting controls.
6. For bounded speech, first check features.methods?.transcribe?.acceptedMedia or features.methods?.synthesize. Use the same request with method: "transcribe" and media (completed Blob, raw base64 with a type, or a page-resolved URL), or method: "synthesize" and text. Transcription ends with done.transcript; synthesis yields encoded audio_delta bytes (MP3 by default; explicit WAV only when methods.synthesize is an object whose outputMediaTypes includes audio/wav) and metadata-only done.audio. Legacy synthesize: true means MP3 only; create playback Blobs using the returned media type. Keep complete and runTools chat-only. Never add method-named API functions, select provider/model/voice IDs from page code, upload live microphone streams, or autoplay output. Stop any page-owned recording before requesting separate IPA transcription approval. Read the speech examples; detect advertised method support before use.

Verify the integration

Exercise the relevant app flow in a secure browser context with a compatible injector such as Inference Bridge. Check the missing-injector state, permission denial, a successful final response (with and without streaming deltas), and cancellation if offered. For each optional feature used, check both advertised and unavailable states. A fallback backend is a separate app choice; do not present its output as IPA.