adding-a-provider-api-feature

Adding a provider API feature

Use this when exposing a new provider API capability through Pydantic AI — prompt caching, strict/structured tool calling, thinking/reasoning effort, service tier, safety settings, logprobs, cache breakpoints, and the like. The output is a change that is consistent with how sibling providers already expose the same concept, defaults deliberately, and gates support with a capability flag.

Not for: adding a new model id (that's add-new-model), a bug fix, or a refactor.

The one rule that prevents the most rework

Before designing anything, find the existing cross-provider abstraction that governs this capability and let its shape decide the API. Most "how should I expose this?" questions are already answered by an abstraction the codebase has — reaching for a new provider-specific knob when one exists is the single most common thing maintainers reject. When a feature routes through an existing abstraction, the abstraction's shape pre-decides the API surface, the opt-out, and often the default.

The tell that you skipped this: you find yourself listing 2-3 "options" for how a user controls the feature. If one of those options duplicates an existing cross-provider control, it isn't a real option — the existing abstraction wins.

Step 0 — Enumerate sibling precedent

For the capability you're adding, list how every provider that already has an analog exposes it, and name the governing existing abstraction. It is one of:

• a per-tool flag — ToolDefinition.strict: bool | None (tools.py), resolved in models/__init__.py::_customize_tool_def;
• a shared ModelSettings field — thinking, service_tier (settings.py), each with per-model resolvers mapping to native concepts;
• a provider-prefixed {Provider}ModelSettings field — anthropic_cache, openai_prompt_cache_key, groq_reasoning_effort;
• a message-stream marker — CachePoint in UserPromptPart.content (messages.py);
• a ModelProfile capability flag — openai_supports_strict_tool_definition, bedrock_supports_prompt_caching.

Read the actual sibling implementations and any review threads on the PRs that added them (gh pr view <n> --comments). Contributors who skipped this got redirected: raw google_tool_config → use strict (#5366); string-prefix model detection → use a profile flag (#4604). Only open a design fork if no existing abstraction covers the capability.

Step 1 — Pick the API shape

1. An existing cross-provider abstraction covers it → reuse it. Add a provider mapping (a _translate_* resolver, a JsonSchemaTransformer subclass, a CachePoint translation). Do not add a provider-prefixed knob for something the shared abstraction already expresses.
2. No shared abstraction, but ≥3 providers now have the concept → promote to a shared ModelSettings field with a deliberately narrow common vocabulary and per-model resolvers, keeping per-provider fields underneath as precedence-winning escape hatches (the service_tier promotion, #4926). Don't delete provider knobs; deprecate only genuinely-misnamed ones with a TODO(v3).
3. The provider's native values can't be expressed by the shared enum (family-disjoint value sets, platform-only request shaping) → a {provider}_* knob is justified, and it coexists with and outranks the unified field (groq_reasoning_effort, #5797).
4. Choose the locus by "can the user naturally point at it?": a boundary inside the message stream → a marker (CachePoint); a structural region (system prompt, tool defs, whole-request setting) → a setting (#3363).
5. Wire it into every request path the provider has (chat and responses APIs); put a setting shared across a provider's APIs on the base settings class (#3678).
6. Type it. Reuse the provider SDK's own types where they exist; type knobs as Literal, never extra_body or untyped **kwargs (models/AGENTS.md; "kwargs are a big no no… I'd rather be repetitive but type safe" — DouweM, #3457).

Step 2 — Default: default-on (silent) vs opt-in

Default the feature on only when enabling it cannot change observable behavior and cannot cost the user — a pure, backward-compatible improvement (caching only lowers cost; a validation mode that needs no schema rewrite and can't reject a previously-valid request). A backward-compatible, purely-better default is welcome and needs no opt-in.

Keep it opt-in when it:

• changes observable output or wire behavior;
• is a preview feature (provider may change semantics);
• can raise cost — and choose the default value so a shared field never silently upgrades anyone to a pricier tier (service_tier 'auto' vs 'default', #4926);
• can hit provider limits at scale (auto-promoting strict silently broke agents with >20 tools; reverted in #5580);
• applies lossy schema rewrites (OpenAI/Anthropic strict transforms).

When the default isn't obvious, decide it with a live probe, not an opinion (#5897 flipped a default on 0/5 → 5/5 recovery). Beware "automatic" language — verify whether it means a Pydantic AI default or just provider-side management of an opt-in feature.

Step 3 — Capability gating

Detect support with a provider-prefixed ModelProfile flag. Where the feature is used, read only the flag or the adapter's own declaration — never an inline isinstance, model-name or base_url check (profiles/AGENTS.md). Gate at the layer(s) that actually vary:

• model/family → a flag in the family's profile function (google_supports_strict_tool_definition in profiles/google.py);
• one provider's API → a flag set in that provider's Provider.model_profile() (bedrock_supports_prompt_caching in providers/bedrock.py);
• transport (the Bedrock, Vertex and Foundry client classes of one SDK, or a client's own mode such as genai.Client.vertexai) → a flag set in the model's _instance_profile_defaults(), or in its profile property when the client can't send the feature at all. Provider.model_profile() is static and sees only the model name (models/AGENTS.md, the client-class bullet);
• per-schema → the JsonSchemaTransformer.is_strict_compatible signal (default conservative unless the mode needs no rewrites);
• SDK version → probe the SDK shape and degrade with a UserWarning (#5580, botocore strict param);
• unknowable client-side → defer to the runtime API error.

Unsupported → silently ignore the setting (best-effort so as many requests as possible succeed), documented in the docstring — never hard-error. Conflicting user settings → UserWarning, not UserError. Profiles are layered: developer-keyed base + thin provider overlay resolved per family, with the provider overlay layering last (#5934, #6231).

Step 3b — Where the wire mapping goes

Read agent_docs/provider-adapters.md before you edit an adapter's mapper. Place the feature by what it adds:

• a new part type → one arm in each message kind's dispatch, calling the part's own method on the model class. When most APIs can't render the part, lower it in Model.prepare_messages instead; an adapter that doesn't render it raises _unprepared_part_error(part) in its one arm.
• a new native tool kind → one entry in the adapter's kind-to-wire-name table. Add a branch only where the wire shape differs.
• a new response wire item → one function, called from both _process_response and the stream.
• a new flag or setting that changes how an existing part maps → read it inside that part's method. Where the adapter still maps that part inline in its loop, read it where the part is mapped today, and leave extracting the method to its own structural PR.

Step 4 — Tests

Pyright does not flag a Literal-keyed dict that misses a key; the lookup raises KeyError only at request time. Add every new mapping keyed by ThinkingEffort to test_every_effort_level_resolves in tests/test_thinking.py, which runs each level through each mapping. When you add a value to another shared Literal, write the same kind of test for it.

Live-recorded wire-contract cassette asserting the exact outbound body (mode/field is sent for supported models, absent/ignored for unsupported), plus a test exercising the new setting. Prefer case-based parametrized VCR over mocks; a unit test is still right for asserting an internal request shape a cassette matcher wouldn't catch. This is also where a review bot's "this will fail" claim gets refuted with a recorded cassette.

Step 5 — Docs & skills

The new public symbol's docstring lists which providers support it and how each interprets the value. Add/refresh the docs/**.md section and any sibling docstrings that now under-claim ("only OpenAI" → the full provider list). Describe the mechanism only as far as the provider documents it — don't assert a mechanism a provider's docs leave unstated. Update the relevant agent skill.

ModelSettings' Supported by: lists are enforced: tests/models/test_model_settings_support.py probes each model class's outgoing request and asserts every list names exactly the classes that send the field. Forwarding a new setting means editing its list, and a new Model class means adding a Case there. tool_choice and thinking are exempt via HAND_MAINTAINED and stay hand-maintained.

Recurring maintainer principles (quoted)

1. Reuse the cross-provider abstraction over a provider knob — "this maps to what the Anthropic and OpenAI APIs call strict… already represented on ToolDefinition… a more complete, consistent, 'doesn't require the user to do something special' implementation would be to automatically use this mode." (#5366)
2. Promote to a shared setting once several providers have the concept; keep per-provider overrides underneath. (#4926)
3. Best-effort — silently ignore unsupported settings, don't error. "we typically do a 'best effort' so that as many requests as possible succeed." (#3438)
4. Hide provider complexity — the feature should be useful to people who don't want to become experts in that provider's limitations.
5. Capability facts belong on ModelProfile flags, layered base + overlay. (#5934)
6. Type safety over repetition; no untyped kwargs. (#3457)
7. Put shared fields on the base settings class; cover every API surface. (#3678)
8. Verify against the real API; defer validation to runtime. "If they're allowed by the SDK types and we can try it out and it doesn't fail, I'm fine with it." (#3678)
9. Name the eventual unification even when deferring it — a {provider}_* knob today can note the future Caching/Thinking-style capability it should fold into. (#4604)

Precedent map

Capability	Reused abstraction	Default	Gating	PR
service tier (cross-provider)	promoted to ModelSettings.service_tier	opt-in, never silent upgrade	map-and-drop	#4926
strict — OpenAI (origin)	ToolDefinition.strict	auto-promote per compatible schema	per-schema is_strict_compatible	#1304
strict — Anthropic	reused strict	conservative opt-in	transformer + profile flag	#3457
strict — Bedrock (+ fix)	reused strict	opt-in (auto-promote reverted)	transformer + profile + SDK probe	#4237, #5580
strict — Gemini VALIDATED	reused strict; rejected raw google_tool_config	default-on (mode needs no schema rewrite)	profile flag; is_strict_compatible = True	#6353
thinking (cross-provider)	ModelSettings.thinking + per-provider maps	opt-in, graceful degradation	supports_thinking flags	#4640
reasoning effort — Groq	provider knob coexists with thinking, outranks it	opt-in	per-family profile flag	#5797, #6231
prompt caching — Anthropic → Bedrock/OpenRouter	CachePoint marker + settings; breakpoint limits through models/_prompt_cache.py	opt-in	{provider}_supports_prompt_caching	#3363, #3438, #4604