data-api-architect
Architect-level design of data-serving APIs — query patterns and pre-aggregation, caching and freshness, pagination and export endpoints, row-level authorisation, and the cost controls that stop a dashboard API from scanning a warehouse on every request. Use when applications or partners consume analytical data.
- License 许可
- MIT
- 最近更新
- 2026-11-13
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- data-api-architect
- license
- MIT
- metadata
- { "crusor-path": "data-engineering-architect/delivery-architect/data-api-architect", "crusor-domain": "data-engineering-architect", "crusor-origin": "original", "crusor-sources": "cube.dev/docs; hasura.io/docs; cloud.google.com/bigquery/docs/best-practices-costs", "crusor-triggers": "data api, analytics api, export endpoint, query caching, pagination, row level security, partner data feed, serving analytics, expose analytics, embedded dashboard, partner feed", "crusor-verified-on": "2026-10-30" }
- description
- Architect-level design of data-serving APIs — query patterns and pre-aggregation, caching and freshness, pagination and export endpoints, row-level authorisation, and the cost controls that stop a dashboard API from scanning a warehouse on every request. Use when applications or partners consume analytical data.
- compatibility
- Any warehouse; GraphQL/REST; Cube, Hasura, custom services
03 SKILL.md 正文
Data API Architect
A data API is a warehouse query behind an HTTP contract. Without pre-aggregation and caching, one dashboard refresh becomes a scan you pay for.
When to use
- Applications, partners, or customers consume analytical data.
- A dashboard product needs an API for embedding.
- Reports must be exported programmatically.
When NOT to use
- Internal analysts can query the warehouse directly.
- The data is transactional and belongs in a service API.
Decision 1 — Query shape
| Access pattern | Design |
|---|---|
| fixed dashboards | pre-aggregated tables or materialised views |
| parameterised slices | curated dimensions, mandatory filters |
| ad-hoc SQL from users | do not expose; offer a fixed query surface |
| large exports | asynchronous job + signed URL |
| single record lookup | serving store, not the warehouse |
- The API should expose metrics and dimensions, not "run this SQL".
- Enforce required filters (date range, tenant) at the API boundary, not in docs.
Decision 2 — Caching and freshness
- Cache per (metric, dimensions, filters) with a TTL matched to the data cadence.
- Return
generated_atand data freshness in every response — consumers must know how stale it is. - Invalidate on model rebuild rather than waiting for TTL where possible.
- Warm the cache for top dashboards after a refresh; cold cache at 09:00 is a latency cliff.
Decision 3 — Cost controls
- Per-tenant query budgets and rate limits.
- Reject unbounded date ranges; cap result rows.
- Prefer pre-aggregated marts over live joins on raw events.
- Monitor bytes scanned per endpoint; a single endpoint dominating cost is a design smell.
Decision 4 — Authorisation
- Row-level filters derived from the caller's identity, applied server-side in the generated query — never trusted from parameters.
- Column visibility per role: PII hidden unless explicitly granted.
- Tenant isolation tested as a first-class case.
- Audit log per request: who, what slice, how many rows.
Decision 5 — Pagination and exports
- Keyset pagination for large result sets; offset pagination degrades at scale.
- Exports as async jobs with a status endpoint and a signed, expiring URL.
- Cap export size and rate; an unbounded export is a scan plus an egress bill.
- Provide stable column names and a versioned response contract.
Failure modes at scale
- API hits the warehouse on every page load; latency 4 s and cost unbounded.
- Row-level filter taken from a request parameter; one tenant reads another's data.
- Offset pagination on 5M rows; late pages time out.
- Exports run synchronously; the gateway times out and users retry, doubling load.
- No freshness metadata; consumers assume real-time.
Verification receipts
- Endpoint inventory with the query shape and required filters.
- Cache design with TTLs and hit-rate measurement.
- Row-level authorisation test across two tenants.
- Export job exercised end to end with an expiring URL.
- Cost per 1,000 requests and the bytes-scanned report per endpoint.
04 同仓库更多 Skills
interaction-patterns-architect
Architect-level interaction design in code — choosing established patterns over invented ones, feedback and latency behaviour, destructive-action safeguards, navigation and overlay conventions, and the state model behind every interactive surface. Use when a screen has to behave, not just look.
design-system-ops-architect
Architect-level operation of a design system in code — token pipelines and versioning, component API and variant policy, the catalogue as the contract, adoption metrics, and the governance that stops a system from forking into three. Use when a component library must survive several teams.
design-handoff-architect
Architect-level design handoff — what must be specified before implementation starts, tokens over raw values, responsive and content-limit contracts, state matrices, and the review loop that catches drift between design and code. Use when a design is about to become code.
content-microcopy-architect
Architect-level UI content — where strings live, voice and terminology consistency, error and empty-state copy as part of the interface, translation-ready authoring, and the content model a product needs before it multiplies screens. Use when the words in the interface matter as much as the pixels.
ui-ux-implementation-architect
Architect-level direction for turning a design into working UI — the handoff contract, component ownership versus screen-level layout, interaction and state conventions, design QA gates, and the review habits that keep an implementation faithful without frozen pixels. Use before any specific UI implementation skill.
test-strategy-architect
Architect-level test strategy — allocating assertions across unit, integration, contract and end-to-end layers, designing for testability through dependency boundaries, fixture and data strategy, and the flake and coverage policies that keep a suite trustworthy. Use when defining how a codebase is tested.