source-connectors

Airweave Source Connectors

Use this sub-skill when a future Airweave task touches connector-facing source classes, source metadata exposed by the registry, auth/config schema behavior, connector validation, browse-tree node selection, access-control extraction, federated search sources, cursor-based incremental sync, or source-specific rate limiting.

Do not use this sub-skill for dashboard UI, Connect widget iframe/OAuth UX, MCP transport, Monke orchestration, or generic backend endpoint lifecycle work. Cross-link to sibling backend-api for source-connection endpoint flows, OAuth verification, search request/response schemas, browse-tree route shapes, and API auth/org semantics.

Route to the right reference

• Read references/source-registry.md before adding a source class, changing @source(...) metadata, changing registry output, hiding/showing feature-flagged sources, or checking entity-definition exposure.
• Read references/auth-and-config.md before changing auth methods, direct credential schemas, OAuth/BYOC/token-injection handling, auth-provider support, template config fields, source connection validation, or runtime token-provider creation.
• Read references/browse-tree.md before changing supports_browse_tree, lazy browse children, browse-node ID encoding, node-selection persistence, or targeted sync behavior.
• Read references/troubleshooting.md when diagnosing source visibility, auth/config errors, browse-tree selection bugs, ACL over/under-grants, federated search failures, cursor regressions, source HTTP errors, concurrency issues, or rate-limit behavior.

Connector operating rules

1. Keep connector identity in one place: the @source(...) decorator sets short_name, display name, auth methods, config classes, capability flags, labels, feature flag, cursor class, federated-search status, browse-tree support, access-control support, and source rate-limit level.
2. Preserve the BaseSource v2 contract: create(*, auth, logger, http_client, config), generate_entities(*, cursor=None, files=None, node_selections=None), and validate() are the primary connector hooks. Do not add ad-hoc credential or logger setup outside these dependencies.
3. Use the injected auth provider and AirweaveHttpClient. Sources should call auth.get_token(), auth.force_refresh() when supported, or direct credential models from DirectCredentialProvider; external HTTP calls should flow through the injected client so SSRF checks and optional source rate limits apply.
4. Put public schema fields in typed Pydantic config/auth models. Use field metadata (is_secret, feature_flag, auth_provider_field, required_for_auth, exclude_from_ui) deliberately because the registry turns those classes into API-visible Fields.
5. For continuous sync, set both supports_continuous=True and cursor_class=...; the decorator rejects continuous sources without a cursor class. Update the cursor only with stable, source-owned state and support full-sync fallback when cursor state is absent or invalid.
6. For federated sources, set federated_search=True, implement search(query, limit), and do not pretend full sync exists. Slack intentionally raises from generate_entities() because it searches Slack at query time.
7. For ACL-aware sources, set supports_access_control=True, set entity.access on yielded entities, and implement generate_access_control_memberships() when group expansion is needed. Principal formats must stay compatible with the search broker.
8. For browse-tree sources, set supports_browse_tree=True, implement get_browse_children(parent_node_id) and parse_browse_node_id(node_id), and keep emitted node IDs parsable into the metadata needed by targeted sync.
9. Keep Monke and manual credentialed workflows external. No bundled script is included for this sub-skill: the only relevant source-maintained manual artifact is the SharePoint browse-tree manual test, and it requires real Microsoft tenant credentials and external state.

Safe validation anchors

When a backend Python 3.13 environment with Airweave and pytest is active, prefer connector-owned unit anchors first:

cd backend
python -m pytest tests/unit/platform/sources/test_http_helpers.py -q
python -m pytest tests/unit/platform/sources/test_process_entities_concurrent.py -q
python -m pytest tests/unit/platform/sources/test_sharepoint_online_acl.py -q
python -m pytest tests/unit/platform/sources/test_sharepoint_online_group_expansion.py -q
python -m pytest tests/unit/platform/sources/test_sharepoint2019v2.py -q
python -m pytest tests/unit/platform/sources/test_sharepoint2019v2_dirsync.py -q
python -m pytest tests/unit/api/test_source_feature_flags.py -q

Credentialed or local-stack E2E anchors are useful but not safe to run unattended: source listing/detail, federated search, source rate limits, continuous sync, entity definitions, OAuth/direct/auth-provider/token source-connection variants, and SharePoint legacy DirSync tests are cataloged in references/troubleshooting.md.

The prepared backend inspection facts for this skill were CPU-only: airweave==0.1.0 imported on Python 3.13.14, connector-relevant imports such as airweave.schemas.source_connection, airweave.platform.sources._base, and airweave.platform.decorators succeeded, no accelerator backend is required for this scope, and settings imports require documented environment variables with sufficiently long STATE_SECRET and SVIX_JWT_SECRET values.