AGENT SKILL

airflow

Apache Airflow: Directed Acyclic Graphs (DAGs), task dependencies, sensors, custom operators, XComs, and Celery/Kubernetes executors

License 许可
MIT
最近更新
2026-11-09
Stars
★ 0
Forks
0

01 前言 Front Matter

name
airflow
risk
low
source
https://github.com/Mr-Nobody-Anonymous/All-skills
license
MIT
version
1.0.0
category
data-engineering
description
Apache Airflow: Directed Acyclic Graphs (DAGs), task dependencies, sensors, custom operators, XComs, and Celery/Kubernetes executors
imported_at
2026-09-20
source_path
awesome_skills/data-engineering/airflow/SKILL.md
source_repository
Mr-Nobody-Anonymous/All-skills
disable-model-invocation
false

03 SKILL.md 正文

Apache Airflow Orchestration

Scope

Apache Airflow authoring, programmatic workflow scheduling, task dependency definition, SLA monitoring, and scalable distributed executor management.

Core Concepts & Invariants

  • DAG Definition Invariants:
    • DAG files must be lightweight and free of top-level code or database calls (parsed repeatedly by scheduler).
    • Deterministic execution date (data_interval_start, data_interval_end).
  • Task Dependencies: Upstream/downstream bitshift operators: extract >> transform >> load.
  • Executors: CeleryExecutor (Redis/RabbitMQ queue), KubernetesExecutor (dynamic ephemeral pod per task).

Tools & Standards

  • Software: Apache Airflow 2.x, Astronomer.

04 同仓库更多 Skills