Build Apache Airflow workflows for data pipelines and scheduled jobs. The skill guides task dependencies, retries, waiting for conditions, testing, and deployment, with patterns for tasks that can be safely rerun.
中文简介正在准备中。当前显示英语简介。
Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.
安装前可以查看交给 AI 代理的指令内容。
Production-ready patterns for Apache Airflow including DAG design, operators, sensors, testing, and deployment strategies.
| Principle | Description |
|---|---|
| Idempotent | Running twice produces same result |
| Atomic | Tasks succeed or fail completely |
| Incremental | Process only new/changed data |
| Observable | Logs, metrics, alerts at every step |
# Linear
task1 >> task2 >> task3
# Fan-out
task1 >> [task2, task3, task4]
# Fan-in
[task1, task2, task3] >> task4
# Complex
task1 >> task2 >> task4
task1 >> task3 >> task4
# dags/example_dag.py
from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.operators.empty import EmptyOperator
default_args = {
'owner': 'data-team',
'depends_on_past': False,
'email_on_failure': True,
'email_on_retry': False,
'retries': 3,
'retry_delay': timedelta(minutes=5),
'retry_exponential_backoff': True,
'max_retry_delay': timedelta(hours=1),
}
with DAG(
dag_id='example_etl',
default_args=default_args,
description='Example ETL pipeline',
schedule='0 6 * * *', # Daily at 6 AM
start_date=datetime(2024, 1, 1),
catchup=False,
tags=['etl', 'example'],
max_active_runs=1,
) as dag:
start = EmptyOperator(task_id='start')
def extract_data(**context):
execution_date = context['ds']
# Extract logic here
return {'records': 1000}
extract = PythonOperator(
task_id='extract',
python_callable=extract_data,
)
end = EmptyOperator(task_id='end')
start >> extract >> end
Detailed pattern documentation lives in references/details.md. Read that file when the navigation tier above is insufficient.
mode='reschedule' - For sensors, free up workersdepends_on_past=True - Creates bottlenecks{{ ds }} macros还没有评价。欢迎分享你的使用感受。
无需登录即可发表。欢迎分享使用感受、提问，或说明在哪些环境下无法运行。
还没有评论。来写第一条吧。
Build and review interfaces against WCAG 2.2 accessibility guidelines. Covers keyboard navigation, screen readers that read interface content aloud, accessible forms, mobile support, and display preferences, helping identify and fix barriers to using a website or app.
中文简介正在准备中。当前显示英语简介。
Review AI-generated code for problems hidden behind successful runs. Check failure handling, resource cleanup, unusual inputs, nonexistent dependencies, and departures from project conventions to identify missing safeguards before merging or accepting the work as complete.
中文简介正在准备中。当前显示英语简介。
Plan and carry out an AngularJS to Angular migration, from choosing a rewrite strategy to converting components, services, and routes. Includes a hybrid approach that runs both frameworks together while individual features are migrated.
中文简介正在准备中。当前显示英语简介。
Identify software protections that hinder inspection, such as debugger detection and obfuscation that makes code harder to understand. Support authorized binary analysis and CTF security challenges with protection findings, bypass strategies, reports, and analysis scripts.
中文简介正在准备中。当前显示英语简介。
Design and review REST and GraphQL APIs with consistent naming, data structures, error responses, pagination for large result sets, and versioning strategies. Helps teams define shared standards and plan changes that existing clients can adopt gradually.
中文简介正在准备中。当前显示英语简介。
Write and maintain Architecture Decision Records (ADRs), documents explaining important technical choices. Capture the context, options, decision, and consequences, then organize records and track their status so teams can revisit the reasoning.
中文简介正在准备中。当前显示英语简介。