Define a maintenance DAG to prune old data from the airflow DB
This DAG needs the following airflow variables to be set to run:
-
db_cleanup_tables
: what tables to clean up -
db_cleanup_retention_days
: the amount of data to keep, in days
We tag this DAG with airflow_maintenance
to keep consistency with
purge_old_logs_from_s3
, and we also set a markdown documentation at
the task level.
Signed-off-by: Balthazar Rouberol brouberol@wikimedia.org Bug: T375878