Merge requests · repos / data-engineering / Airflow DAGs

Run Dumps 2.0 main DAG at a daily cadence rather than hourly.

!888 · created Oct 23, 2024 by Xcollazo
- Merged
- 4
- Approved
updated Oct 24, 2024
Add more parallelism to refine_to_hive_hourly_dag

!874 · created Oct 17, 2024 by Aqu
- Merged
- 1
- Approved
updated Oct 22, 2024
Set wikitext_raw's rewrite_data_files() max file group size.

!884 · created Oct 21, 2024 by Xcollazo
- Merged
- Approved
updated Oct 22, 2024
T376721 Standardization of airflow dag props 5 of 5 checklist items completed

!886 · created Oct 22, 2024 by Andrew McAllister (WMDE)
- Merged
updated Oct 22, 2024
Hotfix: Pickup fix for tuning ingestion.

!885 · created Oct 22, 2024 by Xcollazo
updated Oct 22, 2024
Pick up ingestion tuning changes for Dumps 2.0

!876 · created Oct 17, 2024 by Xcollazo
- Merged
- 1
updated Oct 21, 2024
test_k8s: tweak Datahub_Cleanup to make sure DAGs are pull automatically

!883 · created Oct 21, 2024 by Brouberol
updated Oct 21, 2024
T370851-T376721 Eexternal task sensor and update paths 5 of 5 checklist items completed

!881 · created Oct 18, 2024 by Andrew McAllister (WMDE)
- Merged
- 1
updated Oct 18, 2024
DAG Runs API call bugfix

!882 · created Oct 18, 2024 by Aleksandar Mastilovic
- Merged
- 1
- Approved
updated Oct 18, 2024
Fix refinery-spark artifact config

!880 · created Oct 18, 2024 by Milimetric
- Merged
updated Oct 18, 2024
Clean up obsolete DAGs from Datahub

!872 · created Oct 16, 2024 by Stevemunene
- Merged
- 14
- Approved
updated Oct 18, 2024
Update mwaddlink tags collection to list type

!869 · created Oct 11, 2024 by KCVelaga
- Merged
- 5
- Approved
updated Oct 18, 2024
Update conda package and fixtures

!877 · created Oct 17, 2024 by Milimetric
- Merged
- 1
updated Oct 17, 2024
test-k8s: move start_date and catchup kwargs to the DAG init arguments

!873 · created Oct 16, 2024 by Brouberol
- Merged
- Approved
updated Oct 16, 2024
test_k8s: run an hourly DAG testing the S3 connectivity

!871 · created Oct 14, 2024 by Brouberol
- Merged
- 5
- Approved
updated Oct 16, 2024
Optimization for Refine on staging

!867 · created Oct 10, 2024 by Aqu
- Merged
- 15
- Approved
updated Oct 14, 2024
Automoderator monthly snapshot DAG

!859 · created Oct 04, 2024 by KCVelaga
- Merged
- 1
- Approved
updated Oct 14, 2024
Bump section-topics to 0.16.0-v0.16.0.

!870 · created Oct 11, 2024 by auto-push-from-structured-data
- Merged
- Approved
updated Oct 11, 2024
Automoderator potential vandalism reverted DAG

!866 · created Oct 10, 2024 by KCVelaga
- Merged
- 3
- Approved
updated Oct 11, 2024
Mitigate refine Webrequest concurrency problem

!868 · created Oct 10, 2024 by Aqu
- Merged
- Approved
updated Oct 10, 2024

Admin message

Admin message

Admin message