Skip to content

fix partitioning of wmf_mediawiki_geoeditors_monthly

Mforns requested to merge fix-geoeditors-monthly-partitioning into main

The druid_load_geoeditors_monthly DAG is waiting for the partition wmf.geoeditors_monthly/year=2023/month=4, but that table has partitions like wmf.geoeditors_monthly/month=2023-04.

This MR makes the wmf_mediawiki_geoeditors_monthly dataset snapshot-based. Even if it's not technically snapshot-bases, it follows the same logic. The only problem is that the "snapshot" value is not stored in the snapshot field, rather in the month field. So, this MR also adds the ability to specify which field holds the snapshot value.

Merge request reports