Merge requests · repos / data-engineering / dumps / mediawiki-content-dump

Clarify column descriptions of *_is_visible columns

!28 · created Jun 20, 2024 by Ottomata
- Merged
- 3
- Approved
updated Jul 24, 2024
Pickup recent fixes to workflow_utils to fix the build.

!27 · created Jun 12, 2024 by Xcollazo
updated Jun 14, 2024
Dummy MR

!26 · created Jun 12, 2024 by Xcollazo
- Closed
updated Jun 14, 2024
Use WRITE ORDERED BY in backfill_create_intermediate_table.py.

!25 · created Feb 09, 2024 by Xcollazo
- Merged
- 1
updated Feb 09, 2024
Implementation of data drift checks.

!24 · created Feb 07, 2024 by Xcollazo
- Merged
- 24
- Approved
updated Feb 21, 2024
Ignoring failing test.

!23 · created Feb 07, 2024 by Xcollazo
updated Feb 07, 2024
Notebook explaining an approach to access MariDB replicas from the Hadoop Cluster.

!22 · created Jan 05, 2024 by Xcollazo
updated Jan 05, 2024
Switch to parquet. Use WRITE ORDERED BY.

!21 · created Dec 14, 2023 by Xcollazo
- Merged
- 1
updated Dec 14, 2023
Update file .gitlab-ci.yml

!20 · created Nov 07, 2023 by Xcollazo
updated Nov 07, 2023
Modify visibility SQL to account for canary events.

!19 · created Oct 26, 2023 by Xcollazo
- Merged
- 2
updated Sep 19, 2024
Modify schema to wikitext_raw_rc2.

!18 · created Oct 18, 2023 by Xcollazo
- Merged
- 8
- Approved
updated Oct 23, 2023
T347524 add integration tests to the pyspark jobs 2 of 2 checklist items completed

!17 · created Oct 16, 2023 by Jennifer Ebe
- Merged
- 35
- Approved
updated Nov 12, 2024
Add LOCATION to intermediate table creation.

!16 · created Sep 28, 2023 by Xcollazo
updated Sep 29, 2023
Return valid predicate when duplicate list is empty.

!15 · created Sep 27, 2023 by Xcollazo
- Merged
- 2
updated Sep 27, 2023
Deduplicate intermediate table (wiki_db, revision_id) tuples on read.

!14 · created Sep 26, 2023 by Xcollazo
- Merged
- Approved
updated Sep 27, 2023
Use an intermediate table when backfilling.

!13 · created Sep 19, 2023 by Xcollazo
- Merged
- 12
updated Sep 21, 2023
Add visibility on backfills via broadcast join of wmf_raw.mediawiki_revision.

!12 · created Sep 07, 2023 by Xcollazo
- Merged
- 6
updated Sep 13, 2023
Change Revision Visibility code to conform to the new schema

!11 · created Sep 05, 2023 by Jennifer Ebe
- Merged
- 12
- Approved
updated Sep 12, 2023
Change backfill strategy from months to years.

!10 · created Aug 29, 2023 by Xcollazo
- Merged
- 1
- Approved
updated Sep 08, 2023
Use memory-optimized runners on .gitlab-ci.yml

!9 · created Aug 29, 2023 by Xcollazo
- Merged
- 1
updated Sep 05, 2023

Admin message

Admin message

Admin message