Skip to content

coalesce outputs with default workable values

Marco Fossati requested to merge T347558 into main

Add an optional --coalesce argument to each CLI, with default values based on trade-offs between less output files and longer execution time.

Note that a drastic coalesce to e.g., 4 leads to crashes of Spark executors, due to too few nodes handling the whole computation.

Bug: T347558

Edited by Marco Fossati

Merge request reports