Bump up memoryOverhead for edit_hourly_dag SparkSqlOperator
The SparkSqlOperator task process_edit_hourly
from the DAG edit_hourly
has failed recently with:
23/06/03 01:30:52 ERROR YarnScheduler: Lost executor 61 on an-worker1141.eqiad.wmnet: Container killed by YARN for exceeding physical memory limits. 8.9 GB of 8.8 GB physical memory used. Consider boosting spark.executor.memoryOverhead.
This MR bumps up the memoryOverhead for that task.