Resolve NXCALS-7753 "Introduce disaster flow control in etls"

Merge request template

COMPATIBILITY NOTE

Please make sure that this change is BACKWARDS COMPATIBLE and a new server will work correctly with clients using old APIs versions.

It may be that data sent in the new format will break the old clients.

In case of any potential incompatibility we have to inform our clients and schedule release & deployment with them.

---

@acc-logging-team - please have a look for approval.

Please describe the merge request here.

Closes NXCALS-7753

ansible/group_vars/all/all.yml

@@ -539,6 +539,7 @@ kafka_etl_daily_partitions: '{{service_compaction_staging_partition_splits}}'
 kafka_etl_time_bucket_since: "2024-06-13 00:00:00.000"
 #For alerting
 kafka_etl_max_lag: 100000
+kafka_etl_flow_control_threshold: 100000000
 
 #NXCALS - Kafka-Etl Hdfs
 #################################################################################