[MONIT-4077] Refactor Metrics Flow: Prometheus Remote Write to Fluent Bit

This patch redefines the metrics flow to improve reliability and mitigate errors previously encountered in Fluent Bit components during metrics scraping from Prometheus. It transitions from Fluent Bit scraping Prometheus to a model where Prometheus performs remote writes directly to Fluent Bit, which then forwards data to the monitoring infrastructure as before. Key updates include:

• Addition of a new Service for Fluent Bit to enable Prometheus remote write.
• Enhanced configurations in prometheus.yaml and statefulset.yaml for robust remote write handling.
• Updated values.yaml to support configurable Fluent Bit replicas and advanced remote write settings.
• Cleaned up Prometheus configurations with relabeling and metadata optimizations.

This refactor addresses issue MONIT-4077 and aims to enhance the stability and maintainability of the monitoring pipeline.

marked this merge request as ready

changed title from Draft: [MONIT-4077] otel-metrics: enable remote write from prom to fb to [MONIT-4077] Refactor Metrics Flow: Prometheus Remote Write to Fluent Bit

changed the description

resolved all threads

approved this merge request

merged

mentioned in merge request !45 (merged)