While doing trying to mimic the OpenShift logging setup with the current
Helm chart we noticed that they were using the lua scripts input from
the fluent bit official Helm chart. In our case this was not possible
and had to be done by adding secrets one by one.

We agreed to mimic this behaviour and had our own helm chart have the
option to add lua scripts to the fluent bit via helm chart values.

This commit modifies the configmap used to create the lua script that
we required for the timestampt in such a way that now it creates this
first lua script but also creates any other provided by the users via
the values of the helm chart.

Reported-at: https://its.cern.ch/jira/browse/MONIT-4106


Signed-off-by: Guillermo Facundo Colunga <guillermo.facundo.colunga@gmail.com>
Signed-off-by: Guillermo Facundo Colunga <guillermo.facundo.colunga@cern.ch>

Some users reported that the current implementation of the metrics flow
it is causing errors in the fluent bit components that scrape the metrics
from the local prometheus and forwards to open telemetry.

This commits inverts the paradigm and now it is prometheus the one doing
the remote write into the local fluent bit. After that, fluent bit is
doing a remote write as it was doing previously into the monitoring infra.

Reported-at: https://its.cern.ch/jira/browse/MONIT-4077


Signed-off-by: Guillermo Facundo Colunga <guillermo.facundo.colunga@gmail.com>
Signed-off-by: Guillermo Facundo Colunga <guillermo.facundo.colunga@cern.ch>

Updated all components that use fluentbit to use the 3.1.8 version.

JIRA Refs:
- MONIT-4029

After some feedback from the kubernetes team we notice the need to
point all the images from the chart to use the internal CERN registry.
As an example only those images are exposed to the TN.

This commit changes all the images to use the internal repository from
cern.

JIRA Refs:
- MONIT-4028

Until now all the components that were not a daemonset and needed to be
deployed on a node contained a config option to indicate a node selector
for the allocation. This commits adds a new global option that allows to
set a node selector that will be applied to all the components that do
not declare one. If any component declares a node selector this last one
will have always more priority.

JIRA Refs:
- MONIT-4003

Nacho Barrientos authored 5 months ago and Guillermo Facundo Colunga committed 5 months ago

when fluentbit loses connection to the metrics endpoint (open telemetry), unsent metrics are dropped and permanently lost. This commit enables the local buffering feature in Fluent Bit, allowing metrics to be temporarily stored in a local buffer. When the connection is reestablished, these buffered metrics are retried and sent to the endpoint. This improves reliability by minimizing metric loss during intermittent connection issues.

Proposed-by: Nacho Barrientos <nacho.barrientos@cern.ch>

Signed-off-by: Guillermo Facundo Colunga <guillermo.facundo.colunga@cern.ch>

Signed-off-by: Guillermo Facundo Colunga <guillermo.facundo.colunga@cern.ch>

As part of this commit the following additions have been done:

* removed templates/namespace in favor of `.Release.namespace`
* use `enabled` flags per component
* add metrics server dependency (it was missing)
* add to the readme file all default values with its description

Borja Garrido Bear authored 8 months ago and Guillermo Facundo Colunga committed 8 months ago

This commit adds a new component to forward metrics. A fluent-bit instance devoted to read metrics from the local prometheus and send them to wherever configured. By default this is used to send metrics to OTEL, but can be configured to have multiple flows.

Author: Borja Garrido Bear <borja.garrido.bear@cern.ch>
Signed-off-by: Borja Garrido Bear &lt;borja.garrido.bear@cern.ch&gt;, Guillermo Facundo Colunga <guillermo.facundo.colunga@cern.ch>