Skip to content
Snippets Groups Projects
Commit d3d5aa0a authored by Marta Vila Fernandes's avatar Marta Vila Fernandes :game_die: Committed by Alex Iribarren
Browse files

opensearch alerts

parent f4aa01dd
Branches
No related tags found
1 merge request!138opensearch alerts
The ~lxsoft-alerts channel is where automated system messages are recorded.
These massages are being defined in [Packages Alerts](https://gitlab.cern.ch/linuxsupport/cronjobs/package_alerts) and [Grafana alerts](https://monit-grafana.cern.ch/alerting/list).
### Packages Alerts
It parses Red Hat API content from https://access.redhat.com/management/api/rhsm and based on the rules defined in `prod.packages.yml`, it will inform about, for instance, dangerous packages (new releases).
### Grafana Alerts
Important to understand if Koji, Lxsoft and Nomad services are in good health.
\ No newline at end of file
......@@ -111,3 +111,28 @@ At the moment (Apr. 2022) there's no good way to get logs of failures to help wi
most recent probes at [http://monit-remote.cern.ch/blackbox](http://monit-remote.cern.ch/blackbox), but they're not kept around for long. Another thing you can do
is manually trigger a probe by visiting [http://monit-remote.cern.ch/blackbox/probe?module=linuxsoft/http_ipv4_koji&target=http%3A%2F%2Fkoji22.cern.ch&debug=true](http://monit-remote.cern.ch/blackbox/probe?module=linuxsoft/http_ipv4_koji&target=http%3A%2F%2Fkoji22.cern.ch&debug=true). You can change the module and target parameters of that URL to trigger different probes.
These magical URLs were given to us by Borja Garrido and they may stop working at any time as they're not an "official" service interface.
# Lxsoft-Alerts
The ~lxsoft-alerts channel is where automated system messages are recorded.
These messages are being defined in [Packages Alerts](https://gitlab.cern.ch/linuxsupport/cronjobs/package_alerts), [Grafana alerts](https://monit-grafana.cern.ch/alerting/list) and [OpenSearch Alerting](https://os-linux.cern.ch/dashboards/app/alerting).
### Packages Alerts
It parses Red Hat API content from https://access.redhat.com/management/api/rhsm and based on the rules defined in `prod.packages.yml`, it will inform about, for instance, dangerous packages (new releases).
### Grafana Alerts
Important to understand if Koji, Lxsoft and Nomad services are in good health.
### OpenSearch Alerts
It searches by two different reposync errors in `monit_private_lxsoft_logs_nomad` index:
- `reposync gpg error`
- `reposync error: No packages in upstream repo, aborting`
It returns the list of repositories with issues.
The first error, means that probably there is a new GPG key upstream that needs to be added to the repo. The second may involve more debugging.
The jobs run every day.
\ No newline at end of file
......@@ -98,7 +98,6 @@ nav:
- 'AIMS': logs/aims.md
- 'Koji': logs/lsb.md
- 'Procedures': logs/procedures.md
- 'Lxsoft-Alerts': lxsoft_alerts/introduction.md
- 'Community':
- 'CentOS': community/centos.md
- 'Resources':
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment