Skip to content

investigate audit loggings memory consumption for O(100)+ node clusters

Verify that the audit logging policy is not creating unnecessary load on masters nodes.

We have been using this configuration for a year so on the surface level it seems like the setup is okay. However we should follow up from INC4637672 where the kube-apiserver in the REANA cluster was repeatably restarting and killing the master node due to high memory consumption. Disabling audit logging in the cluster significantly reduced memory consumption and stabilised the cluster.

This is a reasonable large cluster at over a 100 nodes so one thing that we should investigate is how we are handling events that scale with cluster size like watch events on daemonsets for example.

Edited by Jack Charlie Munday