How to monitor Kubernetes audit logs

Datadog has found that audit logs are extremely helpful for tracking user interactions with the API server, debugging issues, and getting clarity into their workloads in Kubernetes clusters. They provide deep insight into cluster operations by recording requests to the Kubernetes API server. These logs can be used to monitor API authentication issues, slow API requests, and anonymous requests to the API server. By persisting audit logs to longer term storage, it's possible to go back in time and answer questions such as, "Why was this pod evicted?" and "What lifecycle operations occur when we update a deployment?". Datadog also provides guidance on how to configure Kubernetes audit logs for optimal results.