Investigate auditing options and needs
Etcd vs ELK stack
Considerations:
- Audit: url, from and to urls, hash of content, timestamp, success state, error cause
- Possible usecases:
- avoid reindexing same results for speed eg key exists and timestamp ==, or hash of content ==?
- avoid reindexing same content for different LOA, eg lightweight < cern, and if content for both is the same, then update acl list of document instead of sending in a duplicate (use hash of content)
- Latency, scalability and effort to implement/integrate
- Python clients
- Deploy or hosted at cern
- Search and visualization, eg percentage of failures, ease to get list of failed endpoints for retry in a different job