Skip to content

NXCALS-1396 Respect file's last-modification time before compaction

Grigorios Avgitidis requested to merge feature-NXCALS-1396-compactor-respect-file-modification-time into develop

#Merge request template @acc-logging-team - please have a look for approval.

This MR introduces a new hadoop-related internal service, that is capable of contacting HDFS and collect file statistics about the to-be-compacted files. If the remote directory contains even 1 file with last modification time, later than the specified allowed duration, the compaction will be avoided, as the checker will result to negative evaluation.

You can have a look on this effort's result on grafana at my personal development environment

Edited by Grigorios Avgitidis

Merge request reports