Skip to content

NXCALS-1620 Store compacted data on a final path, produced by expanding date to nested directories

#Merge request template @acc-logging-team - please have a look for approval.

This MR changes the way we store compacted data (parquet format) to the destination directory layout. Now the end directory layout is constructed by expanding the staged directory date to sub-directories, one for each date unit.

Ex. compacted data that correspond to the date: 29/03/2016 should result to a directory layout of 2016 / 3 / 29

This will provide some well defined grouping of the end data, plus it will enhance the directory browsing experience when searching for particular date targets.

Edited by Grigorios Avgitidis

Merge request reports