Skip to content

Update scaling and writing

Samuel Van Stroud requested to merge svanstro/update-scaling-and-writing into master

Summary

This MR introduces the following changes

  • Use gzip compression during scaling.
  • Use per jet chunking if applying compression to final shuffled train files
  • Check for nan and inf scales

Relates to the following issues

Conformity

Merge request reports