Skip to content

Bug fixing and restructure/simplifying resampling code

Summary

current resampling code still have code duplicates where some of the older code is still bug prone. Such code variations should be merged to fix the bugs persistent in the old code whereas they are fixed in the new. E.g. when chunk_size<len(indices) resampling_generator cuts off the last non-full chunk whereas the sampling_generator append the rest to a previous chunk (which is also a bad practice, as intuitively chunk_size=max_chunk_size so that one is sure not to run out of memory although factor that is less then 2 plays role really seldom)

This MR introduces the following changes

  • fix a BUG in resampling_base:Resampling:resampling_generator that throws away the last not full bath (chunk) ...
  • ... by merging it with resampling_base:sampling_generator that does not have such a bug
  • Bug in resampling_base:write_file made writing the last jet batch doubled into the output file which might have lead to a lot of problems, now it is fixed

Conformity

Edited by Ivan Oleksiyuk

Merge request reports