Skip to content

Fix clear compression stats

Description

The aim of this merge request is to fix the drive::clearCompressionStats() method which deleted all log metrics with log select on page 0x00 (aka reset all metrics). The following changes are introduced:

  • For IBM and LTO drives: We are selectively clearing only the compression statistics (pages 0x38, 0x32 equivalently)
  • For Oracle and MHVTL drives: Wa are making compression statistics stateful, by accumulating values and subtracting on each flush

Moreover, we are piggybacking a fix for overflowing mountTotal{Read,Write}BytesProcessed (drive::xx::getTape{Read,Write}Errors) by bumping up the container map in 64-bit values (enough to cover the cartridge capacity)

Testing

The results of the changes were tested against IBM and Oracle drives.
More specifically, we attempted to write 40 randomly generated files of 2GB in size with the previous CASTOR version (2.1.16-9), extracted the compression metrics and compared them with the values after the changes.

IBM

before:

Oct  6 12:23:09 tpsrv240 tapeserverd[14257]: LVL="Info" TID="14268" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv240.cern.ch" clientPort="30270" mountTransactionId="38279477" volReqId="38279477" driveUnit="I4JD1306" dgn="IBM4JD" TPVID="IW0111" batch_size="16" compressed="33571798016" Non_compressed="33554439168"
Oct  6 12:24:53 tpsrv240 tapeserverd[14257]: LVL="Info" TID="14268" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv240.cern.ch" clientPort="30270" mountTransactionId="38279477" volReqId="38279477" driveUnit="I4JD1306" dgn="IBM4JD" TPVID="IW0111" batch_size="16" compressed="33571798016" Non_compressed="33554439168"
Oct  6 12:25:45 tpsrv240 tapeserverd[14257]: LVL="Info" TID="14268" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv240.cern.ch" clientPort="30270" mountTransactionId="38279477" volReqId="38279477" driveUnit="I4JD1306" dgn="IBM4JD" TPVID="IW0111" batch_size="8" compressed="16785898496" Non_compressed="16777219072"

**after: **

Oct  6 12:34:28 tpsrv240 tapeserverd[14836]: LVL="Info" TID="14847" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv240.cern.ch" clientPort="30247" mountTransactionId="38279497" volReqId="38279497" driveUnit="I4JD1306" dgn="IBM4JD" TPVID="IW0111" batch_size="16" compressed="33571798016" Non_compressed="33554439168"
Oct  6 12:36:12 tpsrv240 tapeserverd[14836]: LVL="Info" TID="14847" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv240.cern.ch" clientPort="30247" mountTransactionId="38279497" volReqId="38279497" driveUnit="I4JD1306" dgn="IBM4JD" TPVID="IW0111" batch_size="16" compressed="33571798016" Non_compressed="33554439168"
Oct  6 12:37:04 tpsrv240 tapeserverd[14836]: LVL="Info" TID="14847" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv240.cern.ch" clientPort="30247" mountTransactionId="38279497" volReqId="38279497" driveUnit="I4JD1306" dgn="IBM4JD" TPVID="IW0111" batch_size="8" compressed="16785898496" Non_compressed="16777219072"

Oracle

before:

Oct  6 12:21:44 tpsrv614 tapeserverd[3062]: LVL="Info" TID="3073" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv614.cern.ch" clientPort="30298" mountTransactionId="38279475" volReqId="38279475" driveUnit="T10D6118" dgn="T10KD6" TPVID="TW8610" batch_size="16" compressed="33624583612" Non_compressed="33554952064"
Oct  6 12:24:02 tpsrv614 tapeserverd[3062]: LVL="Info" TID="3073" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv614.cern.ch" clientPort="30298" mountTransactionId="38279475" volReqId="38279475" driveUnit="T10D6118" dgn="T10KD6" TPVID="TW8610" batch_size="16" compressed="33624583664" Non_compressed="33554952064"
Oct  6 12:25:12 tpsrv614 tapeserverd[3062]: LVL="Info" TID="3073" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv614.cern.ch" clientPort="30298" mountTransactionId="38279475" volReqId="38279475" driveUnit="T10D6118" dgn="T10KD6" TPVID="TW8610" batch_size="8" compressed="16812291836" Non_compressed="16777476032"

after:

Oct  6 12:34:58 tpsrv614 tapeserverd[3691]: LVL="Info" TID="3702" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv614.cern.ch" clientPort="30227" mountTransactionId="38279495" volReqId="38279495" driveUnit="T10D6118" dgn="T10KD6" TPVID="TW8610" batch_size="16" compressed="33624583612" Non_compressed="33554952064"
Oct  6 12:37:16 tpsrv614 tapeserverd[3691]: LVL="Info" TID="3702" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv614.cern.ch" clientPort="30227" mountTransactionId="38279495" volReqId="38279495" driveUnit="T10D6118" dgn="T10KD6" TPVID="TW8610" batch_size="16" compressed="33624583664" Non_compressed="33554952064"
Oct  6 12:38:26 tpsrv614 tapeserverd[3691]: LVL="Info" TID="3702" MSG="Reported to the client that a batch of file was written on tape" thread="ReportPacker" clientHost="tpsrv614.cern.ch" clientPort="30227" mountTransactionId="38279495" volReqId="38279495" driveUnit="T10D6118" dgn="T10KD6" TPVID="TW8610" batch_size="8" compressed="16812291836" Non_compressed="16777476032"

Merge request reports