Skip to content

Add ability to read Enstore large file tapes

Problem to solve

Enstore wrote a type of tape it called CERN. Enstore did this for files exceeding 8GB of size which don't work in CPIO. This was assumed to be a CASTOR formatted tape, but it is similar but not the same. There are differences:

  • An extra tape mark between the VOL1 label and headers before the first file
  • Much longer headers between the files
  • Some of the information in the HDR1/HDR2/UHL1 and EOF1/EOF2/UTL1 is not present (fSec, fSeq, etc).
  • Positioning by fSeq, not blockID

Stakeholders

PIC has already run into this problem in trying to migrate. Fermilab will hit it soon if not addressed

Proposal

  1. Add a new tape format 3, EnstoreLarge.
  2. Add boolean values to the various header/EOF checking functions to skip the parts that the EnstoreLarge tapes would fail
  3. Create new EnstoreLargeReadSession and EnstoreLargeFileReader classes to read these new tapes. Unclear if this should be a subclass of CtaReadSession/FileReader or of ReadSession/FileReader

In investigating this issue a number of other things were noticed which should be changed:

  1. Many *FileReader classes throw exceptions claiming to be from FileReader::position (a copy and paste which as not updated). These should use FUNCTION (gcc) or func (C++11)
  2. The CI tests limit block size to 256kB where these Enstore tapes are 1 MB. We should raise the limit to 1 or 2 MB so a future test for these files can succeed.
Edited by Eric Wayne Vaandering