Skip to content
Snippets Groups Projects
Forked from atlas / athena
148392 commits behind the upstream repository.
  • Graeme Stewart's avatar
    37af8d63
    Improved JSON transform signatures; added "group" field; more robust script. · 37af8d63
    Graeme Stewart authored
    New way to elevate std::bad_alloc message to "CATASTROPHE" (level = fatal + 10)
    Improved GetTfCommand.py
    Wider test coverage
    * Tag PyJobTransforms-02-03-37 (PyJobTransforms-02-03-37)
    
    	* Tag PyJobTransforms-02-03-37
    
    2014-12-18  Azzah Alshehri <azzah.aziz.alshehri@cern.ch>
    	* python/trfArgClasses.py
    	- Convert the member variable self._desc in prodsysDescription method to be a local variable.
    	- Pass the argument groups into the json signature so that the AMI can use it to layout arguments sensibly when defining 	tags
    
    2014-12-17 Graeme Stewart <graeme.andrew.stewart@cern.ch>
    	* python/trfArgs.py
    	- Add new argument group 'Metadata'
    	- Add task, job, attempt arguments in metadata group
    	- Make metadata arguments a part of standard transform arguments
    	- Fix missing group for athenaMPMergeTargetSize
    	* test/test_trfValidation.py
    	- Clean up all test logfiles
    
    2014-12-17 Graeme Stewart <graeme.andrew.stewart@cern.ch>
    	* python/transform.py
    	- Small change in startup message
    	* python/trfExe.py
    ...
    (Long ChangeLog diff - truncated)
    37af8d63
    History
    Improved JSON transform signatures; added "group" field; more robust script.
    Graeme Stewart authored
    New way to elevate std::bad_alloc message to "CATASTROPHE" (level = fatal + 10)
    Improved GetTfCommand.py
    Wider test coverage
    * Tag PyJobTransforms-02-03-37 (PyJobTransforms-02-03-37)
    
    	* Tag PyJobTransforms-02-03-37
    
    2014-12-18  Azzah Alshehri <azzah.aziz.alshehri@cern.ch>
    	* python/trfArgClasses.py
    	- Convert the member variable self._desc in prodsysDescription method to be a local variable.
    	- Pass the argument groups into the json signature so that the AMI can use it to layout arguments sensibly when defining 	tags
    
    2014-12-17 Graeme Stewart <graeme.andrew.stewart@cern.ch>
    	* python/trfArgs.py
    	- Add new argument group 'Metadata'
    	- Add task, job, attempt arguments in metadata group
    	- Make metadata arguments a part of standard transform arguments
    	- Fix missing group for athenaMPMergeTargetSize
    	* test/test_trfValidation.py
    	- Clean up all test logfiles
    
    2014-12-17 Graeme Stewart <graeme.andrew.stewart@cern.ch>
    	* python/transform.py
    	- Small change in startup message
    	* python/trfExe.py
    ...
    (Long ChangeLog diff - truncated)
Code owners
Assign users and groups as approvers for specific file changes. Learn more.
trfFileValidationFunctions.py 3.02 KiB
# Copyright (C) 2002-2017 CERN for the benefit of the ATLAS collaboration

## @package PyJobTransforms.trfFileValidationFunctions
# @brief Transform file validation functions
# @author atlas-comp-transforms-dev@cern.ch
# @version $Id: trfUtils.py 578615 2014-01-15 21:22:05Z wbreaden $

import logging
msg = logging.getLogger(__name__)

import PyJobTransforms.trfExceptions as trfExceptions

## @brief Integrity function for file class argPOOLFile, argHITSFile and argRDOFile
def returnIntegrityOfPOOLFile(fname):
    from PyJobTransforms.trfValidateRootFile import checkFile
    rc = checkFile(fileName = fname, type = 'event', requireTree = True)
    if rc == 0:
        return (True, "integrity of {fileName} good".format(fileName = str(fname)))
    else:
        return (False, "integrity of {fileName} bad: return code: {integrityStatus}".format(fileName = str(fname), integrityStatus = rc))

## @brief Integrity function for file class argNTUPFile
def returnIntegrityOfNTUPFile(fname):
    from PyJobTransforms.trfValidateRootFile import checkFile
    rc = checkFile(fileName = fname, type = 'basket', requireTree = False)
    if rc == 0:
        return (True, "integrity of {fileName} good".format(fileName = str(fname)))
    else:
        return (False, "integrity of {fileName} bad: return code: {integrityStatus}".format(fileName = str(fname), integrityStatus = rc))

## @brief Integrity function for file class argBSFile
def returnIntegrityOfBSFile(fname):
    try:
        from PyJobTransforms.trfUtils import call
        rc = call(["AtlListBSEvents.exe", "-c", fname],
            logger = msg,
            message = "Report by AtlListBSEvents.exe: ",
            timeout = None
        )
    except trfExceptions.TransformTimeoutException:
        return False
    if rc == 0:
        return (True, "integrity of {fileName} good".format(fileName = str(fname)))
    else:
        return (False, "integrity of {fileName} bad: return code: {integrityStatus}".format(fileName = str(fname), integrityStatus = rc))

### @brief Integrity function for file class argTAGFile
def returnIntegrityOfTAGFile(fname):
    from PyJobTransforms.trfFileUtils import AthenaFileInfo
    dictionaryOfAthenaFileInfo = AthenaFileInfo([str(fname),], retrieveKeys = ['nentries',])
    msg.debug("dictionary of Athena file information: {a}".format(a = dictionaryOfAthenaFileInfo))
    eventCount = dictionaryOfAthenaFileInfo[str(fname)]['nentries']
    if eventCount is None:
        return (False, "integrity of {fileName} bad: got a bad event count in {fileName}: {eventCount}".format(fileName = str(fname), eventCount = eventCount))
    else:
        return (True, "integrity of {fileName} good".format(fileName = str(fname)))

## @brief Integrity function for file class argHISTFile
def returnIntegrityOfHISTFile(fname):
    rc = 0 # (default behaviour)
    if rc == 0:
        return (True, "integrity of {fileName} good".format(fileName = str(fname)))
    else:
        return (False, "integrity of {fileName} bad: return code: {integrityStatus}".format(fileName = str(fname), integrityStatus = rc))