- Dec 16, 2016
-
-
Umesh Dharmaji Worlikar authored
Enabled default dump of environment into env.txt for non-interactive jobs. [ATLASJT-346] (PyJobTransforms-03-00-31) * python/trfUtils.py - Added utility method isInteractiveEnv() * python/trfExe.py - Refactored checks for interactive environment using new method in trfUtils. * python/transform.py - Enabled default dump of environment into env.txt for non-interactive jobs. [ATLASJT-346] * Tagging PyJobTransforms-03-00-31 2016-11-17 Marcelo Vogel <mavogel@cern.ch> * scripts/makeTrfJSONSignatures.py - Replace script by updated version provided by AMI team [ATLASJT-284] * Tagging PyJobTransforms-03-00-30 2016-11-14 Marcelo Vogel <mavogel@cern.ch> * python/transform.py - Add property trfPredata to be set from env variable at trf init * python/trfReports.py - Add trfPredata to job report under key 'trfPredata' - Report version 2.0.7 * Tagging PyJobTransforms-03-00-29 ... (Long ChangeLog diff - truncated)
-
- Aug 19, 2016
-
-
Marcelo Vogel authored
Fixed bug that enabled athenaMP when expectedEvents < N cores. [ATLASJT-337] (PyJobTransforms-03-00-20) * python/trfExe.py - Fix bug that was preventing to turn off athenaMP when - expectedEvents < N cores. [ATLASJT-337] * Tagging PyJobTransforms-03-00-20 2016-07-27 Graeme Stewart <graeme.andrew.stewart@cern.ch> * ATLASJT-335 fixed * python/trfArgClasses.py - Add a new "dumpvalue" property to argSubstepSteering to hold the alias value used to set the steering and dump that if requested * python/trfUtils.py - In dumpJSON and dumpPickle use the "dumpvalue" method, if the argument supports it * Tagging PyJobTransforms-03-00-19 2016-07-25 Marcelo Vogel <mavogel@cern.ch> * python/trfArgs.py - Fix bug in getExtraDPDList. Executor for NTUP_MUONCALIB was added twice * Tagging PyJobTransforms-03-00-18 ... (Long ChangeLog diff - truncated)
-
- Jun 14, 2016
-
-
Marcelo Vogel authored
Fixed typo in python/trfExe.py (ATLASJT-316) and updated tests that were failing in RTT (PyJobTransforms-03-00-11) 2016-06-10 Marcelo Vogel <mavogel@cern.ch> * python/trfExe.py - Fix typo 'ahtenaopts' to 'athenaopts' * test/test_TAGMerge_tf.py - Remove test on metadata (beam_type) which is no longer included in jobReport.json * test/test_RAWMerge_tf.py - Remove tests on metadata which is no longer included in jobReport.json (conditions_tag, geometry and beam_type) * Tagging PyJobTransforms-03-00-11 2016-06-03 Marcelo Vogel <mavogel@cern.ch> * python/trfArgs.py - Add missing nargs='+' option to --triggerConfig * Tagging PyJobTransforms-03-00-10 2016-05-31 Graeme Stewart <graeme.andrew.stewart@cern.ch> * python/trfExe.py - Move disableMP argument from transformExecutor class to athenaExecutor class (ATLASJT-318) * python/trfArgClasses.py - Update selfMerge methods for POOL files to use disableMP=True in executor ... (Long ChangeLog diff - truncated)
-
- May 27, 2016
-
-
Scott Snyder authored
2016-05-06 scott snyder <snyder@bnl.gov> * Tagging PyJobTransforms-03-00-05. * test/test_trfReports.py, test/test_trfArgs.py: Update for last changes. 2016-04-29 Graeme Stewart <graeme.andrew.stewart@cern.ch> * python/trfReports.py - Remove the ~useless "argValues" section from the job report - Remove the ~useless 'integrity' and 'file_type'sections from subfile reports - Update report number to 2.0.0 2016-04-28 Graeme Stewart <graeme.andrew.stewart@cern.ch> * python/trfUtils.py - Minor tweaks on reported environment variables for CMake builds 2016-04-28 Graeme Stewart <graeme.andrew.stewart@cern.ch> * python/trfUtils.py - Update asetupReport to use "lstags" (works for both cmt and cmake) 2016-04-27 Graeme Stewart <graeme.andrew.stewart@cern.ch> ... (Long ChangeLog diff - truncated)
-
- May 09, 2016
-
-
Graeme Stewart authored
- Specifically to test fork after first event for memory savings - Also collects many other minor fixes - EXCLUDES lightweight output file metadata patch (r731518) (PyJobTransforms-02-03-76-01) wwwpackage PyJobTransforms 2016-04-15 Graeme Stewart <graeme.andrew.stewart@cern.ch> ** Merge in trunk changes from r740535 2016-04-15 Graeme Stewart <graeme.andrew.stewart@cern.ch> ** Manually patch in trunk fixes athenaMPMergeTargetSize handling (ATLASJT-298) *** WARNING - Do not attempt to merge r731518 onto this branch *** 2016-04-15 Graeme Stewart <graeme.andrew.stewart@cern.ch> ** Merge in trunk changes, excluding lightweight metadata fixes 2016-04-11 Azzah Alshehri<azzah.aziz.alshehri@cern.ch> ** Change the self merge method to save the merge information * python/trfExe.py - Change the stanalone variable 'myMerger'inside the method to a class variable. - Save the 'merge executor instance' you get back in the athenaExecutor _smartMerge method. - Add an accessor to the base executor class, that returns the self._myMerge value. * python/trfArgClasses.py - Re-set 'myMerger' to the real merger executor in the actual merge. - Return myMerger for each of the specific file mergers method. ... (Long ChangeLog diff - truncated)
-
- Aug 05, 2015
-
-
Graeme Stewart authored
- Allow --skipEvents to be multi-valued (needed by FTK transforms) * scripts/makeTrfJSONSignatures.py - Skip beamSpotT0_Vertex_tf.py, which is a not a PyJobTransforms transform and causes an abort when using __import__ * test/test_trfArgs.py - Do configureFromJSON test with Athena_tf.py instead of Reco_tf.py (which is in a different project, this a problem in ATN during full builds) * Tagging PyJobTransforms-02-03-60 (PyJobTransforms-02-03-60) * python/trfArgs.py - Allow --skipEvents to be multi-valued (needed by FTK transforms) * scripts/makeTrfJSONSignatures.py - Skip beamSpotT0_Vertex_tf.py, which is a not a PyJobTransforms transform and causes an abort when using __import__ * test/test_trfArgs.py - Do configureFromJSON test with Athena_tf.py instead of Reco_tf.py (which is in a different project, this a problem in ATN during full builds) * Tagging PyJobTransforms-02-03-60 2015-07-08 Lisa Errenst <lisa.errenst@cern.ch> * python/trfReports.py - add HEPSPEC to machine report. Report version 1.0.5 2015-07-03 Graeme Stewart <graeme.andrew.stewart@cern.ch> * python/transform.py - Refactor handling of end of transform to always try to kill stray children * python/trfArgClasses.py - Move _mergeArgs methog from argAthenaFile to argFile so that argNTUP has access * python/trfArgs.py ... (Long ChangeLog diff - truncated)
-
- May 17, 2015
-
-
Graeme Stewart authored
- Add new auxiliaryFile property to argFile, signaling that a file should not be validated or appear in the jobReport * python/trfExe.py - Add memMonitor property to scriptExecutor, that will try to use the external MemoryMonitor binary to memory monitor the script executor's child process - MemoryMonitor is launched immediately after the scriptExecutor's child process and the memory summary JSON file parsed afterwards and added to the executor memory report - Deprecate the perfMonFile property of athenaExecutor and stop parsing any perfmon outputs - For DBRelease files, dereference all soft links, allowing for use with the pre-release DBRelease-pcache-current.tar.gz link - Change suffix on multiple-merged outputs after AthenaMP has run to _N (as expected by PanDA) * python/trfFileUtils.py - Add inpFileInterestingKeys (not yet used) * python/trfReports.py - Update report version number to 1.0.1 - Modify top level report 'resource' entry to have entries per executor with MemoryMonitor summary information and a separate entry for the transform and its externals - Add machine report information to resource entry, with information also parsed from /proc/cpuinfo * python/trfUtils.py - Add small debug message when setting up a DBRelease from CVMFS * python/trfValidation.py - Ensure that auxilliary files are never validated * test/test_trfUtilsDBRelease.py - Add new test for DBRelease-pcache-current.tar.gz link * Tag PyJobTransforms-02-03-48 (PyJobTransforms-02-03-48) * python/trfArgClasses.py - Add new auxiliaryFile property to argFile, signaling that a file should not be validated or appear in the jobReport * python/trfExe.py - Add memMonitor property to scriptExecutor, that will try to use the external MemoryMonitor binary to memory monitor the script executor's child process - MemoryMonitor is launched immediately after the scriptExecutor's child process and the memory summary JSON file parsed afterwards and added to the executor memory report - Deprecate the perfMonFile property of athenaExecutor and stop parsing any perfmon outputs - For DBRelease files, dereference all soft links, allowing for use with the pre-release DBRelease-pcache-current.tar.gz link - Change suffix on multiple-merged outputs after AthenaMP has run to _N (as expected by PanDA) * python/trfFileUtils.py - Add inpFileInterestingKeys (not yet used) * python/trfReports.py - Update report version number to 1.0.1 - Modify top level report 'resource' entry to have entries per executor ... (Long ChangeLog diff - truncated)
-
- Jan 19, 2015
-
-
Graeme Stewart authored
New way to elevate std::bad_alloc message to "CATASTROPHE" (level = fatal + 10) Improved GetTfCommand.py Wider test coverage * Tag PyJobTransforms-02-03-37 (PyJobTransforms-02-03-37) * Tag PyJobTransforms-02-03-37 2014-12-18 Azzah Alshehri <azzah.aziz.alshehri@cern.ch> * python/trfArgClasses.py - Convert the member variable self._desc in prodsysDescription method to be a local variable. - Pass the argument groups into the json signature so that the AMI can use it to layout arguments sensibly when defining tags 2014-12-17 Graeme Stewart <graeme.andrew.stewart@cern.ch> * python/trfArgs.py - Add new argument group 'Metadata' - Add task, job, attempt arguments in metadata group - Make metadata arguments a part of standard transform arguments - Fix missing group for athenaMPMergeTargetSize * test/test_trfValidation.py - Clean up all test logfiles 2014-12-17 Graeme Stewart <graeme.andrew.stewart@cern.ch> * python/transform.py - Small change in startup message * python/trfExe.py ... (Long ChangeLog diff - truncated)
-
- Nov 05, 2014
-
-
Graeme Stewart authored
- Change default option in hybridMerge to be "slow" merging, due to persistent issues with fast merged metadata * trfReports.py - Fix bad reference to _maxMsgLen in python report generator * trfValidation.py - Add file count validation for DAOD merging transform * test_AODMerge_tf.py - Account for metadata 'compression' in metadata value test case * Tag PyJobTransforms-02-03-29 (PyJobTransforms-02-03-29) * trfExe.py - Change default option in hybridMerge to be "slow" merging, due to persistent issues with fast merged metadata * trfReports.py - Fix bad reference to _maxMsgLen in python report generator * trfValidation.py - Add file count validation for DAOD merging transform * test_AODMerge_tf.py - Account for metadata 'compression' in metadata value test case * Tag PyJobTransforms-02-03-29 2014-09-19 Graeme Stewart <graeme.andrew.stewart@cern.ch> * trfAMI.py - Introduce test for 'new transform' and make sure that command line arguments are prefixed by "--" * trfReport.py - Truncate very long error messages in exitMsg * trfArgClasses.py - Refactor EOS globbing code to improve robustness and error messages * Tag PyJobTransforms-02-03-28 ... (Long ChangeLog diff - truncated)
-
- Sep 19, 2014
-
-
Graeme Stewart authored
- Change format of DAOD output filename to match grid expectations * Tag PyJobTransforms-02-03-26 (PyJobTransforms-02-03-26)
-