Skip to content
Snippets Groups Projects
  1. Apr 20, 2017
  2. Apr 10, 2017
  3. Dec 16, 2016
    • Umesh Dharmaji Worlikar's avatar
      Enabled default dump of environment into env.txt for non-interactive jobs.... · a8a1d431
      Umesh Dharmaji Worlikar authored
      Enabled default dump of environment into env.txt for non-interactive jobs. [ATLASJT-346] (PyJobTransforms-03-00-31)
      
      	* python/trfUtils.py
      	- Added utility method isInteractiveEnv()
      	* python/trfExe.py
      	- Refactored checks for interactive environment using new method in trfUtils.
      	* python/transform.py
      	- Enabled default dump of environment into env.txt for non-interactive jobs. [ATLASJT-346]
      	* Tagging PyJobTransforms-03-00-31
      
      2016-11-17 Marcelo Vogel <mavogel@cern.ch>
      	* scripts/makeTrfJSONSignatures.py
      	- Replace script by updated version provided by AMI team [ATLASJT-284]
      	* Tagging PyJobTransforms-03-00-30
      
      2016-11-14 Marcelo Vogel <mavogel@cern.ch>
      	* python/transform.py
      	- Add property trfPredata to be set from env variable at trf init
      	* python/trfReports.py
      	- Add trfPredata to job report under key 'trfPredata'
      	- Report version 2.0.7
      	* Tagging PyJobTransforms-03-00-29
      ...
      (Long ChangeLog diff - truncated)
      a8a1d431
  4. Aug 19, 2016
    • Marcelo Vogel's avatar
      Fixed bug that enabled athenaMP when expectedEvents < N cores. [ATLASJT-337]... · 105925ea
      Marcelo Vogel authored
      Fixed bug that enabled athenaMP when expectedEvents < N cores. [ATLASJT-337] (PyJobTransforms-03-00-20)
      
      	* python/trfExe.py
      	- Fix bug that was preventing to turn off athenaMP when
      	- expectedEvents < N cores. [ATLASJT-337]
      	* Tagging PyJobTransforms-03-00-20
      
      2016-07-27 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	* ATLASJT-335 fixed
      	* python/trfArgClasses.py
      	- Add a new "dumpvalue" property to argSubstepSteering to hold the alias
      	  value used to set the steering and dump that if requested
      	* python/trfUtils.py
      	- In dumpJSON and dumpPickle use the "dumpvalue" method, if the argument
      	  supports it
      	* Tagging PyJobTransforms-03-00-19
      
      2016-07-25 Marcelo Vogel <mavogel@cern.ch>
      	* python/trfArgs.py
      	- Fix bug in getExtraDPDList. Executor for NTUP_MUONCALIB was added twice
      	* Tagging PyJobTransforms-03-00-18
      
      ...
      (Long ChangeLog diff - truncated)
      105925ea
  5. Jun 14, 2016
    • Marcelo Vogel's avatar
      Fixed typo in python/trfExe.py (ATLASJT-316) and updated tests that were... · 3d71ddb4
      Marcelo Vogel authored
      Fixed typo in python/trfExe.py (ATLASJT-316) and updated tests that were failing in RTT (PyJobTransforms-03-00-11)
      
      2016-06-10 Marcelo Vogel <mavogel@cern.ch>
      	* python/trfExe.py
      	- Fix typo 'ahtenaopts' to 'athenaopts'
      	* test/test_TAGMerge_tf.py
      	- Remove test on metadata (beam_type) which is no longer included in jobReport.json
      	* test/test_RAWMerge_tf.py
      	- Remove tests on metadata which is no longer included in jobReport.json
      	  (conditions_tag, geometry and beam_type)
      	* Tagging PyJobTransforms-03-00-11
      
      2016-06-03 Marcelo Vogel <mavogel@cern.ch>
      	* python/trfArgs.py
      	- Add missing nargs='+' option to --triggerConfig
      	* Tagging PyJobTransforms-03-00-10
      
      2016-05-31 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	* python/trfExe.py
      	- Move disableMP argument from transformExecutor class to athenaExecutor class (ATLASJT-318)
      	* python/trfArgClasses.py
      	- Update selfMerge methods for POOL files to use disableMP=True in executor
      ...
      (Long ChangeLog diff - truncated)
      3d71ddb4
  6. May 27, 2016
    • Scott Snyder's avatar
      'Update tests for last changes.' (PyJobTransforms-03-00-05) · 00312877
      Scott Snyder authored
      2016-05-06  scott snyder  <snyder@bnl.gov>
      	* Tagging PyJobTransforms-03-00-05.
      	* test/test_trfReports.py, test/test_trfArgs.py: Update for last
      	changes.
      
      2016-04-29 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	* python/trfReports.py
      	- Remove the ~useless "argValues" section from the job report
      	- Remove the ~useless 'integrity' and 'file_type'sections from subfile reports
      	- Update report number to 2.0.0
      
      2016-04-28 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	* python/trfUtils.py
      	- Minor tweaks on reported environment variables for CMake builds
      
      2016-04-28 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	* python/trfUtils.py
      	- Update asetupReport to use "lstags" (works for both cmt and cmake)
      
      2016-04-27 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      ...
      (Long ChangeLog diff - truncated)
      00312877
  7. May 09, 2016
    • Graeme Stewart's avatar
      Tag PyJobTransforms-02-03-76-01 · 55c63ade
      Graeme Stewart authored
      - Specifically to test fork after first event for memory savings
      - Also collects many other minor fixes
      - EXCLUDES lightweight output file metadata patch (r731518) (PyJobTransforms-02-03-76-01)
      
      wwwpackage PyJobTransforms
      2016-04-15 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	** Merge in trunk changes from r740535
      
      2016-04-15 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	** Manually patch in trunk fixes athenaMPMergeTargetSize handling (ATLASJT-298)
      	*** WARNING - Do not attempt to merge r731518 onto this branch ***
      
      2016-04-15 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	** Merge in trunk changes, excluding lightweight metadata fixes
      
      2016-04-11 Azzah Alshehri<azzah.aziz.alshehri@cern.ch>
      	** Change the self merge method to save the merge information
      	   * python/trfExe.py
      	   - Change the stanalone variable 'myMerger'inside the method to  a class variable.
      	   - Save the 'merge executor instance' you get back in the athenaExecutor _smartMerge method.
      	   - Add an accessor to the base executor class, that returns the self._myMerge value.
      	   * python/trfArgClasses.py
      	   - Re-set 'myMerger' to the real merger executor in the actual merge.
      	   - Return myMerger for each of the specific file mergers method.
      ...
      (Long ChangeLog diff - truncated)
      55c63ade
  8. Aug 05, 2015
    • Graeme Stewart's avatar
      * python/trfArgs.py · 014ee775
      Graeme Stewart authored
      	- Allow --skipEvents to be multi-valued (needed by FTK transforms)
      	* scripts/makeTrfJSONSignatures.py
      	- Skip beamSpotT0_Vertex_tf.py, which is a not a PyJobTransforms transform and causes
      	  an abort when using __import__
      	* test/test_trfArgs.py
      	- Do configureFromJSON test with Athena_tf.py instead of Reco_tf.py (which is
      	  in a different project, this a problem in ATN during full builds)
      	* Tagging PyJobTransforms-02-03-60 (PyJobTransforms-02-03-60)
      
      	* python/trfArgs.py
      	- Allow --skipEvents to be multi-valued (needed by FTK transforms)
      	* scripts/makeTrfJSONSignatures.py
      	- Skip beamSpotT0_Vertex_tf.py, which is a not a PyJobTransforms transform and causes
      	  an abort when using __import__
      	* test/test_trfArgs.py
      	- Do configureFromJSON test with Athena_tf.py instead of Reco_tf.py (which is
      	  in a different project, this a problem in ATN during full builds)
      	* Tagging PyJobTransforms-02-03-60
      
      2015-07-08 Lisa Errenst <lisa.errenst@cern.ch>
      	* python/trfReports.py
      	- add HEPSPEC to machine report. Report version 1.0.5
      
      2015-07-03 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	* python/transform.py
      	- Refactor handling of end of transform to always try to kill stray children
      	* python/trfArgClasses.py
      	- Move _mergeArgs methog from argAthenaFile to argFile so that argNTUP has access
      	* python/trfArgs.py
      ...
      (Long ChangeLog diff - truncated)
      014ee775
  9. May 17, 2015
    • Graeme Stewart's avatar
      * python/trfArgClasses.py · a59cfdb2
      Graeme Stewart authored
      	- Add new auxiliaryFile property to argFile, signaling that a file should not
      	  be validated or appear in the jobReport
      	* python/trfExe.py
      	- Add memMonitor property to scriptExecutor, that will try to use the external
      	  MemoryMonitor binary to memory monitor the script executor's child process
      	- MemoryMonitor is launched immediately after the scriptExecutor's child process
      	  and the memory summary JSON file parsed afterwards and added to the executor
      	  memory report
      	- Deprecate the perfMonFile property of athenaExecutor and stop parsing any
      	  perfmon outputs
      	- For DBRelease files, dereference all soft links, allowing for use with the
      	  pre-release DBRelease-pcache-current.tar.gz link
      	- Change suffix on multiple-merged outputs after AthenaMP has run to _N (as
      	  expected by PanDA)
      	* python/trfFileUtils.py
      	- Add inpFileInterestingKeys (not yet used)
      	* python/trfReports.py
      	- Update report version number to 1.0.1
      	- Modify top level report 'resource' entry to have entries per executor
      	  with MemoryMonitor summary information and a separate entry for the
      	  transform and its externals
      	- Add machine report information to resource entry, with information
      	  also parsed from /proc/cpuinfo
      	* python/trfUtils.py
      	- Add small debug message when setting up a DBRelease from CVMFS
      	* python/trfValidation.py
      	- Ensure that auxilliary files are never validated
      	* test/test_trfUtilsDBRelease.py
      	- Add new test for DBRelease-pcache-current.tar.gz link
      	* Tag PyJobTransforms-02-03-48 (PyJobTransforms-02-03-48)
      
      	* python/trfArgClasses.py
      	- Add new auxiliaryFile property to argFile, signaling that a file should not
      	  be validated or appear in the jobReport
      	* python/trfExe.py
      	- Add memMonitor property to scriptExecutor, that will try to use the external
      	  MemoryMonitor binary to memory monitor the script executor's child process
      	- MemoryMonitor is launched immediately after the scriptExecutor's child process
      	  and the memory summary JSON file parsed afterwards and added to the executor
      	  memory report
      	- Deprecate the perfMonFile property of athenaExecutor and stop parsing any
      	  perfmon outputs
      	- For DBRelease files, dereference all soft links, allowing for use with the
      	  pre-release DBRelease-pcache-current.tar.gz link
      	- Change suffix on multiple-merged outputs after AthenaMP has run to _N (as
      	  expected by PanDA)
      	* python/trfFileUtils.py
      	- Add inpFileInterestingKeys (not yet used)
      	* python/trfReports.py
      	- Update report version number to 1.0.1
      	- Modify top level report 'resource' entry to have entries per executor
      ...
      (Long ChangeLog diff - truncated)
      a59cfdb2
  10. Jan 19, 2015
    • Graeme Stewart's avatar
      Improved JSON transform signatures; added "group" field; more robust script. · f3e09d81
      Graeme Stewart authored
      New way to elevate std::bad_alloc message to "CATASTROPHE" (level = fatal + 10)
      Improved GetTfCommand.py
      Wider test coverage
      * Tag PyJobTransforms-02-03-37 (PyJobTransforms-02-03-37)
      
      	* Tag PyJobTransforms-02-03-37
      
      2014-12-18  Azzah Alshehri <azzah.aziz.alshehri@cern.ch>
      	* python/trfArgClasses.py
      	- Convert the member variable self._desc in prodsysDescription method to be a local variable.
      	- Pass the argument groups into the json signature so that the AMI can use it to layout arguments sensibly when defining 	tags
      
      2014-12-17 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	* python/trfArgs.py
      	- Add new argument group 'Metadata'
      	- Add task, job, attempt arguments in metadata group
      	- Make metadata arguments a part of standard transform arguments
      	- Fix missing group for athenaMPMergeTargetSize
      	* test/test_trfValidation.py
      	- Clean up all test logfiles
      
      2014-12-17 Graeme Stewart <graeme.andrew.stewart@cern.ch>
      	* python/transform.py
      	- Small change in startup message
      	* python/trfExe.py
      ...
      (Long ChangeLog diff - truncated)
      f3e09d81
  11. Nov 05, 2014
    • Graeme Stewart's avatar
      * trfExe.py · 8dc71161
      Graeme Stewart authored
      	- Change default option in hybridMerge to be "slow" merging, due
      	to persistent issues with fast merged metadata
      	* trfReports.py
      	- Fix bad reference to _maxMsgLen in python report generator
      	* trfValidation.py
      	- Add file count validation for DAOD merging transform
      	* test_AODMerge_tf.py
      	- Account for metadata 'compression' in metadata value test case
      	* Tag PyJobTransforms-02-03-29 (PyJobTransforms-02-03-29)
      
      	* trfExe.py
      	- Change default option in hybridMerge to be "slow" merging, due
      	to persistent issues with fast merged metadata
      	* trfReports.py
      	- Fix bad reference to _maxMsgLen in python report generator
      	* trfValidation.py
      	- Add file count validation for DAOD merging transform
      	* test_AODMerge_tf.py
      	- Account for metadata 'compression' in metadata value test case
      	* Tag PyJobTransforms-02-03-29
      
      2014-09-19 Graeme Stewart <graeme.andrew.stewart@cern.ch>
          * trfAMI.py
          - Introduce test for 'new transform' and make sure that command line
          arguments are prefixed by "--"
          * trfReport.py
          - Truncate very long error messages in exitMsg
          * trfArgClasses.py
          - Refactor EOS globbing code to improve robustness and error messages
          * Tag PyJobTransforms-02-03-28
      ...
      (Long ChangeLog diff - truncated)
      8dc71161
  12. Sep 19, 2014
    • Graeme Stewart's avatar
      * trfExe.py · 4982836c
      Graeme Stewart authored
              - Change format of DAOD output filename to match
                grid expectations
          * Tag PyJobTransforms-02-03-26 (PyJobTransforms-02-03-26)
      4982836c
Loading