The zero bias file currently in use is from 21.0.28 and in consequence has ELReset issues. This MR replaces it with a more recent release.


Former-commit-id: ad60ace7b3e0666df00ba041a548414703f352a6

This MR introduces the following changes to DerivatioFrameworkART:
 * restores HION formats following !10404
 * modifies the RPVLL file used by EXOT23 (data) test
 * adds some new status print messages


Former-commit-id: 128040781386871be3b1c607db94b3271c87dd78

See title


Former-commit-id: 58c76542f70ad1a4e05e9ef9473d1bc527d6348c

See title.


Former-commit-id: 71774f15719d95232e4545e9cf3bfbffdc6b3d99

The trains running in the ART tests are updated to reflect more recent sizes. These trains will be used for a large-scale test before we start 2018 data taking.


Former-commit-id: 3aa9e6910e522e8b462a0b6a40d70e724c16e62a

See above - no changes to C++ or python, just configuration of ART tests


Former-commit-id: ea8416ee0ec259129c927bc015f5f5fd91304a89

This MR extends the list of formats that are blocked from using the AllVariables method of the slimming helper to those formats which are 1-2% of the size of their input AOD. Once accepted, all physics DAODs > 1% of their input will be blocked form using AllVariables. The obsolete HIGG2D2 is also dropped from ART and the list of flags.


Former-commit-id: 11c48f1697f7672d9dd7e94dafff14c40fe6a11a

In this MR, the first batch of formats (those in Higgs and Exotics with a size > 2% of the input AOD) are added to the list that blocks them from using AllVariables. EXOT18 is also removed from the ART tests, as it is currently obsolete.


Former-commit-id: 4efecbb9c4d87db27e83cf9d58d32c5dcb3a18b1

Former-commit-id: 1e6f49d61b7fa55b1e94f957ee51eb10695b19da

Former-commit-id: 164c9f1a544688eebe366d8beb648db42ba6acbb

Heavy ion DAOD formats, which were not previously in the ART tests, are now introduced with this MR, using a dedicated test input file. Furthermore the BPHY10 test on the Main stream is restored.


Former-commit-id: 4098b9759457e9bf08f56cca7cfdd7a7dc2470d2

This MR updates the test AOD file used by the derivation framework ART tests to a file from the Christmas 2017 reprocessing campaign. The dedicated test is dropped. Also checkxAOD test added to ART tests.


Former-commit-id: ec23dc11d7f8e97eabdbb90a9974fa5252964b45

The BPhysLS stream is much richer in events considered by BPHY8 than the other streams, and in consequence it runs much slower. Reducing the number of events accordingly.


Former-commit-id: 40a7d39dcb0912d38dcf855f4a9d6a86cb9f5eec

The current input file is in 2017 period A and so is useless. Switching to a later file.


Former-commit-id: cee2029f096b14f8b3afe83a929142f3b971fdd6

Former-commit-id: 7a9bdd7141f4b5547dfe63005d7b635e55fd022e

In this MR the following changes are made to DerivationFrameworkART
 * EXOT7 and SUSY9 are added to their trains
 * Tests are included for BPHY7,8 and 10 on the BLS and delayedBPhys streams
 * All events are processed for a given input file, unless there is a specific need to process fewer


Former-commit-id: c43737bbb6dae1e08583743f76fdf0b89011b60e

EXOT23, which runs on special RPVLL AODs, is introduced to ART in this MR. To do: the MC file is an unofficial privately produced file. We should replace this with a centrally produced one, when one is available.


Former-commit-id: 65884afab14200101690405d9ff9658e8683d69e

EXOT22 is a new format and in this MR is added to the appropriate train test.


Former-commit-id: b13888a0980002e86fe30685a1f963c417660956

In this MR the following changes are made to the DAOD building ART tests:
 * All available events now processed in the merge test
 * MC test file switched to ttbar (410501, r9778)
 * New train test added for TOPQ, running on MC only


Former-commit-id: 8337fe0f57ef180181e9f59c83e1a3b0de61863d

This MR makes the following adjustments:
 * drops EXOT14 completely
 * adds SUSY9 and adds an ART test for data and MC
 * Adds TOPQ5 to the ART train tests for data and MC
 * Adds BPHY8 to the ART train test for data
 * Reduces BPHY3 to 500 events in the ART tests
 * Amends the test input file used for the PHYSVAL data ART test

Where required the script-maker has been amended to reflect the above as well


Former-commit-id: 45309cc21d645eb9f2b801ff75b531a3a081037a

The same trains are set for MC as for data.


Former-commit-id: a9ef32acb13fb1d75d878c47f90abb253af85b01

This MR adds a line in all DAOD scripts to collect the output of the checkFile test to the tarball. The master script-generator is adjusted to reflect this.


Former-commit-id: 39457bbc9b804475b5cf39a5058e6157f2e7afd9

Former-commit-id: 76fe626bf4fdad5c1d62d22b5386b0fe8cd2a157

The input AOD file for the test_MC16BPHY8 ART test is adjusted to
a file from
mc16_13TeV.300999.Pythia8BEvtGen_A14_CTEQ6L1_Bplus_Jpsi_mu3p5mu3p5_Kplus.recon.AOD.e4397_s3126_r9703
This input file -- although it only contains 2000 events -- is currently the
best available file from a MC16 channel which BPHY8 is actually set up to
process while on the other side it was done with a reconstruction release
recent enough for the btagging xAODFix being run in 21.2 derivations.

The input file AOD.11705353._000001.pool.root.1 has been copied to
/eos/atlas/atlascerngroupdisk/data-art/grid-input/DerivationFrameworkART
so it will be picked up for inclusion in CVMFS.

It has been checked locally that the BPHY8 format works correctly with recent
21.2-AthDerivation and this new input file.

Please note that the file name has also been updated in makeARTScripts.py
to ensure that the adjustment will still be applied after this script is
run.


Former-commit-id: 8f0ba7475709a70220bea43059debe09f1882c57

Fabian Wilk authored 7 years ago and James Catmore committed 7 years ago

Former-commit-id: 93a7f90b486eb5bdcdfda82cd1aab374530d92da

This MR includes a number of additions to the DAOD ART tests
 * increased number of events in the test from 5000 to 8000 (data) and 10000 (MC)
 * adds checkFile.py report
 * adds a new format (HIGG4D6)
 * corrects the NTUP_PHYSVAL command in the PHYSVAL tests


Former-commit-id: c524b403b21f399e7c4aa0ba66d15cff6c32514e

ATLASDPD-976

The following containers were missing from DAOD_PHYSVAL:
 * PFlow: JetETMissNeutralParticleFlowObjects, JetETMissChargedParticleFlowObjects
 * Muons: MuonTruthParticles
 * Egamma: egammaTruthParticles
 * Taus: TauTracks

This MR adds them to the job options. The ART tests are also adjusted to include a test production of NTUP_PHYSVAL from the DAOD_PHYSVAL made in the same test.


Former-commit-id: b8295e093e75e753e4565cb1e9d449e42871e9c0

BPHY8 requires specific MC samples since it uses a look-up list of DSNs. This MR uses such a file in the test to avoid a python-level crash.


Former-commit-id: aba7270c8ab57ca934b54623651c3dc6ffea7cb8

In this MR the following changes are made to DerivationFrameworkART
 * Adding train tests (for data, 100 events without skimming)
 * Fixing the flavour tagging configuration for both data and MC
 * adding ```athenaopts="s"``` to the TRUTH format merging for debugging purposes


Former-commit-id: 63258a3641a07d54c9dd264b18bae403e2ee35a9

This adds the ```set -e``` to all of the ART jobs, such that if either of the tasks in an ART job fail (DAOD building or merging) then the job returns a failure code and is marked red by ART. Without this MR only the merge job is used to detect the success or failure. The script-making script is also updated to reflect this modification.


Former-commit-id: db33bd38f72183a791dd13156dadab941f200f09

This MR appends the pre-exec to the DAOD building jobs in the ART tests, required to pick up the latest flavour tagging calibration file tag. The ART-script making script is coorespondingly updated as well.


Former-commit-id: 3b47352eead93c50f7b8a9892599a7c1e2619354

AGENE-776

This MR introduces the TRUTH0,1 and 3 formats to the ART tests. A new input EVNT file needs to be introduced to CVMFS for this to run. This is being done today.


Former-commit-id: b6ecf280e6ba1ebe224d181f21e3ef6f7501e134

STDM1 and SUSY9 are obsolete and do not need regular testing. Dropping from the ART tests and also from DerivationFrameworkCore.


Former-commit-id: 2e5aae9dc8199bb76fad29fb6a6eff7e38861eb3

FTAG5 is not in current use and so should not be tested on a nightly basis.


Former-commit-id: b407b43a736810552da264dbc5e68ee4ed58eebf

Former-commit-id: 129c2df826ee4ee4f6a93fc4d7289850723398b6

Adding the non-TRUTH formats to the list of ART jobs, one script per format. No merging yet.


Former-commit-id: 635fb15d2d95f4cf6a9dab45693ef83310150f1d