athena merge requestshttps://gitlab.cern.ch/atlas/athena/-/merge_requests2024-03-26T14:10:45+01:00https://gitlab.cern.ch/atlas/athena/-/merge_requests/70088Use APRDefaults to name the Event Data TTree/RNTuple2024-03-26T14:10:45+01:00Marcin NowakUse APRDefaults to name the Event Data TTree/RNTupleMake the AtheaPool Conversion Service use by default the prefix defined in APRDefaults as the name prefix for POOL containers. This prefix in most cases corresponds to the name of the main Event Data TTree/RNTuple.
There should be no...Make the AtheaPool Conversion Service use by default the prefix defined in APRDefaults as the name prefix for POOL containers. This prefix in most cases corresponds to the name of the main Event Data TTree/RNTuple.
There should be no change for the standard "CollectionTree" name in normal Athena jobs. But by configuring the Output.StorageTechnology.EventData="ROOTRNTUPLE" the main RNTuple will be now by default called "EventData" (or whatever is specified in APRDefaults).
These defaults can be still overridden in the usual way by setting the Prefix property of this service.https://gitlab.cern.ch/atlas/athena/-/merge_requests/61980Modularise in-file metadata configuration2024-03-25T13:23:57+01:00Maciej Pawel SzymanskiModularise in-file metadata configurationThis MR builds on !61368 to modularise the in-file metadata configuration. It requires each workflow to explicitly specify which metadata category should be ~~configured~~ created and added to the output stream (nothing ~~added~~ created...This MR builds on !61368 to modularise the in-file metadata configuration. It requires each workflow to explicitly specify which metadata category should be ~~configured~~ created and added to the output stream (nothing ~~added~~ created by default). By default, all the metadata found in the input will be propagated using the relevant tools (can be turned off with `propagateMetadataFromInput=False`).
I decided to keep a single function (although a bit renamed wrt !61368) for propagation and one for creation for each category ~~with an extra argument instead of splitting it into dedicated bits, but I'm open to suggestions. To keep some compatibility with event data configuration, the requested for creation metadata categories are passed as a list of strings, e.g. `["ByteStreamMetaData"]`~~
~~Note that I not yet incorporated !61962 (probably better to wait for merging).~~ [Edit: Done]https://gitlab.cern.ch/atlas/athena/-/merge_requests/70029Moved APRDefaults.h from POOLCore to RootUtils2024-03-22T08:41:02+01:00Marcin NowakMoved APRDefaults.h from POOLCore to RootUtilsThe new location makes the ROOT file structure naming information from this header accessible also in AnalysisBase.
I was also thinking about AthenaKernel as a good location for the header. I can move it there is people think it woul...The new location makes the ROOT file structure naming information from this header accessible also in AnalysisBase.
I was also thinking about AthenaKernel as a good location for the header. I can move it there is people think it would be a better idea.
Relates to ATEAM-969.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69909RNTuple reading improvements2024-03-21T14:31:05+01:00Marcin NowakRNTuple reading improvementsRemove redundant #includes
Cache RNTupleView objects so they are not recreated every read
Fix MetaReader so it understands some objects may be RNTuples and not TTreesRemove redundant #includes
Cache RNTupleView objects so they are not recreated every read
Fix MetaReader so it understands some objects may be RNTuples and not TTreeshttps://gitlab.cern.ch/atlas/athena/-/merge_requests/70003IOVDbFolder: fill object cache when reading data from CREST2024-03-21T11:43:10+01:00Vakhtang TsulaiaIOVDbFolder: fill object cache when reading data from CRESTThis change implements a mechanism for filling the `IOVDbFolder` cache with data
read from CREST. The mechanism works similarly to what we get when reading
data from COOL.
The current version does not yet support CREST folders with mult...This change implements a mechanism for filling the `IOVDbFolder` cache with data
read from CREST. The mechanism works similarly to what we get when reading
data from COOL.
The current version does not yet support CREST folders with multi-iov payloads.
The support for such folders will be added in future revisions.
Also added a new script to the `DataModelRunTests` package: `CondReadWriteCrest.py`.
This is not yet the final version. Currently, it is testing only CREST folders that
cross IOV boundaries during the job. Later we plan to add testing of the so-called
extensible folders, i.e. the folders that get new IOVs while the job is running.
CC the ADAM team members: @gunal @formica @mavogel @nozturk @aleksand @mmineevhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69964RootStorageSvc: Switch RNTuple attribute flags from bool to int2024-03-20T15:58:12+01:00Alaettin Serhan MeteRootStorageSvc: Switch RNTuple attribute flags from bool to intUsing `bool` seems to cause weird issues for some reason. Switch to using `int` instead. Eventually we can consider moving these from the Database to the Container but that's for later (and needs additional changes).
cc: @gemmerenUsing `bool` seems to cause weird issues for some reason. Switch to using `int` instead. Eventually we can consider moving these from the Database to the Container but that's for later (and needs additional changes).
cc: @gemmerenhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69975Cleanup small things left after a recent code refactoring2024-03-20T15:49:08+01:00Marcin NowakCleanup small things left after a recent code refactoringhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69933Compare app2024-03-19T18:18:13+01:00Evgeny AlexandrovCompare appCreate application for compare CREST and COOL data for selected folder and timestampCreate application for compare CREST and COOL data for selected folder and timestamphttps://gitlab.cern.ch/atlas/athena/-/merge_requests/68171Improve the condition for setting the address for proxy and releasing the add...2024-03-19T17:46:35+01:00Maciej Pawel SzymanskiImprove the condition for setting the address for proxy and releasing the address objectIt's a follow-up from https://gitlab.cern.ch/atlas/athena/-/merge_requests/67465/ and https://gitlab.cern.ch/atlas/athena/-/merge_requests/68101
It seems that in 0fc6d9d148f4832d1b058baf07c279ba0863b9b5 we had the ref count off by one w...It's a follow-up from https://gitlab.cern.ch/atlas/athena/-/merge_requests/67465/ and https://gitlab.cern.ch/atlas/athena/-/merge_requests/68101
It seems that in 0fc6d9d148f4832d1b058baf07c279ba0863b9b5 we had the ref count off by one which led to the memory leak reported in https://its.cern.ch/jira/browse/ATLASG-2660. This MR unconditionally releases the addresses, also for the new output objects.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69665Removed special prefix that was being added when naming RNTuples in APR2024-03-18T19:19:25+01:00Marcin NowakRemoved special prefix that was being added when naming RNTuples in APRAt the start RNTuple used the same names as corresponging TTrees, with "RNT:" prefix added to not confuse the 2. Now this prefix is removed. Also, RNTuples will now use different names as defined in POOLCore/APRDefaults - except Collecti...At the start RNTuple used the same names as corresponging TTrees, with "RNT:" prefix added to not confuse the 2. Now this prefix is removed. Also, RNTuples will now use different names as defined in POOLCore/APRDefaults - except CollectionTree, which is mentioned in 115 places in the code. Maybe something to address later.
Athena is now also able to read EventTags in RNTuple format (the previous implementation was not really doing it)https://gitlab.cern.ch/atlas/athena/-/merge_requests/69863Cleanup RNTCollection reading code2024-03-18T12:11:58+01:00Marcin NowakCleanup RNTCollection reading codeJust RNTuple Collection code - changes do not affect any current tests in Athena mainJust RNTuple Collection code - changes do not affect any current tests in Athena mainhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69829Rntuple.reading.through.views2024-03-15T10:12:51+01:00Marcin NowakRntuple.reading.through.viewsChanged RNTuple reading from using RFields to using RNTupleViews.
Based on draft MR !68810 (which is included)Changed RNTuple reading from using RFields to using RNTupleViews.
Based on draft MR !68810 (which is included)Marcin NowakMarcin Nowakhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69809main-coverity-IOVDbSvc2024-03-15T09:37:09+01:00Shaun Roemain-coverity-IOVDbSvcCoverity defect 19799: ensure vector index cannot be negative.Coverity defect 19799: ensure vector index cannot be negative.https://gitlab.cern.ch/atlas/athena/-/merge_requests/697692024-03-13: merge of 24.0 into main2024-03-14T11:00:03+01:00Walter Lampl2024-03-13: merge of 24.0 into mainThis sweep contains the following MRs:
* !69744 athena: unify legacy/CA interactive prompt ~Core
* !69747 Bugfix to ZDC pulse analysis to fix irreproducibility problem ~ForwardDetectors
* !69685 AthenaCommon: package and unit test cle...This sweep contains the following MRs:
* !69744 athena: unify legacy/CA interactive prompt ~Core
* !69747 Bugfix to ZDC pulse analysis to fix irreproducibility problem ~ForwardDetectors
* !69685 AthenaCommon: package and unit test cleanup ~Core
* !69698 Improve monitoring of non truth matched hadronic tau in TauCP PhysVal mon ~Analysis, ~Tau
* !69723 Clean up legacy TrkAlignment configs ~Tracking
* !69679 Add protection against bad vertices in InDetBeamSpotFinder ~DQ, ~InnerDetector
* !69742 Remove broken jobO files from DataQuality/GoodRunsListsUser ~DQ
* !69704 TrigEgammaMonitoring: naming cleanup ~DQ, ~Egamma, ~Reconstruction, ~Trigger
* !69687 TriggerTest: update HI test ~Trigger, ~changes-trigger-counts
* !69681 ATR-28771:PTF (prompt track frac) Trigger based on gFEX ~Trigger, ~TriggerMenu, ~changes-trigger-counts
* !69671 Fixes a typo in the HLT jet preselection monitoring of GNTau chains ~DQ, ~JetEtmiss, ~Trigger, ~TriggerJet, ~TriggerMenu
* !69720 Clean up legacy configs InDetAlignAlgs and Tools ~InnerDetector
* !69684 Clean up legacy config egammaRec package ~Derivation, ~Egamma, ~Reconstruction
* !69676 ZDC RPD centroid max ADC fix ~ForwardDetectors
* !69675 MuonConfig: remove usage of CARunner.py ~MuonSpectrometer
* !69696 AthenaMonitoring+DataQualityTools: remove AthMonitorCfgHelperOld ~Core, ~DQ
* !69706 GaudiSequencer: delete unused legacy job options ~Core
* !69693 athena.py: cleanup interactive mode command line parsing ~Core
* !69703 MM digi: only initialize thresholds db read handle key if conditions thresholds are used ~Digitization, ~MuonSpectrometer
* !69494 PyUtils: delete unused MetaReaderPeekerFull ~Tools
* !69596 Migrate EventIndex Producer to the Component Accumulator ~Database
* !69673 Clean up legacy Jet configs ~JetEtmiss, ~Reconstruction
* !69682 Clean up legacy MuonIdentification configs ~MuonSpectrometer, ~Reconstruction
* !69670 Clean up legacy configs in JetTagCalibration ~Analysis, ~BTagging, ~JetEtmiss, ~Trigger
* !69672 Clean up of legacy configs in HIGlobal ~JetEtmiss, ~Reconstruction, ~Trigger
* !69663 Fixing DQ egamma CP configuration for trigger aware mechanism ~DQ, ~Egamma, ~Reconstruction, ~Trigger
* !68496 Update L1Calo derivation/monitoring now that Offline Copy has been removed [ATR-28688] ~DQ, ~Derivation, ~L1Calo, ~Trigger
* !69331 Updating HI run3 test after L1MuonSim fix ~Triggerhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69764main-coverity-CoolConvUtilities2024-03-13T13:50:08+01:00Shaun Roemain-coverity-CoolConvUtilitiesSolving possible resource leak in case of error condition (coverity defects 19797, 19798 introduced 12 March), use unique_ptr wrapper for curl for resource managementSolving possible resource leak in case of error condition (coverity defects 19797, 19798 introduced 12 March), use unique_ptr wrapper for curl for resource managementhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69596Migrate EventIndex Producer to the Component Accumulator2024-03-13T12:00:26+01:00Francisco Javier Sanchez MartinezMigrate EventIndex Producer to the Component AccumulatorEventIndex Producer Transformation migrated to the Component Accumulator
This merge shouldn't affect anything but the EventIndexProducer packageEventIndex Producer Transformation migrated to the Component Accumulator
This merge shouldn't affect anything but the EventIndexProducer packagehttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69641Obtain smallest run which has not yet been released for Tier-0 bulk and...2024-03-13T12:00:25+01:00Marcelo VogelObtain smallest run which has not yet been released for Tier-0 bulk and...Obtain smallest run which has not yet been released for Tier-0 bulk and current online run from web serviceObtain smallest run which has not yet been released for Tier-0 bulk and current online run from web servicehttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69705Manual sweep of !69641 from main to 24.0: Obtain smallest run which has not y...2024-03-13T12:00:23+01:00John Derek ChapmanManual sweep of !69641 from main to 24.0: Obtain smallest run which has not yet been released for Tier-0 bulk and current online run from web serviceManual sweep of !69641 from main to 24.0: Obtain smallest run which has not yet been released for Tier-0 bulk and current online run from web service
See merge request atlas/athena!69641
(cherry picked from commit cd5c691043cf18cdcfd00...Manual sweep of !69641 from main to 24.0: Obtain smallest run which has not yet been released for Tier-0 bulk and current online run from web service
See merge request atlas/athena!69641
(cherry picked from commit cd5c691043cf18cdcfd003f8ae4a3604db4f426d)
7aa8d7f6 Obtain smallest run which has not yet been released for Tier-0 bulk and...
81276b13 fixed warning in compilationhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/69758RDBAccessSvc: Fix cppcheck warnings.2024-03-13T10:59:05+01:00Scott SnyderRDBAccessSvc: Fix cppcheck warnings.Pass strings by const reference, not by value.
Copying value in range-for.Pass strings by const reference, not by value.
Copying value in range-for.https://gitlab.cern.ch/atlas/athena/-/merge_requests/69749IOVDbFolder: introduced new private function fetchCrestObjects()2024-03-13T10:43:50+01:00Vakhtang TsulaiaIOVDbFolder: introduced new private function fetchCrestObjects()As the name suggests, this function contains code that retrieves data from
the CREST database and stores it in the objects of type `BasicFolder`.
The changes in this MR are mostly moving the existing code around in preparation
for addin...As the name suggests, this function contains code that retrieves data from
the CREST database and stores it in the objects of type `BasicFolder`.
The changes in this MR are mostly moving the existing code around in preparation
for adding support for reading multiple IOVs from the CREST database in a single
Athena job. Also, there are a few shortcuts/temporary solutions that are expected
to be reworked in the next MR-s