athena merge requestshttps://gitlab.cern.ch/atlas/athena/-/merge_requests2023-04-26T13:07:03+02:00https://gitlab.cern.ch/atlas/athena/-/merge_requests/62625EventPrimitives : Purely formatting changes follow up2023-04-26T13:07:03+02:00Christos Anastopouloschristos.anastopoulos@cern.chEventPrimitives : Purely formatting changes follow upEventPrimitives : Purely formatting changes follow up
to !62189 !62565 !62571 !62566EventPrimitives : Purely formatting changes follow up
to !62189 !62565 !62571 !62566https://gitlab.cern.ch/atlas/athena/-/merge_requests/62566EventPrimitivesHelpers : Rm seemingly unused isSymmetric2023-04-25T10:28:19+02:00Christos Anastopouloschristos.anastopoulos@cern.chEventPrimitivesHelpers : Rm seemingly unused isSymmetricEventPrimitivesHelpers : Rm seemingly unused isSymmetricEventPrimitivesHelpers : Rm seemingly unused isSymmetrichttps://gitlab.cern.ch/atlas/athena/-/merge_requests/58980EventPrimitivesHelpers.h cleanup2022-12-05T21:42:35+01:00Christos Anastopouloschristos.anastopoulos@cern.chEventPrimitivesHelpers.h cleanupEventPrimitivesHelpers.h cleanup
- remove unused ``largestDifference``
- use ``constexpr`` rather tempalte metaprogramming for ``CalculateCompressedSize``
- use Eigen ``isNaN`` ``allFinite`` to implement the ``saneVector`` without expli...EventPrimitivesHelpers.h cleanup
- remove unused ``largestDifference``
- use ``constexpr`` rather tempalte metaprogramming for ``CalculateCompressedSize``
- use Eigen ``isNaN`` ``allFinite`` to implement the ``saneVector`` without explicit loop.https://gitlab.cern.ch/atlas/athena/-/merge_requests/61470EventPrimitivesHelpers compress, reserve also in the 2nd overload2023-03-15T15:42:06+01:00Christos Anastopouloschristos.anastopoulos@cern.chEventPrimitivesHelpers compress, reserve also in the 2nd overloadEventPrimitivesHelpers compress, reserve also in the 2nd overloadEventPrimitivesHelpers compress, reserve also in the 2nd overloadhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/35891EventPrimitives: Fix cppcheck warnings.2020-08-27T03:04:16+02:00Scott SnyderEventPrimitives: Fix cppcheck warnings.Pass class instances by const reference, not by value.Pass class instances by const reference, not by value.https://gitlab.cern.ch/atlas/athena/-/merge_requests/62571EventPrimitives : Add fixed capacity dynamic typedefs. Add test. Fix bug2023-04-26T01:17:39+02:00Christos Anastopouloschristos.anastopoulos@cern.chEventPrimitives : Add fixed capacity dynamic typedefs. Add test. Fix bugEventPrimitives :
- Add test for the Amg definitions.
- Fix Eigen assertion on AmgRowVector (seems unused) uncovered by the test.
- Add fixed capacity dynamic typedefs
The last one is a number of dyn allocation vs space trade off.
...EventPrimitives :
- Add test for the Amg definitions.
- Fix Eigen assertion on AmgRowVector (seems unused) uncovered by the test.
- Add fixed capacity dynamic typedefs
The last one is a number of dyn allocation vs space trade off.
- There are cases where we know exactly the number and we use the fixed size
- Then there are cases or largish matrices we can not guess the sizes or anyhow quite largish.
- Then there cases we know the max capacity we need (i.e Local Parameters 5 , Measurements covariance 3) even if we do not know the exact number. So it might be interesting to have for these ones where the max is small.
This is like a "small"/ "fixed capacity" vector but for "Eigen"
![Screenshot_2023-04-26_at_00.16.17](/uploads/e22333a024058fdbcd3445004dc66d3b/Screenshot_2023-04-26_at_00.16.17.png)
ping @abarton @smh for the last onehttps://gitlab.cern.ch/atlas/athena/-/merge_requests/46229EventOverlayJobTransforms+TrigT1EventAthenaPool: Delete obsolete LVL1 job opt...2021-09-03T03:03:55+02:00Frank WinklmeierEventOverlayJobTransforms+TrigT1EventAthenaPool: Delete obsolete LVL1 job optionsDelete unused L1 job options:
- `Event/EventOverlay/EventOverlayJobTransforms/share/L1Signal_jobOptions.py`
- `Trigger/TrigT1/TrigT1EventAthenaPool/share/jobOptions_ReadLVL1_Pool.py`Delete unused L1 job options:
- `Event/EventOverlay/EventOverlayJobTransforms/share/L1Signal_jobOptions.py`
- `Trigger/TrigT1/TrigT1EventAthenaPool/share/jobOptions_ReadLVL1_Pool.py`https://gitlab.cern.ch/atlas/athena/-/merge_requests/47527EventOverlayJobTransforms: initialize ConfigFlags2021-10-25T15:48:09+02:00Frank WinklmeierEventOverlayJobTransforms: initialize ConfigFlagsInitialize `ConfigFlags` (required for Trigger).
Closes ATLASSIM-5459.Initialize `ConfigFlags` (required for Trigger).
Closes ATLASSIM-5459.https://gitlab.cern.ch/atlas/athena/-/merge_requests/35980EventOverlay: cmake cleanup, enable flake82020-09-01T03:03:21+02:00Frank WinklmeierEventOverlay: cmake cleanup, enable flake8Some cmake cleanup of EventOverlay packages, enable flake8 and make code compliant.Some cmake cleanup of EventOverlay packages, enable flake8 and make code compliant.https://gitlab.cern.ch/atlas/athena/-/merge_requests/29261Event: Migrate EXTRA_PATTERNS to LOG_IGNORE_PATTERN2020-01-08T03:02:11+01:00Frank WinklmeierEvent: Migrate EXTRA_PATTERNS to LOG_IGNORE_PATTERN`EXTRA_PATTERNS` is deprecated and replaced by `LOG_IGNORE_PATTERN`. Migrate all remaining uses for packages in `Event/`.`EXTRA_PATTERNS` is deprecated and replaced by `LOG_IGNORE_PATTERN`. Migrate all remaining uses for packages in `Event/`.https://gitlab.cern.ch/atlas/athena/-/merge_requests/26953EventLoop Worker: Fix a corner case with empty files2019-10-08T03:02:40+02:00Tadej Novaktadej.novak@cern.chEventLoop Worker: Fix a corner case with empty filesIn a very specific case tools and algs would not get initialised in EventLoop:
- max events is set
- file is empty
This MR fixes this.In a very specific case tools and algs would not get initialised in EventLoop:
- max events is set
- file is empty
This MR fixes this.https://gitlab.cern.ch/atlas/athena/-/merge_requests/37282EventLoopGrid: minor changes for R22 AnalysisBase2020-10-16T03:02:19+02:00Bertrand Martin Dit LatourEventLoopGrid: minor changes for R22 AnalysisBaseHello,
This MR is tentatively fixing 2 EventLoopGrid-related errors seen when submitting GRID jobs with AnalysisBase 22.2.0:
```
TUnixSystem::GetFromPipe ERROR command "python -c "from pandatools import PsubUtils;print PsubUtils.getN...Hello,
This MR is tentatively fixing 2 EventLoopGrid-related errors seen when submitting GRID jobs with AnalysisBase 22.2.0:
```
TUnixSystem::GetFromPipe ERROR command "python -c "from pandatools import PsubUtils;print PsubUtils.getNickname();" 2>/dev/null" returned 256
```
and
```
File "<string>", line 130, in ELG_prun
File "/cvmfs/atlas.cern.ch/repo/sw/software/22.2/AnalysisBaseExternals/22.2.0/InstallArea/x86_64-centos7-gcc8-opt/lib/python3.7/re.py", line 223, in findall
return _compile(pattern, flags).findall(string)
TypeError: cannot use a string pattern on a bytes-like object
TypeError: an integer is required (got type NoneType)
```
The job submission is "successful" with the proposed changes (well, jobs crash at runtime, but likely due to an unrelated reason...).
Tagging @krumnack for verification, @adbailey and @xiaozhon for information.
Cheers,
Bertrandhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/29025EventLoopGrid: fix flake8 errors2019-12-19T03:02:52+01:00Tadej Novaktadej.novak@cern.chEventLoopGrid: fix flake8 errorsFix some `flake8` errors in `EventLoopGrid` which might also allow running with python 3.Fix some `flake8` errors in `EventLoopGrid` which might also allow running with python 3.https://gitlab.cern.ch/atlas/athena/-/merge_requests/60541EventLoopGrid: Change ELG_jediState.py return type to int to avoid binding pr...2023-02-13T14:00:40+01:00Filip NechanskyEventLoopGrid: Change ELG_jediState.py return type to int to avoid binding problemsSee JIRA ATLASG-2449 , using `TPython::Eval` with string as return type
causes issues with python3.7+. This is used in `PrunDriver.cxx`
to get JEDI task status for wait/retrieve. Changing type to int based on
enum defined both in `ELG_je...See JIRA ATLASG-2449 , using `TPython::Eval` with string as return type
causes issues with python3.7+. This is used in `PrunDriver.cxx`
to get JEDI task status for wait/retrieve. Changing type to int based on
enum defined both in `ELG_jediState.py` and `PrunDriver.cxx` to solve the issue.
I tried to take advantage of existing Enum in PrunDriver.cxx,
not sure if it is the best way.
Also cleaning-up some commented lines.
Note on return of the ELG_jediState:
Task states not used explicitly by PrunDriver.cxx were still added
to nc_ELG_state_details meta information. I had a look and in whole athena
this is used in PrunDriver and only for print-outs so I think its okay
to just say "task status other than done/finished/failed/running" or
something like that?
Actually covering all the cases with enum would be messy and is probably not necessary.
(there is around twenty possible states, see here:
https://github.com/PanDAWMS/panda-bigmon-core/blob/master/core/constants.py#L85 )
Tagging @akraszna and @krumnackhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/66424EventLoop: fix gcc13 compilation2023-10-11T22:43:47+02:00Johannes ElmsheuserEventLoop: fix gcc13 compilationAdd missing `#include <cstdint>` in `EventLoop` package for gcc13 that was missed in MR https://gitlab.cern.ch/atlas/athena/-/merge_requests/65841Add missing `#include <cstdint>` in `EventLoop` package for gcc13 that was missed in MR https://gitlab.cern.ch/atlas/athena/-/merge_requests/65841https://gitlab.cern.ch/atlas/athena/-/merge_requests/25973EventLoop Driver-Manager update2019-08-21T19:30:00+02:00Nils Erik KrumnackEventLoop Driver-Manager updateThis is a fairly large EventLoop update/restructuring, to be followed by more EventLoop updates soon, but right now I just want to get this update in, so that there isn't too much to review at once. This is already a fair amount of chan...This is a fairly large EventLoop update/restructuring, to be followed by more EventLoop updates soon, but right now I just want to get this update in, so that there isn't too much to review at once. This is already a fair amount of changes to review, and it will hopefully make future updates smaller and more focused. As such the main thing this update does is move code around and change how things get called and stored internally, but it does very little beyond that.
The main change is that I introduce a class interface `Manager` which is similar in concept to `Module`, but for work being done on the submission node, instead of the worker node. Even though it is essentially a pluggable infrastructure, but it is meant for expert use only, not as something that the user would directly instantiate or configure himself. As such most of these structures are open to change as need arises, as user code shouldn't directly depend on them.
The general way this works is that at the beginning of each task/operation a series of managers get loaded based on the task at hand. Then we go through a series of steps (ManagerStep) and at each step each manager gets the option to do what it needs to do at that step. That way the different managers can operate more or less independently and have their code interleave while avoiding a fair amount of spaghetti code. Any data that needs to be communicated between different managers or steps gets placed into the ManagerData structure, meaning individual managers don't need to know about each other directly.https://gitlab.cern.ch/atlas/athena/-/merge_requests/28561EventLoop Driver internal restructuring2019-12-07T03:03:14+01:00Nils Erik KrumnackEventLoop Driver internal restructuringSo, this had been meant to contain more stuff, but realistically I haven't even looked at in a while and may as well get it merged now before proceeding with further developments. This should hopefully be uncontroversial. For the most ...So, this had been meant to contain more stuff, but realistically I haven't even looked at in a while and may as well get it merged now before proceeding with further developments. This should hopefully be uncontroversial. For the most part this is just moving some code around in preparation of splitting the `BatchDriver` up into multiple `Manager` objects, each with its own responsibility and hopefully each testable independently.https://gitlab.cern.ch/atlas/athena/-/merge_requests/57783EventLoop: add support for 'avoidVP'2022-10-25T17:23:46+02:00Tadej Novaktadej.novak@cern.chEventLoop: add support for 'avoidVP'EventLoop: add support for 'avoidVP' which disables virtual placement queues.
/cc @krumnackEventLoop: add support for 'avoidVP' which disables virtual placement queues.
/cc @krumnackhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/53986EventLoop: add possiblity to execute final checks on job output files just be...2022-06-14T16:04:32+02:00Romain BouquetEventLoop: add possiblity to execute final checks on job output files just before job completionThis MR enables to add the possibilty for the user to perform some checks/actions on the output files produced by interactive (`DirectDriver`) or batch (`CondorDriver` and so on) jobs just before the jobs finish successfully.
The output...This MR enables to add the possibilty for the user to perform some checks/actions on the output files produced by interactive (`DirectDriver`) or batch (`CondorDriver` and so on) jobs just before the jobs finish successfully.
The output files are all already closed when performing the check.
The command/executable name or path has to be set by the user using the flag `EL::Job::optWorkerPostClosedOutputsExecutable`
```
job->options()->setString(EL::Job::optWorkerPostClosedOutputsExecutable, NameExecutable);
```
For instance a really simple "executable" can be the `echo` command
```
job->options()->setString(EL::Job::optWorkerPostClosedOutputsExecutable, "echo");
```
For each output of the job, the executable will be called and will be provided by `EventLoop` the 2 following arguments:
* the output label name (e.g. `HISTOGRAM` and so on)
* the associated output file name
This optionnal final command line can be useful for instance to check that the output files produced by the job are not corrupted providing a script that check root files and their content
(see comment https://gitlab.cern.ch/atlas/athena/-/merge_requests/53986#note_5637319 below).
I tested successfully the MR with the `CxAODFramework` and it is working as intended both with the `DirectDriver` and the `CondorDriver`
Tagging @krumnack and @akraszna on this MRhttps://gitlab.cern.ch/atlas/athena/-/merge_requests/27574EventLoop: Add additional grid flags2019-10-30T03:03:03+01:00Tadej Novaktadej.novak@cern.chEventLoop: Add additional grid flagsAdding additional grid flags:
- `cpuTimePerEvent`
- `maxWalltime`Adding additional grid flags:
- `cpuTimePerEvent`
- `maxWalltime`