MC Job Options issueshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues2022-05-17T16:03:17+02:00https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/172EVNTtoEVNT and ECM: fails to download input EVNT in CI2022-05-17T16:03:17+02:00Matthew GignacEVNTtoEVNT and ECM: fails to download input EVNT in CIHi,
A few of the CI jobs fail for some E2E jOs in mc21: https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/jobs/21697381
Looks like the $ECMENERGY environment variable is not correctly set. In this type of transform, the ECM is no...Hi,
A few of the CI jobs fail for some E2E jOs in mc21: https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/jobs/21697381
Looks like the $ECMENERGY environment variable is not correctly set. In this type of transform, the ECM is not required and not written into the log, which is probably the underlying issue. Any ideas on how to detect this?
As an aside, I noticed that the log.generate.short is claiming ecmEnergy of 13000 GeV, though the transform was run with 13600 TeV (not that it matters...). I guess the 13000 is a default somewhere, which maybe is causing some issues?
Cheers,
MatthewSpyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/175Follow-up from "Allow to run mc21_13p6 E2E jobs in CI"2022-05-17T11:08:25+02:00Spyros ArgyropoulosFollow-up from "Allow to run mc21_13p6 E2E jobs in CI"Use grep to extract scope of EVNT files to download in E2E jobs instead of hardcoding
The following discussion from !1831 should be addressed:
- [ ] @cgutscho started a [discussion](https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions...Use grep to extract scope of EVNT files to download in E2E jobs instead of hardcoding
The following discussion from !1831 should be addressed:
- [ ] @cgutscho started a [discussion](https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/merge_requests/1831#note_5560365): (+8 comments)
> The energy might not be in the log, but it is in the file, no? If we grep the log for the release and set it up we could run `checkMetaSG.py` on the file, e.g.
> ```
> checkMetaSG EVNT.24960681._002005.pool.root.1 | grep beam_energy
> beam_energy | [6500000.0]
> | beam_energy | int | 6500000
> ```
> which is a bit more faff but should work "in general"?https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/48Dynamically setting allowed to fail jobs2022-05-17T11:05:37+02:00Spyros ArgyropoulosDynamically setting allowed to fail jobsExpected to come with gitlab 12.6: https://gitlab.com/gitlab-org/gitlab/issues/16733
An example usage in our case would be that CI jobs which don't find anything to check dynamically set an "failed but allowed to fail" status -> shows ...Expected to come with gitlab 12.6: https://gitlab.com/gitlab-org/gitlab/issues/16733
An example usage in our case would be that CI jobs which don't find anything to check dynamically set an "failed but allowed to fail" status -> shows up as a WARNING in the pipeline, so that people who approve MRs know that they have to investigate further.FutureSpyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/173logParser crash when using gridpacks2022-05-17T10:36:18+02:00Dominic HirschbuehllogParser crash when using gridpacksWhen running the logParser for a log.generate which used a gridpack, it crashes with:
"/afs/cern.ch/work/d/dhirsch/stop13TeV/mcjoboptions/scripts/logParser.py", line 368, in powhegChecks
if not glob.glob(f"{os.path.dirname(logFile)}/...When running the logParser for a log.generate which used a gridpack, it crashes with:
"/afs/cern.ch/work/d/dhirsch/stop13TeV/mcjoboptions/scripts/logParser.py", line 368, in powhegChecks
if not glob.glob(f"{os.path.dirname(logFile)}/mc_*TeV.*.GRID.tar.gz"):
File "/cvmfs/atlas.cern.ch/repo/sw/software/22.6/sw/lcg/releases/LCG_101_ATLAS_18/Python/3.9.6/x86_64-centos7-gcc11-opt/lib/python3.9/posixpath.py", line 152, in dirname
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not list
The problem is that the function
powhegChecks(logFile)
expects a filename, but gets the file content.
A similar problem should happen for Madgraph, since the function is defined properly
with def madgraphChecks(logContent), but the variable logFile used to check the gridpack
is nowwhere defined.
Cheers
DominicSpyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/159Allow run_athena to run 13p6 TeV2022-05-10T11:04:58+02:00Spyros ArgyropoulosAllow run_athena to run 13p6 TeVCurrently only `mc15_13TeV` and `mc16_13TeV` external LHE files are handled in run_athena. Should be extended to `mc16_13p6TeV` and `mc21_13p6TeV`.
Check if we can automate this so that we don't need to hard-code things as done now.Currently only `mc15_13TeV` and `mc16_13TeV` external LHE files are handled in run_athena. Should be extended to `mc16_13p6TeV` and `mc21_13p6TeV`.
Check if we can automate this so that we don't need to hard-code things as done now.Spyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/171CI download to check for grid input LHEs with mc21_13p6TeV scope2022-05-10T07:28:54+02:00Jan KretzschmarCI download to check for grid input LHEs with mc21_13p6TeV scopeIn https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/merge_requests/1824 I hit the problem, that Grid input LHEs are always trying to be downloaded with mc15_13TeV or mc16_13TeV scope, I think the origin of this is here:
https://gi...In https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/merge_requests/1824 I hit the problem, that Grid input LHEs are always trying to be downloaded with mc15_13TeV or mc16_13TeV scope, I think the origin of this is here:
https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/blob/master/scripts/run_athena.sh#L115-138
First, it would probably be preferable, if we get the CME string from the CME value rather than fixing it to 13 TeV.
Second, we now also need to allow mc21
As shortest patch we could just keep extending the list of allowed scopes, here mc21_13p6TeV being the obvious one to add immediately.
Many thanks, Janhttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/170New commit script failing for directories with arbitrary names2022-05-06T12:59:13+02:00Spyros ArgyropoulosNew commit script failing for directories with arbitrary names![Screenshot_2022-05-06_at_11.16.03](/uploads/a87f3ca9c98c1d19745c55cc15380114/Screenshot_2022-05-06_at_11.16.03.png)
This is because of:
```
newDSID += parseDSIDList(args.DSID)
```
seems to be adding an empty item in the list![Screenshot_2022-05-06_at_11.16.03](/uploads/a87f3ca9c98c1d19745c55cc15380114/Screenshot_2022-05-06_at_11.16.03.png)
This is because of:
```
newDSID += parseDSIDList(args.DSID)
```
seems to be adding an empty item in the listChristian GutschowChristian Gutschowhttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/158Handle purple list in logParser2022-05-05T16:15:51+02:00Spyros ArgyropoulosHandle purple list in logParserTBD if this is something that we can catch and report e.g. in the parser
See !1673TBD if this is something that we can catch and report e.g. in the parser
See !1673Spyros ArgyropoulosSpyros Argyropoulos2022-04-01https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/168logParser unsupported locale setting2022-04-21T20:53:54+02:00Spyros ArgyropouloslogParser unsupported locale settingFull error message:
INFO: New DSID directory: 100xxx/100001 ...
OK: log.generate file found.
Traceback (most recent call last):
File "scripts/logParser.py", line 8, in <module>
locale.setlocale(locale.LC_CTYPE, f'{lang}.UTF-8'...Full error message:
INFO: New DSID directory: 100xxx/100001 ...
OK: log.generate file found.
Traceback (most recent call last):
File "scripts/logParser.py", line 8, in <module>
locale.setlocale(locale.LC_CTYPE, f'{lang}.UTF-8')
File "/usr/lib64/python3.6/locale.py", line 598, in setlocale
return _setlocale(category, locale)
locale.Error: unsupported locale setting
ERROR: logParser run failed.
Need output of
- locale
- locale -a
- env
- which machine you are running on
@yanlin @nishuSpyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/166Add checks for python2/3 compatibility of jO2022-04-21T17:21:24+02:00Spyros ArgyropoulosAdd checks for python2/3 compatibility of jOan example DSID is 830099: https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/blob/master/830xxx/830099/mc.H7EG_jetjet_72_Cluster_JZ1.py
R21: 21.6.85
R22: You can try 22.6.13 (later releases have issues with EvtGen_i — should be ...an example DSID is 830099: https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/blob/master/830xxx/830099/mc.H7EG_jetjet_72_Cluster_JZ1.py
R21: 21.6.85
R22: You can try 22.6.13 (later releases have issues with EvtGen_i — should be fixed soon).Spyros ArgyropoulosSpyros Argyropoulos2022-04-25https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/167Fix logParser bug when running in CI2022-04-20T22:28:20+02:00Spyros ArgyropoulosFix logParser bug when running in CI![Screenshot_2022-04-20_at_21.08.27](/uploads/ed6760a1cbe977212c0904faf484fd7a/Screenshot_2022-04-20_at_21.08.27.png)![Screenshot_2022-04-20_at_21.08.27](/uploads/ed6760a1cbe977212c0904faf484fd7a/Screenshot_2022-04-20_at_21.08.27.png)Spyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/165Decoding issue in commit script / logParser ?2022-04-10T17:20:38+02:00Spyros ArgyropoulosDecoding issue in commit script / logParser ?![Screenshot_2022-04-10_at_11.44.33](/uploads/8e97d70a5350e335d2005d379cc8d393/Screenshot_2022-04-10_at_11.44.33.png)
appears with python 3.6.8 (same as on lxplus) - content at /afs/cern.ch/user/a/aivina/public/JO_TEST_DIR
Not reproduc...![Screenshot_2022-04-10_at_11.44.33](/uploads/8e97d70a5350e335d2005d379cc8d393/Screenshot_2022-04-10_at_11.44.33.png)
appears with python 3.6.8 (same as on lxplus) - content at /afs/cern.ch/user/a/aivina/public/JO_TEST_DIR
Not reproducible on lxplus or locally
@aivinaSpyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/160check_unique_controlFile too loose2022-04-10T13:51:04+02:00Spyros Argyropouloscheck_unique_controlFile too looseI notice in !1694 that due to mistakes in previous MRs there are spurious errors, check https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/jobs/20185882
This is because https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/blob/...I notice in !1694 that due to mistakes in previous MRs there are spurious errors, check https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/jobs/20185882
This is because https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/blob/master/507xxx/507035/MadGraphControl_Py8EG_2HDMa_monoH_common.py is a file rather than a link.
This means that all the future MRs that will point to this control file (and it's almost certain we'll have more) will all have the same spurious errors.
In fact this is going to happen for all files that should be links and weren't so the situation is going to become worse with time. I would therefore propose to make the test stricter, i.e. we should not allow two physical files with the same name and the same content. If the content is different then the name should be adjusted. And we should not allow the CI job to fail.
Is there any objection @jkretz @mgignac @cgutscho ?Spyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/164Some improvements for commit script2022-04-10T13:43:26+02:00Spyros ArgyropoulosSome improvements for commit script- The removal of the `.sh` extension means that 1) the OS thinks it's an executable and will try to run it instead of opening it with an editor 2) the editor doesn't recognise that it's python so there is no highlighting etc. Generally I...- The removal of the `.sh` extension means that 1) the OS thinks it's an executable and will try to run it instead of opening it with an editor 2) the editor doesn't recognise that it's python so there is no highlighting etc. Generally I see no reason for having removed the extension and I think we should put it back (also same for other scripts where extensions have been removed)
- [L164](https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/blob/master/scripts/commit_new_dsid#L164) seems to be a debug statement which was forgotten to be removed, I don't see the reason for having a long dump of dsids that are going to be processed and this is more clearly visible from the following printout statements which are clearer
![Screenshot_2022-04-10_at_11.12.20](/uploads/eeabafaa609b285bd55e375886126cd8/Screenshot_2022-04-10_at_11.12.20.png)Christian GutschowChristian Gutschowhttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/163ecmEnergy in logParser fixed to 13000 GeV?2022-04-06T11:25:01+02:00Jan KretzschmarecmEnergy in logParser fixed to 13000 GeV?I have the impression that logParser fixes the ecmEnergy to 13000 GeV in the log.generate.short irrespective what the given log file gives
https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/blob/master/scripts/logParser.py#L471
Thi...I have the impression that logParser fixes the ecmEnergy to 13000 GeV in the log.generate.short irrespective what the given log file gives
https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/blob/master/scripts/logParser.py#L471
This impacted https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/merge_requests/1748
(I can fix that MR myself now)https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/162Job Failed #208263692022-04-04T13:40:29+02:00Christian GutschowJob Failed #20826369Job [#20826369](https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/jobs/20826369) failed for 06fa2b7f0bbd6b1dd30f9dd4e073b3b2206ff634:Job [#20826369](https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/jobs/20826369) failed for 06fa2b7f0bbd6b1dd30f9dd4e073b3b2206ff634:https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/147Support for 7-digit DSIDs2022-04-01T15:44:37+02:00Christian GutschowSupport for 7-digit DSIDsDraft proposal discussed in [GIT meeting](https://indico.cern.ch/event/1061701/#23-support-for-7-digit-dsids-i):
* Stick to current repo, look into sparse checkouts in case response time becomes unsatisfactory
* keep 6-digit structure ...Draft proposal discussed in [GIT meeting](https://indico.cern.ch/event/1061701/#23-support-for-7-digit-dsids-i):
* Stick to current repo, look into sparse checkouts in case response time becomes unsatisfactory
* keep 6-digit structure as is, group 7-digits in millions, then in thousands
* Retain grouping by generator (?) for 7-digits (`15xxxxx` Madgraph, `16xxxxx` Powheg etc.)
* can use `10xxxxx` - `14xxxxx` for large BSM scans for instance
* keep using 6-digits for now, but introduce simple mechanism to disable 6-digit booking altogether at some pointChristian GutschowChristian Gutschowhttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/161logParser expects randomSeed arg even for E2E jobs2022-03-25T14:51:01+01:00Jan KretzschmarlogParser expects randomSeed arg even for E2E jobsHi,
in https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/merge_requests/1725 I think I 'discovered' an issue that the logParser requires the argument 'randomSeed' to be present also for EVNT-to-EVNT (afterburner filtering) jobs. T...Hi,
in https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/merge_requests/1725 I think I 'discovered' an issue that the logParser requires the argument 'randomSeed' to be present also for EVNT-to-EVNT (afterburner filtering) jobs. The problem is: even if you give the argument, it is not printed to the log file. Not sure if this is an issue of the transform, but in any case these jobs are not really supposed to need ranfdom numbers in most case.
Thanks, JanSpyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/149NLHE check not working well in CI2022-03-12T09:33:06+01:00Christian GutschowNLHE check not working well in CIAs seen in !1304.
Maybe this check could be based on the extrapolated number of events?
> For a sufficiently large `nEventsPerJob` the 10% is more conservative than 4 sigma, so no harm done, albeit a bit more inefficient. For low `nEve...As seen in !1304.
Maybe this check could be based on the extrapolated number of events?
> For a sufficiently large `nEventsPerJob` the 10% is more conservative than 4 sigma, so no harm done, albeit a bit more inefficient. For low `nEventsPerJob` 4sigma is more conservative and people would probably anyway notice this in their local runs and choose a larger margin (e.g. factors of 25 rather than 1.1). So arguably the 4sigma thing is probably not needed in practice, provided people actually test their setups locally...
Maybe @jkretz should indicate what his preference is for the long run, but at the moment this new rule is causing the CI to crash frequently for no good reason, so we need to patch somehow.
Another option is to remove the 4sigma rule and go back to the previous check with 10% extra events, or that we have the 4sigma rule only for jobs that have a small number of requested output events (although it's actually impossible to know if people just test with a low number of events)https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/63logParser: document warnings and errors2022-03-12T09:24:59+01:00Spyros ArgyropouloslogParser: document warnings and errorsI've been thinking that it would help with the approval process to document somewhere (I'd say in the README - to keep everything together) all the cases where `logParser` throws WARNING or ERROR, so that approvers can easily skim throug...I've been thinking that it would help with the approval process to document somewhere (I'd say in the README - to keep everything together) all the cases where `logParser` throws WARNING or ERROR, so that approvers can easily skim through the CI job output and understand if something should be followed up on.Future