MC Job Options issueshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues2024-03-15T15:47:38+01:00https://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/228Add functionality to allow creating log.generate.short without running the co...2024-03-15T15:47:38+01:00Spyros ArgyropoulosAdd functionality to allow creating log.generate.short without running the commit scriptThis can either be done by supplying an extra script or updating the commit script to handle such cases (might be better).
Standalone code is provided below.
The code below works with
`python3 makeLogShort.py ./path/to/log.generate ./p...This can either be done by supplying an extra script or updating the commit script to handle such cases (might be better).
Standalone code is provided below.
The code below works with
`python3 makeLogShort.py ./path/to/log.generate ./path/to/mc.XXX.py`
```python
#! /usr/bin/env python3
import re,sys, subprocess, shlex
def bashExec(cmd, debug = False, checkStatus = False):
if debug:
print(cmd)
return ''
p = subprocess.run(shlex.split(cmd), stdout=subprocess.PIPE)
if checkStatus:
return p.stdout.decode('utf-8').strip(), p.returncode
return p.stdout.decode('utf-8').strip()
tags = [
'estimated CPU for CI job',
'using release',
'ecmEnergy',
'inputGeneratorFile',
'inputEVNT_PreFile',
'randomSeed',
'EVNT to EVNT',
'LHEonly',
'ATHENA_PROC_NUMBER',
'platform'
]
logparser_cmd = f'python3 scripts/logParser.py -i {sys.argv[1]} -j {sys.argv[2]}'
LPO, rc = bashExec(logparser_cmd, checkStatus = True)
# Remove colours from logParserOutput and extract necessary information
LPO_noCols = re.sub(r'\x1b\[[0-9]{1,3}[mK]', '', LPO)
greps = sum((re.compile(f'.*{tag}.*').findall(LPO_noCols)[:1] for tag in tags), [])
# Pipe information into log.generate.short
with open(f'log.generate.short', 'w') as logShort:
logShort.write('\n'.join(greps))
```Spyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/216Allow rucio to download files stored on datadisk2023-09-29T15:48:41+02:00Spyros ArgyropoulosAllow rucio to download files stored on datadiskIn !2641 the CI failed because the input LHE was stored on datadisk
```
rucio list-file-replicas mc23_13p6TeV:TXT.33690939._000003.tar.gz.1
| mc23_13p6TeV | TXT.33690939._000003.tar.gz.1 | 1.345 MB | 6ed2753b | FZK-LCG2_DATADISK: roo...In !2641 the CI failed because the input LHE was stored on datadisk
```
rucio list-file-replicas mc23_13p6TeV:TXT.33690939._000003.tar.gz.1
| mc23_13p6TeV | TXT.33690939._000003.tar.gz.1 | 1.345 MB | 6ed2753b | FZK-LCG2_DATADISK: root://atlasxrootd-kit.gridka.de:1094//pnfs/gridka.de/atlas/disk-only/atlasdatadisk/rucio/mc23_13p6TeV/c5/61/TXT.33690939._000003.tar.gz.1 |
```
Misha said this is not an issue for prodsys because `rucio download` can be used from everywhere.
Apparently we can request special access for `mcgensvc` from mailto:atlas-adc-ddm-support@cern.ch
(we should keep Misha in cc)
We probably also need to request a new grid certificate for `mcgensvc`Spyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/226Automatically handle TestHepMC efficiency warnings and errors2024-02-04T12:02:36+01:00Spyros ArgyropoulosAutomatically handle TestHepMC efficiency warnings and errorsOnce https://gitlab.cern.ch/atlas/athena/-/merge_requests/68566 is merged we should change the hardcoded thresholds in `logParser` and allow the check to be performed dynamically.Once https://gitlab.cern.ch/atlas/athena/-/merge_requests/68566 is merged we should change the hardcoded thresholds in `logParser` and allow the check to be performed dynamically.Spyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/215Catch cases where MadSpin uses BR that don't add up to 12023-09-23T06:51:44+02:00Spyros ArgyropoulosCatch cases where MadSpin uses BR that don't add up to 1From @dhirsch
> today, we have discovered a bug in the aMC@NLO ttbar samples (see discussion on the top reco mailing list) It turned out that the branching rations for the W boson decay in the param_card didn't add up to one. I.e. the i...From @dhirsch
> today, we have discovered a bug in the aMC@NLO ttbar samples (see discussion on the top reco mailing list) It turned out that the branching rations for the W boson decay in the param_card didn't add up to one. I.e. the is a typo in two values. However MadSpin basically reported this problem with this line: INFO: Branching ratio to allowed decays: 0.984064
> We also have the case of samples generated with gridpacks. In that case the parameter card would be taken from the gridpack. This is the case for the ttH sample (I checked 346443). One can see for this sample that no BR is specified in the parameter card used; then in the header of the lhe file, I see the following values:
```
DECAY 24 2.046400e+00
# BR NDA ID1 ID2 ...
3.333659e-01 2 -1 2 # 0.68219997776
3.333659e-01 2 -3 4 # 0.68219997776
1.111171e-01 2 -11 12 # 0.22739003344
1.111171e-01 2 -13 14 # 0.22739003344
1.110340e-01 2 -15 16 # 0.2272199776
```
It's probably these two samples
```
aMC@NLO+Pythia
410465 aMcAtNloPy8EvtGen_MEN30NLO_A14N23LO_ttbar_noShWe_dil
410464 aMcAtNloPy8EvtGen_MEN30NLO_A14N23LO_ttbar_noShWe_SingleLep
```Spyros ArgyropoulosSpyros Argyropouloshttps://gitlab.cern.ch/atlas-physics/pmg/mcjoboptions/-/issues/227Establish CI to handle stale MRs and branches2024-02-19T11:14:22+01:00Spyros ArgyropoulosEstablish CI to handle stale MRs and branchesFollowing !2906 we should add a CI job that uses this functionality to handle MRs that are open for long with failed pipelines and also delete the associated branches when closing the MRsFollowing !2906 we should add a CI job that uses this functionality to handle MRs that are open for long with failed pipelines and also delete the associated branches when closing the MRsSpyros ArgyropoulosSpyros Argyropoulos