MadGraph adaptor for Gauss
This replaces request !730 (merged), that was merged to the wrong branch.
It includes all changes in
- !573 (closed) for including MadGraph as ME generator (by @cvazquez, @philten)
- !730 (merged) for configuring MadGraph (by @brachwal, @philten)
Closes LHCBGAUSS-1374
(cherry picked from commit 1741033a)
- af11ac96 Initial interface for Madgraph.
- e2606070 Actually adding the interface.
- eb9b6f02 Adding module support.
- 9de8e64e add LRSM and RPVMSSM models to LbMadgraph
- f6acc5f0 Migrated modules to data.
- cc463ab3 Added grid pack functionality.
- 75a7e923 Grid pack generation script.
- 008e80a4 Added LHAPDF support.
- 511e98c4 Updated test to check spillover and pileup options.
- a89ec30b Added in John's comments.
- 46bdc8c2 Fixed some memory issues.
- e1b2e34f Added Madgraph data.
- e0b8c873 Updated configuration through CMAKE.
- ebdba2fa Cleaned up some configuration issues.
- 8c1ed530 Updated documentation.
- edab3ce6 Need to handle Madgraph and CMT versions for gridpacks.
- 0af3eb80 Migrating gridpack script to MadgraphData.
- 1ad216e1 Migrated gripack to data repo.
- c34e9a5b Included exception when using gridpacks.
- e7d04f70 Actually compiles now.
- a65e990d Add madgraph test
- a7866c3a Updated gridpack path convention, exception.
- 88817bdd Use new versioning schema for data package
- 882634ea add missing copyright from new files
- 245af2fe Removed extraneous example files.
- 2426c322 init mg configuration
- 56002e83 refactor LbMadgraph python config
- 01bfc91c extend Gauss configuration
- 86f9141d refactor LbMadgraph src as for new configurable scheme
- 654f886d new config flow defined for Gauss.BeamBeam
- 2d667eb7 sugestions included
- df1b04f5 Protected against missing keys.
- 479401e0 Fix typo
- a2865e90 naming polishing
- 8f5bbf10 Removed using std.
- da8ea370 Fixed constructor, removed unused weight variables.
- 67b775d1 Added additional debugging information.
- 7c57c53b Fixed typo in beamname configuration.
- 561216c9 Added additional documentation on seed setting.
- 9273ebd9 Added overestimate documentation.
- a349bf42 std the opts naming convention
- 9580c764 Fixed large initialization and some warnings.
cc: @kreps, @philten, @brachwal, @philten, @cvazquez, @sfarry, @jback, @adavis
Merge request reports
Activity
changed milestone to %Gauss v55r2
added Generators ToMaster new feature + 1 deleted label
- [2021-06-23 00:11] Automatic merge failed in [lhcb-gauss-gen2-dev#351](https://lhcb-nightlies.web.cern.ch/nightly/lhcb-gauss-gen2-dev/351/Gauss/checkout
- [2021-06-23 12:46] Validation started with lhcb-gauss-gen2-dev#352
- [2021-06-24 00:22] Validation started with lhcb-gauss-gen2-dev#353
- [2021-06-24 10:40] Validation started with lhcb-gauss-gen2-dev#354
- [2021-06-25 00:07] Validation started with lhcb-gauss-dev#2806
- [2021-06-25 00:08] Validation started with lhcb-gauss-gen2-dev#355
- [2021-06-26 00:08] Validation started with lhcb-gauss-dev#2807
- [2021-06-26 00:09] Validation started with lhcb-gauss-gen2-dev#356
- [2021-06-27 00:04] Validation started with lhcb-gauss-gen2-dev#357
- [2021-06-27 00:05] Validation started with lhcb-gauss-dev#2808
- [2021-06-28 00:08] Validation started with lhcb-gauss-gen2-dev#358
- [2021-06-28 00:10] Validation started with lhcb-gauss-dev#2809
- [2021-06-29 00:13] Validation started with lhcb-gauss-dev#2810
- [2021-06-29 00:15] Validation started with lhcb-gauss-gen2-dev#359
- [2021-06-30 00:13] Validation started with lhcb-gauss-dev#2811
- [2021-06-30 00:15] Validation started with lhcb-gauss-gen2-dev#360
- [2021-07-01 00:07] Validation started with lhcb-gauss-dev#2812
- [2021-07-01 00:07] Validation started with lhcb-gauss-gen2-dev#361
- [2021-07-02 00:06] Validation started with lhcb-gauss-dev#2813
- [2021-07-02 00:10] Validation started with lhcb-gauss-gen2-dev#362
- [2021-07-02 12:38] Validation started with lhcb-gauss-dev#2814
- [2021-07-03 00:11] Validation started with lhcb-gauss-dev#2815
- [2021-07-03 00:16] Validation started with lhcb-gauss-gen2-dev#363
- [2021-07-04 00:04] Validation started with lhcb-gauss-gen2-dev#364
- [2021-07-04 00:07] Validation started with lhcb-gauss-dev#2816
- [2021-07-05 00:06] Validation started with lhcb-gauss-gen2-dev#365
- [2021-07-05 00:08] Validation started with lhcb-gauss-dev#2817
- [2021-07-06 00:07] Validation started with lhcb-gauss-dev#2818
- [2021-07-06 00:09] Validation started with lhcb-gauss-gen2-dev#366
- [2021-07-07 00:07] Validation started with lhcb-gauss-dev#2819
- [2021-07-07 00:09] Validation started with lhcb-gauss-gen2-dev#367
- [2021-07-08 00:09] Validation started with lhcb-gauss-dev#2820
- [2021-07-08 00:13] Validation started with lhcb-gauss-gen2-dev#368
- [2021-07-09 00:09] Validation started with lhcb-gauss-dev#2821
- [2021-07-09 00:13] Validation started with lhcb-gauss-gen2-dev#369
- [2021-07-10 00:08] Validation started with lhcb-gauss-dev#2822
- [2021-07-10 00:08] Validation started with lhcb-gauss-gen2-dev#370
- [2021-07-11 00:05] Validation started with lhcb-gauss-gen2-dev#371
- [2021-07-11 00:08] Validation started with lhcb-gauss-dev#2823
- [2021-07-12 00:05] Validation started with lhcb-gauss-gen2-dev#372
- [2021-07-12 00:06] Validation started with lhcb-gauss-dev#2824
- [2021-07-13 00:12] Validation started with lhcb-gauss-dev#2825
- [2021-07-13 00:16] Validation started with lhcb-gauss-gen2-dev#373
- [2021-07-14 00:13] Validation started with lhcb-gauss-dev#2826
- [2021-07-14 00:18] Validation started with lhcb-gauss-gen2-dev#374
- [2021-07-15 00:15] Validation started with lhcb-gauss-dev#2827
- [2021-07-15 00:18] Validation started with lhcb-gauss-gen2-dev#375
- [2021-07-16 00:07] Validation started with lhcb-gauss-gen2-dev#376
- [2021-07-16 00:08] Validation started with lhcb-gauss-dev#2828
- [2021-07-17 00:32] Validation started with lhcb-gauss-dev#2829
- [2021-07-17 00:34] Validation started with lhcb-gauss-gen2-dev#377
- [2021-07-18 00:05] Validation started with lhcb-gauss-gen2-dev#378
- [2021-07-18 00:05] Validation started with lhcb-gauss-dev#2830
- [2021-07-19 00:05] Validation started with lhcb-gauss-dev#2831
- [2021-07-19 00:09] Validation started with lhcb-gauss-gen2-dev#379
- [2021-07-20 00:05] Validation started with lhcb-gauss-dev#2832
- [2021-07-20 00:07] Validation started with lhcb-gauss-gen2-dev#380
- [2021-07-21 00:32] Validation started with lhcb-gauss-dev#2833
- [2021-07-21 00:37] Validation started with lhcb-gauss-gen2-dev#381
- [2021-07-22 00:21] Validation started with lhcb-gauss-dev#2834
- [2021-07-22 00:23] Validation started with lhcb-gauss-gen2-dev#382
- [2021-07-23 00:10] Validation started with lhcb-gauss-dev#2835
- [2021-07-23 00:14] Validation started with lhcb-gauss-gen2-dev#383
- [2021-07-24 00:21] Validation started with lhcb-gauss-dev#2836
- [2021-07-24 00:24] Validation started with lhcb-gauss-gen2-dev#384
- [2021-07-25 00:09] Validation started with lhcb-gauss-dev#2837
- [2021-07-25 00:10] Validation started with lhcb-gauss-gen2-dev#385
- [2021-07-26 00:05] Validation started with lhcb-gauss-dev#2838
- [2021-07-26 00:07] Validation started with lhcb-gauss-gen2-dev#386
- [2021-07-27 00:09] Validation started with lhcb-gauss-dev#2839
- [2021-07-27 00:14] Validation started with lhcb-gauss-gen2-dev#387
- [2021-07-28 00:05] Validation started with lhcb-gauss-dev#2840
- [2021-07-28 00:07] Validation started with lhcb-gauss-gen2-dev#388
- [2021-07-29 00:06] Validation started with lhcb-gauss-dev#2841
- [2021-07-29 00:09] Validation started with lhcb-gauss-gen2-dev#389
- [2021-07-30 00:10] Validation started with lhcb-gauss-dev#2842
- [2021-07-30 00:15] Validation started with lhcb-gauss-gen2-dev#390
- [2021-07-31 00:12] Validation started with lhcb-gauss-dev#2843
- [2021-07-31 00:12] Validation started with lhcb-gauss-gen2-dev#391
- [2021-08-01 00:04] Validation started with lhcb-gauss-gen2-dev#392
- [2021-08-01 00:04] Validation started with lhcb-gauss-dev#2844
- [2021-08-02 00:05] Validation started with lhcb-gauss-dev#2845
- [2021-08-02 00:09] Validation started with lhcb-gauss-gen2-dev#393
- [2021-08-03 00:05] Validation started with lhcb-gauss-dev#2846
- [2021-08-03 00:07] Validation started with lhcb-gauss-gen2-dev#394
- [2021-08-04 00:14] Validation started with lhcb-gauss-dev#2847
- [2021-08-04 00:19] Validation started with lhcb-gauss-gen2-dev#395
- [2021-08-05 00:06] Validation started with lhcb-gauss-dev#2848
- [2021-08-05 00:10] Validation started with lhcb-gauss-gen2-dev#396
- [2021-08-06 00:08] Validation started with lhcb-gauss-gen2-dev#397
- [2021-08-06 00:11] Validation started with lhcb-gauss-dev#2849
- [2021-08-07 00:08] Validation started with lhcb-gauss-gen2-dev#398
- [2021-08-07 00:10] Validation started with lhcb-gauss-dev#2850
- [2021-08-08 00:05] Validation started with lhcb-gauss-gen2-dev#399
- [2021-08-08 00:06] Validation started with lhcb-gauss-dev#2851
- [2021-08-09 00:04] Validation started with lhcb-gauss-gen2-dev#400
- [2021-08-09 00:05] Validation started with lhcb-gauss-dev#2852
- [2021-08-10 00:06] Validation started with lhcb-gauss-dev#2853
- [2021-08-10 00:07] Validation started with lhcb-gauss-gen2-dev#401
- [2021-08-11 00:12] Validation started with lhcb-gauss-dev#2854
- [2021-08-11 00:14] Validation started with lhcb-gauss-gen2-dev#402
- [2021-08-12 00:07] Validation started with lhcb-gauss-dev#2855
- [2021-08-12 00:08] Validation started with lhcb-gauss-gen2-dev#403
- [2021-08-13 00:07] Validation started with lhcb-gauss-dev#2856
- [2021-08-13 00:09] Validation started with lhcb-gauss-gen2-dev#404
- [2021-08-14 00:05] Validation started with lhcb-gauss-dev#2857
- [2021-08-14 00:06] Validation started with lhcb-gauss-gen2-dev#405
- [2021-08-15 00:05] Validation started with lhcb-gauss-gen2-dev#406
- [2021-08-15 00:05] Validation started with lhcb-gauss-dev#2858
- [2021-08-16 00:06] Validation started with lhcb-gauss-gen2-dev#407
- [2021-08-16 00:06] Validation started with lhcb-gauss-dev#2859
- [2021-08-17 00:05] Validation started with lhcb-gauss-dev#2860
- [2021-08-17 00:07] Validation started with lhcb-gauss-gen2-dev#408
- [2021-08-18 00:12] Validation started with lhcb-gauss-dev#2861
- [2021-08-18 00:14] Validation started with lhcb-gauss-gen2-dev#409
- [2021-08-19 00:08] Validation started with lhcb-gauss-dev#2862
- [2021-08-19 00:11] Validation started with lhcb-gauss-gen2-dev#410
- [2021-08-20 00:16] Validation started with lhcb-gauss-dev#2863
- [2021-08-20 00:18] Validation started with lhcb-gauss-gen2-dev#411
- [2021-08-21 00:13] Validation started with lhcb-gauss-dev#2864
- [2021-08-21 00:18] Validation started with lhcb-gauss-gen2-dev#412
- [2021-08-22 00:05] Validation started with lhcb-gauss-dev#2865
- [2021-08-22 00:07] Validation started with lhcb-gauss-gen2-dev#413
- [2021-08-23 00:07] Validation started with lhcb-gauss-gen2-dev#414
- [2021-08-23 00:07] Validation started with lhcb-gauss-dev#2866
- [2021-08-24 00:11] Validation started with lhcb-gauss-dev#2867
- [2021-08-24 00:15] Validation started with lhcb-gauss-gen2-dev#415
- [2021-08-25 00:09] Validation started with lhcb-gauss-dev#2868
- [2021-08-25 00:13] Validation started with lhcb-gauss-gen2-dev#416
- [2021-08-26 00:05] Validation started with lhcb-gauss-dev#2869
- [2021-08-26 00:10] Validation started with lhcb-gauss-gen2-dev#417
- [2021-08-27 00:08] Validation started with lhcb-gauss-gen2-dev#418
- [2021-08-27 00:10] Validation started with lhcb-gauss-dev#2870
- [2021-08-28 00:06] Validation started with lhcb-gauss-dev#2871
- [2021-08-28 00:11] Validation started with lhcb-gauss-gen2-dev#419
- [2021-08-29 00:05] Validation started with lhcb-gauss-dev#2872
- [2021-08-29 00:07] Validation started with lhcb-gauss-gen2-dev#420
- [2021-08-30 00:06] Validation started with lhcb-gauss-dev#2873
- [2021-08-30 00:08] Validation started with lhcb-gauss-gen2-dev#421
- [2021-08-31 00:07] Validation started with lhcb-gauss-dev#2874
- [2021-08-31 00:13] Validation started with lhcb-gauss-gen2-dev#422
- [2021-09-01 00:09] Validation started with lhcb-gauss-dev#2875
- [2021-09-01 00:13] Validation started with lhcb-gauss-gen2-dev#423
Edited by Software for LHCbmentioned in merge request !573 (closed)
Thia nneds to be picked up by its own because of merge conflicts woth !573 (closed). Nevertheless all commits are here, so it is a matter of ensurinb proper documentayoon. Will relaunch the nightly build after removing !573 (closed) from it
I checked nightlies and while it builds, it fails to run. Comparing to the nightly from a week ago, the difference is Madgraph version picked up. While before we were picking it up from
/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_97a_LHCB_6/MCGenerators/madgraph/2.7.3.atlas/x86_64-centos7-gcc9-opt/bin/mg5_aMC
, with this new MR we are getting it from/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_100_LHCB_6/MCGenerators/madgraph5amc/2.7.3.py3.atlas/x86_64-centos7-gcc9-opt/bin/mg5_aMC
. From the path it looks like we are getting one compiled for python3 while we are trying to compile on python2 architecture.@kreps LCG_100 is python3, so we are using python3... but I see the problem with lhcb-gauss-gen2-dev where the stack is python2... Let me put back LCG_97a_LHCB_6 in lhcb-gauss-gen2-dev and also add this to lhcb-gauss-dev where python3 is in the whole stack.
We can then compare the two
Edited by Gloria Corti- Resolved by Gloria Corti
This new has no longer inconsistency of python versions between the Gauss stack and Madgraph libraries.
The madgraph test fails in both ~lhcb-gauss-gen2-dev and ~lhcb-gauss-dev when madgraph is called to produce the input events the errors in the
stdout
are the followingLHAupMadgraph::execute "cd 42912009_6500_6500; ./run.sh 1 1" LHAupMadgraph::execute "gunzip -f 42912009_6500_6500/events.lhe.gz" PYTHIA Error from LHAupMadgraph::run: file '42912009_6500_6500/events.lhe.gz' does not exist. PYTHIA Abort from Pythia::init: Les Houches initialization failed ... LHAupMadgraph::execute "cd 42912009_6500_6500; ./run.sh 240 32517562" LHAupMadgraph::execute "gunzip -f 42912009_6500_6500/events.lhe.gz" Generation.Special FATAL Special:: Exception throw: Could not generate event StatusCode=FAILURE Generation FATAL Exception with tag= is caught
and in
stderr
Traceback (most recent call last): File "./bin/generate_events", line 29, in <module> import amcatnlo_run_interface as run File "/workspace/build/Gauss/build/Sim/Gauss/tests_tmp/42912009_6500_6500/bin/internal/amcatnlo_run_interface.py", line 111 raise aMCatNLOError, 'not correct number of argument' ^ SyntaxError: invalid syntax mv: cannot stat 'Events/run/events.lhe.gz': No such file or directory gzip: 42912009_6500_6500/events.lhe.gz: No such file or directory
for more details see https://lhcb-nightlies.web.cern.ch/nightly/lhcb-gauss-gen2-dev/355/Gauss/x86_64-centos7-gcc9-opt/tests#Gauss.gauss-gen-42912009-madgraph
unassigned @gcorti
- Resolved by Gloria Corti
I managed to do bit of debugging with lhcb-gauss-dev slot version and it when I try to run it locally it comes to issue that things are executed with python3 but madgraph python code is not compatible with python 3. I do not have good idea what to do here as I know too little about madgraph to understand what comes from where. We might need help from experts to get this executed with python2 (or hope that newer version would be python3 compatible).
mentioned in merge request lhcb-datapkg/Gen/MadgraphData!10 (merged)
Temporarily taking this off from lhcb-gauss-dev slot to test !761 (merged) as there is small conflict in formatting. Once !761 (merged) is merged, I will fix this and put it back.