Migrate repo lhcb-core/lhcb-benchmark-scripts to PRConfig

This moves the entire repo of lhcb-core/lhcb-benchmark-scripts (as of lhcb-core/lhcb-benchmark-scripts@ac35672f) into PRConfig.
If I didn't screw up, all files should keep their old history.

• python files have been moved into python/MooreTests
• scripts into a subfolder benchmark-scripts in ./scripts

This MR includes some slight modifications but to make review easier these are all in a separate commit.
As you can see it's almost entirely related to changing hardcoded paths variables.

Once this MR has been merged we can then slowly move the nightly tests to use this repo instead of the old one.
Main benefit apart from getting rid of one repo is that we will in the future more easily be able to test changes to these benchmarking scripts in the nightlies.

The old repo https://gitlab.cern.ch/lhcb-core/lhcb-benchmark-scripts/ will be archived.

cc @maszyman @rmatev

added 140 commits

• b09061d9 - Set of scripts to extract and analyze benchmark data
• eacf17ae - Added script for measuring memory footprint
• eddfc625 - Added script to run benchmarks
• 195d55a8 - Added script to plot pie charts of algorithm usage
• 34930297 - First attempt of hatching
• 4ff7e77b - Improved version for TDR plots
• 61a63fc1 - Added light grid to plotSpeedup images
• f538e0a2 - Nicer colors for AlgoUsage plots
• e616ab4d - Allow to run in non hive mode for single threaded case
• e4443477 - tentative implementation of finer event counting, ignoring first and last ones
• 1cc7c735 - Fixed colors of plot
• 56b31c45 - Plot multijob and no hive by default in plotSpeedup
• c9ba5857 - Dropped HLT1 sequences from hivetimeline plots
• c242f95b - Adapted plotMem to new layout of extracted data
• 5cdf0ac3 - Multiprocess version of extract to speed up extraction
• 3a075d98 - fixes for pr test
• df0d28f0 - fixes for non-interactive running
• 92f2cce0 - plot best hive perf
• 64bd2646 - prettier plot
• d4a52679 - cleanup
• dfccdab8 - don't plot best hive if fails
• 257538e6 - Added readme
• 5c5a35d9 - Be agnostic of where we run, rely on external configuration
• 3af519e8 - Extended runTest for supporting numa and for guessing best configurations
• 430b0ffb - Added script computing max throughput from the extractedData coming from extract.py
• 7e487b19 - Added a script to measure throughput of a machine
• a95f689c - Use new HLT1BestPerf attribute of MiniBrunel Configurable
• 711da9fc - Dropped memory measurements by default and added --memory flag to get it back
• cb921720 - Proper naming of tar file compressed with bzip
• 903475f4 - Enable HLTEventLoopMgr for benchmark tests
• 68de43ce - Ignore checksum checking when reading MDF in benchmarks
• 95453eef - get time between algorithms and within as extracted from timeline.csv
• c7267b6b - fixes for pr test
• 1e43701e - fixes for non-interactive running
• d9b8fe5e - plot best hive perf
• 4bddb813 - prettier plot
• b5ead7b6 - cleanup
• d6b0b1af - don't plot best hive if fails
• 626f73d7 - Merge branch 'mergeRepos' into 'master'
• 27ecbfad - First version of new config files for the nightlies
• abd2c3b7 - Merge branch 'sponce_newConfigFiles' into 'master'
• 3c27625c - Fixed default setup of AlgoConfig in PrPixelTracking
• 3efa9021 - Fixed qlgos input/output and use PatPV3DFuture
• c728ec09 - Rework benchmark to use new scheduler and new options
• 28eef975 - Merge branch 'setupNewBenchmark' into 'master'
• 81850872 - Some small tunings to the new benchmark scripts
• 6b3f0799 - Merge branch 'tunebenchscripts' into 'master'
• 1f8a1310 - fix for missing slash in path
• 672f63c6 - Merge branch 'maszyman-missing-slash' into 'master'
• c0a4377a - add wrapper scripts
• 34cc714f - Merge branch 'maszyman-wrap-scripts' into 'master'
• de3136a8 - add --cp flag to flamegraph command to use same colormap for all graphs
• 8f9c090b - Merge branch 'patch-1' into 'master'
• 7139f57f - adding palette.map file
• 06a464bf - Merge branch 'maszyman-add-palette' into 'master'
• 72440c7a - HLT2 Throughput test with old configuration
• e4bf0acd - Merge branch 'master' into 'master'
• 18d3f6a3 - Fixed argument passing when calling runTest in SetupThroughputTest
• 14190125 - Merge branch 'sponce_fixArgsInThroughput' into 'master'
• 84840a73 - Hlt2 throughput: fix location of Brunel opts file for PR tests. Run on more...
• 2696b8a1 - Merge branch 'master' into 'master'
• 6f20307a - fix dangling profile jobs
• 080a6548 - Merge branch 'mstahl_Hlt2ProfileFix' into 'master'
• 0deddf98 - get number of threads and event slots from lscpu
• 5bdcbef8 - use loop instead of repeating the same name
• f8215df0 - use variable for the input file name
• 6c5992e1 - fix calculation of number of threads
• d330c7f9 - Merge branch 'maszyman-get-threads' into 'master'
• cb89a9aa - fix problem with globbing whithout wildcards
• c1192bb6 - Merge branch 'gunther_globfix' into 'master'
• 1c3c1247 - Modify RunThroughputJobs.py to select a file based on the numa domain
• 906d19fc - Merge branch 'ahennequ_onefilepernuma' into 'master'
• 004618ec - Update runHlt2Profile.sh
• b34b5ae8 - Merge branch 'patch-1' into 'master'
• 5fd19589 - Add throughput test runner for Moore
• c4d2874e - Merge branch 'mstahl_MooreTP' into 'master'
• 3f2adbc2 - adding explicit bash scripts for running nightly tests as discussed in...
• d0581c5c - Merge branch 'mstahl_scripts_for_nightlies' into 'master'
• e256b4c3 - make Moore scripts executable
• 3338f716 - Merge branch 'maszyman-moore-exec' into 'master'
• 65291962 - fix path to the actual script
• c7dbc9dd - Merge branch 'maszyman-fix-paths' into 'master'
• cd995fff - proper call for scripts
• 96781717 - Merge branch 'maszyman-fix-paths' into 'master'
• b92f60f4 - update for Moore throughput tests
• 6228f23e - Merge branch 'mstahl_MooreTP' into 'master'
• 17349d02 - Hardcode input file paths on lbhltperf01
• 80a466a1 - Merge branch 'fix-hlt1-moore' into 'master'
• d9585417 - Update Moore_hlt1_reco_baseline.sh
• ba9c8e48 - Merge branch 'patch-1' into 'master'
• 27a54a3c - fixing arguments of the profile job
• 9488337c - Apply suggestion to scripts/Moore_hlt1_reco_baseline.sh
• 4ad70c3c - Revert "fixing arguments of the profile job"
• a3fc45da - Merge branch 'maszyman-profile-argument' into 'master'
• d5ab17d5 - New hlt1 throughput tests
• f61d05bd - Merge branch 'mstahl_new_hlt1_throughput_tests' into 'master'
• 6696a996 - Adding script to do throughput testing for FTv6
• 058c0c99 - Merge branch 'mstahl_FTv6ThrouhghputTest' into 'master'
• c9e94bc2 - giving correct permissions to scripts/Moore_hlt1_reco_ftv6.sh
• bbc804e7 - Merge branch 'mstahl_permissions_FTv6' into 'master'
• b0caf862 - Add breakdown plot from vtune result plus a bit of refactoring and cleanup
• 516eb55e - Merge branch 'mstahl_MooreTP' into 'master'
• 2be3d4f3 - Adding back cleanup.sh
• bee8a615 - Merge branch 'mstahl_cleaning_fix' into 'master'
• b220e913 - fix for throughput numbers of ftv6 tests
• e4ee681c - Merge branch 'mstahl_ftv6throughput_fix' into 'master'
• a0b4f6f8 - use integer division from python3
• 4247be4e - Increasing number of events for v6 throughput tests
• 0d035d74 - Merge branch 'patch-1' into 'master'
• 2b393cf6 - increasing number of events for Brunel profiling jobs
• 257f615e - Merge branch 'mstahl_hotfix_forBrunel' into 'master'
• 678f7240 - Option to parse number of events for profiling jobs
• 8c145061 - Merge branch 'mstahl_parse_nevts' into 'master'
• 292374d5 - Add velo_only and prforward throughput tests, run hlt2_reco_brunelesque single-threaded
• 77ae4c3b - Merge branch 'mstahl_new_tests' into 'master'
• 4ed37004 - Scaling tests for Moore
• d9f069f1 - Merge branch 'mstahl_scaling' into 'master'
• 15cf764f - Remove scripts for brunel tests
• 78ee47d1 - Fix for scaling job
• f6da329c - Merge branch 'mstahl_scaling' into 'master'
• d286ce42 - Merge branch 'maszyman-integer-division' into 'master'
• 2027b505 - Remove exception for running on lbhltperf01
• 78d35a41 - Merge branch 'rmatev-remove-exception' into 'master'
• 1b8fcdfd - Update Readme, Move import
• ad1474b8 - Merge branch 'mstahl_readme' into 'master'
• 2b0e5112 - bugfix if number of events has been set by user
• 5f0e4722 - Merge branch 'mstahl_n_events_bug' into 'master'
• 0ca148f8 - Add script for pp_comparison options
• c28a64ee - Merge branch 'mstahl_cleanup' into 'master'
• ab9290b5 - Change Hlt2 samples to a GEC filtered sample
• 1aed3b94 - Fix regex for cases where there's a `<cycles 3>` (or another number) after `operator()`
• 5105b533 - Merge branch 'sstahl_change_hlt2_samples' into 'master'
• 70fd0292 - Merge branch 'gunther_fixRegex' into 'master'
• a7ce6a6e - Update Readme
• 407509a5 - Merge branch 'mstahl_doc' into 'master'
• 74af490a - Change creation of FlameBars to be based on parsing of log file.
• ac35672f - Merge branch 'lohenry-readTiming' into 'master'
• 187c7b86 - Merge remote-tracking branch lhcb-benchmark-scripts/master into benchmark_scripts
• 1664bb15 - move stuff around
• cd2ecd69 - adapt paths to new home in PRConfig

Compare with previous version

changed the description

great stuff, thanks! (I would've been lazier and not copied the history, but it's nice to have it!)

I think we might be able to improve locating of things (such as doing python -m MooreTests.run_throughput_jobs ...) but that depends a bit on how we're going to run this.

@maszyman could you remind us where's the code that deals with cloning lhcb-benchmark-scripts and running the job?

@chasse can you please update the Moore docs (doc/tutorials/developing.rst)

mentioned in merge request lhcb/Moore!489 (merged)

mentioned in merge request lhcb-core/LbNightlyTools!307 (merged)

added 1 commit

• 60d0d191 - try to launch via python module

Compare with previous version

• [2020-05-07 17:58] Validation started with lhcb-master-ref#606
• [2020-05-07 18:02] Validation started with lhcb-master-mr#784
• [2020-05-07 18:13] Validation started with lhcb-master-mr#785
• [2020-05-07 18:17] Validation started with lhcb-master-mr#786
• [2020-05-07 21:55] Validation started with lhcb-master-mr#787
• [2020-05-08 00:03] Validation started with lhcb-g4-dev#454
• [2020-05-08 00:04] Validation started with lhcb-gauss-lamarr#115
• [2020-05-08 00:04] Validation started with lhcb-sim10#409
• [2020-05-08 00:04] Validation started with lhcb-lcg-dev4#1271
• [2020-05-08 00:04] Validation started with lhcb-test-throughput#510
• [2020-05-08 00:04] Validation started with lhcb-test-throughput2#338
• [2020-05-08 00:04] Validation started with lhcb-reco14-patches#1037
• [2020-05-08 00:04] Validation started with lhcb-sim09-cmake#981
• [2020-05-08 00:04] Validation started with lhcb-sanitizers#560
• [2020-05-08 00:04] Validation started with lhcb-run2-patches-dev4#319
• [2020-05-08 00:04] Validation started with lhcb-hlt2012-patches#599
• [2020-05-08 00:05] Validation started with lhcb-test-efficiency#70
• [2020-05-08 00:06] Validation started with lhcb-gaudi-head-py3#351
• [2020-05-08 00:06] Validation started with lhcb-lcg-dev3#1258
• [2020-05-08 00:07] Validation started with lhcb-dd4hep#504
• [2020-05-08 00:08] Validation started with lhcb-gauss-gen-dev#363
• [2020-05-08 00:08] Validation started with lhcb-gauss-dev#2379
• [2020-05-08 00:10] Validation started with lhcb-head#2584
• [2020-05-08 00:10] Validation started with lhcb-gaudi-head#2594
• [2020-05-08 00:11] Validation started with lhcb-lcg-test#31
• [2020-05-08 00:13] Validation started with lhcb-master#1096
• [2020-05-08 03:37] Validation started with lhcb-lcg-dev3#1256
• [2020-05-08 09:52] Validation started with lhcb-master-ref#607
• [2020-05-08 09:57] Validation started with lhcb-dd4hep#505
• [2020-05-08 10:04] Validation started with lhcb-master-mr#788
• [2020-05-08 10:13] Validation started with lhcb-master-mr#789

added 1 commit

• 55b70f12 - adapt all benchmark scritps to launch via python module syntax

Compare with previous version

added 1 commit

• a7df0c39 - move flamegraph related script into python package MooreTests

Compare with previous version

approved this merge request

resolved all threads

So do you agree if I'll update all the content fields in options now?

changed the description

resolved all threads