HLT2 online throughput test

Still draft as some more cleanup is needed.

Credits for setting up the throughput test go to @fsouzade!

The throughput difference compared to the offline throughput test is down from ~20% to now ~5%. (with Online!746 (merged) it's now down to ~3%)
I think that we are loosing some throughput due to the I/O of data, but I'm not yet sure why. The offline test reads the same files so why should be pay more for I/O?

Performance fixes applied:

• Fix AlgFlowScheduler threading issue, see Online!745 (merged)
• ensure that we use tcmalloc: 2d02396f
• ensure that we don't stall while waiting for events by increasing the amount of MBM connections: 1e0b6ccb

added enhancement label

assigned to @chasse

added 1 commit

• 083ffc25 - simple python script to measure throughput of HLT2 jobs

Compare with previous version

added RTA label

added 1 commit

• 89ad8c4a - remove some old changes

Compare with previous version

added 1 commit

• f974f58b - remove some old changes

Compare with previous version

requested review from @rmatev

@rmatev, @fsouzade quite a few files e.g. tags.py and online.py are currently modified in place. Are these changes some that we want to apply for all configurations or should we create special options for the throughput test?

requested review from @fsouzade

changed the description

added 1 commit

• 53e9334c - Cleanup changes for throughput test

Compare with previous version

@chasse @fsouzade I've pushed a cleanup commit, also adding some instructions to the readme. I think this is now mergeable.

marked this merge request as ready

Hi @chasse, can I run a test or some further development is expected?

resolved all threads

merged

mentioned in commit 43d9080c

mentioned in issue Moore#456 (closed)

mentioned in issue Moore#456 (closed)