Tags

Tag Notes

o Fixes:
  - Remove double call of BeginRun for some algorithms in the AlgresourcePool
  - Remove some deprecated headers
  - Automatically adjust tbb thread pool size in order to create exactly as many threads as requested and not one less

o Features:
  - Display RSS and seconds in loop in the HiveSlimEventLoopManager
  - Define GOD_NOALLOC variable in CMake configuration to avoid the overriding of new and delete operators for the event data model. Indeed, this behaviour brought to the usage to the boost pool allocators. They are extremely fast and appropriate for a serial execution but suboptimal for the mt one when spanning over a Numa architecture. The pool can be localised on a DRAM and the algorithm accessing it on a socket "nearer" to a different bank. A performance penalty in terms of scaling could be measured.
  - Add to the cpu crunchers a parameter to repeat the read and write onto the whiteboard. This allows to emulate high lock contention increasing the probability of collision.
  - Minimal improvement of the ToolSvc that allows to have "clones" of the same tool provided that the parent is not the ToolSvc

o Optimisations:
  - Add a boolean flag not to schedule multiple updates of the alg state machine, therefore reducing the calls to the function by a factor 2/3

Tag notes

o Fixes:
 - DataFlowManager: remove segfault due to improper use of bitsets
 - Cmake: Treat Root as all other externals
 - HiveEventLoopMgr: (legacy component): check for WB existence and not DataSvc only
 - AIDA: properly found also with the new externals infrastructure
 - HiveReadAlgorithm: Can be used also with old Gaudi infrastructure

o ForwardScheduler: Improved output info in case of stall.

o HiveWhiteBoard: use recursive mutex instead of a spin mutex in order to
support MiniBrunel.

o HiveSlimEventLoopMgr: run the 1st event alone even if multiple events in
flight are selected (eases measurements since some loading happens in
minibrunel at the first event). Add per event RSS and timing measurements.

o IDataSvc: allow to pass a particular IDataProviderSvc in order to properly
use the slots in the whiteboard.

o MDF input: It is now possible to read multiple events simultaneously starting
from MDF files. RawDataAddresses now do not share the same buffer for banks.
The fix (code mainly from M. Frank) consists in the following steps:
  1) StreamBuffer class: add an adopt 'method' which corresponds to the 'data'
  one but passing the ownership of the banks to the client
  2) RawDataAddress class: add an adoptData method which corresponds to the
  setData one. If data is adopted, the destructor of the class will trigger the
  deletion of the data
  3) RawDataSelector class: add a flag to decide if the data is to be adopted
  by the address or not. It should be an option in the future.
  4) MDFSelector class: wire into the method getDataSpace the buffer adoption.
  As for 3, this should be dynamically chosable.
As for the ROOT I/O, the createAddress and next methods of the selector must be
called serially and in this order! In any case, this is ensured by the
structure of the HiveSlimEventLoopManager.

o DetDataSvc: Replace the DataSvc with a version with thread safe retrieve
methods:'TsDataSvc'. The DetDataSvc inherits from this TS version of DataSvc.
The RegistryEntry had to be made friend of this new svc. Recusrive lock is used
given the "richness" of the IDataManager interface. Reentrant methods should be
identified and recursive lock eliminated in favour of a rapid spin lock
(or no lock at all?)

o TimerForSequencer: Eliminate doubles to accumulate statistics in favour of
uint64_t-s. Add sigma of the timing measurements

o CPUCruncher scenario for MiniBrunel

o First iteration of coding conventions

Tag notes:

o Algorithm class modified in order to store the algorithm type, store the data
dependencies in form of data object handles, host the event context, switch to
the correct whiteboard slot in presence of the event context in sysExecute.

o HiveWhiteBoard service added. This service acts as a "manyfold" event store
and allows the treatment of multiple events simultaneously. It provides thread
safe object registration and access.

o HiveFwdSchedulerSvc added. Implementing the IScheduler interface,
this service is a simple state machine which takes
care of the submission of the algorithms (wrapped in a AlgoExecutionTask)
to the TBB runtime according to the presence of their data dependencies and
decisions linked to the control flow. Two helper classes are used to achieve
this goal, the DataFlowManager and the ControlFlowManager.

o AlgResourcePool service added. Provides algorithms instances and clones.
Manages the states of the algorithms (calls to initialise, resetExecuted,
finalize...). It "flattens" the top algorithms list and deduces the control
flow from the sequencers provided.

o HiveSlimEventLoopMgr class added. This is a simple version of the
eventLoopMgr which just acts as an event factory and communicate with the
scheduler svc via the IScheduler interface.

o HiveReadAlgorithm algorithm added. A fundamental piece to achieve forward
scheduling. With no data nor control flow dependencies, this algorithm is used
to preload the content of an input file into the whiteboard.

o DataObjectHandler class added. It allows the declaration of the input and
output data objects of an algorithm.

o concurrentRun.exe executable added to avoid to steer the application
execution with Python.

o CPUCruncher algorithm added. It crunches CPU for a certain amount of time
finding prime numbers using an inefficient algorithm. It is calibrated to emulate
a real machine load.

o HiveEventLoopMgr class added. Legacy: the class was part of the very first
prototype, able only to run CPUCrunchers. It is in the repository to perform
comparisons and reproduce the old results if needed.

o Athena Reco, Brunel and CMSSW Reco scenarios added. Emulate these real
workflows with number crunching algorithms. Necessary to study  the projected
application behaviour.

o InertMsgSvc service added. A thread safe, lightweight msg service not using
the TBB runtime.

o compareRootHistos.py refurbished, made faster (no pvalues spurious
recalculation) and and able to compare histograms bin by bin.

o SeqSchedSvc service added. A scheduler that schedules sequentially the
algorithms. It works with the HiveSlimEventLoopMgr for testing purposes.