Skip to content

Draft: New simplified Transient Store interface

Alejandro Perez Casasrequested to merge
alperezc/Gaudi:alperezc_newTES into master

// --- VERY EARLY DRAFT --- //

The current Transient Event Store stores DataObjects (or AnyDataWapper as a way of storing any other type) on a hierarchical way.

The current TES interface is split in two:

  • IDataProviderSvc.h is the interface that gives access to the TES for the 'users'
  • IDataManagerSvc.h is the interface that manages the hierarchical tree itself

DataSvc.h/cpp is the one implementing both of those interfaces. It holds a unique_ptr to the root node of the tree, a RegistryEntry.

Some characteristics of the current TES:

  • (1) The current TES has a lot of duplicated methods: for working with a string with the full path of the object, with an object itself, with a RegistryEntry, etc. In reality, all calls use the string representation
  • (2) The search for an object is recursive given the hierarchical nature of the tree that holds the objects
  • (3) Each node has an IOpaqueAddress and a DataObject (among other things). This allows algorithms to register an object on TES without having the object itself loaded. When the object is needed, the TES uses the converters to load the object 'on-demand'. A producer RootIOAlg was introduced by Sebastien in the LHCb code for starting the data dependency graph (lhcb/LHCb!2993 (merged)), and a consumer algorithm is being developed as end of the data dependency graph (lhcb/LHCb!5418), also in LHCb code

With this implementation agnostic interface, we could use a (std::string,std::any) map for storing the objects, avoiding (1) by only having string set/get/has/etc. Better yet, using StringKeys for efficiency. Having each path as an entry on the map, we avoid the recursive traversal of the TES (2), making it faster. Furthermore, as all EventBranches are known for RootIOAlg, we could create a perfect hash for the map. This new TES would hold the std::any objects itself, so the functionality of (3) could be changed to load everything before the algorithms start to execute. This massive load could be implemented more efficiently then.

Another point of the new interface would be the use of EventContext for all operations.

Tentative timeline:

  • Create a simple interface for new TES
  • Make the new interface work with current DataSvc methods, and slowly migrate old interface calls with new interface calls. This step would highlight what are the minimum methods required in the new interface (eg. are hard/soft links necessary?)
  • Add a new implementation of the TES in parallel to the current one
  • Cleanup by porting the use of the old TES to the new one
  • Then we could drop the old TES interface/implementation
Edited by Alejandro Perez Casas

Merge request reports