History is not preserved when reading with ROOTObjectReader
Summary
Bug noticed by @kdort when applying digitization in a separate step. The links from PixelHits (created in the new step) to the MCParticles (created earlier) are not instantiated correctly.
The problem is when using the ROOTObjectReader to read a data file with linked history, the internal references are not preserved. The recreated objects propagated by the messenger, and later stored with ROOTObjectWriter, do not have their history pointing to the correct other recreated object, but instead to the original object.
Steps to reproduce
- Run the first simulation with ROOTObjectWriter, MCParticles are in the produced tree
- Run the second simulation with ROOTObjectReader to read the previous data and ROOTObjectWriter to write the final data
- The final tree contains both MCParticles and PixelHits which can be accessed afterwards, but fetching via PixelHit->getMCParticles does no longer work.
Example Configuration
Step 1: tcad_e_field_noDigitization.conf
Step 2: tcad_e_field_digitize.conf
(an arbitrary detector file should suffice)
What is the current bug behavior?
Accessing MCParticles directly from the PixelHits gives nullptr's.
What is the expected correct behavior?
The PixelHits should have a link to the MCParticles that produce .
Detailed problem
The problem is that TRefs are bound to unique instances of objects (and not to any copies of this object). When reading the data in the ROOTObjectReader, messages are internally created by the framework to be passed on to the listening modules. However while doing this the data stored in the trees is copied here during the construction of those new messages, essentially creating new objects that are not part of any history.
The result in this case is that both the PixelCharge and the MCParticle are copied but the MCParticles in the PixelCharge keep pointing to the old MCParticles (instead of the new one which will be stored in the new output tree later)!
Workaround
There is an easy workaround for the problem: by loading the branch in the old tree together with the new tree the required data can be retrieved from the old tree without problem.
Suggestions for fixes
- In the current architecture it should (as far as my understanding goes) be impossible to pass the objects to the messenger without requiring a copy of the object. Preventing this copy would therefore require new type of Messages that would link to references/pointers to objects (instead of having the primary copy necessary in the message itself), probably requiring a significant rework of the current architecture. Then it remains difficult to ensure the original object is not lost during framework executing, and it also depends on ROOT correctly handling storage of the 'same' object in different trees.
- Alternatively history links should be fixed on the fly when reading the objects. This might be the more promising approach as the UniqueID of the objects can theoretically be retrieved from the TObject. This should technically allow to fetch the unique id of the old and new objects and creating a conversion table, to fix the history (see here details about how TRefs links are created). Implementation details for this approach need to be figured out.