Split track containers by type throughout the reconstruction (and beyond)

Currently, the track types are merged at some point (where exactly depending on the sequence) into a Best container. This should be changed to having split containers up to and including the selections and persistency.

There are several tasks included:

• Write a configuration that keeps the containers separate in Moore
• Study the physics impact of this, e.g. the NShared in the muonPID looks at all tracks. Is it enough to just combine the tracks before running the algorithm that needs all tracks?
• What is the impact on the v2::Particle, e.g. for Downstream tracks different states need to be used compared to Long tracks.
• What is the impact on persistency?

@cmarinbe @mvesteri @poluekt @jonrob @sesen @sstahl

closed

reopened

From persistency, there won't be any limitation of number of track containers to be persisted by default.

Here I just added muon tracks to be persisted as well as a test: https://gitlab.cern.ch/lhcb/Moore/-/blob/sevda-packers2/Hlt/RecoConf/python/RecoConf/reconstruction_objects.py#L55

FYI @rvazquez

@decianm could you expand the description a bit to explain where and how you imagine clones being dealt with by analysts? In particular for T-segments used in both long and downstream/T-tracks?

Let me also tag @graven @ahennequ here so we can discuss prioritisation of this within the broader v2::Particle effort. Do you all consider this necessary for datataking this year? If so, why?

The clone killing does not change with respect to now. Already now the clones are killed before the containers are merged (in the fast sequence), by comparing the containers and only storing the decloned tracks. So the analysts will not notice it.

What it will do is to essentially stop the current, slightly odd situation that we have split containers, merge them into one, and when one makes particles, one loops over the merged container to only select the ones that were in one of the split containers.

v3::Tracks are designed to only hold one track type, so essentially they were designed with this in mind. It's software, so one can always change things, but to me it feels easier to go through with this than to reintroduce the complexity of merged containers in everything dealing with v3::Tracks and friends.

I'm not objecting, I just want to make sure I understand.

What happens if we want to reintroduce redundancy between forward and match tracks in the future, because we have the resources available in HLT2? Would these still end up in a single container?

I assume the fact that for each track type you have one container before the clone killing and one after the clone killing is just an implementation detail?

Indeed, as from an analyst perspective it does not matter if the track was made by forward or match, but the tracks are distinguishable, as the v3::Tracks carry a history field which was introduced for this purpose (and is useful for monitoring). I mean one could in principle also split by these two algorithms if needed, but I assume the use case is not so strong.

We actually have a sequence running in the nightlies that exactly does that, i.e. runs the full forward and match (and puts them in a single container), but still uses the other elements of the fast sequence. And as long as we do have the resources at least for the start, I think it would be good to run this for commissioning.

Yes, it is just fiddling a bit with what containers are used how in a clone killer.

OK thanks for clarifying, that all makes sense. Agree with which sequence to run for commissioning.

The concept of "best" and clone killing always scared me w.r.t. tracking efficiencies with special* probe types. It feels clearer to me if these probes are in their own containers.

* I would argue that velo-muon,muon-UT et al. should not be seen as "special". They should be part of the design.

@jonrob @cmarinbe

I was thinking a bit about this in the last days. If the PID algorithms can run individually on each track, this is no problem. It will also work, if it just requires a basic set of tracks, e.g. Long and Downstream and UnfittedSeed and all other types can be fed in additional algorithms to obtain PID information for them.

The reasoning is: We can set up a sequence which fits all tracks, but given the smartness of the scheduler, the fitters will only be called if a line asks for it. But this requires that the track containers are not merged, which breaks this.

@rvazquez if I understood correctly, the muon PID is done individually for each track type, so this should already fulfill this.

@decian, yes the MuonID runs for Long and Downstream tracks separately. I think there was some interest in doing it also for TTracks, but I don't know the status of this developement.

mentioned in merge request !1844 (merged)

assigned to @mveghel

added to epic &5

mentioned in merge request !2141 (closed)

@mveghel What is the status on this?

It is messier than I thought. I could speed it up by not trying to clean up the config.

What do you mean?

Meaning improving consistency in the track algo configurations. I discussed this also with Andrii. To give an example: the RICH has its private split-per-tracktype algo of best track container. So this would be one to clean up.

The RICH algorithm is only used because I already wanted the tracks split by type, and thus had to do it myself. In the simplest case it might be enough to just remove the use of that algorithm and feed the already split containers directly in (there might be an issue with the container types expected at the c++ level, but that shouldn't be complex to fix).

Given that Calo, Muon and apparently also RICH use split containers already, I would actually have hoped it does not introduce too much complication (I mean, it is work, but I was hoping there would be no fundamental problem with it).

There is no fundamental problem, this was just an example.

Probably you are already aware, one complication I hit some time ago in the selections part was the compatibility with old Brunel samples which have only one track container. Things might be much easier if this compatibility is dropped.

Yes, good point, that would help a lot.

I honestly think this can be dropped (we already implicitly dropped it by changing the LHCbID, where the current software would not give useful results for old samples)

@mveghel What is the status of this (I saw that #560 mentions it)?

mentioned in issue #560

mentioned in merge request !2338 (merged)

mentioned in issue LHCb#312 (closed)

closed with merge request !2338 (merged)

mentioned in commit 0c2387cd