WIP: Fix incorrect default feature value for catboost and added isMuon check with catboost

@dovombru @nkazeev I have tested this code with several different configuration you can check it below and I fix a bug with incorrect default features. Fixing #93 (closed)

Catboost 0:
Muon matching:
Muon fraction in all MCPs: 59295/ 4095296 0.01%
Muon fraction in MCPs to which a track(s) was matched: 5365/ 212348 0.03% \Correctly identified muons with isMuon: 4044/ 5365 75.38%
Correctly identified muons from strange decays with isMuon: 1/ 3 33.33%
Correctly identified muons from B decays with isMuon: 264/ 315 3.81%
Tracks identified as muon with isMuon, but matched to non-muon MCP: 5647/ 206983 2.73%
Ghost tracks identified as muon with isMuon: 492/ 18092 2.72%

Catboost -10:
Muon matching:
Muon fraction in all MCPs: 59295/ 4095296 0.01% \ Muon fraction in MCPs to which a track(s) was matched: 5365/ 212348 0.03%
Correctly identified muons with isMuon: 4780/ 5365 89.10%
Correctly identified muons from strange decays with isMuon: 3/ 3 100.00%
Correctly identified muons from B decays with isMuon: 290/ 315 92.06%
Tracks identified as muon with isMuon, but matched to non-muon MCP: 26297/ 206983 12.70%
Ghost tracks identified as muon with isMuon: 2520/ 18092
13.93%

Catboost -100:
Correctly identified muons with isMuon: 4780/ 5365 89.10%
Correctly identified muons from strange decays with isMuon: 3/ 3 100.00%
Correctly identified muons from B decays with isMuon: 290/ 315 92.06%
Tracks identified as muon with isMuon, but matched to non-muon MCP: 26297/ 206983 12.70%
Ghost tracks identified as muon with isMuon: 2520/ 18092 13.93%

Without catboost:
Muon matching:
Muon fraction in all MCPs: 59295/ 4095296 0.01%
Muon fraction in MCPs to which a track(s) was matched: 5365/ 212348 0.03%
Correctly identified muons with isMuon: 4780/ 5365 89.10%
Correctly identified muons from strange decays with isMuon: 3/ 3 100.00% Correctly identified muons from B decays with isMuon: 290/ 315 92.06%
Tracks identified as muon with isMuon, but matched to non-muon MCP: 26297/ 206983 12.70%
Ghost tracks identified as muon with isMuon: 2520/ 18092 13.93%

added 3 commits

• 76ac4786 - Adding catboost switch
• b70594d6 - adding null feature value
• 75be17fa - seams working

Compare with previous version

changed the description

@mbelous thank you for fixing the default value in the case of no hits in the muon stations.

For the efficiency values: Could you please add a separate category for muons identified by Catboost? That will make it easier to compare the Catboost performance with respect to isMuon. At the moment it seems that "Correctly identified muons with isMuon" means "isMuon and Catboost".

It seems that a value of -10 for the Catboost discrimination works well already. Maybe @nkazeev can comment on that as well.

Sure, it was a bit complicated to make new category without making to much copy pasting. I will doing shortly

@dovombru Do we need to apply catboost to this part as well?

Until we have finished the studies on the Catboost muon identification / misidentification as well as throughput performance, I would not add it to any selection lines.

This can be a second step, probably with input from @thboettc and maybe from the muon group. @nkazeev are you aware of HLT1 selection lines in the x86 implementation which use Catboost?

@dcampora Could you please tell, how do you usually do this switch in Allen?

Ah, this is tricky. For now, in the case catboost is not executed, the muon_catboost_output test should pass (ie. be defaulted to > 0.0f).

In the future, the validation algorithms should become just like regular algorithms and we can do better.

Hi @mbelous ! What is the status of this MR?

ping @mbelous . What is the status of separating the Catboost category from isMuon in the checker output?

pong,@dovombru, sorry I fall out of a context, I would need to update branch, fix docker environment, then I would need to copy-paste checker output stuff, if there is no better way and put catboost cut-off as a command-line parameter. I think it would be done by the end of next week, maybe two weeks from now. Do you think it's plausible?

I suggest that you use a parameter for the catboost cut-off instead of a command-line argument (see this readme for explanations: https://gitlab.cern.ch/lhcb/Allen/-/blob/master/contributing.md#parameters-and-properties).

Then yes, it would be great if you can add the additional output to the checker. A time scale of one to two weeks sounds good.

mentioned in issue Moore#203 (closed)

mentioned in issue Moore#205 (closed)

mentioned in issue Moore#206 (closed)

marked as a Work In Progress

closed

changed the description