plotting_umami not compatible with multiple signal classes
Summary
I think the support for multiple signal classes is not yet there yet.
I think the calculation of the discriminant scores (and rejections) is adapted, but not the code for the evaluation plots in umami/evaluation_tools/PlottingFunctions.py
.
The link above for example shows the part in the code where multiple signal classes lead to a problem in the pt vs efficiency plot.
Steps to reproduce
I ran plotting_umami.py -c plotting_umami_crosscheck.yaml
with the following config:
# Evaluation parameters
Eval_parameters:
Path_to_models_dir: /home/fr/fr_fr/fr_jb666/b-tagging/packages/umami/umami
model_name: training_bb_dips_loose_16M_jets
epoch: 200
epoch_to_name: True
Dips_pT_vs_beff:
type: "pT_vs_eff"
models_to_plot:
dips:
evaluation_file: "/home/fr/fr_fr/fr_jb666/b-tagging/packages/umami/umami/training_bb_dips_loose_16M_jets/results/results_large_eval_stats-200_bb.h5"
data_set_name: "ttbar_r21"
label: "$bb$-DIPS loose"
tagger_name: "dips_bb_b_discriminant"
plot_settings:
bin_edges: [0, 20, 30, 40, 60, 85, 110, 140, 175, 250, 400, 1000]
flavour: "cjets"
class_labels: ["ujets", "cjets", "singlebjets", "bbjets"]
main_class: ["singlebjets", "bbjets"]
WP: 0.70
WP_Line: True
Fixed_WP_Bin: False
figsize: [7, 5]
logy: False
UseAtlasTag: True
AtlasTag: "Internal Simulation"
SecondTag: "\n$\\sqrt{s}=13$ TeV, PFlow Jets,\n$t\\bar{t}$ Test Sample"
yAxisAtlasTag: 0.9
yAxisIncrease: 1.3
What is the current bug behavior?
The plots are not produced, since specifying a list as main_class
is not supported
What is the expected correct behavior?
Specifying a list as main_class
should take all entries in the list as part of the signal.
Relevant logs and/or screenshots
>> Singularity >> fr_jb666@login1 ~/b-tagging/submission_scripts/evaluation_bb_DIPS (master) $ plotting_umami.py -c plotting_umami_crosscheck.yaml
0%| | 0/2 [00:00<?, ?it/s]WARNING:umami: You specified the argument yAxisAtlasTag, which is no longer supported and will be ignored.
50%|███████████████████████████████████████████████████████████████████████████████████████████████████ | 1/2 [00:02<00:02, 2.24s/it]
Traceback (most recent call last):
File "/home/fr/fr_fr/fr_jb666/b-tagging/packages/umami_dev/python_install/bin/plotting_umami.py", line 7, in <module>
exec(compile(f.read(), __file__, 'exec'))
File "/home/fr/fr_fr/fr_jb666/b-tagging/packages/umami_dev/umami/plotting_umami.py", line 1486, in <module>
main(parser_args)
File "/home/fr/fr_fr/fr_jb666/b-tagging/packages/umami_dev/umami/plotting_umami.py", line 1475, in main
SetUpPlots(
File "/home/fr/fr_fr/fr_jb666/b-tagging/packages/umami_dev/umami/plotting_umami.py", line 1411, in SetUpPlots
plot_pT_vs_eff(
File "/home/fr/fr_fr/fr_jb666/b-tagging/packages/umami_dev/umami/plotting_umami.py", line 1036, in plot_pT_vs_eff
uet.plot_pt_dependence(
File "/home/fr/fr_fr/fr_jb666/b-tagging/packages/umami_dev/umami/evaluation_tools/PlottingFunctions.py", line 799, in plot_pt_dependence
is_signal = df_results["labels"] == index_dict[main_class]
TypeError: unhashable type: 'list'
Possible fixes
I think there are quite a few occurrences in umami/evaluation_tools/PlottingFunctions.py
which should be changed to the list
syntax instead of str