Umami merge requestshttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests2022-08-17T18:39:05+02:00https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/619Fixing multiple issues with the fraction contour plots2022-08-17T18:39:05+02:00Alexander FrochFixing multiple issues with the fraction contour plots## Summary
This MR introduces the following changes
* Fixing an issue with the permutation calculation of the fraction values.
* Fixing marker position check. If the marker is not on the plotted line, an error will be raised.
* Fixing ...## Summary
This MR introduces the following changes
* Fixing an issue with the permutation calculation of the fraction values.
* Fixing marker position check. If the marker is not on the plotted line, an error will be raised.
* Fixing the plotting for multiple rejections with one or more `fixed_rejections`.
* Changing the `plottype` for the contour plots from `Frac_contour` to `fraction_contour`. For backward compatibility reasons, the old `Frac_contour` is still supported but a TODO is added to remove it in future versions.
* A little clean up in the `evaluate_model.py`
## Conformity
- [X] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/618Adding truth label to results file + Fix flavour retrieval in plotting2022-08-16T13:33:43+02:00Alexander FrochAdding truth label to results file + Fix flavour retrieval in plotting## Summary
This MR introduces the following changes
* **Note:** Unit test files changed! No pipeline will success until this is merged!
* Adding the jet truth labels to the results file produced by `evaluate_model.py` (Like `HadronCone...## Summary
This MR introduces the following changes
* **Note:** Unit test files changed! No pipeline will success until this is merged!
* Adding the jet truth labels to the results file produced by `evaluate_model.py` (Like `HadronConeExclTruthLablID`)
* Removing error prone flavour retrieval from plotting code. This was based on the order of `class_labels` (which needed to be similar to the ones in the preprocess/train config, otherwise this would lead to wrong flavour retrieval). Now it is based on the truth labels which are stored in the results file.
* Adding dummy empty `label` values for plotting. If you only want to plot one model without a model `label` and only the flavour names, this would produce (at the moment) an error. When no `label` is given for the model, an empty string is added to the `model_labels` list to keep the lists dimensionality correct.
* Fixing working point values in the discriminant plots. The x-value was given as label to `puma` which lead to wrong numbers above the vertical lines.
* Sorting the working points which are to be plotted in the discriminant plot in decreasing order so they are correctly printed when `same_height_WP` is `False`.
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/611Removing var_dict from train config2022-08-10T15:15:24+02:00Manuel GuthRemoving var_dict from train config## Summary
This MR introduces the following changes
* removing `var_dict` from training config, this needs to be always the same as specified in the preprocessing config, i.e. since we can access the preprocessing config from the train...## Summary
This MR introduces the following changes
* removing `var_dict` from training config, this needs to be always the same as specified in the preprocessing config, i.e. since we can access the preprocessing config from the training config, we don't need it anymore
* loading preprocessing config directly in train config
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Manuel GuthManuel Guthhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/604Splitting configuration into base class and some preprocessing adaptions2022-08-08T12:53:11+02:00Manuel GuthSplitting configuration into base class and some preprocessing adaptions## Summary
This MR introduces the following changes
* Splits the configuration into base class (needs to be also applied to training config later)
* Some preprocessing changes in docs and in configuration
* Speeding up scaling and shif...## Summary
This MR introduces the following changes
* Splits the configuration into base class (needs to be also applied to training config later)
* Some preprocessing changes in docs and in configuration
* Speeding up scaling and shifting
* adding new option to preprocessing config `n_jets_scaling` which allows to use only a part of the jets to calculate scaling and shifting values
* removing unnecessary variable loading into memory to get shape of jets
Relates to the following issues
* first step towards #181
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Documentation](https://umami-docs.web.cern.ch)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Manuel GuthManuel Guthhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/599Merging evaluate_model script funtions + adapt pt_vs plots to be var_vs plots2022-08-05T13:45:06+02:00Alexander FrochMerging evaluate_model script funtions + adapt pt_vs plots to be var_vs plots## Summary
This MR introduces the following changes
* Merging the (mostly for all functions) doubled code of `evaluate_model.py` in one function.
* Rename `add_variable_eval` to `add_eval_variables` and make this option available for a...## Summary
This MR introduces the following changes
* Merging the (mostly for all functions) doubled code of `evaluate_model.py` in one function.
* Rename `add_variable_eval` to `add_eval_variables` and make this option available for all taggers.
* Adapting `pT_vs` plots so that you can now define which variable you want to plot the efficiency/rejection against.
## Conformity
- [X] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/596Adding x_axis_granularity argument + Fixing evaluation_file plotting issue2022-08-01T13:06:30+02:00Alexander FrochAdding x_axis_granularity argument + Fixing evaluation_file plotting issue## Summary
This MR introduces the following changes
* Adding the `x_axis_granularity` option to the `evaluate_model.py` scripts. The value defines how many steps of b-efficiency are calculated for the ROC curves.
* Fixing an issue when...## Summary
This MR introduces the following changes
* Adding the `x_axis_granularity` option to the `evaluate_model.py` scripts. The value defines how many steps of b-efficiency are calculated for the ROC curves.
* Fixing an issue when giving an `evaluation_file` in the `plotting_umami.py` config where the file was not correctly handed to the `n_jets` extraction.
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/592Adding metadata information to training file2022-07-28T13:46:23+02:00Alexander FrochAdding metadata information to training file## Summary
This MR introduces the following changes
* Adapting function name from `GetVariableDict` to `get_variable_dict`.
* Removing some useless `self.`
* Adds metadata information to the final training file. For the jets and the tr...## Summary
This MR introduces the following changes
* Adapting function name from `GetVariableDict` to `get_variable_dict`.
* Removing some useless `self.`
* Adds metadata information to the final training file. For the jets and the tracks, the variables are now shown as a list with the correct order (corresponding to the columns of the variables in the file) when running `h5ls -v` on the file. This now looks for the jet and track variables like this:
<img src="/uploads/b262a57fe7fed7a76d82bb2dd19335ea/Bildschirmfoto_von_2022-07-27_01-16-40.png" alt="metadata" width="700"/>
Relates to the following issues
* Closes #70
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/590non transparent backgrounds2022-07-26T15:57:12+02:00Manuel Guthnon transparent backgrounds## Summary
This MR introduces the following changes
* making all plots by default with non-transparent background
* updating puma version to `v0.1.6` (needs also this puma PR https://github.com/umami-hep/puma/pull/106)
Relates to the ...## Summary
This MR introduces the following changes
* making all plots by default with non-transparent background
* updating puma version to `v0.1.6` (needs also this puma PR https://github.com/umami-hep/puma/pull/106)
Relates to the following issues
* Closes https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/issues/183
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Manuel GuthManuel Guthhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/585Fixing track masking for the input variable plots2022-07-23T01:32:14+02:00Alexander FrochFixing track masking for the input variable plots## Summary
This MR introduces the following changes
* Adding `valid` as the default track mask variable. If `valid` is not available, `ptfrac` will be used instead.
* Adding new unit test R22 test file (with `tracks_loose` and `valid` ...## Summary
This MR introduces the following changes
* Adding `valid` as the default track mask variable. If `valid` is not available, `ptfrac` will be used instead.
* Adding new unit test R22 test file (with `tracks_loose` and `valid` flag present) (**Will be added when this MR is approved, otherwise all other pipelines will fail.**)
Relates to the following issues
* Closes #196
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/584Removing casefold in tagger name retrieval2022-07-20T10:57:28+02:00Alexander FrochRemoving casefold in tagger name retrieval## Summary
This MR introduces the following changes
* Removing the `.casefold()` functionality when retrieving the `tagger_name` from the plotting config file in `plotting_umami`
Relates to the following issues
* Closes #197
## Con...## Summary
This MR introduces the following changes
* Removing the `.casefold()` functionality when retrieving the `tagger_name` from the plotting config file in `plotting_umami`
Relates to the following issues
* Closes #197
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/582Fixing lazy formatting2022-07-07T15:04:10+02:00Manuel GuthFixing lazy formatting## Summary
This MR introduces the following changes
* fixes the recommended lazy formatting for logging (removing `logging-fstring-interpolation` errors)
Relates to the following issues
* first step towards #194
## Conformity
- [x...## Summary
This MR introduces the following changes
* fixes the recommended lazy formatting for logging (removing `logging-fstring-interpolation` errors)
Relates to the following issues
* first step towards #194
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Manuel GuthManuel Guthhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/570Adding consistent n_jets naming2022-07-06T18:17:33+02:00Alexander FrochAdding consistent n_jets naming## Summary
This MR introduces the following changes
* Renames all `nJets` and `njets` to `n_jets`. This will break all kind of backward compatability, so maybe before merging this, we should make another tag.
Relates to the following ...## Summary
This MR introduces the following changes
* Renames all `nJets` and `njets` to `n_jets`. This will break all kind of backward compatability, so maybe before merging this, we should make another tag.
Relates to the following issues
* Step towards #167
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/576Updating docs and renaming hybrids folder2022-07-06T14:58:27+02:00Alexander FrochUpdating docs and renaming hybrids folder## Summary
This MR introduces the following changes
* Updating the docs for the preprocessing and the training
* Renaming the `hybrids` folder. There are no hybrids inside, only the prepared samples.
* Removing hardcoded `DL1` as namin...## Summary
This MR introduces the following changes
* Updating the docs for the preprocessing and the training
* Renaming the `hybrids` folder. There are no hybrids inside, only the prepared samples.
* Removing hardcoded `DL1` as naming from the `evaluate_model.py`
* Adapt plotting config accordingly
## Conformity
- [X] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/569Splitting CADS and DIPS Attention2022-06-23T14:55:28+02:00Alexander FrochSplitting CADS and DIPS Attention## Summary
This MR introduces the following changes
* Splitting up the training with and without conditional information into two taggers.
* cads: DIPS with conditional and attention
* dips_attention: DIPS without conditional but with ...## Summary
This MR introduces the following changes
* Splitting up the training with and without conditional information into two taggers.
* cads: DIPS with conditional and attention
* dips_attention: DIPS without conditional but with attention.
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/572Switch to puma 0.1.32022-06-23T14:40:35+02:00Joschka BirkSwitch to puma 0.1.3## Summary
This MR introduces the following changes
* update the version of `puma-hep` in the requirements
* small adjustments to make umami work with the new version
* add the `ratio_group` argument to some histogram plots (this was...## Summary
This MR introduces the following changes
* update the version of `puma-hep` in the requirements
* small adjustments to make umami work with the new version
* add the `ratio_group` argument to some histogram plots (this was included in the `flavour` argument before)
* update the expected images for roc cuves with vertical lines drawn in the plot. The new default colour for these new lines was changed in `puma` (from darkred to black, since the colour of bb-jets is now darkred)
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Documentation](https://umami-docs.web.cern.ch)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Joschka BirkJoschka Birkhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/566Fixing uncertainty calculation for the ROC curves2022-06-21T10:47:32+02:00Alexander FrochFixing uncertainty calculation for the ROC curves## Summary
This MR introduces the following changes
* Changes the `njets` handed to the ROC plots so that the uncertainties are correctly calculated. The correct calculation of these uncertainties is part of `puma`
Relates to the foll...## Summary
This MR introduces the following changes
* Changes the `njets` handed to the ROC plots so that the uncertainties are correctly calculated. The correct calculation of these uncertainties is part of `puma`
Relates to the following issues
* Closes #144
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/556Rework saliency plots to use puma2022-06-16T13:16:04+02:00Alexander FrochRework saliency plots to use puma## Summary
This MR introduces the following changes
* Using puma to create saliency plots
* Fix disabled unit test for saliency plots
* Re-implement saliency plots (remove hard-coded stuff)
* Adding docs for saliency
* Removing PyATLAS...## Summary
This MR introduces the following changes
* Using puma to create saliency plots
* Fix disabled unit test for saliency plots
* Re-implement saliency plots (remove hard-coded stuff)
* Adding docs for saliency
* Removing PyATLASStyle. Umami now works completely with Atlasify
Relates to the following issues
* Closes #176
* Closes #155
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Documentation](https://umami-docs.web.cern.ch)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/557Adding verbose option to executable scripts2022-06-08T17:49:27+02:00Alexander FrochAdding verbose option to executable scripts## Summary
This MR introduces the following changes
* Adding verbose option to all executable scripts.
* Adding `plot_input_variables.py` to executable scripts.
Relates to the following issues
* Closes #154
## Conformity
- [x] [Cha...## Summary
This MR introduces the following changes
* Adding verbose option to all executable scripts.
* Adding `plot_input_variables.py` to executable scripts.
Relates to the following issues
* Closes #154
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/554Moving Plotting Files in one folder2022-06-07T17:26:45+02:00Alexander FrochMoving Plotting Files in one folder## Summary
This MR introduces the following changes
* Moving all plotting functions of umami (the eval_plots, train_plots and preprocessing_plots) in one module `plotting_tools`.
* Renaming plotting functions so they comply with the PE...## Summary
This MR introduces the following changes
* Moving all plotting functions of umami (the eval_plots, train_plots and preprocessing_plots) in one module `plotting_tools`.
* Renaming plotting functions so they comply with the PEP8 guidelines.
Relates to the following issues
* Closes #186
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/549Fixing some small issues2022-05-31T11:10:38+02:00Alexander FrochFixing some small issues## Summary
This MR introduces the following changes
* Fixing some naming issues in the `plotting_umami.py` example configs + docs.
* Init a default train config with default values if the loaded one does not have these.
* Re-ordering t...## Summary
This MR introduces the following changes
* Fixing some naming issues in the `plotting_umami.py` example configs + docs.
* Init a default train config with default values if the loaded one does not have these.
* Re-ordering the lines in the Rejection per epoch plots from `plotting_epoch_performance`. Taggers loaded from files are now plotted first, then the taggers from local trainings and last the model which is to be plotted.
* Fixing some types and remove some pylint ignores which are not needed anymore.
* Fixed the usage of `trained_taggers` and `taggers_from_file`. `trained_taggers` was not used correctly and the label from `taggers_from_file` was not correctly loaded.
## Conformity
- [X] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Froch