Umami merge requestshttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests2023-10-30T20:22:17+01:00https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/743Upp v0032023-10-30T20:22:17+01:00Ivan OleksiyukUpp v003## Summary
This MR introduces the following changes
* Change to a version upp that we need for integration
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changel...## Summary
This MR introduces the following changes
* Change to a version upp that we need for integration
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Documentation](https://umami-docs.web.cern.ch)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Rewrite PreprocessingIvan OleksiyukIvan Oleksiyukhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/742Upp version update2023-10-30T12:07:44+01:00Ivan OleksiyukUpp version update## Summary
This MR introduces the following changes
* Updates upp version to 0.0.3 that is required for upp integration
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/m...## Summary
This MR introduces the following changes
* Updates upp version to 0.0.3 that is required for upp integration
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [ ] [Documentation](https://umami-docs.web.cern.ch)
- [ ] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [ ] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/686Draft: Adding example configs for Xbb/Xcc preprocessing and fixing a bug with...2023-04-01T21:50:28+02:00Osama Karkouto.karkout@nikhef.nlDraft: Adding example configs for Xbb/Xcc preprocessing and fixing a bug with the boosted QCD flavour in the global config.## Summary
This MR introduces the following changes
Adding example configs for Xbb/Xcc preprocessing and fixing a bug with the boosted QCD flavour in the global config.
*
*
Relates to the following issues:
* closes #223
## Conformity...## Summary
This MR introduces the following changes
Adding example configs for Xbb/Xcc preprocessing and fixing a bug with the boosted QCD flavour in the global config.
*
*
Relates to the following issues:
* closes #223
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [ ] [Documentation](https://umami-docs.web.cern.ch)
- [ ] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [ ] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Osama Karkouto.karkout@nikhef.nlOsama Karkouto.karkout@nikhef.nlhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/715Draft: Merge current Preprocessing Rewrite2023-03-28T12:15:49+02:00Alexander FrochDraft: Merge current Preprocessing Rewrite## Summary
This MR introduces the following changes
* Merge the current status of the preprocessing rewrite into master
Relates to the following issues
* Closes #232 #235 #237
* Touches #60 #152 #233 #234
## Conformity
- [ ] [Chang...## Summary
This MR introduces the following changes
* Merge the current status of the preprocessing rewrite into master
Relates to the following issues
* Closes #232 #235 #237
* Touches #60 #152 #233 #234
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/704Draft: Sweep Master into Preprocessing-Rewrite2023-02-27T11:07:48+01:00Alexander FrochDraft: Sweep Master into Preprocessing-Rewrite## Summary
This MR introduces the following changes
* Sweeping master into preprocessing-rewrite## Summary
This MR introduces the following changes
* Sweeping master into preprocessing-rewriteAlexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/673Moving/adding checks for options to train_config configuration2022-11-23T15:11:58+01:00Alexander FrochMoving/adding checks for options to train_config configuration## Summary
This MR introduces the following changes
* Moving check for `working_point` and `n_jets` to train_config configuration for validation/evaluation with a freshly trained tagger.
* Adding check for `frac_values` to train_config...## Summary
This MR introduces the following changes
* Moving check for `working_point` and `n_jets` to train_config configuration for validation/evaluation with a freshly trained tagger.
* Adding check for `frac_values` to train_config configuration for validation/evaluation with a freshly trained tagger.
* Adding checks for `tagger` and `frac_values_comp` to the train_config configuration for the validation/evaluation without a freshly trained tagger.
* Removing unused checks in various places for the moved/added checks.
* Adding unit tests for the configuration for the new checks
Relates to the following issues
* Closing #208
## Conformity
- [X] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/602Draft: Adding Track Classification Auxiliary Loss for CADS2022-08-09T11:07:34+02:00Alexander FrochDraft: Adding Track Classification Auxiliary Loss for CADS## Summary
This MR introduces the following changes
* Adding Track classification auxiliary loss for CADS
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog...## Summary
This MR introduces the following changes
* Adding Track classification auxiliary loss for CADS
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/603Draft: Fixing tensorflow import errors2022-08-05T11:33:44+02:00Alexander FrochDraft: Fixing tensorflow import errors## Summary
This MR introduces the following changes
* Fixing the import errors from `tensorflow`. The lazy import tensorflow uses is not accepted by the linters/interpreters correctly, so you get errors although this works fine. Adding...## Summary
This MR introduces the following changes
* Fixing the import errors from `tensorflow`. The lazy import tensorflow uses is not accepted by the linters/interpreters correctly, so you get errors although this works fine. Adding the `.python` fixes the issue for the linters/interpreters.
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/589Draft: soft electron features2022-07-27T17:10:37+02:00Junwen XiongDraft: soft electron features## Summary (work in progress)
This MR introduces the following changes to enabling training with soft electrons and ttbar-only samples for the latest umami master branch (d428da88):
* Preprocessing: add config file for the preprocessi...## Summary (work in progress)
This MR introduces the following changes to enabling training with soft electrons and ttbar-only samples for the latest umami master branch (d428da88):
* Preprocessing: add config file for the preprocessing with soft electrons (TBF)
* Training: add config file for the training with soft electrons (TBF)
* Evaluating: add functions to evaluate training with soft electrons (TBF)
* Plotting: add functions to evaluate training with soft electrons and evaluate PHF roc curves
(TBF)
Relates to the following issues:
* The issue about making the code and documentation available can be found here: https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/17
## Conformity
(TBD)
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [ ] [Documentation](https://umami-docs.web.cern.ch)
- [ ] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [ ] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/578Mtanasin master patch 685132022-07-04T17:50:09+02:00Martino TanasiniMtanasin master patch 68513## Summary
This MR introduces the following changes
*
*
Relates to the following issues
*
*
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [...## Summary
This MR introduces the following changes
*
*
Relates to the following issues
*
*
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Documentation](https://umami-docs.web.cern.ch)
- [ ] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [ ] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/523Draft: ci improvements2022-05-10T10:50:08+02:00Manuel GuthDraft: ci improvements## Summary
This MR introduces the following changes
* simplifying some CI jobs## Summary
This MR introduces the following changes
* simplifying some CI jobshttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/492Resolve "mkdocs job crashes"2022-03-25T13:27:26+01:00Manuel GuthResolve "mkdocs job crashes"## Summary
This MR introduces the following changes
* updating medics material version to `8.2.7`
Related issues
* closes #161
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umam...## Summary
This MR introduces the following changes
* updating medics material version to `8.2.7`
Related issues
* closes #161
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Documentation](https://umami-docs.web.cern.ch)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)
Closes #161Manuel GuthManuel Guthhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/489Draft: Small fix in histogram plotting API2022-03-22T18:00:16+01:00Joschka BirkDraft: Small fix in histogram plotting API## Summary
This MR introduces the following changes
* Set ratio uncertainty to zero for reference ratio (horizontal line at 1)
## Conformity
- ~~[ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/...## Summary
This MR introduces the following changes
* Set ratio uncertainty to zero for reference ratio (horizontal line at 1)
## Conformity
- ~~[ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)~~
- ~~[] [Documentation](https://umami-docs.web.cern.ch)~~
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Joschka BirkJoschka Birkhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/486Small fix in plotting_epoch_performance script2022-03-22T10:04:34+01:00Joschka BirkSmall fix in plotting_epoch_performance script## Summary
This MR introduces the following changes
* small fix for the following error
```bash
$ plotting_epoch_performance.py -c training_bb_dips_loose_16M_jets/metadata/Dips-PFlow-Training-config.yaml --dict training_bb_dips_loose_1...## Summary
This MR introduces the following changes
* small fix for the following error
```bash
$ plotting_epoch_performance.py -c training_bb_dips_loose_16M_jets/metadata/Dips-PFlow-Training-config.yaml --dict training_bb_dips_loose_16M_jets/validation_WP0p77_300000jets_Dict.json
INFO:umami: Using train config file training_bb_dips_loose_16M_jets/metadata/Dips-PFlow-Training-config.yaml
INFO:umami: Using config file /home/fr/fr_fr/fr_jb666/b-tagging/packages/umami/umami/training_bb_dips_loose_16M_jets/metadata/PFlow-Preprocessing.yaml
Traceback (most recent call last):
File "/home/fr/fr_fr/fr_jb666/b-tagging/packages/umami_dev/python_install/bin/plotting_epoch_performance.py", line 7, in <module>
exec(compile(f.read(), __file__, 'exec'))
File "/home/fr/fr_fr/fr_jb666/b-tagging/packages/umami_dev/umami/plotting_epoch_performance.py", line 207, in <module>
main(parser_args, training_config, preprocessing_config)
File "/home/fr/fr_fr/fr_jb666/b-tagging/packages/umami_dev/umami/plotting_epoch_performance.py", line 180, in main
for comp_tagger in comp_tagger_list
UnboundLocalError: local variable 'comp_tagger_list' referenced before assignment
```
## Conformity
- ~~[x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)~~
- ~~[x] [Documentation](https://umami-docs.web.cern.ch)~~
- ~~[x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)~~
- ~~[x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)~~Joschka BirkJoschka Birkhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/437Master2022-02-20T10:11:23+01:00Helena SantosMasterThese 3 files suitable to train heavy ion data. They are:
The config file for Preprocessing: HI-Preprocessing.yaml
The Dictfile for the scaling and shifting (json)
# List of variables for training (yaml)
Dips_Variables-HI.yaml
and the ...These 3 files suitable to train heavy ion data. They are:
The config file for Preprocessing: HI-Preprocessing.yaml
The Dictfile for the scaling and shifting (json)
# List of variables for training (yaml)
Dips_Variables-HI.yaml
and the config file for training
Dips-HI-Training-config.yamlhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/423Draft: Downgrade pandas (Not supported for Pytorch)2022-02-15T16:46:57+01:00Alexander FrochDraft: Downgrade pandas (Not supported for Pytorch)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/394WIP: Fix evaluation of trained tagger2022-02-03T17:18:44+01:00Joschka BirkWIP: Fix evaluation of trained taggerThis fixes bugs which made the `evaluate_model.py` script unusable for evaluating trained taggers.
Maybe we should also add an integration test for this?This fixes bugs which made the `evaluate_model.py` script unusable for evaluating trained taggers.
Maybe we should also add an integration test for this?Joschka BirkJoschka Birkhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/389WIP: Generalise output directory of evaluation results2022-02-02T14:54:26+01:00Joschka BirkWIP: Generalise output directory of evaluation resultsThis adds another argument to the `evaluate_model.py` script, which allows to specify the folder where the results should be saved.
The default value is `results`, which was the hardcoded value until now.This adds another argument to the `evaluate_model.py` script, which allows to specify the folder where the results should be saved.
The default value is `results`, which was the hardcoded value until now.https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/368Adding check of output of the taggers as Integration test2022-01-25T10:04:22+01:00Alexander FrochAdding check of output of the taggers as Integration testThis MR addresses #121. The output of the train integration tests is now plotted **and** checked against control plots.
* Adding possibility to set the random seed of `tensorflow` via a command line argument.
* Adding control plots for ...This MR addresses #121. The output of the train integration tests is now plotted **and** checked against control plots.
* Adding possibility to set the random seed of `tensorflow` via a command line argument.
* Adding control plots for the results of the taggers.
* Adding function in `plotting_umami` integration test, which checks the produced plots against the control plots.
Closes #121Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/362Merging master in protected branch2022-01-21T14:44:14+01:00Alexander FrochMerging master in protected branch