Umami merge requestshttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests2022-03-09T18:23:19+01:00https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/455Fixing issue with integration test + Removing old namings2022-03-09T18:23:19+01:00Alexander FrochFixing issue with integration test + Removing old namings## Summary
This MR introduces the following changes
* Removing old `MC16d` from all examples and documentation.
* Adding correct names (or more self explanatory names) of validation- and test files to the train configs in `examples/`.
...## Summary
This MR introduces the following changes
* Removing old `MC16d` from all examples and documentation.
* Adding correct names (or more self explanatory names) of validation- and test files to the train configs in `examples/`.
* Renaming integration tests `test_train_umami_cond_att`->`test_train_cond_att_umami` and `test_train_tfrecords_umami_cond_att`->`test_train_tfrecords_cond_att_umami`. When running `pytest` with `-k` to only run a certain test, the name of the tests are checked. If the given string is in the test name, the test will be run. So when running `test_train_umami` (which IS a integration test), this will trigger not only `test_train_umami` but also `test_train_umami_cond_att`. To prevent this, the test for `umami_cond_att` are renamed.
* Adding fix for merge conflicts in the changelog.md
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Documentation](https://umami-docs.web.cern.ch)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/468Changelog do not interfere anymore with just doc changes2022-03-09T15:55:12+01:00Manuel GuthChangelog do not interfere anymore with just doc changes## Summary
This MR introduces the following changes
* umamibot approves MR if only docs are concerned even if `changelog.md` was changed
## Conformity
- [x] ~[Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorith...## Summary
This MR introduces the following changes
* umamibot approves MR if only docs are concerned even if `changelog.md` was changed
## Conformity
- [x] ~[Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)~
- [x] ~[Documentation](https://umami-docs.web.cern.ch)~
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/451update 3d significance sv variable2022-03-09T13:43:25+01:00Manuel Guthupdate 3d significance sv variable## Summary
This MR introduces the following changes
* Updating jet training variable from `SV1_significance3d` to `SV1_correctSignificance3d` for r22
* Adding a check for `replaceLineInFile` if leading spaces stay same, if not a warnin...## Summary
This MR introduces the following changes
* Updating jet training variable from `SV1_significance3d` to `SV1_correctSignificance3d` for r22
* Adding a check for `replaceLineInFile` if leading spaces stay same, if not a warning is raised
* Allowing that no `cuts` are provided for samples in the preprocessing step
Relates to the following issues
* closes #141
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Documentation](https://umami-docs.web.cern.ch)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/463Restructuring gitlab pipeline folders2022-03-08T14:35:55+01:00Manuel GuthRestructuring gitlab pipeline folders## Summary
This MR introduces the following changes
* changes the folder name from `pipelines` to `.gitlab/workflow`
* updates MR and issue templates
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-to...## Summary
This MR introduces the following changes
* changes the folder name from `pipelines` to `.gitlab/workflow`
* updates MR and issue templates
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] ~[Documentation](https://umami-docs.web.cern.ch)~
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/454small todo issue improvement2022-03-01T18:18:49+01:00Manuel Guthsmall todo issue improvementhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/453small fixes2022-03-01T16:17:39+01:00Manuel Guthsmall fixesThis MR fixes the changelog to be also available in master docs and CI improvements
for plotting using now 77% WP to allow to decrease stats in CI filesThis MR fixes the changelog to be also available in master docs and CI improvements
for plotting using now 77% WP to allow to decrease stats in CI fileshttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/450Organising ToDos in sections2022-02-28T16:30:00+01:00Manuel GuthOrganising ToDos in sectionsThis MR organises the automated TODO issue in sections, divided into `general` TODOs and the TODOs related to newly introduced features in python 3.9 and 3.10.
The TODO related to `vr_overlap` is outdated since this can be realised usin...This MR organises the automated TODO issue in sections, divided into `general` TODOs and the TODOs related to newly introduced features in python 3.9 and 3.10.
The TODO related to `vr_overlap` is outdated since this can be realised using the Cuts in preprocessing or evaluationhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/444Update TFRecord reader/writer + Adding support for CADS and Umami Cond Att2022-02-28T15:59:10+01:00Alexander FrochUpdate TFRecord reader/writer + Adding support for CADS and Umami Cond AttThis MR adds the following:
* Adding multiple track collections in TFRecords is now available. Closes #109
* Adapting the reader/writer of TFRecords so the correct track collection, conditional info etc. is loaded.
* Adding possibility...This MR adds the following:
* Adding multiple track collections in TFRecords is now available. Closes #109
* Adapting the reader/writer of TFRecords so the correct track collection, conditional info etc. is loaded.
* Adding possibility to train CADS with TFRecords.
* Adapting unit tests of the TFRecords reader/writer.
* Adding Integration tests for CADS with TFRecords, Umami Cond Att and Umami Cond Att with TFRecords. Related to #142.
* Fixing issues in the evaluation chain of Umami Cond Att. Closes #143
* Update documentation of preprocessing config section `convert_to_tfrecord`.Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/448restructure docs2022-02-25T11:56:50+01:00Manuel Guthrestructure docsThis MR restructures the docs a bit
closes #137 #138 #134 #52This MR restructures the docs a bit
closes #137 #138 #134 #52https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/430Fixing all issues of darglint and removing unused functions2022-02-16T16:50:23+01:00Alexander FrochFixing all issues of darglint and removing unused functionsThis MR fixes all `darglint` issues and removes the `MakePlots` and `Plot_vars` fuctions (outdated). Related to #61 and #145
The file `umami/train_tools/Plotting.py` is now excluded from darglint due to issues of darglint with the sphi...This MR fixes all `darglint` issues and removes the `MakePlots` and `Plot_vars` fuctions (outdated). Related to #61 and #145
The file `umami/train_tools/Plotting.py` is now excluded from darglint due to issues of darglint with the sphinx documentation style in the docstrings
closes https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/issues/97 #145Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/432ci tag docker - another one2022-02-16T16:40:54+01:00Manuel Guthci tag docker - another onelet's hope it is fixed now ....let's hope it is fixed now ....https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/431ci tag docker fix2022-02-16T15:13:47+01:00Manuel Guthci tag docker fixhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/429ci tag docker2022-02-16T14:18:18+01:00Manuel Guthci tag dockerhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/428Increasing umami version to 0.62022-02-16T13:21:50+01:00Manuel GuthIncreasing umami version to 0.6The following new features were added since [release 0.5](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/tags/0.5):
- CI improvements
- latest samples added to documentation
- packages were upgraded
- new Python AP...The following new features were added since [release 0.5](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/tags/0.5):
- CI improvements
- latest samples added to documentation
- packages were upgraded
- new Python API added for plotting of ROC curves
- Added normalisation option to input plotting
- logging level for all tests are set by default to debug
- Added optional results filename extension
- Added docs for pdf method and parallelise pdf method
- Possibility to modify names of track variables in config files
- Added new sphinx documentation
- Black was added in CI
- fraction contour plots were added
- bb-jets category colour was changed
- Copying now config files during pre-processing
- several doc string updates
- docs update for taggers (merged them)
- save divide added
- flexible validation sample definition in config addedhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/427Making sure also in master pipeline the base image is used which is built in CI2022-02-16T12:58:12+01:00Manuel GuthMaking sure also in master pipeline the base image is used which is built in CIhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/417Adding increased code coverage2022-02-16T11:04:39+01:00Jackson BarrAdding increased code coveragehttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/421Updating doc strings for yaml tools2022-02-15T21:21:58+01:00Manuel GuthUpdating doc strings for yaml toolshttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/422CI improvements2022-02-15T18:51:20+01:00Manuel GuthCI improvementsThis MR
- fixes pylint, which started to fail after !418
- updates until test CI definition making it easier
- updates CI for pushing to hub removing unnecessary duplicationThis MR
- fixes pylint, which started to fail after !418
- updates until test CI definition making it easier
- updates CI for pushing to hub removing unnecessary duplicationhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/418Upgrading packages2022-02-15T16:48:54+01:00Alexander FrochUpgrading packagesThis MR updates multiple packages. The main package change here will be switching from `tensorflow` 2.7.0 to 2.8.0This MR updates multiple packages. The main package change here will be switching from `tensorflow` 2.7.0 to 2.8.0Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/419Updating packaging and small issues2022-02-15T12:55:12+01:00Manuel GuthUpdating packaging and small issuesThis MR updates some small issues with packaging and now switches to `pip` instead of `python setup.py install/develop`
closes #131This MR updates some small issues with packaging and now switches to `pip` instead of `python setup.py install/develop`
closes #131