Umami merge requestshttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests2022-03-16T15:18:47+01:00https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/480Add example plots to docs2022-03-16T15:18:47+01:00Joschka BirkAdd example plots to docs## Summary
This MR introduces the following changes
* adds a job to the CI pipeline which runs the example scripts in `examples/plotting` and hands the resulting plots as artifacts to the job which creates the docs
* updates in the doc...## Summary
This MR introduces the following changes
* adds a job to the CI pipeline which runs the example scripts in `examples/plotting` and hands the resulting plots as artifacts to the job which creates the docs
* updates in the docs, adding the above mentioned plots
* adds docs section for pt_vs_eff plot
* adds optional `add_pt` argument to dummy data
* fixing small bug in `replace_placeholders_in_md.py` which resulted in one line missing at the end
Relates to the following issues
* first step towards #159
The result is that we can add the output plot of an example script directly to the docs, which then also corresponds to the most recent version of the code.
<img src="/uploads/4ad17c915dd654bd2c15dbb7bd840225/image.png" height="800">
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- ~~[x] [Documentation](https://umami-docs.web.cern.ch)~~
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Joschka BirkJoschka Birkhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/341Add helper function to extract validation file identifier2022-01-17T13:30:24+01:00Joschka BirkAdd helper function to extract validation file identifierAs a first step I added a helper function which helps to obtain the identifiers of different validation files later on.
E.g. we want to be able to extract the identifier "ttbar" from the string "X_valid_ttbar".As a first step I added a helper function which helps to obtain the identifiers of different validation files later on.
E.g. we want to be able to extract the identifier "ttbar" from the string "X_valid_ttbar".Joschka BirkJoschka Birkhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/364adding automated todo issue update in master ci2022-01-24T09:40:28+01:00Manuel Guthadding automated todo issue update in master cicloses #118closes #118https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/407adding black in ci2022-02-09T14:48:11+01:00Manuel Guthadding black in cihttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/476Adding code blocks to docs directly from repo2022-04-05T14:25:06+02:00Joschka BirkAdding code blocks to docs directly from repo## Summary
This MR introduces the following changes
* Automated replacement of file-placeholders in documentation using a simple python script. This allows to get rid of code blocks in cases where we copy over whole files.
Relates to ...## Summary
This MR introduces the following changes
* Automated replacement of file-placeholders in documentation using a simple python script. This allows to get rid of code blocks in cases where we copy over whole files.
Relates to the following issues
* First step towards #111
### Example
The file placeholders of the form `§§§<filename>:<start>:<end>§§§` in the markdown file on the left will be replaced with the content of the actual files (from line `<start>` to line `<end>`), resulting in the docs page on the right. This all happens in the CI pipeline, such that the actual files of the repo are untouched.
![image](/uploads/f73b87361a151e70561f8a40a3ab0126/image.png)
## ToDos
- [x] make script more general such that we don't have to modify the ci config every time a new script needs replacements (probably the script should just go throught all documentation files)
- [x] add support for line numbers
- [x] change to syntax `§§§<filename>:start:end§§§`
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [x] [Documentation](https://umami-docs.web.cern.ch)
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Joschka BirkJoschka Birkhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/254Adding conditional attention2021-11-24T09:52:39+01:00Alexander FrochAdding conditional attentionThis MR adds the following:
- Adding new train script for DIPS Conditional Attention + example train config.
- Adding support for new DIPS Conditional Attention model in the validation/evaluation chain.
- Adding integration test for tra...This MR adds the following:
- Adding new train script for DIPS Conditional Attention + example train config.
- Adding support for new DIPS Conditional Attention model in the validation/evaluation chain.
- Adding integration test for train/validation/evaluation of DIPS Conditional Attention.
- Fixing some issues in the `tf_tools/models` with the masking (also adapted the unit test).
- Adding new generator for DIPS Conditional Attention.
- Adding some flexibility to the loading of the `loading_validation_data` functions.
- Adding compatibility of the `evaluate_model.py` for DIPS and DIPS Conditional Attention.
- Adding DIPS Conditional Attention to possible models for `plotting_epoch_performance.py`
- Adding DIPS Conditional Attention train script to the `setup.py`.
- Make the calculation of the Saliency maps steerable in the train config of the DIPS models.Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/291adding doc string linting2021-12-07T14:04:56+01:00Manuel Guthadding doc string lintingfirst step towards #97
So far the doc string check test is only performed if there was at all any doc string in the function and the CI stage is allowed to fail for now
This also means the CI will from now on display that it ended wit...first step towards #97
So far the doc string check test is only performed if there was at all any doc string in the function and the CI stage is allowed to fail for now
This also means the CI will from now on display that it ended with a warning until all the docstring errors are fixedhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/519Adding error message if file in placeholder does not exist2022-04-14T15:25:24+02:00Joschka BirkAdding error message if file in placeholder does not exist## Summary
This MR introduces the following changes
* adds that the md-placeholder script raises an error if the file specified in a placeholder is not found (to avoid moving files without changing the path in the placeholders)
* fixin...## Summary
This MR introduces the following changes
* adds that the md-placeholder script raises an error if the file specified in a placeholder is not found (to avoid moving files without changing the path in the placeholders)
* fixing wrong path in markdown placeholder
## Example
The invalid placeholder below (fixed in this MR) did not raise an error so far.
Now it makes the job fail (yay)
<img src=/uploads/27fd4da9fecdf8fe664ffa10f0d8c7ff/image.png width=1200>
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- ~[ ] [Documentation](https://umami-docs.web.cern.ch)~
- [x] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [x] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Joschka BirkJoschka Birkhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/331Adding fix for DIPS/UMAMI integration test2022-01-11T15:39:44+01:00Alexander FrochAdding fix for DIPS/UMAMI integration testThis MR adds:
* Fixes a small issue with the integration tests of DIPS and UMAMI. The tests where called multiple times due to wrongly specifying which test should run (contributes to #107).
* Adding an exception to pylint (too-many-inst...This MR adds:
* Fixes a small issue with the integration tests of DIPS and UMAMI. The tests where called multiple times due to wrongly specifying which test should run (contributes to #107).
* Adding an exception to pylint (too-many-instance-attributes).Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/417Adding increased code coverage2022-02-16T11:04:39+01:00Jackson BarrAdding increased code coveragehttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/660Adding integration test for DL1* with tfrecords2022-10-31T17:46:49+01:00Alexander FrochAdding integration test for DL1* with tfrecords## Summary
This MR introduces the following changes
* Enable hybrid validation sample creation for all integration tests
* Adding integration test for DL1* with tfrecords
* Fixing issue in the tfrecord writing/loading for DL1*
## Conf...## Summary
This MR introduces the following changes
* Enable hybrid validation sample creation for all integration tests
* Adding integration test for DL1* with tfrecords
* Fixing issue in the tfrecord writing/loading for DL1*
## Conformity
- [X] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [X] [Documentation](https://umami-docs.web.cern.ch)
- [X] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [X] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/405Adding integration test for evaluation without a freshly trained tagger2022-02-09T13:30:04+01:00Alexander FrochAdding integration test for evaluation without a freshly trained taggerThis MR adds a integration test for the evaluation without a freshly trained tagger.This MR adds a integration test for the evaluation without a freshly trained tagger.Alexander FrochAlexander Frochhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/339Adding more flexibility for validation and test file usage2022-01-27T19:28:09+01:00Manuel GuthAdding more flexibility for validation and test file usageThis is a kind of epic MR collecting intermediate changes towards the resolution of the issue
Closes #112This is a kind of epic MR collecting intermediate changes towards the resolution of the issue
Closes #112Manuel GuthManuel Guthhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/217adding more labels in MR check2021-10-29T09:52:54+02:00Manuel Guthadding more labels in MR checkhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/396adding plotting module2022-02-11T13:37:58+01:00Manuel Guthadding plotting modulethis MR introduces a new module `umami.plotting` (open for other name suggestions) which moves the plotting into one module using class syntax making everything more modular and then easier to call via python API
in this first step the ...this MR introduces a new module `umami.plotting` (open for other name suggestions) which moves the plotting into one module using class syntax making everything more modular and then easier to call via python API
in this first step the roc curves are being introduced
- `figure_base` class having all the information about the figure and its over all style
- `roc` class containing info about one roc curve and allows to calculate the ratio to another roc curve etc.
do people haven thoughts about this new way of doing things? @alfroch @pgadow @svanstro @birk
first step towards #100
An example how the roc plotting is done now
```python
from umami.plotting import roc, roc_plot
plot_roc = roc_plot(
n_ratio_panels=2, ylabel="background Rejection", xlabel="b-jets efficiency"
)
plot_roc.add_roc(
roc(
sig_eff,
rnnip_ujets_rej,
n_test=100000,
rej_class="ujets",
signal_class="bjets",
label="RNNIP",
colour="#AA3377",
),
reference=True,
)
plot_roc.add_roc(
roc(
sig_eff,
dips_ujets_rej,
n_test=100000,
rej_class="ujets",
signal_class="bjets",
label="DIPS",
colour="#228833",
),
)
plot_roc.add_roc(
roc(
sig_eff,
rnnip_cjets_rej,
n_test=100000,
rej_class="cjets",
signal_class="bjets",
label="RNNIP",
colour="#AA3377",
),
reference=True,
)
plot_roc.add_roc(
roc(
sig_eff,
dips_cjets_rej,
n_test=100000,
rej_class="cjets",
signal_class="bjets",
label="DIPS",
colour="#228833",
),
)
plot_roc.set_ratio_class(1, "ujets")
plot_roc.set_ratio_class(2, "cjets")
plot_roc.draw(
rlabel=[
"light-flavour jets ratio",
"c-jets ratio",
],
leg_class_labels=[
"light-flavour jets rejection",
"c-jets rejection",
],
)
plot_roc.savefig("roc.pdf")
```https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/503Adding prepare_docs stage to CI2022-04-06T14:26:02+02:00Joschka BirkAdding prepare_docs stage to CI## Summary
This MR introduces the following changes
* Adds a stage `prepare_docs` to the CI pipeline, since we now have multiple jobs which should be run before building the docs
* Move placeholder rendering to separate job
## Conform...## Summary
This MR introduces the following changes
* Adds a stage `prepare_docs` to the CI pipeline, since we now have multiple jobs which should be run before building the docs
* Move placeholder rendering to separate job
## Conformity
- [x] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- ~~[ ] [Documentation](https://umami-docs.web.cern.ch)~~
- ~~[ ] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)~~
- ~~[ ] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)~~Joschka BirkJoschka Birkhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/413Adding rejection and efficiency function and plotting examples2022-02-14T11:22:20+01:00Manuel GuthAdding rejection and efficiency function and plotting examplesThis MR adds
- `calc_eff` and `calc_rej`
- more plotting docs with examplesThis MR adds
- `calc_eff` and `calc_rej`
- more plotting docs with exampleshttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/338adding rules for image build in CI2022-01-14T16:51:14+01:00Manuel Guthadding rules for image build in CIRunning the docker image build in the MR pipeline is only ran if packages changed
adresses https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/issues/89Running the docker image build in the MR pipeline is only ran if packages changed
adresses https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/issues/89https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/473adding slim umami image2022-03-11T13:41:28+01:00Manuel Guthadding slim umami image## Summary
This MR introduces the following changes
* adding a slim image - mainly for plotting
Relates to the following issues
* #151
*
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algor...## Summary
This MR introduces the following changes
* adding a slim image - mainly for plotting
Relates to the following issues
* #151
*
## Conformity
- [ ] [Changelog entry](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/blob/master/changelog.md)
- [ ] [Documentation](https://umami-docs.web.cern.ch)
- [ ] [Development guidelines](https://umami-docs.web.cern.ch/setup/development/)
- [ ] [Style guides](https://umami-docs.web.cern.ch/setup/development/good-practices/)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/umami/-/merge_requests/344Adding sphinx package2022-01-18T10:46:29+01:00Manuel GuthAdding sphinx packageThis MR adds several features
- a new API sphinx reference guide is added in the 'read the docs' style which is deployed with the docs and will appear under [https://umami.docs.cern.ch/sphinx-docs](https://umami.docs.cern.ch/sphinx-docs...This MR adds several features
- a new API sphinx reference guide is added in the 'read the docs' style which is deployed with the docs and will appear under [https://umami.docs.cern.ch/sphinx-docs](https://umami.docs.cern.ch/sphinx-docs) once this MR is merged
- the docs are now also built in the MR and deployed as a separate website for each MR under the following URL https://umami-ci-provider.web.cern.ch/mr-docs/{mr_id}
- these URLs are posted to the MR with the `umamibot`
for this particular MR you can find the new reference guide under [https://umami-ci-provider.web.cern.ch/mr-docs/344/sphinx-docs](https://umami-ci-provider.web.cern.ch/mr-docs/344/sphinx-docs) as well as the temporary docs under [https://umami-ci-provider.web.cern.ch/mr-docs/344](https://umami-ci-provider.web.cern.ch/mr-docs/344)