algorithms issueshttps://gitlab.cern.ch/groups/atlas-flavor-tagging-tools/algorithms/-/issues2024-02-26T13:17:11+01:00https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/issues/60Use mock data generation from atlas-ftag-tools2024-02-26T13:17:11+01:00Samuel Van StroudUse mock data generation from atlas-ftag-toolsremove https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/blob/main/salt/utils/inputs.py?ref_type=heads
in favour of https://github.com/umami-hep/atlas-ftag-tools/blob/main/ftag/mock.pyremove https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/blob/main/salt/utils/inputs.py?ref_type=heads
in favour of https://github.com/umami-hep/atlas-ftag-tools/blob/main/ftag/mock.pyhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/issues/57Support for feature wise transformations2024-02-26T13:19:14+01:00Emil HainesSupport for feature wise transformationsImplement alternate mechanism for model conditioning through feature-wise transformations, as described here https://distill.pub/2018/feature-wise-transformations/. This extends the current parameterisation implementation, which only inc...Implement alternate mechanism for model conditioning through feature-wise transformations, as described here https://distill.pub/2018/feature-wise-transformations/. This extends the current parameterisation implementation, which only includes the standard concatenation based approach, providing the user with greater flexibility, and possible performance gains.
- [x] Basic implementation with the option to add feature wise transformations to the inputs / global track representations
- [ ] Implement the option to apply feature wise transformations to each layer in the encoder, and investigate its performanceEmil HainesEmil Haineshttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/issues/56Loss logging issue with Comet2024-03-12T14:09:27+01:00Leonardo SplendoriLoss logging issue with CometOn Comet multiple different values of the loss are logged per epoch. This can be seen by opening the panels of a training, like [this one](https://www.comet.com/lesplend/salt/7c53e719ad344f55b3b9c64c7473cd0d?experiment-tab=panels&showOut...On Comet multiple different values of the loss are logged per epoch. This can be seen by opening the panels of a training, like [this one](https://www.comet.com/lesplend/salt/7c53e719ad344f55b3b9c64c7473cd0d?experiment-tab=panels&showOutliers=true&smoothing=0&xAxis=epoch).
Selecting `epoch` to be displayed on the x axis, navigating to the train_loss and exporting the graph data as a JSON will show repeated values for the x axis.
I included two JSONs showing a training loss and a valuation loss log exported like this. The first shows the issue reported and the second shows what one would expect the data for a 2D graph to look like.
[loss_chart_data.json](/uploads/68f8f496604d81aba8667fbc5f5e7ad4/loss_chart_data.json)
[val_loss_chart_data.json](/uploads/b37e07b2bb6cb408097a313c78b907b7/val_loss_chart_data.json)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/41Update jet labelling docs2024-02-22T11:33:36+01:00Samuel Van StroudUpdate jet labelling docsOur jet labelling docs are a bit confusing: https://ftag.docs.cern.ch/algorithms/labelling/jet_labels
We describe between delta R hadron labelling, and ghost parton labelling. but there is a full 2x2 matrix of possibilities
- whether to...Our jet labelling docs are a bit confusing: https://ftag.docs.cern.ch/algorithms/labelling/jet_labels
We describe between delta R hadron labelling, and ghost parton labelling. but there is a full 2x2 matrix of possibilities
- whether to label hadrons or partons (in FTAG we generally use hadrons not partons)
- whether to associated hadrons or partons to jets using deltaR cones or ghost association (in FTAG we want to switch from dR to GA)
We should also mention the additional labels for pT, dR, pdgid, etc that we have for the hadron labelling schemeWei Sheng LaiWei Sheng Laihttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/40Move tutorials to relevant repos2024-03-01T11:57:49+01:00Samuel Van StroudMove tutorials to relevant reposTutorials hosted here can get outdated in two ways
1. becoming irrelvant (when based on tag)
2. becoming broken when not (e.g. https://ftag.docs.cern.ch/software/tutorials/tutorial-tdd/ - thanks to @wlai for reporting this one)
It migh...Tutorials hosted here can get outdated in two ways
1. becoming irrelvant (when based on tag)
2. becoming broken when not (e.g. https://ftag.docs.cern.ch/software/tutorials/tutorial-tdd/ - thanks to @wlai for reporting this one)
It might be easier to move tutorials in their respective repos, and just link to them from these documentation pages.
(also someone should update the dumpster tutorial)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/39More permissive access rights in some parts of ftag docs2024-03-09T01:04:27+01:00Dan GuestMore permissive access rights in some parts of ftag docsI'd like to make at least the main page, possibly other ones, public. It should help people search for for our info, and is also a good way to get us thinking more about what _should_ be public.
Our good friend and mentor @feickert figu...I'd like to make at least the main page, possibly other ones, public. It should help people search for for our info, and is also a good way to get us thinking more about what _should_ be public.
Our good friend and mentor @feickert figured out how to do this with [the ML forum docs](https://atlasml.web.cern.ch/atlasml/), so it should be possible. But I can't figure out how he did it.https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/38Algo recommendations improvements2024-02-26T11:22:42+01:00Samuel Van StroudAlgo recommendations improvements- [ ] Add GN2v01 perf #36
- [ ] Refactor [recs code](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/docs-infrastructure/atlas-ftag-management) to use Yuma https://umami-hep.github.io/puma/main/examples/yuma.html
- [ ] Add...- [ ] Add GN2v01 perf #36
- [ ] Refactor [recs code](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/docs-infrastructure/atlas-ftag-management) to use Yuma https://umami-hep.github.io/puma/main/examples/yuma.html
- [ ] Add Z' high pT plots
- [ ] Add c-tagging plots
- [ ] Generate as much of the markdown as possible automaticallyAdam Leonard WarnerbringAdam Leonard Warnerbringhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/issues/53Maskformer improvements2024-02-09T14:32:34+01:00Nikita Ivvan PondMaskformer improvementsMaskformer is being added in !221 . The current implementation runs but there are several possible improvements/changes
- [ ] Fix ONNX exports for NodeGAP: The normal approach of simply concatenating a dummy track before pooling doesn't ...Maskformer is being added in !221 . The current implementation runs but there are several possible improvements/changes
- [ ] Fix ONNX exports for NodeGAP: The normal approach of simply concatenating a dummy track before pooling doesn't seem to work. The error suggests that something in the Decoder layers themselves are giving the issue, but everything works if we don't apply this pooling.
- [ ] Refactor decoder tasks into main tasks list: Currently, the decoder requires 2 network definitions for the mask prediction net and object classification net. It would be nicer to have all tasks listed in the same area, but this requires some thinking, as these task outputs are used in several places, and some (such as mask prediction) can utilise multiple loss functions.
- [ ] Refactoring of regression scaling: !221 Introduces `RegressionTargetScaler`, which allows for function-based scaling of regression targets. The other scaling methods can be treated as a sub-set of this, possibly allowing a cleanup of the code in RegressionTaskBase. The main challenge with this is if we can automatically treat the 'stds' in GaussianRegression properly with this.https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/37Consolidate information on grid usage and samples2024-02-08T11:15:53+01:00Samuel Van StroudConsolidate information on grid usage and samplesAt the very least these two should link to each other, ideally we would probably put them on a single page.
- https://ftag.docs.cern.ch/samples/grid/
- https://ftag.docs.cern.ch/calibrations/tech/#storage
@vvecchioAt the very least these two should link to each other, ideally we would probably put them on a single page.
- https://ftag.docs.cern.ch/samples/grid/
- https://ftag.docs.cern.ch/calibrations/tech/#storage
@vvecchiohttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/36Add GN2v01 Performance and CDI working points file2024-02-19T13:40:10+01:00Dan GuestAdd GN2v01 Performance and CDI working points file- [ ] We should update https://ftag.docs.cern.ch/recommendations/algs/r22-preliminary/ with GN2v01 performance
- [ ] CDI is here `/cvmfs/atlas.cern.ch/repo/sw/database/GroupData/xAODBTaggingEfficiency/13p6TeV/20
21-22-13TeV-MC21-CDI_GN2v...- [ ] We should update https://ftag.docs.cern.ch/recommendations/algs/r22-preliminary/ with GN2v01 performance
- [ ] CDI is here `/cvmfs/atlas.cern.ch/repo/sw/database/GroupData/xAODBTaggingEfficiency/13p6TeV/20
21-22-13TeV-MC21-CDI_GN2v01_Test_smooth.root`. It's not showing up on https://ftag.docs.cern.ch/recommendations/calib/cdi_overview/ at the moment.Adam Leonard WarnerbringAdam Leonard Warnerbringhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/35Integrate software docs?2024-02-05T13:48:04+01:00Samuel Van StroudIntegrate software docs?Totally up to @bdong @ligang if they would like to do this, but I think we could pretty easily fit the content of https://ftag-sw.docs.cern.ch/ in a single tab on the main docs page: https://ftag.docs.cern.ch/software/.
The benefits are...Totally up to @bdong @ligang if they would like to do this, but I think we could pretty easily fit the content of https://ftag-sw.docs.cern.ch/ in a single tab on the main docs page: https://ftag.docs.cern.ch/software/.
The benefits are discussed in #34
Seems to have worked ok for calibrations: https://ftag.docs.cern.ch/calibrations/https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/34Adding calibration section2024-02-08T11:17:24+01:00Valentina VecchioAdding calibration sectionCreating a section for the calibration documentation to unify everything under ftag.docs.cern.chCreating a section for the calibration documentation to unify everything under ftag.docs.cern.chValentina VecchioValentina Vecchiohttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/33Add calendar subscription links to subgroup meetings2024-02-01T18:29:42+01:00Dan GuestAdd calendar subscription links to subgroup meetingsThree possible implementations here:
- Some kind of [cron job](https://github.com/dguest/calcomb) that filters agenda into `.ics` files
- Something similar integrated into the docs
- Reorganize indico into categories and use the indico l...Three possible implementations here:
- Some kind of [cron job](https://github.com/dguest/calcomb) that filters agenda into `.ics` files
- Something similar integrated into the docs
- Reorganize indico into categories and use the indico links
I'm a big fan of 3 because it's less technical work for us, and also lets us appoint managers for the categories. I think @pgadow was reluctant, although I promised that that wouldn't break anything in existing indicombs. Other thoughts from @svanstro (the other owner of this repo)?https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/issues/47Revist masking convention2024-01-23T10:27:49+01:00Samuel Van StroudRevist masking conventionAt the moment we negative the `valid` flag twice, we could probably make this cleanerAt the moment we negative the `valid` flag twice, we could probably make this cleanerhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/issues/45Allow checkpoints to be exported to ONNX after transferring between machines2024-02-06T14:50:02+01:00Samuel Van StroudAllow checkpoints to be exported to ONNX after transferring between machinesCurrently in `to_onnx` the model is loaded directly, rather than going through the CLI. This means that the `norm_config` parameters are not correctly set. We should either offload stuff from the CLI into the model files themselves, or l...Currently in `to_onnx` the model is loaded directly, rather than going through the CLI. This means that the `norm_config` parameters are not correctly set. We should either offload stuff from the CLI into the model files themselves, or load the model using the CLI in the `to_onnx` script
possibly related to https://github.com/Lightning-AI/pytorch-lightning/pull/18105
cc @npondhttps://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/issues/44Improve batch/grid submission2024-02-01T15:17:33+01:00Samuel Van StroudImprove batch/grid submission@pgadow added great support for submitting to HTCondor batch systems in !169. Some related tasks
- [ ] add support to submit to the GRID
- [ ] improve the slurm batch support to match the functionality of the HTCondor submission (i.e. a...@pgadow added great support for submitting to HTCondor batch systems in !169. Some related tasks
- [ ] add support to submit to the GRID
- [ ] improve the slurm batch support to match the functionality of the HTCondor submission (i.e. automatically write the submission script from python). Ideally this would use a common backend with a submission script writer for either HTCondor or Slurm.https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/30Configurable recommendations2023-11-03T17:53:19+01:00Samuel Van StroudConfigurable recommendationsFor the tagger description pages and CDI pages, we should highlight the latest recommendation.
We could have a `recommendations.yaml` file in this reco, like
```yaml
recommendation-name:
year: 2022
taggers:
GN2:
model_pat...For the tagger description pages and CDI pages, we should highlight the latest recommendation.
We could have a `recommendations.yaml` file in this reco, like
```yaml
recommendation-name:
year: 2022
taggers:
GN2:
model_path: BTagging/path/to/model.onnx
cdis:
- cdi_path: path/to/cdi.room
```https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/ftag-docs/-/issues/28Consolidate deployment tools in this repo2024-01-08T12:07:47+01:00Samuel Van StroudConsolidate deployment tools in this repoThe idea is that the CI in this repo should scrape/build as much of the docs as possible on the fly, rather than having to rely on code/CI from other repos (which makes things confusing and lets automated).
Todo:
- [x] [CDI Dumper](ht...The idea is that the CI in this repo should scrape/build as much of the docs as possible on the fly, rather than having to rely on code/CI from other repos (which makes things confusing and lets automated).
Todo:
- [x] [CDI Dumper](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/docs-infrastructure/cdi-dump)
- [ ] cleanup old cron job and eos space
- [x] Tagger metadata cron job
- [x] cleanup old cron job
- [x] keep pushing json metadata to the eos space for now?
- [x] [sample lists](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/docs-infrastructure/atlas-ftag-management/-/tree/master/ftag_management/samplelists?ref_type=heads)
- [x] [ftag indicomb](https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/docs-infrastructure/ftag-indicomb)https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/issues/42ONNX Export for multiple input / output types2024-01-11T14:19:13+01:00Samuel Van StroudONNX Export for multiple input / output typesThe ONNX model should eventually be able to support multiple input types. We would also need to make sure that the auxiliary tasks know which input type to decorate their outputs to.
We might need to add this to our custom json configur...The ONNX model should eventually be able to support multiple input types. We would also need to make sure that the auxiliary tasks know which input type to decorate their outputs to.
We might need to add this to our custom json configuration, I'm not sure we could put it directly on the ONNX model itself (though we should certainly try).
This was discussed a bit here https://gitlab.cern.ch/atlas/athena/-/merge_requests/66718#note_7242673https://gitlab.cern.ch/atlas-flavor-tagging-tools/algorithms/salt/-/issues/41Augmentation pipeline2023-10-23T12:14:12+02:00Samuel Van StroudAugmentation pipelineIt would be nice to be able to apply transformations on data on the fly as it is loaded. This could be used to add systematics to the model.It would be nice to be able to apply transformations on data on the fly as it is loaded. This could be used to add systematics to the model.