Skip to content

Heterogeneous Transformer Updates

Jackson Barr requested to merge jabarr/salt:multitask into main

This is a WIP MR to add a fully heterogeneous architecture, with cross attention layers between different input types interspersed with regular self attention updates.

It would also be useful to support tasks with multiple input types, e.g. if we include flow constituents, we will have two sets of track inputs - those with associated calo clusters and those without. We would want to perform the vertexing on both sets of tracks. Update: Upon further thinking on this point, I think the solution for this case is to simply merge the two sets of track inputs and treat them as one input type. This would also require us to dump the neutral flow objects separately from the charged flow objects.

See #15 (closed)

Edited by Samuel Van Stroud

Merge request reports