Autoweights
Implements the GLS automatic weighting, studied by @isanai for MaskFormer, but which can hopefully be used in other models!
TLDR: Instead of applying a weighted sum for the loss, instead we apply a geometric mean
Description
Review checklist:
-
CI Passing -
Comments addressed -
Source branch is up to date with target
Edited by Samuel Van Stroud