Skip to content

Draft: MaskFormer Changes

Theresa Reischrequested to merge
treisch/salt:hits_mf_fixes into main

Summary

This MR introduces the following changes

  • disabling the error if the maskformer_config.constituent.name is not in the labels already since that labels are gather via the input-names of the tasks. Since I do not have a task which uses the same input as my maskformer configuration, that caused an issue. (see changes in cli.py)
  • changes the way the dimension of pad_mask are being checked in the forward pass of the maskformer. I had the issue that my pad_mask would be a dictionary with two keys: maskformer_config.constituent.name and REGISTERS, now picking only the one with the constituent name (see maskformer.py)
  • change the way the pad_mask is being padded to match dimensionality issues I had (see maskformer.py)
  • cast obj_class_tgt * valid_obj_mask to long because torch.gather complained (see matcher.py)
  • change the ways the weights are parsed to the cross entropy in the maskformer because there was in issue that I had less classes in my mask former config than in my classification task (see maskformer_loss.py)

I have been trying to debug to my and ChatGPT's best knowledge, but I would appreciate if an expert can have a look if the changes make sense.

Conformity

Merge request reports