Draft: Fix for flavor tagging ip sign flips: 23.0 edition

This is rebase of @yinmiao's work in !61905 (closed), but pointing to the correct branch. Tagging @bdong and @pgadow. As soon as we have some validation we should mark one (or both) of these as urgent so we can get this in to the nightly.