Blinding data in potential new signal regions
Given these NN studies will be in part used to define new signal regions, we should make sure to blind any actual data distributions in these regions. This can be (easily, I hope) implemented as turning off data points above NN output>0.5.
I know arguments have been made about this before, that we've already done a Full Run 2 analysis and looked at the unblinded data, but these studies will likely add new data events to the mix. To be safe, we shouldn't show unblinded NN output score plots with data for now, until any new such strategy has been officially approved.
This can also be seen as a safeguard on the user side: don't evaluate a NN based on how poor/good the data/MC agreement is in the sensitive region!