Only search for additional hits in X layers if there is actually an empty one

Does not affect efficiencies, saves a tiny bit of time (< 1%) @jihu @chasse @sstahl