SubmissionController sends off jobs when it can't talk to the scheduler

Hi,

So I ran into a very weird issue this weekend thanks to HTCondor on lxplus not behaving nicely. The scheduler would not be reachable sometimes. I would try and submit using submit.py and I would get a message along the lines "Cannot contact scheduler, assuming the queue empty." So when this would happen, the submit script would submit every single job, even if it was already actually running. And then I ran into a case where a job that was previously running finished while the 'submitting task blahblah' was crawling. Then the job was submitted and queued to run on the batch, even though it was already finished and available in my batchOutput directory.

I'm not sure how much of an issue this is, but I think the submission following through when communication isn't fully available isn't very safe.

Thanks, Nate