process hangs when getting errors from couchdb
If there's a temporary problem with httpd or couchdb, the build job may hang in some cases (e.g. while exiting for an unhandled exception).
The stack trace we got in one of those cases was:
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner
self.run()
File "/usr/lib64/python2.7/threading.py", line 765, in run
self.__target(*self.__args, **self.__kwargs)
File "/workspace/venv/centos7/lib/python2.7/site-packages/LbNightlyTools/Utils.py", line 525, in worker
action(*args, **kwargs)
File "/workspace/venv/centos7/lib/python2.7/site-packages/LbNightlyTools/Scripts/Common.py", line 679, in send
receiver.accept(msg)
File "/workspace/venv/centos7/lib/python2.7/site-packages/LbNightlyTools/Scripts/Common.py", line 498, in accept
self.project_build_completed(msg)
File "/workspace/venv/centos7/lib/python2.7/site-packages/LbNightlyTools/Scripts/Common.py", line 439, in project_build_completed
self.dashboard.update(self.doc_name, data)
File "/workspace/venv/centos7/lib/python2.7/site-packages/LbNightlyTools/Utils.py", line 487, in update
self.db[key] = new
File "/workspace/venv/centos7/lib/python2.7/site-packages/couchdb/client.py", line 447, in __setitem__
status, headers, data = resource.put_json(body=content)
File "/workspace/venv/centos7/lib/python2.7/site-packages/couchdb/http.py", line 578, in put_json
**params)
File "/workspace/venv/centos7/lib/python2.7/site-packages/couchdb/http.py", line 596, in _request_json
headers=headers, **params)
File "/workspace/venv/centos7/lib/python2.7/site-packages/couchdb/http.py", line 592, in _request
credentials=self.credentials)
File "/workspace/venv/centos7/lib/python2.7/site-packages/couchdb/http.py", line 429, in request
raise ServerError((status, error))
ServerError: (500, (u'unknown_error', u'undefined'))