Skip to content

Adding catch for binary file reading in Python3

A change in python3 (as I understand it) is that the default file encoding is utf-8. That means by default files are interpreted as text files, and if they are binary, reading causes a crash. As a simple example, this works in python2 but not in python3:

>>> x = f.read(1024*1024)

In this particular case, the job transform is validating data integrity by trying to read through the entire output file (as I understand it) and looking for read errors. I've added a catch in the case that we encounter this particular unicode decode error, and then restarted the file reading in binary mode so that we can still check the integrity. This should bring the behavior back to matching what it does in 21.X.

This should fix the crash reported at the top of AGENE-1962.

Merge request reports