Skip to content
Snippets Groups Projects

fix(workflow): UploadMC TypeError exception with DateTime objects

I randomly found an issue in the UploadMC module while reading the logs of the PushJobAgent:

  File "/opt/dirac/versions/v11.0.46-1723623328/Linux-x86_64/lib/python3.11/site-packages/LHCbDIRAC/Workflow/Modules/UploadMC.py", line 176, in execute
    output.write(str(json.dumps(jsonData)))
                     ^^^^^^^^^^^^^^^^^^^^
  File "/opt/dirac/versions/v11.0.46-1723623328/Linux-x86_64/lib/python3.11/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dirac/versions/v11.0.46-1723623328/Linux-x86_64/lib/python3.11/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/dirac/versions/v11.0.46-1723623328/Linux-x86_64/lib/python3.11/json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
  File "/opt/dirac/versions/v11.0.46-1723623328/Linux-x86_64/lib/python3.11/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type datetime is not JSON serializable

The UploadMC module tries to serialize a datetime object in a JSON file.

  • Note1: from what I can see, this object was introduced a few months ago in 3740f91e, and current tests did not spot the issue. The workflow tests I introduced in devel would have spotted it from what I can see. Should we backport the tests to master?
  • Note2: I tried to spot similar issues in other MC simulation jobs that were running on "classic" sites (not managed by the PushJobAgent), and I noticed that there was no log related to the UploadMC step (std.out generally stops at UploadOutputData). Is this expected?

BEGINRELEASENOTES

*Workflow FIX: UploadMC TypeError exception with datetime objects serialized

ENDRELEASENOTES

Merge request reports

Pipeline #8050386 passed

Pipeline passed for 2a584b99 on alboyer:master_fix_uploadMC-datetime-error

Merged by Federico StagniFederico Stagni 3 months ago (Sep 11, 2024 8:30am UTC)

Loading

Pipeline #8093166 passed

Pipeline passed for 1424db01 on master

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
Please register or sign in to reply
Loading