Skip to content

Feature/unpacker skeleton

Mykhailo Dalchenko requested to merge feature/unpacker_skeleton into develop

Description

Detailed description with some implementation details has been presented at the GEM DAQ SW meeting The general structure remains the same, however important improvements were made:

  • fixed bug with arbitrary hardware block ordering
  • pandas.DataFrame conversion is not made once for all events instead of event-by-event (@lmoureau)
  • bitstring package was replaced by cbitstruct (@lpetre)
  • input data is read as numpy.ndarray (fromfile method)
  • formatting strings (dictionaries) are now compiled

Related Issue

Closes #3 (closed), partially addresses #4 and #2

How Has This Been Tested?

Tested by running on verified test data set, consisting of 9306 events

Test on gem904daq01 

CPU info:
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    2
Core(s) per socket:    4
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 58
Model name:            Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz

C++ code:
time ./gemtreewriter/bin/linux/x86_64_centos7/unpacker /tmp/run000434_Cosmic_CERNQC7_2020-03-02_chunk_0.dat sdram
[GEMUnpacker]: ---> Main()

real	0m0.704s
user	0m0.556s
sys	0m0.151s

python code:
python -m timeit -n 1 "from gdh import unpacker; unpacker.run('/tmp/run000434_Cosmic_CERNQC7_2020-03-02_chunk_0.dat','sdram')"
***
INFO:gdh.unpacker:Finished unpacking. Processed 9306 events.
1 loops, best of 3: 3.36 sec per loop

Jupiter notebook:
CPU times: user 3.15 s, sys: 78.1 ms, total: 3.23 s
Wall time: 3.22 s

Test on MBP 13" Fall 2014

CPU info:
machdep.cpu.max_basic: 13
machdep.cpu.max_ext: 2147483656
machdep.cpu.vendor: GenuineIntel
machdep.cpu.brand_string: Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
machdep.cpu.family: 6
machdep.cpu.model: 69
machdep.cpu.extmodel: 4
machdep.cpu.extfamily: 0
machdep.cpu.stepping: 1

python code:
python -m timeit -n 1 "from gdh import unpacker; unpacker.run('/tmp/run000434_Cosmic_CERNQC7_2020-03-02_chunk_0.dat','sdram')"
***
INFO:gdh.unpacker:Finished unpacking. Processed 9306 events.
1 loop, best of 5: 3.9 sec per loop

Jupiter notebook:
CPU times: user 3.66 s, sys: 122 ms, total: 3.78 s
Wall time: 3.84 s

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Merge request reports