Feature/unpacker skeleton
Description
Detailed description with some implementation details has been presented at the GEM DAQ SW meeting The general structure remains the same, however important improvements were made:
- fixed bug with arbitrary hardware block ordering
- pandas.DataFrame conversion is not made once for all events instead of event-by-event (@lmoureau)
-
bitstring
package was replaced bycbitstruct
(@lpetre) - input data is read as
numpy.ndarray
(fromfile
method) - formatting strings (dictionaries) are now compiled
Related Issue
Closes #3 (closed), partially addresses #4 and #2
How Has This Been Tested?
Tested by running on verified test data set, consisting of 9306 events
gem904daq01
Test on CPU info:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 58
Model name: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
C++ code:
time ./gemtreewriter/bin/linux/x86_64_centos7/unpacker /tmp/run000434_Cosmic_CERNQC7_2020-03-02_chunk_0.dat sdram
[GEMUnpacker]: ---> Main()
real 0m0.704s
user 0m0.556s
sys 0m0.151s
python code:
python -m timeit -n 1 "from gdh import unpacker; unpacker.run('/tmp/run000434_Cosmic_CERNQC7_2020-03-02_chunk_0.dat','sdram')"
***
INFO:gdh.unpacker:Finished unpacking. Processed 9306 events.
1 loops, best of 3: 3.36 sec per loop
Jupiter notebook:
CPU times: user 3.15 s, sys: 78.1 ms, total: 3.23 s
Wall time: 3.22 s
Test on MBP 13" Fall 2014
CPU info:
machdep.cpu.max_basic: 13
machdep.cpu.max_ext: 2147483656
machdep.cpu.vendor: GenuineIntel
machdep.cpu.brand_string: Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
machdep.cpu.family: 6
machdep.cpu.model: 69
machdep.cpu.extmodel: 4
machdep.cpu.extfamily: 0
machdep.cpu.stepping: 1
python code:
python -m timeit -n 1 "from gdh import unpacker; unpacker.run('/tmp/run000434_Cosmic_CERNQC7_2020-03-02_chunk_0.dat','sdram')"
***
INFO:gdh.unpacker:Finished unpacking. Processed 9306 events.
1 loop, best of 5: 3.9 sec per loop
Jupiter notebook:
CPU times: user 3.66 s, sys: 122 ms, total: 3.78 s
Wall time: 3.84 s
Types of changes
-
Bug fix (non-breaking change which fixes an issue) -
New feature (non-breaking change which adds functionality) -
Breaking change (fix or feature that would cause existing functionality to change)
Checklist:
-
My code follows the code style of this project. -
My change requires a change to the documentation. -
I have updated the documentation accordingly. -
I have read the CONTRIBUTING document. -
I have added tests to cover my changes. -
All new and existing tests passed.