On the data formats...
Summary
There were a few informal discussions and ideas thrown about the file formats to use in the longer term. Let's try to formalize it better.
-
XML
to be minimized as much as possible:-
Libraries:
xdata
,xoap
, andxerces
- SOAP calls to be replaced with a JSON API when possible:
- Better unification with the Web interface and reduction of the code duplication
- xDAQ static configuration to be kept to its minimum
-
Libraries:
-
YAML
for the configuration files:-
Library:
yaml-cpp
- Hardware layout tree
- Text configuration database
- Possibly application configuration
-
Library:
-
JSON
for information exchange:-
Libraries:
nlohmann/json
andcereal
- Web interface
- Import-export of configuration parameters
- Export of the monitorables
- ...
-
Libraries:
-
cereal
binary format for the RPC calls:-
Library:
cereal
- Can it be replaced with
JSON
? - Anyway, no duplication of code since the same library would be used for
JSON
-
Library:
Then, the data-taking file formats:
-
GEM DAQ
data format within the CMS Raw file format:- See #50 (closed) for more details
- Implementation in progress in !145 (merged)
- Compressed
numpy
arrays for the non-tracking data scans:- Library: To be developed
- Currently
CSV
-like, but storage hungry - Could a simple compression of the CSV files be enough?