Abstract data sink interface

Adds the following:

• New libDataSink library.
• Abstract IDataSink interface
• ConsoleSink implementation for printing data to standard output
• ps_monitor tool that monitors a power supply and saves data to a sink.
• datasink_example example that saves randomly generated values to a sink.

A series of data points, grouped by time into measurements, can be streamed into a data sink.

Data Points and Measurements

Data Points

A data point is a collection of basic types (string, double, int) corresponding to different variables. It can be thought of as a row.

The end of a data point is indicated using the recordPoint.

Measurement

A measurement is a collection of data points grouped by a timestamp. All data points in a measurement must have the same set of variables. The timestamp shared by all data points in a given measurement is defined at the beginning of the said measurement. It is up to the implementation on how to deal with the timestamp.

A set of measurements can be thought of a large table with the timestamp as a column. The rows with the same time value belong to the same measurement.

The start of a new measurement is indicated using startMeasurement and the end with endMeasurement. At the start, the name of the measurement and the corresponding time needs to be specified.

Specific sink implementations can require that measurements sharing the same name use the same set of variables. It is up to the implementation to enforce such a requirement.

Variables

There are two types of variables and are treated differently; fields and tag.

Tags correspond to meta-data and are constant for a measurement. They also remain set for the reminded of the data-taking, although values can be changed at any time (outside of a measurement).

Fields correspond to data and are only valid for the duration of a measurement. Once set, a field exists until the end of a measurement. A field is not removed by recordPoint. Thus data that is constant during a measurement only needs to be set once.

The following variable names are reserved and cannot be be used: time.

Implemetations can use the checkReserved function to check whether a variable name is reserved.

Saving Data

It is up to a specific sink implementation to decide when to save data. For example, the ConsoleSink prints out data as soon as a data point completes. But the JSONSink, due to the more complex structure, writes data to disk once a measurement ends.

The recordPoint and endMeasurement are used to indicate to the implementation at which point a data point or measurement ends. However there is no requirement in which function the actual data saving needs to happen.

Example

A typical loop could look like this.

std::shared_ptr<IDataSink> datasink=std::make_shared<SpecificSink>("name");
while(forever)
 {
   // Run IV scan
   datasink->setTag("module", "myFirstModule"); // present for all data points in ivscan
   datasink->startMeasurement("ivscan", std::chrono::system_clock::now());
   datasink->setField("Ilimit", ps->getCurrentProtect()); // present for all data points in ivscan
   for(double V=0; V<1000; V+=100)
     {
       ps->setVoltageLevel(V);
       datasink->setField("I", ps->measureCurrent());
       datasink->setField("V", ps->measureVoltage());
       datasink->recordPoint();
     }
   datasink->endMeasurement();

   // Monitor climate for the next minute
   for(every second for minute)
     {
       // Record a data point
       datasink->startMeasurement("climate", std::chrono::system_clock::now());
       datasink->setField("temperature", measureTemperature());
       datasink->setField("humidity", measureHumidity());
       datasink->recordPoint();
       datasink->endMeasurement();
     }
 }

IDataSink Factory

Data sinks can be specified at runtime using the DataSinkConf factory. The factory uses a hardware configuration JSON file with a datasink entry containing a dictionary of available sinks. An example can be seen inside configs/input-hw.json.

The hardware configuration file declaring the sinks can be, but does not have to be, the same as used for declaring hardware.

ConsoleSink

When a data point is recorded, all fields are printed to the standard output as a pretty table. The format is as follows:

Tag1: value
...
Tagn: value
FIELD1 FIELD2 FIELD3
  VAL1   VAL2   VAL3

For fields, the columns are taken to have 10 characters.

The header (tags and the column names) are printed when any of the following happens:

• The name of the measurement changes.
• A tag value changes.
• The set of fields changes.

The header is not printed just because a previous measurement has been stopped. If the field definitions and tags are the same, the previous header still applies.