NXCALS-3567: HBaseDatasetCreator can now extract datasets from a collection of HBaseResources
#Merge request template @acc-logging-team - please have a look for approval.
Query
Dataset<Row> dataset = DataQuery.builder(sparkSession)
.byEntities().system("CMW")
.startTime("2010-08-09 01:30:46.000").endTime("2020-08-09 01:30:45.000")
.entity()
.keyValue("device", "RPCAB.361.BR4.RXSK6L4")
.keyValue("property", "SUB")
.buildDataset();
dataset.printSchema();
Result:
root
|-- EVENT_GROUP: integer (nullable = true)
|-- I_DIFF_MA: integer (nullable = true)
|-- I_EARTH_PCNT: double (nullable = true)
|-- I_ERR_MA: integer (nullable = true)
|-- I_MEAS: float (nullable = true)
|-- I_REF: float (nullable = true)
|-- STATE_OP: string (nullable = true)
|-- STATE_PC: string (nullable = true)
|-- STATE_PLL: string (nullable = true)
|-- STATE_VS: string (nullable = true)
|-- ST_ADC_A: string (nullable = true)
|-- ST_ADC_B: string (nullable = true)
|-- ST_ADC_C: string (nullable = true)
|-- ST_ADC_D: string (nullable = true)
|-- ST_DCCT_A: string (nullable = true)
|-- ST_DCCT_B: string (nullable = true)
|-- ST_FAULTS: string (nullable = true)
|-- ST_LATCHED: string (nullable = true)
|-- ST_UNLATCHED: string (nullable = true)
|-- ST_WARNINGS: string (nullable = true)
|-- V_MEAS: float (nullable = true)
|-- V_REF: float (nullable = true)
|-- __record_timestamp__: long (nullable = true)
|-- __record_version__: long (nullable = true)
|-- acqStamp: long (nullable = true)
|-- class: string (nullable = true)
|-- cyclestamp: long (nullable = true)
|-- device: string (nullable = true)
|-- property: string (nullable = true)
|-- selector: string (nullable = true)
|-- nxcals_entity_id: long (nullable = true)
total values:8274053
Closes NXCALS-3567
Edited by Grigorios Avgitidis