Skip to content

NXCALS-1844 Fix schema incompatibility exception

Grigorios Avgitidis requested to merge feature-NXCALS-1844-fix-schema-incompatibility-exception into develop

#Merge request template @acc-logging-team - please have a look for approval.

In case of non primitive type and different schema we throw IncompatibleSchemaPromotionException instructing the client to extract separate datasets for a set of time windows.

Applied test with spark shell:

Query

scala> import cern.nxcals.data.access.builders._
scala> import java.time.Instant
scala> val ds = KeyValuesQuery.builder(spark).system("PM").startTime(Instant.ofEpochSecond(1531322200)).endTime(Instant.ofEpochSecond(1531322600)).entity().keyValue("pmsystem","TEST").keyValue("pmclass","invalidArraySize").keyValue("pmsource","rosimpso").buildDataset()

Result

cern.nxcals.data.access.api.exception.IncompatibleSchemaPromotionException: Unsupported type promotion for field value. 
Please extract separate datasets for time windows : 2018-07-11T15:16:40Z - 2018-07-11T15:23:20Z; 2018-07-11T15:16:40Z - 2018-07-11T15:23:20Z
  at cern.nxcals.data.access.api.DataAccessServiceImpl.getRequiredSchemaFieldsForOrThrow(DataAccessServiceImpl.java:256)
  at cern.nxcals.data.access.api.DataAccessServiceImpl.createDataSetFor(DataAccessServiceImpl.java:85)
  at cern.nxcals.data.access.builders.QueryData.buildDataset(QueryData.java:161)
  at cern.nxcals.data.access.builders.FluentQuery.buildDataset(FluentQuery.java:91)
  ... 50 elided
Caused by: cern.nxcals.data.access.api.exception.IncompatibleSchemaPromotionException: Unsupported type promotion for field value
  at cern.nxcals.data.access.api.FieldTypeResolver.getPromotedFieldTypeFor(FieldTypeResolver.java:73)
  at cern.nxcals.data.access.api.FieldTypeResolver.getPromotedFieldTypeWithAliases(FieldTypeResolver.java:57)
  at cern.nxcals.data.access.api.DataAccessServiceImpl.lambda$getRequestedFieldsWithPromotedTypes$17(DataAccessServiceImpl.java:280)
  at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
  at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1553)
  at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
  at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
  at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
  at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
  at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
  at cern.nxcals.data.access.api.DataAccessServiceImpl.getRequestedFieldsWithPromotedTypes(DataAccessServiceImpl.java:281)
  at cern.nxcals.data.access.api.DataAccessServiceImpl.getRequiredSchemaFieldsFor(DataAccessServiceImpl.java:265)
  at cern.nxcals.data.access.api.DataAccessServiceImpl.getRequiredSchemaFieldsForOrThrow(DataAccessServiceImpl.java:250)
  ... 53 more
Caused by: cern.nxcals.data.access.api.exception.IncompatibleSchemaPromotionException: 
Unsupported type promotion of schemas [{long rows of schema dumps go here}]

Closes NXCALS-1844

Merge request reports