Arcadia support for non-flat metrics

Hello there,

I am trying to build a visualization based on top of KSQL (with avro as my data format). My topic has metrics that are not flat, and contain inner record fields. When loading my table into Arcadia, these fields get shown as just [object Object] with no way (as far as I can tell) to process on the fields on the object.

Are there plans to add support for this use case? I can get around it by creating a new stream per inner (complex) field, but this is a pretty big annoyance.

@Haris_Godil are you referring to complex types like STRUCT and ARRAYS

If yes, then right now we do not support it out of the box. Flattening the data as you suggested is the workaround.

Our current complex type support is for HDFS storage in parquet format.

In the future we will look to extend this to real-time / streaming connections.