Process data in Arcadia


I have a data table like this:


Right now, I process this data in Python and use matplotlib to visualize the result. The processing of the data is something like:

For each ID:
Example 1) Category “A”: Date1 - Category “B”: Date2
Example 2) If Category “A” Date2 is null, then ActDate - Category “B”: Date3

How can I implement this data processing in Arcadia, what is the best approach? I don’t want to process the data in Python and upload results to do the visualization in Arcadia


@Georgio ideally when building your semantic model within Arcadia datasets, your processing/calculations are based on values on the same row.

It seems in your case your referencing values in other rows. Which make the processing much more complex and best suited with ETL scripts as you are doing with python.

Alternatively, have you considered using complex data types like ARRAYS or STRUCTS to store you data?

In your case it would be perfect to store the category dates in a struct. Arcadia supports complex structures, and it’s very easy to drag and drop items in shelfs to process & visualize.

This way you can focus your data ingestion & ETL to prep your data in efficient data structures which you can then use within Arcadia to do the semantic modeling & business rules that you can change frequently as needed.

Does that make sense?