Accessing complex types MAP types through Arcadia

Arcadia support complex types such as arrays when stored in parquet format.
http://documentation.arcadiadata.com/4.4.0.0/#pages/topics/complex-datatypes.html

Here’s an example of what MAP type looks like within hive:

studentid           	int                 	                    
name                	string              	                    
class_grade         	map<string,string>  	

select * from shaun.test_map;
OK
0 mary {"mathemtics":"85","english":"82","history":"70"}
1 jason {"mathemtics":"95","english":"72","history":"90"}
2 sean {"chemistry":"91","art":"80","history":"66"}

Within Arcadia accessing the same MAP type will appear pivoted down. The reason being, Arcadia translate the key , value pairs into separate columns. The key being one column and the value being another.

Here’s how the above MAP field will appear. Notice each student will now appear 3 times once for each class.

If instead you want similar layout as what appears in hive, you can use a calculate column to “pivot up” the rows into a singular entry.

Example expression & resulting table view:

 group_concat(concat_ws (':',[class_grade.key],[class_grade.value]),'; ')

2 Likes