Analytical View refresh


I see log when refresh analytical view command triggered, insert overwrite command is triggered automatically to insert data to Arcadia view, can below insert overwrite command be used directly instead of using command refresh analytical view? will there be any performance issue for long term perspective?

INSERT OVERWRITE database.tablename PARTITION(X1, X2) SELECT period period, product product, rcao rcao,currency2 currency2, rm_soeid rm_soeid, mrc mrc, country country, mis_dt mis_dt FROM database2.basetablename WHERE‘XX’ ;

When refreshing Analytical Views you should not try to manually run the INSERT OVERWRITE command. Refresh Analytical View does a series of commands, not just the INSERT OVERWRITE and it’s possible running the command manually could cause some issues with the Analytical View integrity.

Hi taddwood

Thanks a lot for your update, there is one requirement in our daily work, it is using table A join table B on some joining keys to populate data to Arcadia Analytical view, although Arcadia Analytical view is created to have the same partition with table A, let’s say table A is a Fact table and table B is a dimention table.
after the Arcadia view created successful in the first time, whenever there is data insertion in the base have table, when we execute Refresh Analytical view command in the target table, it is always giving error , and I checked log in url, it is not like Arcadia refresh with one single table only , it is dropping all the partitions in the AV, then doing insert overwrite for all partitions. currently we have no way to resolve this issue except drop AV and recreate it each night, any good solution about this case?

@Astrid if the AV is dropping and re-inserting all partitions, then its likely that something similar is happening with the base table its created from. For example, if the base table is dropped and recreated where each partition is created again, this behavior would happen with the AV too. Also if an INSERT OVERWRITE is being run on the base table that would also cause the partitions to be overwritten, and then trigger the AV to drop and re-insert each partition again.

it’s not like that, this is one table left join the other 2 tables with primary key in the hive base table, every day one more partition will be added in the driving hive table, but in AV view, when we are trying to refresh analyitcal view, in the back end log, it is printing AV would drop all the historical partitions and do insert overwrite from the begging.
Ideally it should only do insert overwrite newly added partitions from base table, correct?

Any update for this scenario? can you please help?