Error: Disk I/O error - Root cause: BlockMissingException: Could not obtain block

When running Arcadia Analytics Engine (Arcengine) on a new environment you might see an error like this:

Visual 1000 - p2.1_1560443785: Could not load data
(HY000, None) Disk I/O error: Error reading from HDFS file: hdfs://<file path>
Error(255): Unknown error 255
Root cause: BlockMissingException: Could not obtain block: BP-378631197-15.234.30.149-1458602780341:blk_1860035581_789602233 file=/<file path>/000016_0

The first thing to check is to make sure that your table’s metadata has been updated in Arcadia if new data has been loaded by running - INVALIDATE DATA <db>.<tablename> .

If that doesn’t resolve the issue, then its likely that this is a symptom of Linux open file limits not being configured properly. Configuring the Linux open file limits for Arcadia can be done by following these instructions.