Why won't Arcadia service start in Ambari when I reboot a node?

It has been observed on Hortonworks Data Platform (HDP) 2.x installations that Arcadia services are not always able to restart in the event of a node rebooting due to the absence of a folder named “arcadia” in the “/run” folder.

It is believed that this is due to a change in the filesystem layout between Red Hat versions 6 and 7 whereby version 7 changed the /run filesystem to be a temporary filesystem, tmpfs, thus causing it to be cleared down on reboots.

We can use the tmpfiles.d functionality to ensure the folder is recreated after a restart by creating a configuration file in the /usr/lib/tmpfiles.d folder with a suitable name e.g. arcadia-reboot-fix.conf using your text editor of choice. This file should contain contents as follows :-

#Type Path             Mode UID      GID      Age Argument
d     /var/run/arcadia 0775 arcadia  hadoop   -   -

Alternatively, the following commands can be issued to create the script direct from the command line :-

echo “#Type Path Mode UID GID Age Argument” > /usr/lib/tmpfiles.d/arcadia-reboot-fix.conf
echo “d /var/run/arcadia 0775 arcadia hadoop - -” >> /usr/lib/tmpfiles.d/arcadia-reboot-fix.conf