How to fix the "Too many open files" issue

After setting up and starting to use Arcadia on a new environment, you may see an error like this appear in the Arcadia UI:

This error is related to the linux open file limits set on the server(s). To fix this issue, the limit should be modified to allow the “arcadia” user or daemon processes to have a higher number of soft and hard file limit settings.

Below is an example of how this limit can be set in /etc/security/limits.conf:

arcadia      hard    nofile      64000
arcadia      soft    nofile      64000

Another option is to create a configuration file (i.e. arcadia.conf) in limits.d ( /etc/security/limits.d ) for the arcadia daemon processes with the same limits shown above.

Its important to check each server’s linux file limits to ensure that each is updated with the suggested configuration for the arcadia daemon processes. For any server not running Arcadia services, this update would not be required.

After making the updates you may also need to restart the cloudera or ambari agent processes to ensure that the new settings take hold.

1 Like