Understanding nscd daemon for hosts cache

nscd stands for Name Service Cache Daemon and is used to provide cache for common name service request. For providing hosts cache nscd daemon uses /etc/hosts file as it's database and any changes made to the database is immediately noticd ny nscd and it will flush the cache once these are changed. However, this will happen only after a short delay (unless the inotify(7) mechanism is available and glibc 2.9 or later is available)This daemon is used in most environments where a lookup for various database and tables are needed in a frequent time to time base to build a cache and increase the end performance of the application for eg. LDAP uses nscd to process any bind request from clients to the server, webservers etc
nscd provides caching for different databases using standard libc interfaces like for hosts database it uses GETHOSTBYADDR, GETHOSTBYNAME and others.
There are two caches for each database: a positive one for items found, and a negative one for items not found. Each cache has a separate TTL (time-to-live) period for its data. These parameters are configurable using /etc/nscd.conf file.
Let us look at the several options and variables available for hosts cache
To collect the statistics of nscd execute the below command
NOTE: Since for this article we are concentrating on hosts cache I will grep the output which only shows hosts cache details

hosts cache:
            yes  cache is enabled
             no  cache is persistent
            yes  cache is shared
            211  suggested size
         216064  total data pool size
            320  used data pool size
            600  seconds time to live for positive entries
              2  seconds time to live for negative entries
              5  cache hits on positive entries
              0  cache hits on negative entries
              9  cache misses on positive entries
              1  cache misses on negative entries
             33% cache hit rate
              2  current number of cached values
              4  maximum number of cached values
              1  maximum chain length searched
              0  number of delays on rdlock
              0  number of delays on wrlock
              0  memory allocations failed
            yes  check /etc/{hosts,resolv.conf} for changes

'cache is enabled' - informs about the status of the hosts cache which here means we have enabled this cache. If any cache is disabled this will be 'no'
'cache is persistent' - This is if you want the cache to be persistent across daemon restart i.e. the stored statistics will be saved in the memory and will not be refreshed for most of the stat values.
For eg:
My existing stats with 'enabled' persistent caching

hosts cache:
            yes  cache is enabled
            yes  cache is persistent
            yes  cache is shared
            211  suggested size
        3244035  total data pool size
              0  used data pool size
            600  seconds time to live for positive entries
              2  seconds time to live for negative entries
              0  cache hits on positive entries
              0  cache hits on negative entries
              0  cache misses on positive entries
          88180  cache misses on negative entries
              0% cache hit rate
              0  current number of cached values
          30889  maximum number of cached values
            185  maximum chain length searched
              0  number of delays on rdlock
              0  number of delays on wrlock
              0  memory allocations failed
            yes  check /etc/{hosts,resolv.conf} for changes

Restarted nscd service

# /etc/init.d/nscd restart
Shutting down Name Service Cache Daemon                                                                                 done
Starting Name Service Cache Daemon    

Post restart of the daemon the values are still same

hosts cache:
            yes  cache is enabled
            yes  cache is persistent
            yes  cache is shared
            211  suggested size
        3244035  total data pool size
              0  used data pool size
            600  seconds time to live for positive entries
              2  seconds time to live for negative entries
              0  cache hits on positive entries
              0  cache hits on negative entries
              0  cache misses on positive entries
          88180  cache misses on negative entries
              0% cache hit rate
              0  current number of cached values
          30889  maximum number of cached values
            185  maximum chain length searched
              0  number of delays on rdlock
              0  number of delays on wrlock
              0  memory allocations failed
            yes  check /etc/{hosts,resolv.conf} for changes

After disabling 'cache is persistent' and restarting nscd daemon service

hosts cache:
            yes  cache is enabled
             <span style="color: #339966;">no  cache is persistent</span>
            yes  cache is shared
            211  suggested size
         216064  total data pool size
              0  used data pool size
            600  seconds time to live for positive entries
              2  seconds time to live for negative entries
              0  cache hits on positive entries
              0  cache hits on negative entries
              0  cache misses on positive entries
              0  cache misses on negative entries
              0% cache hit rate
              0  current number of cached values
              0  maximum number of cached values
              0  maximum chain length searched
              0  number of delays on rdlock
              0  number of delays on wrlock
              0  memory allocations failed
            yes  check /etc/{hosts,resolv.conf} for changes

So all the cache entries are cleared.
cache is shared - If this is enabled any client nodes connecting to the the server will perform lookup themself in the nscd cache rather than asking nscd daemon which makes the lookup process faster. nscd daemon would be needed only to update the cache if the client host entry is unavailable in the hosts cache. Once nscd is in shared mode the nascd cache hit rate is mostly shown as 0% as nscd is mostly not use and the reverselookup is performed from the cache
suggested size - (From the man page) This is the internal hash table size, value should remain a prime number for optimum efficiency.  The default is 211.
total data pool size - This accounts for the total list of cache host entry which has been looked up by nscd (both positive and negative)
used data pool size - Hosts cache used in the current session of nscd. Everytime nscd daemon is restarted this value will reset to "0" and a fresh used data pool size is built using the existing hosts file.
seconds time to live for positive entries - (From the man page) Sets the TTL (time-to-live) for positive entries (successful queries) in the specified cache for service. Value is in seconds. Larger values increase cache hit rates and reduce mean response times, but increase problems with cache coherence.
seconds time to live for negative entries - (From the man page) Sets the TTL (time-to-live) for negative entries (unsuccessful queries) in the specified cache for service.  Value is in seconds.  Can result in significant performance improvements if there are several files owned by UIDs (user IDs) not in system databases (for example untarring the Linux kernel sources as root); should be kept small to reduce cache coherency problems.
cache hits on positive entries - This value will be populated only if nscd daemon is running in non shared mode i.e. 'cache is shared' variable is 'no'. In such case nscd performs all the lookups and will increment the value for any lookup from a target host which manages to establish a ESTABLISHED network connection with the client hosts.
For eg.
I have added below entry in hosts file

192.169.32.10 cc01-nds-ins

Next I attempt ssh from 192.169.32.10 to the target node and observe the nscd stats

# nscd -g | grep "hosts cache" -A 22 | grep "cache hits on positive entries"
             13  cache hits on positive entries

So we have an increment in the cache hit rate for positive entries since 192.169.32.10 was present in our hosts file
cache hits on negative entries - This value will be populated only if nscd daemon is running in non shared mode i.e. 'cache is shared' variable is 'no'. In such case nscd performs all the lookups and will increment the value for any lookup from a target host which fails to establish a ESTABLISHED network connection with the client hosts.
memory allocations failed - If persistent mode is not enabled then there is very less chance that you will see this value incrementing unless the alloted database size for nscd goes out of space. When persistent mode is enabled all the caches are stored in memory which might run out of space when you will start see incrementing values of memory allocation failures
For eg:
I reduced my database size to below size

       max-db-size             hosts           335511

and restarted nscd services
After a while I started receiving multiple memory allocation failures

hosts cache:
            yes  cache is enabled
             no  cache is persistent
            yes  cache is shared
            211  suggested size
         334559  total data pool size
         334544  used data pool size
            600  seconds time to live for positive entries
              2  seconds time to live for negative entries
              0  cache hits on positive entries
              0  cache hits on negative entries
              1  cache misses on positive entries
           3483  cache misses on negative entries
              0% cache hit rate
              1  current number of cached values
           3484  maximum number of cached values
             28  maximum chain length searched
              0  number of delays on rdlock
              0  number of delays on wrlock
         100418  memory allocations failed
            yes  check /etc/{hosts,resolv.conf} for changes
I hope the article was useful.