• VMware

    Learn about VMware virtualization for its products like vsphere ESX and ESXi, vCenter Server, VMware View, VMware P2V and many more

  • Linux

    Step by step configuration tutorials for many of the Linux services like DNS, DHCP, FTP, Samba4 etc including many tips and tricks in Red Hat Linux.

  • Database

    Learn installation and configuration of databases like Oracle, My SQL, Postgresql, etc including many other related tutorials in Linux.

  • Life always offers you a second chance ... Its called tomorrow !!!

    Friday, April 21, 2017

    What is paranoia mode in nscd?

    This variable can be used in nscd.conf file if you wish nscd daemon to restart itself after a provided amount of time interval. This argument is used in combination with "restart-interval"

    Open nscd.conf in edit mode
    # vi /etc/nscd.conf
    # enable paranoia for One NDS for 1h restart interval
            paranoia                yes
            restart-interval        3600

    Here as per my configuration nscd daemon will be restarted automatically by default every hour.

    Follow below link for more information on nscd
    Understanding nscd daemon for hosts cache

    Follow the below links for more tutorials

    How to find the path of any command in Linux
    How to configure a Clustered Samba share using ctdb in Red Hat Cluster
    How to delete an iscsi-target from openfiler and Linux
    How to perform a local ssh port forwarding in Linux
    How to use yum locally without internet connection using cache?
    What is umask and how to change the default value permanently?
    Understanding Partition Scheme MBR vs GPT
    How does a successful or failed login process works in Linux
    How to find all the process accessing a file in Linux
    How to exclude multiple directories from du command in Linux
    How to configure autofs in Linux and what are its advantages?
    How to resize software raid partition in Linux
    How to configure Software RAID 1 mirroring in Linux
    How to prevent a command from getting stored in history in Linux

    Understanding nscd daemon for hosts cache

    nscd stands for Name Service Cache Daemon and is used to provide cache for common name service request. For providing hosts cache nscd daemon uses /etc/hosts file as it's database and any changes made to the database is immediately noticd ny nscd and it will flush the cache once these are changed. However, this will happen only after a short delay (unless the inotify(7) mechanism is available and glibc 2.9 or later is available)

    This daemon is used in most environments where a lookup for various database and tables are needed in a frequent time to time base to build a cache and increase the end performance of the application for eg. LDAP uses nscd to process any bind request from clients to the server, webservers etc

    nscd provides caching for different databases using standard libc interfaces like for hosts database it uses GETHOSTBYADDR, GETHOSTBYNAME and others.

    There are two caches for each database: a positive one for items found, and a negative one for items not found. Each cache has a separate TTL (time-to-live) period for its data. These parameters are configurable using /etc/nscd.conf file.

    Let us look at the several options and variables available for hosts cache

    To collect the statistics of nscd execute the below command

    NOTE: Since for this article we are concentrating on hosts cache I will grep the output which only shows hosts cache details

    hosts cache:
                yes  cache is enabled
                 no  cache is persistent
                yes  cache is shared
                211  suggested size
             216064  total data pool size
                320  used data pool size
                600  seconds time to live for positive entries
                  2  seconds time to live for negative entries
                  5  cache hits on positive entries
                  0  cache hits on negative entries
                  9  cache misses on positive entries
                  1  cache misses on negative entries
                 33% cache hit rate
                  2  current number of cached values
                  4  maximum number of cached values
                  1  maximum chain length searched
                  0  number of delays on rdlock
                  0  number of delays on wrlock
                  0  memory allocations failed
                yes  check /etc/{hosts,resolv.conf} for changes

    'cache is enabled' - informs about the status of the hosts cache which here means we have enabled this cache. If any cache is disabled this will be 'no'

    'cache is persistent' - This is if you want the cache to be persistent across daemon restart i.e. the stored statistics will be saved in the memory and will not be refreshed for most of the stat values.

    For eg:
    My existing stats with 'enabled' persistent caching

    hosts cache:
                yes  cache is enabled
               
    yes  cache is persistent
                yes  cache is shared
                211  suggested size
            3244035  total data pool size
                  0  used data pool size
                600  seconds time to live for positive entries
                  2  seconds time to live for negative entries
                  0  cache hits on positive entries
                  0  cache hits on negative entries
                  0  cache misses on positive entries
              88180  cache misses on negative entries
                  0% cache hit rate
                  0  current number of cached values
              30889  maximum number of cached values
                185  maximum chain length searched
                  0  number of delays on rdlock
                  0  number of delays on wrlock
                  0  memory allocations failed
                yes  check /etc/{hosts,resolv.conf} for changes

    Restarted nscd service
    # /etc/init.d/nscd restart
    Shutting down Name Service Cache Daemon                                                                                
    done
    Starting Name Service Cache Daemon    

    Post restart of the daemon the values are still same
    hosts cache:
                yes  cache is enabled
                yes  cache is persistent
                yes  cache is shared
                211  suggested size
            3244035  total data pool size
                  0  used data pool size
               
    600  seconds time to live for positive entries
                  2  seconds time to live for negative entries
                  0  cache hits on positive entries
                  0  cache hits on negative entries
                  0  cache misses on positive entries
              88180  cache misses on negative entries
                  0% cache hit rate
                  0  current number of cached values
              30889  maximum number of cached values
                185  maximum chain length searched
                  0  number of delays on rdlock
                  0  number of delays on wrlock
                  0  memory allocations failed
                yes  check /etc/{hosts,resolv.conf} for changes


    After disabling 'cache is persistent' and restarting nscd daemon service
    hosts cache:

                yes  cache is enabled
                 
    no  cache is persistent
                yes  cache is shared
                211  suggested size
             216064  total data pool size
                  0  used data pool size
                600  seconds time to live for positive entries
                  2  seconds time to live for negative entries
                  0  cache hits on positive entries
                  0  cache hits on negative entries
                  0  cache misses on positive entries
                  0  cache misses on negative entries
                  0% cache hit rate
                  0  current number of cached values
                  0  maximum number of cached values
                  0  maximum chain length searched
                  0  number of delays on rdlock
                  0  number of delays on wrlock
                  0  memory allocations failed
                yes  check /etc/{hosts,resolv.conf} for changes

    So all the cache entries are cleared.

    cache is shared - If this is enabled any client nodes connecting to the the server will perform lookup themself in the nscd cache rather than asking nscd daemon which makes the lookup process faster. nscd daemon would be needed only to update the cache if the client host entry is unavailable in the hosts cache. Once nscd is in shared mode the nascd cache hit rate is mostly shown as 0% as nscd is mostly not use and the reverselookup is performed from the cache

    suggested size - (From the man page) This is the internal hash table size, value should remain a prime number for optimum efficiency.  The default is 211.

    total data pool size - This accounts for the total list of cache host entry which has been looked up by nscd (both positive and negative)

    used data pool size - Hosts cache used in the current session of nscd. Everytime nscd daemon is restarted this value will reset to "0" and a fresh used data pool size is built using the existing hosts file.

    seconds time to live for positive entries - (From the man page) Sets the TTL (time-to-live) for positive entries (successful queries) in the specified cache for service. Value is in seconds. Larger values increase cache hit rates and reduce mean response times, but increase problems with cache coherence.

    seconds time to live for negative entries - (From the man page) Sets the TTL (time-to-live) for negative entries (unsuccessful queries) in the specified cache for service.  Value is in seconds.  Can result in significant performance improvements if there are several files owned by UIDs (user IDs) not in system databases (for example untarring the Linux kernel sources as root); should be kept small to reduce cache coherency problems.

    cache hits on positive entries - This value will be populated only if nscd daemon is running in non shared mode i.e. 'cache is shared' variable is 'no'. In such case nscd performs all the lookups and will increment the value for any lookup from a target host which manages to establish a ESTABLISHED network connection with the client hosts.

    For eg.
    I have added below entry in hosts file
    192.169.32.10 cc01-nds-ins
    Next I attempt ssh from 192.169.32.10 to the target node and observe the nscd stats
    # nscd -g | grep "hosts cache" -A 22 | grep "cache hits on positive entries"
                 13  cache hits on positive entries

    So we have an increment in the cache hit rate for positive entries since 192.169.32.10 was present in our hosts file

    cache hits on negative entries - This value will be populated only if nscd daemon is running in non shared mode i.e. 'cache is shared' variable is 'no'. In such case nscd performs all the lookups and will increment the value for any lookup from a target host which fails to establish a ESTABLISHED network connection with the client hosts.

    memory allocations failed - If persistent mode is not enabled then there is very less chance that you will see this value incrementing unless the alloted database size for nscd goes out of space. When persistent mode is enabled all the caches are stored in memory which might run out of space when you will start see incrementing values of memory allocation failures

    For eg:
    I reduced my database size to below size
           max-db-size             hosts           335511
    and restarted nscd services

    After a while I started receiving multiple memory allocation failures
    hosts cache:

                yes  cache is enabled
                 no  cache is persistent
                yes  cache is shared
                211  suggested size
             334559  total data pool size
             334544  used data pool size
                600  seconds time to live for positive entries
                  2  seconds time to live for negative entries
                  0  cache hits on positive entries
                  0  cache hits on negative entries
                  1  cache misses on positive entries
               3483  cache misses on negative entries
                  0% cache hit rate
                  1  current number of cached values
               3484  maximum number of cached values
                 28  maximum chain length searched
                  0  number of delays on rdlock
                  0  number of delays on wrlock
             100418  memory allocations failed
                yes  check /etc/{hosts,resolv.conf} for changes



    Follow the below links for more tutorials

    Tutorial for Monitoring Tools SAR and KSAR with examples in Linux
    How to secure Apache web server in Linux using password (.htaccess)
    Red hat Enterprise Linux 5.5 Installation Guide (Screenshots)
    15 tips to enhance security of your Linux machine
    Why is Linux more secure than windows and any other OS
    How to log iptables messages in different log file

    quotacheck: Something weird happened while scanning. Error 2133571361

    Error

    quotacheck: Something weird happened while scanning. Error 2133571361

    Explanation:

    Other day I was attempting to enable quota on one of my vms and I observed the quotachecl failed with above error.

    This error most likely means that quotacheck was unable to perform read write operation on the disk where the provided partition was mounted.

    Below errors are expected as seen for my case inside /var/log/messages
    Apr 21 15:18:49 deep-test kernel: [179264.576665] sd 0:0:0:0: [sda]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
    Apr 21 15:18:49 deep-test kernel: [179264.576671] sd 0:0:0:0: [sda]  Sense Key : Medium Error [current]
    Apr 21 15:18:49 deep-test kernel: [179264.576674] Info fld=0x70148b
    Apr 21 15:18:49 deep-test kernel: [179264.576676] sd 0:0:0:0: [sda]  Add. Sense: Unrecovered read error
    Apr 21 15:18:49 deep-test kernel: [179264.576679] sd 0:0:0:0: [sda] CDB: Read(10): 28 00 09 fd 31 d0 00 02 00 00
    Apr 21 15:18:49 deep-test kernel: [179264.576691] end_request: critical target error, dev sda, sector 167588304
    Apr 21 15:18:49 deep-test kernel: [179264.576699] Buffer I/O error on device dm-4, logical block 18087994
    Apr 21 15:18:49 deep-test kernel: [179264.576705] Buffer I/O error on device dm-4, logical block 18087995
    Apr 21 15:18:49 deep-test kernel: [179264.576707] Buffer I/O error on device dm-4, logical block 18087996
    Apr 21 15:18:49 deep-test kernel: [179264.576710] Buffer I/O error on device dm-4, logical block 18087997
    Apr 21 15:18:49 deep-test kernel: [179264.576713] Buffer I/O error on device dm-4, logical block 18087998
    Apr 21 15:18:49 deep-test kernel: [179264.576715] Buffer I/O error on device dm-4, logical block 18087999
    Apr 21 15:18:49 deep-test kernel: [179264.576717] Buffer I/O error on device dm-4, logical block 18088000
    Apr 21 15:18:49 deep-test kernel: [179264.576720] Buffer I/O error on device dm-4, logical block 18088001
    Apr 21 15:18:49 deep-test kernel: [179264.576722] Buffer I/O error on device dm-4, logical block 18088002
    Apr 21 15:18:49 deep-test kernel: [179264.576725] Buffer I/O error on device dm-4, logical block 18088003
    Apr 21 15:18:57 deep-test kernel: [179272.674406] sd 0:0:0:0: [sda]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
    Apr 21 15:18:57 deep-test kernel: [179272.674414] sd 0:0:0:0: [sda]  Sense Key : Medium Error [current]
    Apr 21 15:18:57 deep-test kernel: [179272.674418] Info fld=0x70148b
    Apr 21 15:18:57 deep-test kernel: [179272.674421] sd 0:0:0:0: [sda]  Add. Sense: Unrecovered read error
    Apr 21 15:18:57 deep-test kernel: [179272.674425] sd 0:0:0:0: [sda] CDB: Read(10): 28 00 09 fd 32 88 00 00 08 00
    Apr 21 15:18:57 deep-test kernel: [179272.674434] end_request: critical target error, dev sda, sector 167588488
    Apr 21 15:18:57 deep-test kernel: [179272.674443] quiet_error: 54 callbacks suppressed
    Apr 21 15:18:57 deep-test kernel: [179272.674446] Buffer I/O error on device dm-4, logical block 18088017Solution
    Perform a HDD check using fsck or get the disk replaced with a healthy one then the issue should be fixed

    Follow the below links for more tutorials

    Tutorial for Monitoring Tools SAR and KSAR with examples in Linux
    How to secure Apache web server in Linux using password (.htaccess)
    Red hat Enterprise Linux 5.5 Installation Guide (Screenshots)
    15 tips to enhance security of your Linux machine
    Why is Linux more secure than windows and any other OS
    How to log iptables messages in different log file

    Thursday, November 24, 2016

    How to Remove Duplicate Rows from a Table

    Use rowid pseudo column. All you have to do is to keep the latest data (i.e. highest ROWID) and remove other duplicated rows.
    SELECT * FROM table1 a
    WHERE rowid < (SELECT max(rowid) FROM table1 b
    WHERE a.column1 = b.column1 AND etc...);

    OR

    create table testtt (num number);

    insert into testtt values(111);
    insert into testtt values(111);
    insert into testtt values(111);
    insert into testtt values(111);
    insert into testtt values(222);
    insert into testtt values(222);
    insert into testtt values(333);
    insert into testtt values(333);
    insert into testtt values(333);


    select * from testtt;

    delete from testtt
    where (rowid, num) not in (select max_rid, num
    from (select num,
    count(num) over (partition by num) cnt,
    max(rowid) over (partition by num) max_rid
    from testtt)
    where cnt > 1);

    select * from testtt;

    OR

    While I doubt this method has any advantages over another, it's an example:
    DELETE FROM table_a
    WHERE rowid IN
    ( SELECT rowid FROM table_a
    MINUS
    SELECT MAX( rowid ) FROM table_a
    GROUP BY column_list )

    OR

    delete from table_name where rowid not in (select max(rowid) from table group byduplicate_values_field_name);
    OR

    highest rowid does not necessarily mean latest data... since space freed from deleting rows might be reused.
    SQL> CREATE TABLE t AS SELECT level l FROM DUAL CONNECT BY LEVEL <= 5000;
    SQL> DELETE FROM t WHERE l < 5000;
    SQL> COMMIT;
    SQL> INSERT INTO t VALUES (5001);
    SQL> COMMIT;
    SQL> SELECT max(l) KEEP(DENSE_RANK LAST ORDER BY rowid) as maxrid, max(l) KEEP(DENSE_RANK FIRST ORDER BY rowid) minrid FROM t;

    MAXRID MINRID
    ---------- ----------
    5000 5001


    Sunday, October 02, 2016

    Why is Linux considered more safer than windows or any other OS ?

    Now we hear many times from most of the people around us claiming that Linux is much more secure than Windows or Linux is very secure.

    But what are the things in Linux that makes it secure than Windows or any other Operating System.

    Here I will try to throw some light on some of the security features which I know in Linux and believe that these points add up to the construction of a much more secure wall all around the Linux Operating System.

    1. Execshield

    ExecShield is designed to prevent security breaches caused by software programs written to crawl
    through the Internet looking for systems with common vulnerabilities such as worms and viruses. It is enabled in the kernel and works in a way that is non intrusive to the user.

    Its goal is not to defend against the expert hacker who has broken into your local network or an employee inside the company who already has access to parts of the network instead its goal is to prevent against intruders using scripts that look for vulnerabilities in the way a program running with root privileges is written.

    For more knowledge on Execshield follow the below link
    Prevent security breaches with Execshield

    2. SElinux (Security Enhanced Linux)

    SELinux is an implementation of a flexible mandatory access control architecture in the Linux operating system.  The SELinux architecture provides general support for the enforcement of many kinds of  mandatory  access control  policies,including those based on the concepts of Type Enforcement(R), Role- Based Access Control, and Multi-Level Security.

    SELinux can potentially control which activities a system allows each user, process and daemon, with very precise specifications. However, it is mostly used to confine daemons like database engines or web servers that have more clearly-defined data access and activity rights. This limits potential harm from a confined daemon that becomes compromised. Ordinary user-processes often run in the unconfined domain, not restricted by SELinux but still restricted by the classic Linux access rights.

    3. IPtables

    With the enhanced features available with the IPtables you can implement a greater level of security for your Linux machine.

    IPtables  is used to set up, maintain, and inspect the tables of IP packet filter rules in the Linux kernel.  Several different tables may be defined.  Each table contains a number of built-in chains and may also contain user-defined chains.

    Each chain is a list of rules which can match a set of packets.  Each rule specifies what to do with a packet that matches.   This  is called a 'target', which may be a jump to a user-defined chain in the same table.

    For more information on iptables please follow the below link
    iptables tutorials for Linux I
    iptables tutorial for Linux II

    4. PAM (Pluggable Authentication Modules)

    Linux-PAM is a system of libraries that handle the authentication tasks of applications (services) on the system.  The principal feature of the PAM approach is that the nature of the authentication is dynamically configurable. In other words, the system administrator is free to choose how individual service-providing applications will authenticate users.

    Linux-PAM separates the tasks of authentication into four independent management groups: account management; authentication management; password management; and session management.

    account - provide account verification types of service: has the user's password expired?; is this user permitted access to the requested service?

    authentication - authenticate a user and set up user credentials. Typically this is via some challenge-response request that the user must satisfy: if you are who you claim to be please enter your password.

    password - this group's responsibility is the task of updating authentication mechanisms. Typically, such services are strongly coupled to those of the auth group. Some authentication mechanisms lend themselves well to being updated with such a function. Standard UN*X password-based access is the obvious example: please enter a replacement password.

    session - this group of tasks cover things that should be done prior to a service being given and after it is withdrawn. Such tasks include the maintenance of audit trails and the mounting of the user's home directory. The session management group is important as it provides both an opening and closing hook for modules to affect the services available to a user.

    5. Audit

    The 2.6 Linux kernel has the ability to log events such as system calls and file access. These logs can then be reviewed by the administrator to determine possible security breaches such as failed login attempts or a user failing to access system files. This functionality, called the Linux Auditing System.

    auditd  is  the  userspace component to the Linux Auditing System. It's responsible for writing audit records to the disk. Viewing the logs is done with the ausearch or aureport utilities. Configuring the audit rules is done with the auditctl utility.  During  startup, the  rules  in  /etc/audit/audit.rules are read by auditctl.

    For more information please follow the below link
    Using audit in Linux to track changes

    These are some of the front level security features available in Linux for a better security. I would be glad if someone would like to review or add any of the missing points in this article.

    Apart from these there are a numerous options which can be utilized to enhanced the level of security for your Linux machine.

    You can follow the below link to find out tips on improving the security of your Linux server
    15 tips to improve security of your Linux server