- High availability
- Load balancing
- High performance
- Storage clusters provide a consistent file system image across servers in a cluster, allowing the servers to simultaneously read and write to a single shared file system.
- A storage cluster simplifies storage administration by limiting the installation and patching of applications to one file system.
- The High Availability Add-On provides storage clustering in conjunction with Red Hat GFS2
- High availability clusters provide highly available services by eliminating single points of failure and by failing over services from one cluster node to another in case a node becomes inoperative.
- Typically, services in a high availability cluster read and write data (via read-write mounted file systems).
- A high availability cluster must maintain data integrity as one cluster node takes over control of a service from another cluster node.
- Node failures in a high availability cluster are not visible from clients outside the cluster.
- High availability clusters are sometimes referred to as failover clusters.
- Load-balancing clusters dispatch network service requests to multiple cluster nodes to balance the request load among the cluster nodes.
- Load balancing provides cost-effective scalability because you can match the number of nodes according to load requirements. If a node in a load-balancing cluster becomes inoperative, the load-balancing software detects the failure and redirects requests to other cluster nodes.
- Node failures in a load-balancing cluster are not visible from clients outside the cluster.
- Load balancing is available with the Load Balancer Add-On.
- High-performance clusters use cluster nodes to perform concurrent calculations.
- A high-performance cluster allows applications to work in parallel, therefore enhancing the performance of the applications.
- High performance clusters are also referred to as computational clusters or grid computing.
- Better performance for heavy usage in a single directory
- Faster synchronous I/O operations
- Faster cached reads (no locking overhead)
- Faster direct I/O with preallocated files (provided I/O size is reasonably large, such as 4M blocks)
- Faster I/O operations in general
- Faster Execution of the df command, because of faster statfs calls
- Improved atime mode to reduce the number of write I/O operations generated by atime when compared with GFS
- GFS2 supports the following features.
- extended file attributes (xattr)
- the lsattr() and chattr() attribute settings via standard ioctl() calls
- nanosecond timestamps
- GFS2 uses less kernel memory.
- GFS2 requires no metadata generation numbers.
- Allocating GFS2 metadata does not require reads. Copies of metadata blocks in multiple journals are managed by revoking blocks from the journal before lock release.
- GFS2 includes a much simpler log manager that knows nothing about unlinked inodes or quota changes.
- The gfs2_grow and gfs2_jadd commands use locking to prevent multiple instances running at the same time.
- The ACL code has been simplified for calls like creat() and mkdir().
- Unlinked inodes, quota changes, and statfs changes are recovered without remounting the journal.
- GFS2 is based on 64 bit architecture, which can theoretically accommodate an 8 EB file system.
- However, the current supported maximum size of a GFS2 file system for 64-bit hardware is 100 TB.
- The current supported maximum size of a GFS2 file system for 32-bit hardware for Red Hat Enterprise Linux Release 5.3 and later is 16 TB.
- NOTE: It is better to have 10 1TB file systems than one 10TB file system.
- A journaling filesystem is a filesystem that maintains a special file called a journal that is used to repair any inconsistencies that occur as the result of an improper shutdown of a computer.
- In journaling file systems, every time GFS2 writes metadata, the metadata is committed to the journal before it is put into place.
- This ensures that if the system crashes or loses power, you will recover all of the metadata when the journal is automatically replayed at mount time.
- GFS2 requires one journal for each node in the cluster that needs to mount the file system. For example, if you have a 16-node cluster but need to mount only the file system from two nodes, you need only two journals. If you need to mount from a third node, you can always add a journal with the gfs2_jadd command.
- Quorum Disk is a disk-based quorum daemon, qdiskd, that provides supplemental heuristics to determine node fitness.
- With heuristics you can determine factors that are important to the operation of the node in the event of a network partition
- This is a service termed as Resource Group Manager
- RGManager manages and provides failover capabilities for collections of cluster resources called services, resource groups, or resource trees
- it allows administrators to define, configure, and monitor cluster services. In the event of a node failure, rgmanager will relocate the clustered service to another node with minimal service disruption
- luci is the server component of the Conga administration utility
- Conga is an integrated set of software components that provides centralized configuration and management of Red Hat clusters and storage
- luci is a server that runs on one computer and communicates with multiple clusters and computers via ricci
- ricci is the client component of the Conga administration utility
- ricci is an agent that runs on each computer (either a cluster member or a standalone computer) managed by Conga
- This service needs to be running on all the client nodes of the cluster.
- This is an abbreviation used for Cluster Manager.
- CMAN is a distributed cluster manager and runs in each cluster node.
- It is responsible for monitoring, heartbeat, quorum, voting and communication between cluster nodes.
- CMAN keeps track of cluster quorum by monitoring the count of cluster nodes.
IP Port no.
dlm (Distributed Lock Manager)
- The use of NetworkManager is not supported on cluster nodes. If you have installed NetworkManager on your cluster nodes, you should either remove it or disable it.
- # service NetworkManager stop
- # chkconfig NetworkManager off
- The cman service will not start if NetworkManager is either running or has been configured to run with the chkconfig command
- We say a cluster has quorum if a majority of nodes are alive, communicating, and agree on the active cluster members. For example, in a thirteen-node cluster, quorum is only reached if seven or more nodes are communicating. If the seventh node dies, the cluster loses quorum and can no longer function.
- A cluster must maintain quorum to prevent split-brain issues.
- If quorum was not enforced, quorum, a communication error on that same thirteen-node cluster may cause a situation where six nodes are operating on the shared storage, while another six nodes are also operating on it, independently. Because of the communication error, the two partial-clusters would overwrite areas of the disk and corrupt the file system.
- With quorum rules enforced, only one of the partial clusters can use the shared storage, thus protecting data integrity.
- Quorum doesn't prevent split-brain situations, but it does decide who is dominant and allowed to function in the cluster.
- quorum can be determined by a combination of communicating messages via Ethernet and through a quorum disk.
- Tie-breakers are additional heuristics that allow a cluster partition to decide whether or not it is quorate in the event of an even-split - prior to fencing.
- With such a tie-breaker, nodes not only monitor each other, but also an upstream router that is on the same path as cluster communications. If the two nodes lose contact with each other, the one that wins is the one that can still ping the upstream router.That is why, even when using tie-breakers, it is important to ensure that fencing is configured correctly.
- CMAN has no internal tie-breakers for various reasons. However, tie-breakers can be implemented using the API.
- Fencing is the disconnection of a node from the cluster's shared storage.
- Fencing cuts off I/O from shared storage, thus ensuring data integrity.
- The cluster infrastructure performs fencing through the fence daemon, fenced.
- When CMAN determines that a node has failed, it communicates to other cluster-infrastructure components that the node has failed.
- fenced, when notified of the failure, fences the failed node.
- Power fencing — A fencing method that uses a power controller to power off an inoperable node.
- storage fencing — A fencing method that disables the Fibre Channel port that connects storage to an inoperable node.
- Other fencing — Several other fencing methods that disable I/O or power of an inoperable node, including IBM Bladecenters, PAP, DRAC/MC, HP ILO, IPMI, IBM RSA II, and others.
- DLM is a short abbreviation for Distributed Lock Manager.
- A lock manager is a traffic cop who controls access to resources in the cluster, such as access to a GFS file system.
- GFS2 uses locks from the lock manager to synchronize access to file system metadata (on shared storage)
- CLVM uses locks from the lock manager to synchronize updates to LVM volumes and volume groups (also on shared storage)
- In addition, rgmanager uses DLM to synchronize service states.
- without a lock manager, there would be no control over access to your shared storage, and the nodes in the cluster would corrupt each other's data.