Understanding Load Average in Linux and when to be worried about it?

The one of the most important task of any Linux Admin includes performance monitoring which includes a parameter "Load Average" or "CPU Load".
Load Average is the value which represents the load on your system for a specific period of time. Also it can be considered the ratio of the number of active tasks to the number of available CPUs.
 

How to check?

You can use either top or uptime command to view the load average. The output would look like as shown below

# uptime
00:07:00 up 4 days, 6:14, 1 user, load average: 0.11, 0.14, 0.09
# top
top - 00:07:12 up 4 days, 6:15, 1 user, load average: 0.09, 0.13, 0.09

 

What are the three values?

As you can see three values representing the load average column. These show the load on your system over a significant period of time (one or current, five and fifteen minutes averages).
 

How do you know your system has a high load?

The most important question as in most cases I have seen how do you determine your system has high load.
Does a high value represents high load average and that your system requires atention?
What is the threshold value for load average?
How can we conclude if the load average value is good or bad?
Before I answer these question first let us understand about multi core and multi processor CPU.
A Central Processing Unit in earlier days used to be having only one processor and the core concept was not their in those days. But with the advancement in technology and the urge of higher speed to meet up demands of IT industry multiple processor were integrated in the same CPU making it multi-processor.
However increasing the no. of processor did increased the working speed of many tasks and performance but it also leads to increase in size, complexity and heat issues. So, in order to continue improvement of performance the core concept was introduced.
Instead of having two CPUs and a motherboard capable of hosting them, two CPUS are taken together and combined to form a dual core processor which will utilize an individual socket using less power and size capable of performing the same amount of task as dual processor CPU.
Bottom Line is that Load value depends on the no. of cores in your machine. For example a dual core is relevant to 2 processor or 2 cores and quad core is relevant to 4 processor or four cores as the maximum value for load.
 

How do I check the no. of cores on my Linux system?

Number of physical CPUs
The following command will show how many active physical processors a system has.
Example: If this number is 2, one could potentially open up the system chassis and remove 2 physical processors with one's hands.

# grep physical.id /proc/cpuinfo | sort -u | wc -l
2

 
Number of cores per CPU
On a system with multi-core processors, the following command should report the number of CPU cores per physical processor (though in rare cases it might not).
Example: If this number is 8 and physical CPUs is 2, then each of the 2 physical processors has 8 CPU cores, leading to a total of 16 cores.

# grep cpu.cores /proc/cpuinfo | sort -u
cpu cores : 8

 
Number of logical processors
This last command will show the total number of "logical" processors seen by the Linux kernel. This number is usually the most important of the three stats. It is the effective number of processors -- as far as the operating system is concerned, this is the number of distinct CPUs that can do work at any given microsecond.
Example: Continuing with the above example scenario, the number seen below could be 16 instead of 8. Simply put, if this command shows a different number than the total number of CPU cores, it's because hyper-threading is enabled on the CPUs, further dividing each core (in this example, into 2 usable "threads").

# grep processor /proc/cpuinfo | wc -l
16

How to check if Hyper Threading is enabled or disabled?
Coming back to our primary question
 

Understanding Load Average

If the number of active tasks utilising CPU is less as compared to available CPU cores then the load average can be considered normal but if the no. of active tasks starts increasing with respect to available CPU cores then the load average will start rising.
For example in my case I am using a dummy script to utilise all my CPU and increase the load

top - 20:29:27 up  1:41,  2 users,  load average: 11.11, 3.74, 1.44
Tasks: 257 total,  17 running, 240 sleeping,   0 stopped,   0 zombie
%Cpu(s):100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 13173656+total, 27243444 free, 10403308+used,   460040 buff/cache
KiB Swap:  4190204 total,  4190204 free,        0 used. 26855740 avail Mem
PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
22360 root      20   0  196724   5684    736 R 100.0  0.0   1:07.32 python
22361 root      20   0  196724   5684    736 R 100.0  0.0   1:07.33 python
22363 root      20   0  196724   5688    736 R 100.0  0.0   1:07.32 python
22364 root      20   0  196724   5688    736 R 100.0  0.0   1:07.32 python
22369 root      20   0  196724   5700    736 R 100.0  0.0   1:07.30 python
22371 root      20   0  196724   5704    736 R 100.0  0.0   1:07.29 python
22373 root      20   0  196724   5704    736 R 100.0  0.0   1:07.28 python
22374 root      20   0  196724   5708    736 R 100.0  0.0   1:07.26 python
22359 root      20   0  196724   5684    740 R 100.0  0.0   1:07.30 python
22362 root      20   0  196724   5688    740 R 100.0  0.0   1:07.32 python
22365 root      20   0  196724   5688    736 R 100.0  0.0   1:07.32 python
22366 root      20   0  196724   5692    736 R 100.0  0.0   1:07.32 python
22367 root      20   0  196724   5692    736 R 100.0  0.0   1:07.32 python
22368 root      20   0  196724   5692    736 R 100.0  0.0   1:07.32 python
22370 root      20   0  196724   5700    736 R 100.0  0.0   1:07.23 python
22372 root      20   0  196724   5704    736 R 100.0  0.0   1:07.29 python

So as per the no. of cores I calculated i.e 16 cores and seeing the value 11.11 Although this is high but I shouldn't be worried much unless it crosses the red line value i.e. 16 for my case.
I hope the article was useful.
 
Related Articles
Tutorial for Monitoring Tools SAR and KSAR with examples in Linux
9 examples to help you understand top command usage in Unix/Linux
15 tips to enhance security of your Linux machine
How to check all the currently running services in Linux