- Types of HugePages
- How to disable Transparent Hugepages on runtime
- How to disable Transparent Hugepages permanently using GRUB2
- How to disable Transparent Hugepages permanently using tuned profile
- How to check if HugePages is disabled
- How to check if HugePages is enabled
- How to disable Explicit Transparent Hugepages on runtime
- How to disable Explicit Transparent Hugepages permanently
- How to enable and assign explicit hugepages permanently
What is transparent hugepage (THP) and how to check THP usage per process in Linux (Explained)
Types of HugePages:
There can be two types of HugePages in the system
- Explicit Huge Pages which are allocated explicitly by vm.nr_hugepages sysctl parameter i.e. the pages that are used as huge pages are reserved inside the kernel and cannot be used for other purposes.
- Transparent Huge Pages which are allocated automatically by the kernel.
How to disable Transparent Hugepages on runtime?
Running the below commands will stop only creation and usage of the new THP.
The THP which were created and used at the moment the above commands were run would not be disassembled into the regular memory pages.
To get rid of THP completely the system should be rebooted with THP disabled at boot time.
# echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
"defer" means that an application will wake kswapd in the background to reclaim pages and wake kcompactd to compact memory so that THP is available in the near future. It's the responsibility of khugepaged to then install the THP pages later.
"madvise" will enter direct reclaim like "always" but only for regions that are have used madvise(MADV_HUGEPAGE). This is the default behaviour.
"never" means do not allocate THP at all
By default kernel tries to use huge zero page on read page fault to anonymous mapping. It's possible to disable huge zero page by writing 0 or enable it back by writing 1
echo 1 >/sys/kernel/mm/transparent_hugepage/use_zero_page
How to disable Transparent Hugepages permanently using GRUB2?
You can change the sysfs boot time defaults of Transparent Hugepage support by passing the parameter "transparent_hugepage=always" or "transparent_hugepage=madvise" or "transparent_hugepage=never" to the kernel command line.
To disable THP use "transparent_hugepage=never" to the kernel command line in the grub configuration file.
For my setup I am using GRUB2 (/etc/sysconfig/grub)
Next rebuild your grub configuration file
Reboot the node for the changes to take affect. Once the node is up make sure the newly added entry exists in your loaded grub configuration
BOOT_IMAGE=/vmlinuz-3.10.0-862.6.3.el7.x86_64 root=/dev/mapper/os-root ro novga console=ttyS0,115200 panic=1 numa=off elevator=cfq rd.md.uuid=d265dd3d:9ee4d53a:597b8c08:8201b9af rd.lvm.lv=os/root rd.md.uuid=5398452a:ab1b8e91:4307b53b:5c3cccbd rd.md.uuid=131bc1e7:7c9087c3:03f3ad4a:7cde170c noht biosdevname=0 net.ifnames=0 rhgb quiet transparent_hugepage=never
So the configuration is correctly loaded.
How to check if CPU supports HugePages and change default hugepage size in RHEL/CentOS 7
How to check Transparent HugePage usage per process in Linux with examples
How to disable Transparent Hugepages permanently using tuned profile?
You can either use your existing tuned profile or you can create a new profile.
Use your existing tuned profile.
My node is configured with throughput-performance profile, you can check your loaded profile using below command
Current active profile: throughput-performance
Next edit your tuned profile configuration file and append below value under '[vm]'
[vm]
transparent_hugepage=never
Next activate your profile
Validate your changes
always madvise [never]
Create a new customised tuned profile
With this we will create a customized version of the currently running profile. The customized version will only disable THP.
Find out which profile is active, create a copy. In the following example we currently use the throughput-performance profile:
Current active profile: throughput-performance
To create customized profile, create a new directory in /etc/tuned directory with desired profile name.
Then create a new tuned.conf file for myprofile-disablethp, and insert the new tuning info:
[main]
include= throughput-performance
[vm]
transparent_hugepages=never
Make the script executable
Enable myprofile like
Validate your changes
always madvise [never]
How to check if HugePages is disabled?
In the above steps we disabled the THP to make sure kernel doesnot allocates or reserves any hugepage.
If the value of HugePages_Total is "0" it means HugePages is disabled on the system.
HugePages_Total: 0
Check below file
always madvise [never]
This must show that [never] is selected for hugepage as above
How to check if HugePages is enabled?
If the value of HugePages_Total is greater than "0", it means HugePages is enabled on the system
HugePages_Total: 23090
Check below file
[always] madvise never
This must "not" show highlighted option as [never].
How to disable Explicit Transparent Hugepages on runtime?
nr_hugepages indicates the current number of "persistent" huge pages in the kernel's huge page pool. "Persistent" huge pages will be returned to the huge page pool when freed by a task.
When multiple huge page sizes are supported, /proc/sys/vm/nr_hugepages indicates the current number of pre-allocated huge pages of the default size.
Thus, one can use the following command to dynamically allocate/deallocate default sized persistent huge pages. If the value in /proc/sys/vm/nr_hugepages file or vm.nr_hugepages sysctl parameter is "0" it means HugePages is disabled on the system
# cat /proc/sys/vm/nr_hugepages
0
# sysctl vm.nr_hugepages
vm.nr_hugepages = 0
How to disable Explicit Transparent Hugepages permanently?
To disable the explicit hugepage permanently add below entry in "/etc/sysctl.conf"
vm.nr_hugepages=0
Refresh the values
Below command will show the updated value
vm.nr_hugepages = 0
How to enable and assign explicit hugepages permanently?
One allocate persistent huge pages on the kernel boot command line by specifying the "hugepages=N" parameter, where 'N' = the number of huge pages requested. This is the most reliable method of allocating huge pages as memory has not yet become fragmented.
Append "hugepages=N" to "/etc/sysconfig/grub" to GRUB_CMDLINE_LINUX as shown below
Rebuild the initramfs
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-693.21.1.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-693.21.1.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-fae7c244c3134771a10cc7c3ace3edcb
Found initrd image: /boot/initramfs-0-rescue-fae7c244c3134771a10cc7c3ace3edcb.img
done
Reboot the node for the changes to take affect
Validate your changes post reboot
vm.nr_hugepages = 2048
You can also reserve hugepages using sysctl.conf
Refresh the values
Reboot the node for the changes to take affect.