• VMware

    Learn about VMware virtualization for its products like vsphere ESX and ESXi, vCenter Server, VMware View, VMware P2V and many more

  • Linux

    Step by step configuration tutorials for many of the Linux services like DNS, DHCP, FTP, Samba4 etc including many tips and tricks in Red Hat Linux.

  • Database

    Learn installation and configuration of databases like Oracle, My SQL, Postgresql, etc including many other related tutorials in Linux.

  • How to enable or disable transparent (THP) and explicit (nr_hugepages) hugepage and check the status in Linux with examples (explained in detail)


    Below list of topics are covered in this article:


    Types of HugePages:

    There can be two types of HugePages in the system
    • Explicit Huge Pages which are allocated explicitly by vm.nr_hugepages sysctl parameter i.e. the pages that are used as huge pages are reserved inside the kernel and cannot be used for other purposes.
    • Transparent Huge Pages which are allocated automatically by the kernel.

      How to disable Transparent Hugepages on runtime?

      Running the below commands will stop only creation and usage of the new THP.
      The THP which were created and used at the moment the above commands were run would not be disassembled into the regular memory pages.

      To get rid of THP completely the system should be rebooted with THP disabled at boot time.
      # echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
      # echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag

      "always" means that an application requesting THP will stall on allocation failure and directly reclaim pages and compact memory in an effort to allocate a THP immediately.
      "defer" means that an application will wake kswapd in the background to reclaim pages and wake kcompactd to compact memory so that THP is available in the near future. It's the responsibility of khugepaged to then install the THP pages later.
      "madvise" will enter direct reclaim like "always" but only for regions that are have used madvise(MADV_HUGEPAGE). This is the default behaviour.
      "never" means do not allocate THP at all

      By default kernel tries to use huge zero page on read page fault to anonymous mapping. It's possible to disable huge zero page by writing 0 or enable it back by writing 1
      echo 0 >/sys/kernel/mm/transparent_hugepage/use_zero_page
      echo 1 >/sys/kernel/mm/transparent_hugepage/use_zero_page






      How to disable Transparent Hugepages permanently using GRUB2?

      You can change the sysfs boot time defaults of Transparent Hugepage support by passing the parameter "transparent_hugepage=always" or "transparent_hugepage=madvise" or "transparent_hugepage=never" to the kernel command line.

      To disable THP use "transparent_hugepage=never" to the kernel command line in the grub configuration file.
      For my setup I am using GRUB2 (/etc/sysconfig/grub)
      GRUB_CMDLINE_LINUX="novga console=ttyS0,115200 panic=1 numa=off elevator=cfq rd.md.uuid=d265dd3d:9ee4d53a:597b8c08:8201b9af rd.lvm.lv=os/root rd.md.uuid=5398452a:ab1b8e91:4307b53b:5c3cccbd rd.md.uuid=131bc1e7:7c9087c3:03f3ad4a:7cde170c noht rhgb quiet transparent_hugepage=never"

      Next rebuild your grub configuration file
      # grub2-mkconfig -o /boot/grub2/grub.cfg

      Reboot the node for the changes to take affect. Once the node is up make sure the newly added entry exists in your loaded grub configuration
      # grep transparent_hugepage /proc/cmdline
      BOOT_IMAGE=/vmlinuz-3.10.0-862.6.3.el7.x86_64 root=/dev/mapper/os-root ro novga console=ttyS0,115200 panic=1 numa=off elevator=cfq rd.md.uuid=d265dd3d:9ee4d53a:597b8c08:8201b9af rd.lvm.lv=os/root rd.md.uuid=5398452a:ab1b8e91:4307b53b:5c3cccbd rd.md.uuid=131bc1e7:7c9087c3:03f3ad4a:7cde170c noht biosdevname=0 net.ifnames=0 rhgb quiet transparent_hugepage=never

      So the configuration is correctly loaded.



      How to disable Transparent Hugepages permanently using tuned profile?

      You can either use your existing tuned profile or you can create a new profile.

      Use your existing tuned profile.
      My node is configured with throughput-performance profile, you can check your loaded profile using below command
      # tuned-adm active
      Current active profile: throughput-performance

      Next edit your tuned profile configuration file and append below value under '[vm]'
      # vim /usr/lib/tuned/throughput-performance/tuned.conf
      [vm]
      transparent_hugepage=never

      Next activate your profile
      # tuned-adm profile throughput-performance

      Validate your changes
      # cat /sys/kernel/mm/transparent_hugepage/enabled
      always madvise [never]

      IMPORTANT NOTE: This does not requires reboot and the changes will get activated immediately

      Create a new customised tuned profile
      With this we will create a customized version of the currently running profile. The customized version will only disable THP.

      Find out which profile is active, create a copy. In the following example we currently use the throughput-performance profile:
      # tuned-adm active
      Current active profile: throughput-performance

      To create customized profile, create a new directory in /etc/tuned directory with desired profile name.
      # mkdir /etc/tuned/myprofile-disablethp

      Then create a new tuned.conf file for myprofile-disablethp, and insert the new tuning info:
      # cat /etc/tuned/myprofile-disablethp/tuned.conf
      [main]
      include= throughput-performance

      [vm]
      transparent_hugepages=never

      Make the script executable
      # chmod +x /etc/tuned/myprofile-disablethp/tuned.conf

      Enable myprofile like
      # tuned-adm profile myprofile-disablethp

      Validate your changes
      # cat /sys/kernel/mm/transparent_hugepage/enabled
      always madvise [never]

      IMPORTANT NOTE: This does not requires reboot and the changes will get activated immediately



      How to check if HugePages is disabled?

      In the above steps we disabled the THP to make sure kernel doesnot allocates or reserves any hugepage.

      If the value of HugePages_Total is "0" it means HugePages is disabled on the system.
      # grep -i HugePages_Total /proc/meminfo
      HugePages_Total:       0

      Check below file
      # cat /sys/kernel/mm/transparent_hugepage/enabled
      always madvise [never]

      This must show that [never] is selected for hugepage as above

      How to check if HugePages is enabled?

      If the value of HugePages_Total is greater than "0", it means HugePages is enabled on the system
      # grep -i HugePages_Total /proc/meminfo
      HugePages_Total:   23090

      Check below file
      # cat /sys/kernel/mm/transparent_hugepage/enabled
      [always] madvise never

      This must "not" show highlighted option as [never].

      How to disable Explicit Transparent Hugepages on runtime?

      nr_hugepages indicates the current number of "persistent" huge pages in the kernel's huge page pool.  "Persistent" huge pages will be returned to the huge page pool when freed by a task.

      When multiple huge page sizes are supported, /proc/sys/vm/nr_hugepages indicates the current number of pre-allocated huge pages of the default size.

      Thus, one can use the following command to dynamically allocate/deallocate default sized persistent huge pages. If the value in /proc/sys/vm/nr_hugepages file or vm.nr_hugepages sysctl parameter is "0" it means HugePages is disabled on the system
      # echo 0 > /proc/sys/vm/nr_hugepages

      # cat /proc/sys/vm/nr_hugepages
      0

      # sysctl vm.nr_hugepages
      vm.nr_hugepages = 0



      How to disable Explicit Transparent Hugepages permanently?

      To disable the explicit hugepage permanently add below entry in "/etc/sysctl.conf"
      # grep -i nr_hugepages /etc/sysctl.conf
      vm.nr_hugepages=0

      Refresh the values
      # sysctl -p

      Below command will show the updated value
      # sysctl -a | grep nr_hugepages
      vm.nr_hugepages = 0

      NOTE: A reboot is required to activate the changes.

      How to enable and assign explicit hugepages permanently?

      One allocate persistent huge pages on the kernel boot command line by specifying the "hugepages=N" parameter, where 'N' = the number of huge pages requested.  This is the most reliable method of allocating huge pages as memory has not yet become fragmented.

      Append "hugepages=N" to "/etc/sysconfig/grub" to GRUB_CMDLINE_LINUX as shown below
      GRUB_CMDLINE_LINUX="novga console=ttyS0,115200 panic=1 numa=off elevator=cfq rd.md.uuid=c0dd5b9a:ff7440ae:3bfc1616:defc0291 rd.lvm.lv=os/root rd.md.uuid=05e8d321:d3191ad9:48a0aba2:efdd0b6c rd.md.uuid=2826bc19:9266a29e:e0989473:987348ad rhgb quiet console=tty0 hugepages=2048"

      Rebuild the initramfs
      # grub2-mkconfig -o /boot/grub2/grub.cfg
      Generating grub configuration file ...
      Found linux image: /boot/vmlinuz-3.10.0-693.21.1.el7.x86_64
      Found initrd image: /boot/initramfs-3.10.0-693.21.1.el7.x86_64.img
      Found linux image: /boot/vmlinuz-0-rescue-fae7c244c3134771a10cc7c3ace3edcb
      Found initrd image: /boot/initramfs-0-rescue-fae7c244c3134771a10cc7c3ace3edcb.img
      done

      Reboot the node for the changes to take affect
      Validate your changes post reboot
      # sysctl -a | grep hugepages
      vm.nr_hugepages = 2048

      You can also reserve hugepages using sysctl.conf
      # echo "vm.nr_hugepages = 2048" >> /etc/sysctl.conf

      Refresh the values
      # sysctl -p

      Reboot the node for the changes to take affect.

      I hope the article was useful.

      Deepak Prasad

      is a techie and an author who is still trying to survive in this IT generation with very little knowledge he has on Linux/Unix, VMware, SAN Storage, Automation, networking etc

      You can follow him on Facebook or Google+


      Do you also have something to share here?
      Join GoLinuxHub Team as an Author, Click here for more information
      How to enable or disable transparent (THP) and explicit (nr_hugepages) hugepage and check the status in Linux with examples (explained in detail) How to enable or disable transparent (THP) and explicit (nr_hugepages) hugepage and check the status in Linux with examples (explained in detail) Reviewed by Deepak Prasad on Sunday, August 05, 2018 Rating: 5

      No comments:

      Powered by Blogger.