• VMware

    Learn about VMware virtualization for its products like vsphere ESX and ESXi, vCenter Server, VMware View, VMware P2V and many more

  • Linux

    Step by step configuration tutorials for many of the Linux services like DNS, DHCP, FTP, Samba4 etc including many tips and tricks in Red Hat Linux.

  • Database

    Learn installation and configuration of databases like Oracle, My SQL, Postgresql, etc including many other related tutorials in Linux.

  • How to view and increase the default pipe buffer size in Linux (F_GETPIPE_SZ, F_SETPIPE_SZ)

     A pipe has a limited capacity.  If the pipe is full, then a write(2) will block or fail. Applications should not rely on a particular capacity: an application should be designed so that a reading process consumes data as soon as it is available, so that a writing process does not remain blocked.

    In Linux versions before 2.6.11, the capacity of a pipe was the same as the system page size (e.g., 4096 bytes on i386). Since Linux 2.6.11, the pipe capacity is 16 pages (i.e., 65,536 bytes in a system with a page size of 4096 bytes). Since Linux 2.6.35, the default pipe capacity is 16 pages, but the capacity can be queried and set using the fcntl(2) F_GETPIPE_SZ and F_SETPIPE_SZ operations.

           F_SETPIPE_SZ (int; since Linux 2.6.35)
                  Change the capacity of the pipe referred to by fd to be at least arg bytes.  An unprivileged process can adjust the pipe capacity to any value between the
                  system page size and the limit defined in /proc/sys/fs/pipe-max-size (see proc(5)).  Attempts to set the pipe capacity below the page  size  are  silently
                  rounded  up  to the page size.  Attempts by an unprivileged process to set the pipe capacity above the limit in /proc/sys/fs/pipe-max-size yield the error
                  EPERM; a privileged process (CAP_SYS_RESOURCE) can override the limit.

                  When allocating the buffer for the pipe, the kernel may use a capacity larger than arg, if that is convenient for the  implementation.   (In  the  current
                  implementation,  the  allocation is the next higher power-of-two page-size multiple of the requested size.)  The actual capacity (in bytes) that is set is
                  returned as the function result.

                  Attempting to set the pipe capacity smaller than the amount of buffer space currently used to store data produces the error EBUSY.

           F_GETPIPE_SZ (void; since Linux 2.6.35)
                  Return (as the function result) the capacity of the pipe referred to by fd.

    To validate this let us create a fifo pipe file
    # mkfifo /tmp/testfifo

    Next below python script (/tmp/change_fifo_size.py) can be used to get the default size and also to increase the pipe buffer size to any new value

    #!/usr/bin/env python

    import signal
    import os
    import fcntl
    import sys

    F_SETPIPE_SZ = 1031  # Linux 2.6.35+
    F_GETPIPE_SZ = 1032  # Linux 2.6.35+

    def open_fifo(fifo):
            print "Checking fifo file ..."
            fifo_fd = open(fifo, "rb+")
            print "Pipe size            : "+str(fcntl.fcntl(fifo_fd, F_GETPIPE_SZ))
            fcntl.fcntl(fifo_fd, F_SETPIPE_SZ, 1000000)
            print "Pipe (modified) size : "+str(fcntl.fcntl(fifo_fd, F_GETPIPE_SZ))
            return fifo_fd
        except Exception, e:
            print "Unable to create fifo, error: "+str(e)

    fifo_fd = open_fifo("/tmp/testfifo")

    Here you can replace the highlighted sections as per your requirement i.e. the location and size of the pipe file

    Execute the above script
    # /tmp/change_fifo_size.py
    Checking fifo file ....
    Pipe size            : 65536
    Pipe (modified) size : 1048576

    So the default fifo file size was 64KB which is increased to 1MB.

    I hope the article was useful.

    How to view and increase the default pipe buffer size in Linux (F_GETPIPE_SZ, F_SETPIPE_SZ) How to view and increase the default pipe buffer size in Linux (F_GETPIPE_SZ, F_SETPIPE_SZ) Reviewed by admin on Sunday, May 06, 2018 Rating: 5


    1. question on the change in FIFO file size from 65K to 1MB....is this persistent change or only within the Python process. in testing this, after making the pipe file...running the script twice shows the same output twice. shows before 65KB and after 1MB. this leads me to believe the change is not persistent

      1. You have to create a daemon with these changes so that the pipe is always open, if the daemon stops the changes will go back to default


    Powered by Blogger.