Thursday, February 21, 2013

Overflow in datagram type sockets

When using Datagram type sockets, the packets may not be sent when the sending buffer is full. If the sending buffer is full, the send function returns error EAGAIN (usually with an integer value of 11 in Linux). The EAGAIN error message prints "Resource temporarily unavailable".

If packet loss happens, we can use the system function setsockopt() with SO_RCVBUF to increase the receiving buffer. Similarly, if EAGAIN error happens, we can use setsockopt() with SO_SNDBUF to increase the sending buffer.

According to the manpage of socket(7), when you use setsockopt() to set the buffer size, the kernel doubles the value you specify. And getsockopt() will return the doubled value. For example, if you do:
        int new_size = 10240;
        setsockopt(sock, SOL_SOCKET, SO_RCVBUF, &new_size, sizeof(new_size));

getsockopt() will return 20480 as the new buffer size.
These function calls cannot increase the buffer size to as much larger as you want. The maximum size it can reach is restricted by the system parameters net.core.rmem_max and net.core.wmem_max -- for the maximum of receiving and sending respectively. You can check the current value of these system parameters by bringing up a console and issuing these commands:
        cat /proc/sys/net/core/rmem_max
        cat /proc/sys/net/core/wmem_max

To increase them, you need to have the admin privilege to run the sysctl command. e.g.
        sysctl -w net.core.rmem_max=256000 
        sysctl -w net.core.wmem_max=256000  

For datagram-oriented Unix Domain Socket, setting the SO_SNDBUF socket option has an effect, but the SO_RCVBUF option does not (manpage unix(7)). And even the buffer is big enough, we can still experience the packet overflow problem. That is because there is another limitation in the datagram type Unix Domain Socket. The backlog of the packets in the buffer is not unlimited. For example, the default value of the maximum backlog in most Linux systems is set to 10, i.e. only 10 packets can wait in the queue. The commands to check and increase the backlog limit are:
        cat /proc/sys/net/unix/max_dgram_qlen
        sysctl -w net.unix.max_dgram_qlen=128   

And certainly, you need to be an administrative user to change it.

These parameters should be well planned to avoid the waste of the memory space. And since these are system parameters, changing them will affect any processes and programs that use sockets, not just your program.

No comments:

Get This <