The OnixS directConnect: Market Data Handler SDKs provide Direct Market Access solutions for a number of venues that use multicast market data feeds. Debugging issues with high multicast network packet loss rates can be complicated: What reasons are there for high multicast network packet loss rates? What tweaks and adjustments can we perform to minimize such losses? Read more to explore diagnostics, tuning techniques, useful tools and reference materials to minimize such losses.
Why Do Lost Multicast Packets Occur, and How Can We Minimize Losses?
There are several reasons for multicast packet loss. For example:
- The UDP protocol itself trades reliability of performance and does not guarantee the datagrams delivery. Therefore, the packet could be lost during the network transmission.
- Even if the packet reaches the network node, it does not always mean that the application receives it because during processing the received packet goes through several levels, on each level there could be a loss.
Understanding the typical path of a network packet can help to identify and address key areas where loss may occur. The typical path of a network packet is shown on Figure 1.
Figure 1
In this image, first, the network adapter (NIC) receives and processes the network packet. The NIC has its own hardware ring buffer. When the network data flow is higher than the NIC can process, the newest coming data will overwrite the oldest ones. The possibility of this happening depends on the NIC characteristics such as computing performance and hardware buffer size.
Next, after processing by the NIC it comes to the operating system buffer, which can also overflow. All the packets from all the NICs for all the applications and auxiliary packets go through this buffer.
Therefore, the possibility of the loss at the operating system level depends on:
- the size of the operating system buffer
- the general system performance
- the general system load
- the network-related system load
It also depends on the number of NICs even if an application does not use the NIC: it creates some additional load by auxiliary protocols like ARP or ICMPv6.
Then, it comes to the socket buffer from which the application takes the packet. If the application is unable to take the packet from the socket in time, the buffer will overflow and the packet will be lost. Therefore, the possibility of the loss on the application level depends on:
- the socket buffer size
- how fast the application takes the data.
For a technical walk through of troubleshooting diagnostic techniques and tuning across different operating system environments, see this OnixS technical guide reference.
