We look at the origins and evolution of FIX Simple Binary Encoding, and explore why FIX Protocol SBE standards usage by CME and for Brokertec is a key consideration for success today.
The Evolution of Exchanges
For the past few years, we have seen Exchanges upgrading their technology for the transmission of real time market data. Simple Binary Encoding (SBE) is the latest standard for those Exchanges striving to deliver market data with the lowest latencies, whilst offering their members an open standard based on the FIX Protocol.
Looking back, the FIX Protocol version 4.2 released in 2001 was where FIX started supporting market data messages for real time dissemination of a venue’s order book. It was ASCII based and you could send entire order books or incremental price updates. The data was verbose, and messages could be quite large if a deep order book was transmitted. This was entirely suitable for the demands and requirements of traders at the time.
Around 2007, a few Exchanges were already offering improvements to the FIX Protocol with the adoption of FIX FAST (FIX Adapted for Streaming) compression. FAST introduced the concept of a template published by the Exchange describing how the data was compressed. When compressed, these smaller messages reduced the bandwidth required as well as reduced latencies. With use of the templates these FAST messages could then be uncompressed by the receiving system. During this period CME launched Market Data Protocol 2 (MDP 2), LSE had MITCH and Eurex used FIX FAST for Enhanced Market Data Interface (EMDI).
Additionally, data is no longer sent on a conventional point to point TCP/IP based connection; instead data is broadcast via a network protocol called multicast UDP thus everyone is now sent the data at the same time on a publish/subscribe model. Because the data is broadcast these packets do not involve send/acknowledgement network handshakes and, therefore, are delivered with lower latency. There is a catch though; UDP doesn’t guarantee delivery of every network packet as there can be network loss.
The Exchange platforms further augmented their market data feed implementations to mitigate against, and handle, missing data so in a modern Exchange specification you will normally see the following features:
- Data being sent on dual multicast UDP feeds, commonly known as A and B feeds, which gives you opportunities to receive the same data packet at the same time;
- TCP based recovery feeds, to enable specific missing messages to be resent by the Exchange on a request basis
- Snapshot feeds where you are able to get the state of the entire order book for an instrument in a single message. This aids order book synchronisation with the Exchange.
Around 2015, we saw the introduction of FIX SBE (Simple Binary Encoding), and it was immediately adopted by CME (the pioneer Exchange for this new standard) with the updated CME MDP 3.0 market data feed.
Simple Binary Encoding (SBE) - a New Standard
Networking technology had improved to the extent that message size was less of a limiting factor for message latency. SBE focussed more on the representation of data for decoding and encoding efficiency, and binary is the obvious solution here as it’s the native machine representation. In terms of the CME as a venue, it is more efficient to encode and send prices in binary, and for the trading firm to receive it and decode it into their platforms. SBE still requires A and B feeds, recovery and snapshots since multicast UDP is still the favoured transport; an SBE feed still looks very familiar to anyone who has experience integrating FAST.
Beyond 2015 - Continued Evolution
Of course, standards continue to evolve and change today.
What has definitely changed is the complexity of the development required since the early FIX days. A significant aspect of a trading system’s performance comes down to how fast it can receive and decode messages without any data loss, which if not optimal and deterministic can mean the trading strategy is behind the market. Any missed message recovery needs to take place, which adds latency. The issue is further compounded by Exchanges now publishing entire categories of instrument data whether needed or not. If you are not interested in most of the data, then your system may be expending cycles purposefully ignoring it.
Compression of messages has always been important for market data feeds, but now Exchanges have recognised that order entry messages sent as ASCII via standard FIX Protocol can also be optimised with resultant improvements in bandwidth utilisation and latencies. This is why the CME has adopted SBE encoding for the new CME iLink 3 Binary Order Entry for CME Globex. This has been a significant change as firms now have to seek totally new solutions to this order entry component of their platforms, which has been relatively untouched by Exchanges up to now.
The State of Exchanges Today
As of 2020, a few Exchanges have followed CME and are using SBE for order entry APIs such as Euronext and Moscow. In the MENA region, Oman and Tunisia Exchanges also have incarnations of SBE, because they have licensed the Euronext owned Optiq platform.
In the pipeline, Brazil’s B3 have launched its own SBE programme as part of the exchange platform modernisation project. The B3 specifications are now published, and we wait for the schedule to be confirmed nearer the end of 2020. Formerly known as the BM&FBovespa, B3 has had close relations with CME and have historically shared technology, so it should be no surprise that this may have helped their decision to go in a similar technology direction as CME.
Venues and Messaging Standards
The remaining question now is what does a venue, whether starting up or established, choose as its messaging standard? Outside of regulated Exchanges, we think FIX Protocol will remain the standard with its simplicity of implementation and interoperability for the venue and trading members. For regulated Exchanges, those offering FAST will continue to do so unless there is member demand to change. The choice of SBE is the option for those Exchanges wanting to offer its members the lowest latency platform and be on the forefront of such technology. Some Exchanges will still want to build their own proprietary standards from the ground up in order to suit their specific needs and, of course, others may wish to join the NASDAQ franchise by adopting the Genium platform which offers the ITCH and OUCH standards.
For trading firms, there are choices. There are good quality open-source FIX FAST and SBE Encoder/Decoder implementations of the open FIX Standards available. These components can be used for the specific requirements of encoding/decoding message payloads. But overall this is just one element of the direct market access requirements for both market data feeds and order entry APIs.
Right now, we do see a number of firms looking for an SBE Encoder/Decoder codec specifically to support the Brokertec migration to the CME Globex platform. This may indeed be the only additional component needed by some firms who have correctly analysed the implementation requirements. However the landscape is not that simple.
There are lower level network optimisations using alternative network stacks optimised for multicast UDP and TCP network traffic. For example Solarflare TMultra-low latency ef_vi and TCPDirect ultra-low latency network stack API respectively.
And there are higher level requirements based on the market data feed decoded message payloads regarding A/B channel arbitration, security filtering, order book build management, recovery, replay and back-testing. For order entry considerations, including session security/authentication, gap/retransmission handling, fail over and optimal threading models for the lowest latency/minimal jitter.
OnixS direct market access SDKs include the OnixS high performance SBE encoder/decoder along with the lower and higher-level functions to provide a fully supported set of direct market access solutions. These include:
- OnixS directConnect: CME MDP 3.0 Market Data Handler SDK
- OnixS directConnect: CME MDP Conflated TCP feed Market Data Handler SDK
- OnixS directConnect: CME iLink 3 Binary Order Entry Handler SDK supporting CME Globex MSGW access
Ensure Success Through Every Evolution
All OnixS directConnect: CME Globex Direct Market Access SDK solutions are pre-certified with the Exchange and are production ready, fully maintained and quick to implement the CME Direct Market Access SDK solutions.
Request Your Evaluation Download
To request a free evaluation of CME Globex Direct Market Access SDK solutions, download an evaluation of your choice of direct market access solutions here.
