Hardwired: FX trading in a chip When every microsecond counts cutting code into specialised hardware offers real time saving, but has standard chip evolution caught up? Dan Barnes weighs up the benefits of hardware-accelerated appliances for electronic FX trading applications. If you stub your toe against a wall, you will feel the impact of brickwork one or two seconds before you feel the pain. The reason for this is that there are different nerves to conduct the sensations of touch and pain, each having a different structure. A trader seeing the yuan plummet against the dollar may equally feel pain several seconds after he has registered the changing digits on a screen. To avoid this dislocation of input and reaction time, many traders prefer to employ digital nervous systems that will react almost instantly to such market shifts, whether to generate alpha or to avoid losses. Choosing which components to use when building this circuitry can make a crucial impact on how fast it is. In a technology stack a firm will have both hardware and software that is used to pass information about. The more components there are, the more time is taken to move information from one part to another, albeit in fractions of a second. Those fractions will count, nevertheless, when trying to beat other traders in the market. At the very core of high-speed trading is automated decision-making, which requires a program that takes in data and triggers an appropriate response. Mark Skalabrin, CEO at Redline Trading Solutions, a provider of TECHNOLOGY ultra-low latency market data and order execution systems, says, In terms of hardware acceleration, you have to look at what you are trying to achieve. In FX trading it is to reduce latency on the information from a counterparty and the market, then make a decision and get that back to the liquidity provider and have it executed. We design to reduce latency while improving determinism so we can make the response time very consistent. A consequence of fast and consistent latency is an improvement in the aggregation and accuracy of pricing data feeding your trading strategies. FPGA S Typically a program is software and the processor - central processing unit (CPU) or a graphics processing unit (GPU), is hardware. The software contains instructions on what to do based on incoming data, and those are held in one chip. They have to be grabbed and translated into commands by the processor, which then distributes those commands across the CPU s transistors, binary switches set to 1 or 0. The setting of these switches executes the instructions and then the outcome of those instructions is written into memory. There are alternatives to that model. The field-programmable gate array (FPGA) is a chip that can have instructions 138 october 2015 e-forex october 2015 e-forex 139
written into the hardware itself, removing the need for the movement of instructions in and out of the processor. It does this by having the logic written in a program that is translated into a binary file that is downloaded in the chip. For performance and latency you are never going to beat the FPGA, says Craig Davis, senior product marketing manager at FPGA supplier Altera. Any other standard processing unit such as a GPU, will never be as close to the data as an FPGA. You have to take data from the accelerator card, to express bus to your processor then you have to do the return journey as well with your data. That is usually done with a block copy and takes a while each time. The use of software, such as an operating system (OS) Mark Skalabrin can also impede speed. An OS has to effectively keep an eye on everything, from how to paint a pixel on the screen to any other activity on other programs, and so sometimes for no particular reason while a trading algo is being run the CPU could disappear for 10 microseconds. Careful BIOS, firmware and OS tuning for low latency applications is required to minimize this effect. However the advantages conferred by FPGAs are challenged by Redline, which in its paper Thinking of FPGAs for Trading? Think Again!, issued in February 2015, says that developments in other technologies can offer considerable advantages. Skalabrin says that there is a wide range of application performance optimization expertise, from firms writing software that is logically correct but ignorant of the underlying hardware to the other end of the spectrum where firms are writing applications for FPGAs using a hardware programming language. Redline asserts using industry-standard chips can be highly effective when programmed optimally. What we have said is writing applications to the capabilities A consequence of fast and consistent latency is an improvement in the aggregation and accuracy of pricing data feeding your trading strategies. of mainstream hardware can give you the advantages of reducing latency and very much improved determinism without all the disadvantages of having to write applications into an FPGA, Skalabrin says. LATENT NOISE The application of low-latency technology is valuable in several areas of FX trading. Marcus Perrett, director of Technology & Development at Fixnetix provider of multi-functional lowlatency technology and services, says that one of the firm s first projects was in FX, addressing two aspects of concern. The first of these was building a consolidated market-data feed handler. Because FX markets have a distributed network and they run 24-hours a day, then traders want a market data appliance that sits in the corner, and doesn t need too much interaction as regards software that might need to be monitored, he says. Consequently we could see that FPGAs have value running in the background from a maintenance and liability point of view. The handler took sources of data from various venues and markets around the world, and combined them together into a single consolidated feed. By using raw data the firm could manipulate how the consolidated feed appeared, which provided the specific Fixnetix - for global FX infrastructure providing bespoke, low latency, fully managed, co-location, liquidity venue connectivity, networking, hardware, monitoring and 24x7x365 support. Fixnetix provides you with performance, speed and agility to compete in today s electronic trading world. www.fixnetix.com sales@fixnetix.com LONDON - NEW YORK - BOSTON - CHICAGO - TOKYO 140 october 2015 e-forex
value of the market data feed. One of the benchmarks of the systems was not simply whether it worked, but would it work over a big figure such as non-farm payroll? says Perrett. What does the latency look like during that period? Can it maintain three 9s reliability and its latency profile over nonfarm payroll? The key is always about testing a technology s limits. In any marketplace where an event causes volatility - for equities that could be an initial public offering, for FX it could be a macro number like nonfarm payroll - that s when you need your technology to work and if that s the time when the technology lets you down historically, it is the software that lets you down. That is where you will see the adoption A Taxonomy of the Total Latency Budget in an Electronic Trading Platform of technology like FPGAs. Redline s market data software not only includes feed handlers that normalize the data but also builds a consolidated order level view across multiple venues. Lee Fisher, vice president of marketing at Redline Trading Solutions says, We distribute market data to the trading applications that our customers are writing and then we have an execution gateway which processes their orders. It runs pre-trade risk checks,, manages the orders, and sends them to the selected liquidity provider. Crucially it does all of that in the accelerated software it is running on general purpose Intel processors, speeding timeto-market. Some people are trying to do each one of those steps with FPGAs the market data handling, order execution and the pre-trade risk. We feel very strongly that there really isn t much benefit doing that, particularly in Forex where data rates are relatively low. When you look at the whole end-toend application performance ( tick-to-trade as shown in the taxonomy diagram) there is not a material gain for using FPGA over the highly accelerated software versions that we develop and there is just a lot of downside associated with it, explains Fisher. THE USE CASE WITH FX AND HFT Within foreign exchange trading, the capacity to use low-latency trading strategies is less than in some other asset classes such as equities and futures. High-frequency trading (HFT) is optimal in a very stable infrastructure where the timing of data flows and order placement is very predictable. With exchange-traded products which trade at high volume and use a centralised infrastructure such trading patterns tend to work well. FX does not tend to have the same centralized infrastructure because of the over-the-counter (OTC) trading model. In our opinion one of the drivers to use new technology is regulation, says Perrett. The current round of regulation, not just in equities as in 2008 but all asset classes including FX, is forcing clients towards the adoption and implementation of compliance and credit checks to meet internally and externally imposed rules. If performance and determinism above all else is the focus then FPGAs are the only viable solution. Regulation has changed that. By imposing a series of checks on traders of FX derivatives a clash occurs; the need for speed contrasts with the implementation of checks thereby slowing the process. The adoption of a [low latency] pre-trade risk gateway can be based on a use case of the broker making make more money if they speed the process up. TECHNOLOGY the more regulation that comes in to the FX space the more you are going to see FPGAs and GPUs and other acceleration technology start to crop up because then you have actually got a problem to solve. That s generally a harder sell internally; if you build a solution to address regulation everybody is going to take it in your marketplace, Perrett says. If you build a bespoke solution for one set of clients that think they can make money by accelerating and then it doesn t work out, then you have both wasted time. So I think regulation is a big part and I think the more regulation that comes in to the FX space the more you are going to see FPGAs and GPUs and other acceleration technology start to crop up because then you have actually got a problem to solve. An FPGA is in the same place as the data, so it could be in the place of an accelerator card, and it could accelerate algorithmic trading, for example, and send the order straight back out without really having to touch the processor. Another aspect to the sell side use of FPGAs is in aggregation of data where some value adding is needed for clients. Perrett says, One proposal we saw was to normalise all of Marcus Perrett Any other standard processing unit such as a GPU, will never be as close to the data as an FPGA. Craig Davis 142 october 2015 e-forex october 2015 e-forex 143
Another aspect to the sell side use of FPGAs is in aggregation of data the anticipated performance advantage may evaporate if the trading application is slowed down by unnecessary processing. If a more featurerich ticker plant can minimise the workload on the trading application by delivering only the market data of interest, in the form that is most efficient for that application to consume, then the trading application will run faster and overall tickto-trade performance may be improved over a barebones feed handler. the different clients, including historical clients that would be trading on bespoke application programming interfaces (APIs) or market specific APIs. This institution was going to push at some point to get clients and it had up to 20 APIs to manage. The idea was to have the FPGA normalise all those on the fly into a single protocol which was then used internally within that bank. TOO RIGID Skalabrin says that there is definitely a push in FX to get to the next level of performance, as systems are moving from milliseconds and hundreds of microseconds down into the single digit micro second range It makes sense that there are gains in terms of getting more consistent, lower latency performance and we see that happening, he says. But what we see is you can get that performance out of standard hardware with the right kind of architecture and that you don t need to endure the costs of FPGAs. One of the challenges identified by Redline in the use of FPGAs is the matching of a trading application to the digestion of incoming market data. In order to make a trading decision the order decision latency needs to be incorporated in the platform selection analysis. In its paper, Redline wrote, While some FPGA-based solutions may be able to flood a trading application with raw updates at impressive speed, COMPLEXITY A further challenge in the use of FPGAs, also identified by Redline in its paper, is the complexity or specialised skills needed when the chips have to be reprogrammed. Lee Fisher When you look at the whole of the application performance end-to-end there is not a material gain for using FPGA over the highly accelerated software versions that we run 144 october 2015 e-forex
There are certain situations where standard chips with optimised software can be advantageous and others where specialised hardware has an edge, argues Perrett. The problem with FPGA is that it takes a long time to get it to talk to a complicated market protocol. Some of the European markets have incredibly complicated protocol specs, similarly to arrowhead in Tokyo. It is actually easier and quicker and better to maintain that kind of operation with software. In circumstances where tuning a system to within 10 to 15 micro seconds and to a reliability of 99% is acceptable, using an FPGA can add work beyond the necessary return. Non-FPGA solutions really come into their own because if you don t want to have the ultimate fastest solution but you want to drastically reduce your time to market, then those solutions fit really really nicely, says Perrett. I am not so hard-nosed as to say that they are not good for those purposes, because they fill a niche. If you need to be absolute fastest you would always have to do it in hardware. If I am writing algorithm and I am writing it in Java well probably I should do that in CPU because that s where it s going to run best. However hardcoding FPGAs is not the only way to use them for advantage adds Davis. If you want to make sure things are easy to do, you wouldn t have to write that from scratch, you could buy an IP box in. That tends to be a Lego block of logic that can be incorporated into a system, he concludes.