HIGH VELOCITY VOLUME COMPLEXITY. Addressing the challenges of Big Data in FX

Transcription

1 HIGH VELOCITY VOLUME COMPLEXITY Addressing the challenges of Big Data in FX The big data challenges that face traders in foreign exchange are the sheer volume of trading, amount of data and number of execution venues that are available. Following the FX fixing scandal, many large sell-side firms have found their clients turning to automated trading systems while senior traders were suspended during the various investigations. Automated trading There are enormous advantages available for FX trading firms that are able to capture and analyse the universe of available data, but, as Dan Barnes reports, getting to grips with the massive wealth of information that s out there is a considerable technical challenge. drove up the demand for data that could be used to support price formation and transaction cost analysis to support trade decision making. Analyst firm Tabb Group estimates US$15.1 billion was spent on market data and market data infrastructure globally in 2013, with US$11.4 billion of that spent on data that had been aggregated in a feed or on a screen. Peter Durkan, chief technology officer at financial infrastructure provider Lucera, says, The market has certainly accelerated its use of technology and the onus is now on firms to keep up with this. The rise of application programming interface (API) driven trading and the increasing number of liquidity venues has made capturing, managing and analysing market data a difficult problem. Firms need access to a large number of liquidity sources to ensure best price and deep liquidity but this has the downside of making effective transaction cost analysis (TCA) that much more difficult as it results in huge daily market data volumes that need to be stored, analysed and archived in order to provide clients with intelligent and effective TCA. Moreover as a trading firm it s important that internal systems can access this data for a variety of reason - risk, compliance to name a few. This type of analysis requires an investment in hardware, software and the analytical skill to interpret all of this data. In some cases it s a big investment, but in our opinion this is fast becoming a requirement to run a successful FX trading operation. Analysis by FIX Global Trading magazine in April 2014 found that capital markets firms considered big data technology to be either essential (50%) or important (50%). The data that is required by FX market participants, whether they are the price-making algorithms at banks or trading desks at buy-side firms, is vast and varied making its collection and analysis quite a challenge. MARKET FRAGMENTATION James Sinclair, CEO of Marketfactory, which provides market participants with a single point of access to multiple trading destinations, says that market fragmentation in FX has challenged the existing price formation process fuelling a demand for improved sourcing of and quality of data. In the US equity market a consolidated tape of price data supports price formation, a model which has not yet been implemented in European equities and for which there is no direct equivalent in FX. My definition of big data is More data than you are used to, he says. Data in FX has ballooned over TECHNOLOGY My definition of big data is More data than you are used to, James Sinclair the last few years, while in equities for example, it has not. In equities there was this explosion of data 10 years ago but trading volumes have actually decreased since. There are not that many more exchanges in equities; in the US there are 12 exchanges and 42 ECNs. In FX the key distinction is that five or six years ago you could price your engines off of two platforms - Reuters and EBS. Today that is no longer true because, relative to the market as a whole, they have lost market share. Joel Steinmetz, managing director, at FX trading automation specialist Fluent Trade Technologies says that this fragmentation creates the greatest technical challenge, with data coming from different places. It s not like you can go to an exchange or three exchanges and you have all the data, he says. We are going to 40 banks and 45 trading venues, it is all different kinds of data coming in and each one of those places deals with data differently and has a different protocol. Somehow from the customer s perspective that has to be gathered, normalised and distributed so that its readable as Source: Global Trading 118 january 2015 e-forex january 2015 e-forex 119

2 Source: Global Trading a single unit rather than as loads of different feeds. The second challenge is that you have orders and executions to account for, so you have to include the way the markets are moving orders from 80 markets. The third challenge is that you want to incorporate a lot of different factors in to the book that you are building. LAST LOOK Durkan says the complexity is compounded in some cases by the prevalence of last look in the FX market, which allows a dealer the option not to execute an order at Joel Steinmetz an otherwise agreed and published price. As an electronic market maker in FX last look can make certain aspects of the market microstructure difficult to model resulting in difficulties in building efficient liquidity and inventory management strategies, he says. In order to get the exact picture of the market required, the data may have to be carefully sliced and diced. For example, a firm may take data from 20 different feeds but want to exclude counterparties that had the last look, or to rank counterparties set against average response time for all participants, to determine how likely they are to trade if they are top of book compared with a counterparty at the next level below. Taking all of that information and gathering those historical trends so that they can understand every single feed and incorporating that in to the book building that you are doing is a challenge enough in itself, says Steinmetz. Incorporating all of those variations comes from the different players that are delivering that data to you. From the customer s perspective [data] has to be gathered, normalised and distributed so that its readable as a single unit rather than as loads of different feeds. The scale of the data means efficiency becomes increasingly important across the processes of gathering that data, storing it, formalising it, and manipulating it in a way that makes it effective for a firm s specific usage. Steinmetz adds, What s particularly interesting is those efficiencies that are delivered to the market place or have been delivered to the market place have been related specifically to execution. The way we have looked at it is that we need to leverage those capabilities - the low latency capabilities, the design of unification, all of the pieces of the puzzle - into areas other than execution. To that extent we focus on risk management and big data. COPING MECHANISMS Where many market participants come unravelled is in their ability to provide the systems that can store and retrieve large data sets with the necessary flexibility says Bryan Hunter, head of foreign exchange at Interactive Data. Traders need data on both a raw basis and on a normalised basis, and when you look across at the number of updates per second it creates more of a data management issue than for any other asset class except perhaps equity options he says. Other than for the biggest fund managers, it becomes a data 120 january 2015 e-forex

3 Traders need data on both a raw basis and on a normalised basis, and when you look across at the number of updates per second it creates more of a data management issue than for any other asset class except perhaps equity options, Bryan Hunter management issue that traders are unable to manage internally because of the sheer cost. Not only do they need to store the data, they need to figure out how to have real-time access to it. Reuters and EBS until a few years ago only showed top of book and the amount a trader could buy up or sell down to get a given amount, Sinclair observes. Furthermore data was refreshed every 250 milliseconds or every 500 milliseconds. Today the demands on data infrastructure are considerably greater. When you look at a Currenex, a Hotspot or an FXAll you have an ITCH data feed which gives you a hosepipe, he says. It is tick-bytick and it is full depth of book, plus there is CME which is less voluminous but has higher peaks. Volume has increased by an order of magnitude, says Sinclair. Rob Lane, general manager for consolidated feeds & 7ticks EMEA at Interactive Data says that the firm has seen demand from clients to get more access to trading venues and counterparties - whether that be the exchanges or the banks themselves which is supported through 7Ticks, a global managed services offering from Interactive Data Real- Time Services for the electronic trading community. months, he adds. In December last year we were around 19 million updates just on the FX spot rate on our consolidated feed and today we are at 55 million. So we are seeing huge volumes that we have to manage and then downstream clients are going to have to manage that themselves through proprietary applications with tick data history or through a third party. At the base level for technology, trading infrastructure is needed to move data and bring it in to a data centre, but the more fluid part, the data itself, creates real challenges for the trader and the processor if the data that is feeding through is altered for example by changing a feed which in turn affects the calculations that rely upon the data. Steinmetz says: If I were a hedge fund, perhaps I switched prime brokers and now they are allowing me to interact with a different group of feeds; perhaps one feed has shown that they are too active on their book or they are not responding quickly enough. Do I want to get a full book from that feed? Maybe I switch that around tomorrow. The changing demands of a firm is one of the biggest challenges because each day creates a new historical reality that might impact the overall trend and how you want to interact with the market. What is important for the trader in any one of those circumstances is to ensure that flexibility exists in the technology was there to effectively implement whatever changes were required on the fly. Peter Durkan We have also seen higher volumes on our normalised feed, which has been considerable over the last 12 Having the technology and skills to analyse and make sense of social media trends or real-time news feeds is now a real advantage. WORKING THE DATA Putting the data to use within a trading operation requires a rethink of traditional systems. For structured data, relational databases provide a solid way of storing, retrieving and analysing data, assuming it has been stored in such a way that the type of query is predictable. However it 122 january 2015 e-forex

4 Putting the data to use within a trading operation requires a rethink of traditional systems. is not the fastest or most flexible model for storage and retrieval. Gil Tene, CTO of Azul Systems, a specialist in boosting the Gil Tene performance of Java-based systems to cope with big data and low latency challenges, says The main thing we are seeing as a trend, although it s not universal, is a lot more in-memory processing of data in latency sensitive paths, as opposed to the database approach to processing that are generally more batch-based. We are seeing that for two purposes. The first is simply speed - people get more crunching done and it gets done faster. They can run more models in parallel. The second, and more exciting one, is where you see the large data computation coming closer and closer to affecting real-time trade decisions. This is something that we are seeing across in-memory data is seriously affecting behaviour. The more sophisticated models are often based on substantially larger amounts of up-to-date data being available right in the latency critical paths. asset classes. We are seeing trading algos evolving in complexity, and playing with a lot more space as large amounts of in-memory data is seriously affecting behaviour. Simple knee-jerk reaction algos controlled by only a a few parameters are being overtaken by algos that have Gigabytes of up-to-second information at their fingertips. The more sophisticated models are often based on substantially larger amounts of up-to-date data being available right in the latency critical paths. Transaction cost analysis (TCA) is being used by more sophisticated buy-side firms to provide automated pre-trade decision making in order to deliver trading efficiency and potentially greater alpha. However Rosario Ingargiola President & CEO at FXone, a provider of execution services and connectivity to traders, says that delivering the right decision can be challenged if the data used as input is not up to scratch. 124 january 2015 e-forex

5 While the use of TCA for FX trading is growing rapidly, quality of data can be an issue as can choice of a relevant benchmark, he says. For the maximum benefit a TCA feedback loop needs to be incorporated into the trade management and even alpha generating processes. Ideally this feedback loop is automated and execution algos themselves can be selected dynamically based on TCA and execution objectives for maximum efficiency. There are many considerations including bank neutrality vs. bank provided algos and TCA and choice of execution algos themselves as well as granularity and frequency of the TCA metrics. Using common strategies that are easy to understand like VWAP all of the time is not going to provide the best execution result. The markets are dynamic and the algos need to be also. TCA is the real-time report card that smart algos can leverage. Buy-side firms alternative investment managers and highfrequency trading proprietary shops are becoming independent from the traditional sell-side dealers in their Rosario Ingargiola technology requirements, say IDC s Lane. They access multiple prices in the market and, while there is some variation by execution strategy and time arrangement he says. They also need to be co-located globally with the major liquidity venues, a service that we provide to a number of them. So they need low-latency connections, they want primary access to all of the pricing and all of the tick data; they have access to all of many venues that they see as valuable.. Our services to them often include managed co-location, low-latency venue connectivity, and interconnection between data centres, very much like what we are offering to the sell-side institutions.. As firms explore the potential that existing market data offers them, they are also beginning to reach out into alternative sources of While the use of TCA for FX trading is growing rapidly, quality of data can be an issue as can choice of a relevant benchmark, We are seeing increasing data volumes coming from the FX Market, which we help clients to manage and connect to third party and/or proprietary applications. Rob Lane information including unstructured information such as text which allows news sources to feed into the trading and execution process. Durkan says, As in other asset classes sources of information are not just limited to exchanges and ECN data. More and more we are seeing new filtering and social media playing a large part in driving algorithms. Having the technology and skills to analyse and make sense of social media trends or real-time news feeds is now a real advantage. LOOKING AHEAD In the future there are also some interesting sources of data that might become available through regulatory or commercial means, says, Ingarglio. For example, at some point in the future there could be regulatory change that requires mandatory reporting of all trades to single consolidated Volume repository that could potentially be opened up for access by all market participants. Something like this would be very interesting from a TCA and even alpha generation point of view. Durkan concurs, noting that there is always data available that is overlooked by a segment of the market. Having access to this data and recognising its value is the difficult thing, he says. Once you start acting on it and trading due to signals generated by this data it will not be long until someone else also sees the value in the same data and the efficiencies of the market will make it such that the advantage will no longer exist. What that data is? Well, it would be silly of us to share if we knew! 126 january 2015 e-forex