WHITE PAPER. GemFireTM on Intel for Financial Services (GIFS) Reference Architecture for Trading Systems. Overview

Transcription

1 WHITE PAPER GemFireTM on Intel for Financial Services (GIFS) Reference Architecture for Trading Systems Overview The GIFS solution provides a powerful platform for the development of both high throughput/low latency automated trading systems like direct market access (DMA) & algorithmic trading as well as more traditional managed order flow such as program trading. Increasingly stringent regulatory and financial control requirements are also driving fundamental infrastructure changes. The GIFS solution can eliminate or greatly improve traditional performance bottlenecks such as those incurred by database transactions, high availability (HA), middleware messaging, thread contention, and more. The GemFireTM on Intel for Financial Services (GIFS) solution from GemStone Systems and Intel Corporation represents an ensemble of software, hardware, tools and services meant to address such growing challenges in the financial markets. Please refer to the white paper titled Powering the Next Generation Data Infrastructure: GemFire on Intel for Financial Services (GIFS) for more details about this solution offering.

2 WHITE PAPER Business and Technology Challenges In a world of ever-increasing trade volumes and reduced margins, trading technology must deliver the highest possible performance characteristics while at the same time reducing existing hardware and software infrastructure costs. As investment banks, institutions, and hedge funds try to scale and diversify both business departments and technology systems, associated cost increases must be predictable, incremental, and always able to take advantage of economies of scale. Unfortunately, the trading systems that exist in these firms are often plagued with the following problems: The GIFS solution powered by the GemFire TM Enterprise Data Fabric (EDF) and Intel s Xeon - based hardware platform provides key technical features that enable deployment of high performance trading systems. High Latency Transactions: Persistence models that rely on a relational database and/or underlying disk persistence are at least an order of magnitude slower than memory-based models. Algorithmic, DMA, and program trading systems all depend on ultra low-latency transactional updates to achieve both performance and resiliency targets. Database Contention: Database connections cannot scale cost effectively to service a large number of active concurrent connections. When trading systems are scaled-up by adding CPU horsepower and parallelization, database resource contention quickly causes I/O bottlenecks. High Availability Data Replication: Disk-based vendor solutions are slow (file system replication) or expensive, and custom solutions are difficult to build and maintain. No serious trading operation can forgo these requirements, so this problem generally ends-up taking a large part of budgets (for hardware, vendor software, and development time). Middleware Messaging: This usually introduces three inefficiencies: 1. Object marshalling/un-marshalling (either through serialization or XML parsing) 2. Buffer copies between applications and middleware daemon processes 3. Context switching between application and middleware processes. Scalability: Most of today s systems require extensive hardware and/or software configuration work to scale up to greater volumes. Adding capacity dynamically during the trading day is almost unheard of. Recovery Time: Often overlooked, recovery time for trading systems frequently causes trading firms to lose valuable additional time after a failure. Many firms have not moved key trading applications to an architecture using hot backups. Visibility/Manageability: Building visibility into your order flow for the purposes of customer service, compliance monitoring, and operational risk management introduces additional risks of system slowdown and failure.

3 GIFS for Trading: Solution Overview The GIFS solution powered by the GemFire TM Enterprise Data Fabric (EDF) and Intel s Xeon -based hardware platform provides key technical features that enable deployment of high performance trading systems. GemFire EDF, from GemStone systems, provides a scalable, distributed platform to manage increasing volumes of data and streaming events with almost zero latency. It is best envisioned as a middle-tier, operational data management layer that sits in between information sources and consumers as shown in Figure 1. Applications written in different languages like Java, C#, C++, XML or SQL and deployed on different hardware nodes and platforms can access, store, distribute and analyze data and events represented within the GemFire fabric. With advanced data virtualization, distributed caching and complex event processing capabilities, the GemFire EDF enables the delivery of actionable information to the right application at the right time. GemFire s main memory based distributed caching offers sub-millisecond data read and update capabilities over a LAN. Contention for data update resources is dramatically reduced by co-locating data in an application process and/or distributing data in our highly scalable cache server model. Data replication is an inherent part of GemFire EDF and controlled through simple API s and/or tuning options. Analytics Portals Execution Reporting Query Grid Computing SOA Event-Processing SQL XML Java C++/ C# Event Streams GEMFIRE TM ENTERPRISE DATA FABRIC Disk Figure 1: GemFire TM Enterprise Data Fabric (EDF)

4 WHITE PAPER GemFire on Intel Xeon-based hardware guarantees better performance and reliability than other more expensive hardware/software solutions at a fraction of the cost and complexity. Direct data update propagation and event notification between the trading system components eliminates the need for middleware in most inter process communications scenarios, while GemFire s optimized object serialization and XML parsing greatly reduces the time and CPU resources needed to move data over the wire. Finally, GemFire s tunable and highly concurrent distributed collections remove unnecessary performance-killing contention. GemFire EDF offers a cohesive set of features that leverage the Intel platform and address operational bottlenecks in trading environments. These include: GemFire TM on Intel Xeon - based hardware guarantees better performance and reliability than other more expensive hardware/software solutions at a fraction of the cost and complexity. Distributed Caching High Availability Models Data & Load Partitioning Update Propagation/Guaranteed Event Notification Distributed Event Notification & Complex Continuous Queries Distributed Caching and High Availability Models GemFire EDF s implementation of distributed caching removes all of the performance limitations inherent in database persistence models. While GemFire offers both write-through and write-behind file persistence, it is the ability to transparently guarantee the integrity of synchronous memory copies between instances of a distributed cache that drives some of the biggest performance gains. GemFire promotes an architecture where application data is colocated within the application for instant access, and where updates to the data are transparently mirrored to one or more hot backups. For greater location transparency, GIFS also supports a cache-server model where data is hosted in a HA cluster and the application updates data through a simple driver and API. When developers build new or modify existing trading systems, they can use one of the three techniques to achieve maximum performance and absolute cache consistency (depending on the scenario). Synchronous Memory Copies: This approach works when all of the changes that need to be applied in a logical update are to a single object (for example, a change to an order property or a margin trading credit limit). Operating on a GemFire distributed Map interface, a developer would get() the relevant object, modify it, and then put() it back. Upon completion of the put(), you would be guaranteed that the change was propagated to backup. GemFire Distributed Transactions: In this scenario, one would invoke the GemFire Transaction Manager to group a logical unit of work on several different objects (for example, handling a new execution and updating the parent order). A developer implementing this approach would: Begin() a transaction. Create an execution object and put() it into the cache. Get() the parent order object, modify it, and put() it back into the cache. Commit() the transaction.

5 Upon completion of the commit() method, all changes are guaranteed to be propagated to backup. This technique has slightly more overhead than the first (two network hops instead of one), but all operations are still performed in main memory and consistency is assured. GemFire Distributed Transaction Service can be used simply as an efficient data replication mechanism (for multi-object updates) to a hot backup server, or to assure that different independently operating actors in the trading system topology never corrupt data due to dirty reads or writes. Command Objects: This is a mechanism to propagate multiple cache operations in a single hop while still assuring the consistency of the transaction approach. The trade-off is that some custom logic is required. A good example of this is found in program trading, where one often creates a wave of child orders from a basket of parent orders. A developer would create a command object that, for instance, instructs the system to slice a wave of child orders from a client basket that represents 10% of each order s original quantity. When using this command object to initiate the wave logic, only the command is propagated to the hot backup. Each instance of the system then independently generates and locally caches the resulting wave of orders before the primary instance finally sends it to the market. This propagation is accomplished through GemFire s CacheListener API callback interface. Data & Load Partitioning GemFire EDF provides options for both static and dynamic partitioning of data across multiple instances of a trading system. Data partitioning allows transparent and dynamic load balancing of trade traffic from counterparty gateways to trading servers. Once any given OrderID is routed to a server, all subsequent traffic is also automatically routed to the same server for the fastest possible processing through a sophisticated distributed hashing algorithm. Such a partitioning scheme allows for handling order data volumes beyond the physical memory (RAM) limits of a single machine. Moreover, it also dynamically re-balances the data load across nodes, when new members get added or deleted. GemFire TM EDF provides options for both static and dynamic partitioning of data across multiple instances of a trading system. Static partitioning of data and load is easily accomplished by the simple mapping of symbols (i.e. A-F, etc.) or customer groups to namespaces. Static and dynamic partitioning can also be combined to dynamically scale platinum infrastructure dedicated to your most important customers. Overall, GemFire s data partitioning options allow for much easier scaling and parallelization of trade traffic. Update Propagation/Guaranteed Event Notification Most high volume trading systems today rely on either middleware solutions or proprietary sockets-based technologies. With middleware, there are inherent drawbacks such as memory buffer copies, process context-switching from application to middleware daemons, and often inefficient object serialization methods. When using GemFire s distributed caching framework, API callback notifications produced by the creation or update of any object in the distributed cache takes the place of middleware technologies. Quite simply, if you attach a class that implements GemFire s CacheListener interface to the GemFire Cache, it receives all event notifications that might previously have come through messaging middleware directly from GemFire. The data and associated event distribution technologies of GemFire utilizes Intel s I/O Acceleration technology (I/O AT) and Native I/O options in both Java and C++ to eliminate low-level buffer copies and provide extremely high message throughput. This also guarantees successful

6 WHITE PAPER API callback completion with both memory-copy speed and disk write-behind backup, which is a faster functional equivalent of the ledger guaranteed messaging solution (currently available in the market). Highly optimized object serialization and XML parsing techniques also compliment these messaging advantages. The GemFire TM Enterprise Data Fabric solves most of the difficult technology challenges faced by trading systems in Brokerage Services, Arbitrage Trading Systems, Exchange Matching, Routing, and Execution systems. It is important to note that while file-based ledger benchmark results look impressive, these are usually done without synchronizing data writes to disk, thus leaving a window of vulnerability for message loss in the case of operating system or hardware failure. GemFire s memory-copy technology protects against such failure by copying message delivery statistics to another process on another machine. This model is both faster and more reliable than synchronized file I/O and provides almost instant recovery time when a primary process fails (there is no disk replication lag or need to mount the file system to a backup machine). This is essentially an extension of the advantages GemFire provides over transactional data persistence to an RDBMS. Distributed Event Notification & Complex Continuous Queries GemFire EDF also provides an extremely robust platform for connecting trader GUI s, compliance, clearing, and service monitoring applications. GemFire Real-Time Events (RTE), a component of GemFire EDF, allows easy development of GUI applications to display trade blotters or any desired view of an order flow. RTE is based on a continuous querying (CQ) paradigm that allows a user to register SQL queries that retrieve an initial result set from a data store much like you might expect from a relational database. But, it then maintains the results set automatically as and when new events occur that match your query where clause or older data drops out. Push-based API callback notifications remove any need for polling and allow you to build trade blotter views by simply binding a GUI component to a dynamically changing query result set. All these capabilities are also designed to have a negligible impact on trading system performance whilst guaranteeing that slow or dead consumers never cause publisher blockage or memory overflows. Traders simply looking for activity from a given client or more complex patterns such as monitoring for limit orders too far from live market data prices are equally easy to implement. Compliance can continuously query for orders that violated regulatory or client-driven rules, while the back-office can selectively listen for trades, and the trading service desk can monitor for exceptions such as latency spikes or unacknowledged orders. Such components can be developed for object-oriented applications as well by simply attaching applications to a GemFire EDF-based distributed cache and requesting key or namespace based subscriptions. Event notification policies allow users to balance performance impact, delivery guarantees, and other factors according to business and application requirements. Who the Solution Will Benefit The GemFire Enterprise Data Fabric solves most of the difficult technology challenges faced by trading systems in Brokerage Services, Arbitrage Trading Systems, Exchange Matching, Routing, and Execution systems. Any system where latency needs to be reduced, throughput multiplied, availability increased, cross-asset visibility enabled, and risk controls improved and/or introduced are excellent candidates for GemFire EDF. Enterprises like Bear Stearns have adopted the GemFire EDF on Intel Xeon platform as a data backbone for multiple applications like equity

7 trading, bond pricing portals, and clearing & settlement. Firms that adopt measures to improve these areas gain a competitive advantage in the marketplace by improving their overall service and the diversity of products for customers, reducing operational risk, and achieving a lower, more predictable cost for scaling their business. Those that fail to implement such improvements will either be marginalized or forced to spend much more to meet the objectives as they watch business opportunities bleed to competitors. Summary The GIFS solution provides a truly transformative approach to solving the challenges inherent in building high-throughput, ultra low-latency, resilient, and manageable electronic trading systems. GemFire EDF s caching, distribution, replication, serialization, and complex event processing capabilities coupled with Intel Xeon platform s Virtualization and I/O acceleration provides the best possible technology infrastructure platform for trading systems available on the market today. Distributed Cache Event Notification, reducing costs and increasing efficiency. Orders from Investors Execution Results Sent back to Clients Data may be co-located with processing or hosted in cache servers Order Routing Order Routing Program Trading Platform (Basket Trading) ECN Orders Sent to Exchanges Ack s, Excutions, & Market Data The GIFS solution provides a truly transformative approach to solving the challenges inherent in building high-throughput, ultra low-latency, resilient, and manageable electronic trading systems. Execution/order data hierarchical representation in cache synchronous asynchronous both Mirrored GemFire Caches (n-way Replication) GEMFIRE Replication Models: Single Object Multi-update transactions Command object design pattern GemFire Persistence RDBMS Figure 2: GemFire TM in a Trading Workflow

8 Information Links and Resources T H E E N T E R P R I S E D A T A F A B R I C T H E E N T E R P R I S E D A T A F A B R I C GemStone Corporate Information Corporate Headquarters: 1260 NW Waterhouse Ave., Suite 200 Beaverton, OR Phone: Fax: info@gemstone.com Regional Sales Offices: New York 90 Park Avenue 17th Floor New York, NY Phone: Washington D.C. 3 Bethesda Metro Center Suite 778 Bethesda, MD Phone: Santa Clara 2880 Lakeside Drive Suite 331 Santa Clara, CA Phone: Copyright 2006 by GemStone Systems, Inc. All rights reserved. GemStone, GemFire, and the GemStone logo are trademarks or registered trademarks of GemStone Systems, Inc. Information in this document is subject to change without notice. Copyright 2006 Intel Corporation. All rights reserved. Intel, the Intel logo, and Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. *Other names and brands may be claimed as the property of others US