Moving at the speed of Markets Gyan Aggarwal Sr. Software Architect #cassandra12

Transcription

1 Moving at the speed of Markets Gyan Aggarwal Sr. Software Architect #cassandra12 TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM

2 Overview Commodity Trading Platform (based on Java Technology) Ø Valuation Engine Ø End of Day Valuation Process Ø Position Manager TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 2

3 Relationship of Core Modules Market Data Trades Ref Data Can change rapidly Delivery Events Pricing Events New Trade Edit Trade Mostly static Valuation Engine End of Day Process It is a distributed batch process that runs every day, it fetches all the open trades and sends them to valuation engine Risk Data (position, p/l etc.) Risk data contains - A set of attribute values and - A set of results values Position Manager Fetches risk data using a combination of attribute values (ad-hoc query) TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 3

4 Market Data Market data can change very rapidly during the day depending upon the volatility of the market. Market data includes commodity price, fx rate, interest rate etc. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 4

5 Risk Data Consists of trade attributes, valuation attributes and valuation results Normally, attributes are used to make queries to view a set of valuation results (can also be aggregated) Consistency and accuracy are absolute necessity Valuation results depend upon market data, trade data, state of the trade, processing date etc. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 5

6 Risk Data contd. Market price, trade price of a trade may depend upon market data. They can have a very simple dependence (single market data access) or can have a very complex dependence (multiple market data dependence and involve some computation) There are some valuation results that can be derived easily. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 6

7 Valuation Engine Works in 2 mode intraday and end of day A trade has a life cycle and it enters into different states with time, market activity, trade change events, delivery events etc. Valuation engine in intraday mode keeps the risk data current with any change in market activity or trade. Trade change automatically invokes this engine but market data change requires manual invocation from users. Valuation logic for a trade depends upon the type of trade, its state and it may access a few market data points or thousands of market data points. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 7

8 Valuation Engine contd. In intraday mode, market data change affects valuation results where as trade data change affects valuation attributes End of day mode is invoked by a distributed batch process (EOD process). This process runs the valuation engine for all open trades. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 8

9 Valuation Engine contd. In End of Day mode, market data is frozen (EOD market data), valuation results are re-computed to use EOD market data. Valuation attributes can also change because of time factor. End of day process can generate anything between a few GB to a few TB of data every day depending upon the volume of trades being processed. This is an official snapshot of risk data. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 9

10 Position Manager Makes ad-hoc queries on Risk data based on attributes. Fetches attributes and results and aggregates them. Converts many attributes to more user-friendly display string. Data fetched can be very large. Should be able to return the data within a reasonable time. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 10

11 Expectation As Production Platform Ø Real-time with market data changes Ø High scalability Ø High volume of writes at short period of time (EOD process) Ø Fast response time for a ad-hoc query on large data set Ø Data consistency and accuracy As Development Platform Ø Easy to make new features Ø Easy to make enhancements and changes Ø Easy to maintain and debug Ø Layered architecture and function/service encapsulation TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 11

12 In a nutshell What is our Data Model to support our requirement Ø Wide row design and in our case it happens to be skinny rows also. Ø Lots of additional secondary data (mainly, secondary indexes) for attributes to support ad-hoc query. Ø Expressions instead of values in valuation result columns to support realtime movement with market. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 12

13 Secondary Indexes Native secondary index, maintained by Cassandra. It is recommended when selectivity is low. Custom secondary index, maintained by user application. There are different design pattern based on selectivity Ø One to one mapping, this could be maintained by a single row per column family. You can have a single column family to maintain all the one-to-one mapping in your application (Keyspace). Ø One to many mapping, you need to create a separate column family per mapping TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 13

14 Values stored in valuation attribute and result columns Valuation attribute columns, are normally used in query and they are indexed. We create either a native index or custom index depending upon selectivity. Valuation result columns, normally change with trade data, market etc. We store formula, data expression and fetch/ derive the value when we access these columns. Example, Ø trade_price (val double 85.0) Ø market_price (exp entity = MKT_CURVE_DETAIL, curve_num = 1850, curve_dt = ) Ø risk_qty (val double 150.0) Ø risk_mtm (mul double (ref risk_qty) (sub double (ref market_price) (ref trade_price))) TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 14

15 Cassandra API Abstraction - Keyspace A simple abstraction of Cassandra API Keyspace abstraction public class Keyspace { public static void add_keyspace(string keyspace) throws Exception {.. } public static void drop_keyspace(string keyspace) throws Exception {. } } When a Keyspace is created, it also creates the following Column Families SYS_ONE_TO_ONE_SECONDARY_COLUMN_INDEX This is being used to store one-to-one mapping column value to rowkey. SYS_GEN_NEXT_NUM This is a counter column family and it can be used to generate surrogate rowkey. SYS_XACT_LOG This will used to manage transaction in Cassandra. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 15

16 Cassandra API Abstraction - Column Family Interface public interface UserColumnFamilyIFace { public String getkeyspace(); public String getname(); public void create(object object) throws Exception; public void drop(object object, String context) throws Exception; public void insert(object object, String rowkey, Map datamap) throws Exception; public void delete(object object, String rowkey, String sname, List columnlist) throws Exception; public void update(object object, String rowkey, Map datamap) throws Exception; public long loadfromdb(string context, List columnlist) throws Exception; } TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 16

17 Column Family ADDRESS_BOOK2 public class AddressBook2 extends BaseColumnFamily { public static final String CF_NAME private static final String KEY_SPACE = "ADDRESS_BOOK2"; = SystemConfig.keyspace(CF_NAME); private static ColumnFamilyDefinitionIFace getcolumnfamilydefinition(string name) { ColumnFamilyDefinition cfdefinition = BaseColumnFamily.columnFamilyDefinition(); cfdefinition.set(key_space, name, BaseColumnFamily.getCfType(), ColumnFamilyDefinition.PRIMARY); cfdefinition.add(new ColumnDefinition("username", DataConstants.ROW_KEY+DataConstants.DATA_TYPE_STR)); cfdefinition.add(new ColumnDefinition("state", DataConstants.SECONDARY_COLUMN+DataConstants.DATA_TYPE_STR)); cfdefinition.add(new ColumnDefinition("profession", DataConstants.SECONDARY_COLUMN+DataConstants.DATA_TYPE_STR)); cfdefinition.addseccolumnfamily(new OneToOneSecondaryColumnIndexDefinition(name, new String[] {"fname", "lname"}, 0)); cfdefinition.addseccolumnfamily(new SecondaryColumnIndexDefinition(name, new String[] {"state", "city", "lname"}, 0)); cfdefinition.addseccolumnfamily(new SecondaryColumnIndexDefinition(name, new String[] {"state", "city"}, 0)); cfdefinition.setsecondarycolumnindexdefinition(); } return cfdefinition; } public AddressBook2() { this.setcolumnfamilydefinition(getcolumnfamilydefinition(cf_name)); } TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 17

18 Column Family ADDRESS_BOOK2 contd. Our API creates this column family with appropriate metadata to also create native secondary index on state and profession. It also creates secondary column family ADDRESS_BOOK2_state_city_lname and ADDRESS_BOOK2_state_city It uses username value as rowkey, if you do not have a column (a set of columns) that can be designated as rowkey then you need to extends BaseUUIDRowkeyColumnFamily or BaseNextNumRowkeyColumnFamily class. If we insert the a row to ADDRESS_BOOK2, example, username=ja, fname=john, lname=adam, state=tx, city=houston, profession=computer It will use JA as rowkey and insert all the column/values in ADDRESS_BOOK2, At the same time it will also upsert a row in SYS_ONE_TO_ONE_SECONDARY_COLUMN_INDEX with rowkey as ADDRESS_BOOK2_fname_lname,column name as John Adam and column value as JA TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 18

19 Column Family ADDRESS_BOOK2 contd. It will upsert a row in ADDRESS_BOOK2_state_city_lname with rowkey as TX_Houston_Adam, column name as JA and column value null. Similarly, it will also upsert a row in ADDRESS_BOOK2_state_city with rowkey as TX_Houston, column name as JA and column value null. Similarly, it manages the data delete and update also. All the 4 column families are being updated as a result of a single operation. Updates to all the 4 column families happen in a single batch operation. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 19

20 Column Family Accessor Interface public interface AccessorIFace { public Map get(storeaccess access, Map querymap, List columnlist) throws Exception; public long get_count(storeaccess access) throws Exception; public long get_count(storeaccess access, Map querymap, List columnlist) throws Exception; } TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 20

21 ADDRESSBOOK2Accessor public class AddressBook2Accessor extends AbstractAccessor { public AddressBook2Accessor() { super(new AddressBook2()); } } TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 21

22 ADDRESSBOOK2Accessor contd. When a query is submitted using querymap (Map<String, Set<String>>), it will automatically utilize rowkey information (if username is given), native secondary index (if state or profession is given), and custom secondary index column families. TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 22

23 Questions Thank you #cassandra12 TRIPLE POINT TECHNOLOGY WESTPORT HOUSTON PARSIPPANY LONDON VIENNA GENEVA CAPE TOWN PUNE CHENNAI SINGAPORE SAO PAULO SYDNEY TPT.COM 23