GLOBAL FINANCIAL SERVICES COMPLIANCE & RISK MANAGEMENT with

Transcription

1 GLOBAL FINANCIAL SERVICES COMPLIANCE & RISK MANAGEMENT with Bombuz A Big Data & Semantic Web Solution Insigma Hengtian Software Ltd. Bayshore Management Consultants, LLC

2 Table of Contents OVERVIEW... 1 KEY CHALLENGES IN THE RISK AND COMPLIANCE DOMAIN.. 2 The Regulatory Environment Today s Approach to Solving the Challenge The Big Data Solution RISK & COMPLIANCE BUSINESS CASE 7 Exposure Risk Assessment....6 Client Organizational Description..6 BOMBUZ A UNIQUE BIG DATA SOLUTION... 7 Data Synchronization....7 Semantic Mapping.. 8 Performance Testing...10 Dashboard.11 CONCLUSION AND FUTURE WORK.12

3 OVERVIEW The globalization of the financial industry has resulted in firms that operate 24/7 in multi-country, multi-currency and multicultural environments. Complex organizational models comprised of centralized, regional and local operations must somehow function seamlessly day in and day out. Overlapping responsibilities, matrix reporting, independent outposts and outsourced operations must be monitored not only for business performance but to ensure compliance with a vast system of rules and regulations. And in the aftermath of the financial crisis, risk management and compliance responsibilities have exploded. There are literally hundreds of new rules and regulations from Basel III and Solvency II, to individual country rules, and to those coming from the alphabet soup of U.S. regulatory agencies. In fact, the House Financial Services Committee estimates that it will take private industry 24 million man-hours annually to comply with the first 185 new rules emanating from Dodd-Frank. At this rate, it will cost the industry somewhere around 50 million man-hours to comply with all 400 proposed rules. This translates to approximately 25,000 additional personnel and $2.5B in annual expenses. This is coming at a time when firms in our industry are still engaged in cost-cutting and productivity-enhancing initiatives. With IT budgets slashed, there are more projects competing for fewer resources. And with most of the data needed to monitor risk and demonstrate compliance with these new rules resident in silos whether that be in functional business units or in specific applications it still takes a significant amount of human intervention to answer a simple request. To manage risk and demonstrate compliance in the future, new combinations of systems, technologies and delivery mechanisms will be needed from mobile and web-based applications to legacy systems, spreadsheets and stand-alone applications. Furthermore, they will need immediate and targeted access to vast amounts of data in a variety of forms, from hard-copy documents to digitally-stored images, to calculated values, to tweets and blogs, to audio and video clips, and to any number of other data feeds to satisfy the everexpanding measures of risk. This paper discusses the business challenges of finding a cost-effective way to rapidly develop and deploy management tools that can adapt to the changes in the regulatory environment over the next several years, demonstrate compliance, and mitigate risks. We will look at the compliance function across the supply chain from the front, to the middle, and to the back-office operations. We will identify the challenges of enhancing current processes at each step in the process, from changes required to enhance the quality of the data, to the analytics needed to test the rule, and to the delivery of the final results. Although this is a complex domain, we feel that there are enough similarities surrounding the challenges to core processes that can benefit from a new approach and be applied across the supply chain to demonstrate compliance. 1

4 KEY CHALLENGES IN THE RISK AND COMPLIANCE DOMAIN The Regulatory Environment There are a number of challenges in developing new tools to address the changing requirements in the areas of risk and compliance the first being regulatory clarity. This is an issue no matter what regulatory jurisdiction you re working in. Rules are proposed, then they are issued in draft, and then you have the comment period. This entire process can eat up a lot of time. And businesses just can t sit around and wait. They need to plan and budget, they need to prioritize initiatives, and they need to know where and how to deploy their resources. Let s take as an example the recently implemented IRS Regulation Cost Basis Reporting. It took several years for the IRS to issue final regulations in fact, it wasn t until November of 2011 that they were finalized just one month prior to implementation for equity securities in January of During this time, industry user groups were formed to sort through the draft regulations, comment on certain provisions, then wait for responses from the IRS and attempt to develop best practices so that firms could begin developing the new functionality required to address the mandate. It s clear that large portions of IT budgets over the next few years will be devoted to meeting new risk, regulatory and compliance requirements. IDC Financial Insights estimates that growth in IT spending on risk management will top 15% of the total IT spending in financial services in And these estimates are only for the known rules. That is why we believe a new approach is needed one that allows for a quick and efficient response to change, and doesn t burn through the IT budget for the year. The Systems, Data and Delivery 2

5 In today s world, managing and monitoring risk requires a complex systems and data environment that may or may not include Enterprise Risk Management (ERM) applications, Compliance Systems, Security Masters, Data Warehouses and functional applications for Managing Orders, Trade Routing and Execution, Accounting, Reporting, etc. Most risk measures are driven off compliance rules that are either integrated into a specific application or are maintained in a compliance rule engine. But regardless of what tools are used, they all require data in order to function. Much of the data that is needed for testing rules or running a risk monitor is spread across multiple sources from the actual application, to the Security Master File, to massive data warehouses. Security information can be sourced from vendors, offering documents (in the case of certain asset types), direct market feeds, etc. And depending on whether the system is in-house or at a vendor, the same data may come with different identifiers and in different formats. With over 400 rules emanating from Dodd-Frank alone, just think of the additional data requirements that will be needed. The Gartner Group has estimated that data will grow 800% over the next five years and 80% of that data will be unstructured. In addition, they also estimate that 85% of currently deployed data warehouses will, in some respect, fail to address new issues around extreme data management by How will firms who are squeezed for resources be able to keep pace with these new demands? Today s Approach to Solving the Challenge Any solution trying to address today s risk and compliance challenges has to deal with these issues: Distributed Data Unstructured Data Large Data Volume Less Operating Expenses Forever Changing Rules Typical solutions include a central data mart or warehouse model. Data are stored with a pre-defined schema in a structural format. With the help of ETL tools, data are extracted, transformed and loaded into the data repository to be queried by some type of business intelligence software (BI). In most scenarios, this central repository model is a vertical scaling platform. This means more powerful machines will have to be used as the volume of data grows. This, in turn, requires upgrades to the hardware and software in the data center limiting flexibility and increasing costs in answering demands. Another major challenge is the ability to mine unstructured data in a distributed environment, while at the same time trying to keep pace with all the new rule requirements. And whenever a quick solution is demanded for a complex problem, it usually comes with a high cost attached. 3

6 The Big Data Solution Horizontal scaling solutions with commodity hardware running distributed and parallel processes have become very popular today. BigData, which refers to the technologies that handle large volumes of data, is more of a general term, covering open source tools like Hadoop, MapReduce and NoSQL databases. These evolving technologies provide a relatively lower cost and a more flexible alternative to a central data repository model. The solution described here is built on the generally-accepted distributed computing framework, Hadoop, which leverages the MapReduce programming model to achieve a highly scalable, distributed processing capacity. Instead of collecting specific data into a central data warehouse through an ETL process, BigData systems extract the raw data from operational systems into a NoSQL database, HBase, avoiding repetitive ETL process when regulatory rules change. HBase is based on HDFS (Hadoop Distributed File System). It is able to store very large files with streaming data access and runs on clusters of commodity hardware. Furthermore, there is a semantic data mapping layer presented in a BigData system, building a virtual RDF graph that links the data stored in HBase. This RDF graph helps find the corresponding records in HBase in terms of the query subject, and feeds them as the input to the MapReduce task. In this case, changes in data semantics can be automatically reflected in a BigData system by updating the mapping files that define how the virtual RDF graph is generated. A Big Data system solves many of these challenges and is also an economical solution for big data analysis. For one thing, a Big Data system can either run on clusters of commodity machines or on an elastic cloud like Amazon EC2. Another benefit of a Big Data system is that the technologies are built on open source tools/applications, providing the opportunity for customization with relatively low development/maintenance overhead. 4

7 RISK & COMPLIANCE BUSINESS CASE To illustrate the challenges that an investment manager faces in responding to a relatively straightforward compliance and/or risk management query, we will use a case study. We developed the business case based on real-life experience within a complex diversified manager responding to an event. It is illustrative of the many pain points encountered in responding to a typical request. Exposure Risk Assessment Client Organizational Description: American Alliance (AA) is a large diversified global financial service company providing both asset management and insurance products to individuals and institutions. AA s product line-up includes mutual funds, separate accounts and collective trusts, along with annuity products, life insurance and alternative investments. AA has grown significantly over the past ten years, mostly through acquisitions. They have done significant work to integrate their operations, but still maintain several separate legal entities. Up until the recent financial crisis, AA had been working on a number of data warehouse and consolidated reporting initiatives to provide management with better dashboard tools to monitor and measure key indicators of risk and compliance across their businesses. On the investment side, AA has four Investment Advisors, over 30 Custodians and three Trading Desks. On the operations side, AA has a hybrid operating model: Mutual Fund Accounting, Custody and Transfer Agent operations are outsourced, while Institutional Portfolios (SMAs), CTFs, Commingled Pools, Fixed Annuities and the Insurance General Fund are accounted for in-house, as is Annuity Administration. Hedge Funds are supported by a Prime Broker. On the technology side, AA uses both internally developed and vendor-supplied applications. They have different trading platforms for Equity and Fixed Income, as well as different applications for Pre- and Post-Trade Compliance. They have a home -grown portfolio accounting system and a vendor-supplied annuity administration system. AA also has a Data Warehouse that takes nightly feeds from the internal portfolio accounting system and the Mutual Fund accounting system of holdings at the individual security level. At the current time, holdings data for the General Fund, Fixed Annuities, Sub-Advised Portfolios, and Alternative Products are not available in the warehouse. 5

8 Challenges Because AA has a hybrid operating model with some functions performed in-house and some outsourced there isn t a straightforward solution to this relatively simple request to run a report to see what the exposure to XYZ Corporation is across the complex. Multiple accounting platforms, back-end compliance applications, data downloads and data warehouses must be queried in order to retrieve the data required to satisfy the report. Each application stores data in different formats and some level of translation is required in order to be able to satisfy the requirements of the report request. The primary challenge of an enterprise solution to this business scenario includes the capability to handle large volume of historical data as well as distributed operating data stores. A common practice is to build a centralized data warehouse with legacy system codes to handle ETL (extract, transform, load) jobs at the back office. Compliance reports will then be generated at the end of day, week or month, after all the data have been transformed and stored in the data warehouse. It is not difficult to understand that similar solutions would require a huge commitment in the infrastructure and system development expenditure. More important is the lengthy project duration in planning and deployment of the data warehouse, which can easily become obsolete prior to production, simply because of a regulatory requirement change. The Business Problem: There has been a significant event related to XYZ Corporation, a leading supplier of power generation equipment. Coming on the heels of poorer than expected earnings, the wire services are reporting that the US Military has nixed their plans to use XYZ as their primary supplier of generators. A bankruptcy filing could be coming. The stock price is declining rapidly as traders rush to dump shares The AA investment committee has called a meeting to determine the overall exposure to XYZ in order to develop an action plan one that includes how to provide updates and answer inquiries from their institutional clients. The key questions that need to be answered are: How much do we own? Where are the holdings what Funds, SMA, Trust/Commingled Pools, etc.? How many clients are impacted? Can we sell out of position, if needed do we have any issues? (Settlements, shares out on loan, etc.) What Needs to Be Done? AA needs to determine their overall exposure to XYZ Corporation across the entire complex. Because of the complex operating and technical environment at AA, requests for information must go out to multiple constituents simultaneously. The information will have to be pieced together from source systems, data warehouses, s, spreadsheets and manual reports. This takes time and time is of the essence. A secondary requirement will be to determine the holdings for XYZ at the individual client level. The SMAs, CTFs and Commingled Pools are all accounted for on the Portfolio Accounting system which has aggregate reporting functionality. In addition to the holdings data, we will need information on pending trades, settlement issues and/or shares on loan for XYZ Corporation. This information is helpful in determining a trading strategy that will minimize any potential issues should they decide to authorize selling out of all positions in XYZ. 6

9 BOMBUZ A UNIQUE BIG DATA SOLUTION To solve the business scenario quoted in this paper, a proof of concept utilizing the Bombuz framework is being developed. The sample financial company AA acts as a custodian for vendors that provides various financial products, such as mutual funds, SMA, CTF, etc., to individuals as well as institutions. The transaction data is distributed on different systems, such as data warehouse, portfolio account system or , depending on the asset class of the transaction. To explore the exposure across all the product lines, all the transaction data will be extracted first into HBase in RAW format, upon which queries can be made at the issuer level, and also be further drilled down to the customer level. In addition, the semantic connection among customers can be customized in the user interface, and the resulting dataset presented in dashboard will change accordingly. Data Synchronization The Thrift service solution is designed to synchronize data from ODS to HBase. The reason we don t use Hadoop Sqoop, an open source tool for bulk loading, is that this solution requires the HBase server to pull data from source systems, thus imposing a great burden on the server to manage and coordinate with various clients. The Thrift service solution, instead, lets the clients push data to the server whenever they are available. The data can be stored in various ways as structured data from MySQL and Oracle, semi-structured data as found in spreadsheet tables, or unstructured data as found in and can be synchronized into HBase easily. 7

10 Semantic Mapping As mentioned previously, Bombuz synchronizes raw data from source operating data stores to HBase without an ETL process, as in traditional data warehousing. Thus, the same data may present in different identifiers and different formats. Semantic web technology is leveraged to establish the logic link amongst related data. Generally, there are two ways of synchronizing that can serve the purpose. The first method is to map all the data in HBase, together with the domain ontology that maintains the relationships among them, to a consolidated virtual RDF (Resource Description Framework) graph, as shown in the following chart. This unified data view can be queried through standard RDF query language, such as SPARQL. Currently, there is no out-of-box tool geared towards mapping a non-relational database to RDF. However, there are tools mapping RDB (Relational Database) to RDF, such as D2RQ, which is an open source tool capable of mapping SQL-92 compatible database to RDF. And combined with Hive, a SQL-like query engine, D2RQ can successfully execute most of the RDF queries with the data in HBase. The second method is to merely maintain the data relationships as domain ontology, and leave the raw data in HBase intact. As illustrated in the chart below, a client query should be broken down into a SPARQL query and a MapReduce program, interacting with RDF graph and HBase tables respectively. 8

11 Both methods are implemented and then executed on a small Hadoop cluster, just to demonstrate the performance comparison between them. The results chart below shows that it takes much more time for the unified RDF method (the first method) to process the query than the second method. This is because the first method attempts to translate SPARQL to SQL with D2RQ, and then translate SQL to MapReduce with Hive. Clearly the process suffers from these two transitions. Another disadvantage of the first method is that Hive does not support all the features of SQL-92. Also, Hive requires major upgrades that will allow for all the features of SPARQL to be supported through D2RQ. Therefore, the second method is adopted as more suitable for Bombuz. The architecture diagram below shows how the system is implemented under this philosophy. The query service always retrieves data result sets that are pre-processed and stored in Hadoop. The semantic logic is maintained in a triple store through the rule service, which also triggers a re-calculation of certain queries as long as some data link changes occur. 9

12 Performance Testing In order to measure the potential performance of the solution, especially how many nodes are needed for the cluster to achieve acceptable response latency, we conducted a series of in-house tests upon several clusters with different numbers of slave nodes. All the cluster nodes were commodity machines and each was equipped with one Intel G GHZ 2- core processor, one 2 GB RAM, 1T SATA hard disk. The size of effective data to be processed in the test case is 1 TB (the physical data stored is 3 TB with a default setup of three replications in HBase). The chart below shows the average processing time required to compute the total amount of holdings and pending transactions at the vendor level, upon two clusters with different numbers of nodes. 10

13 According to the results above, the predictive response time resembles the curve below (assuming the processing time will drop linearly when the number of nodes expands). This gives us a ballpark estimation of a 10-minute response time, if the number of nodes reaches 500. Dashboard Following are sample dashboards of Pending Transactions and Holding Reports running in ipad as HTML5: 11

14 CONCLUSION AND FUTURE WORK This POC proves that Bombuz, as a Big Data and Semantic Web framework, is a feasible solution for processing a huge volume of data scattered within various data stores. Built upon Hadoop / MapReduce and Semantic Web, this system can scale without physical limits. And compared to the potential downtime triggered by semantic rule changes (e.g. several days) for repetitive ETL by a traditional data warehouse solution, our solution can achieve a response time of several hours or even several minutes as long as enough commodity machines are configured to the cluster. Still, there is more work to be done. Firstly, the performance testing for data extraction / synchronization from ODS to HBase will be conducted and analyzed to determine a practical mechanism with a high payload. Secondly, a scalable solution for large triple stores may need to be explored, since the size of RDF triples could be too large to manage in current triple stores, such as Jena TDB and SDB. A potential solution is to use HBase to store a large volume of triples, then develop methods to execute triple queries in parallel. 12