RiskMetrics RiskBurst : Scaling the Private Cloud out to the Public Cloud

Transcription

1 RiskMetrics RiskBurst : Scaling the Private Cloud out to the Public Cloud Dr Rob Fraser RiskMetrics Group rob.fraser@riskmetrics.com 1

2 Business Drivers of RiskMetrics Cloud Use RiskMetrics Group Offers industry-leading products and services in the disciplines of risk management, corporate governance and financial research & analysis Our use case an example of Cloud Bursting Scaling private cloud computation to the public cloud especially to meet peak or unpredicted demand. Integration of our extensive on-premise SOA-based capability with Windows Azure We are running on 2,000 compute instances on Windows Azure We have plans to use 10,000+ compute instances Calculation of financial risk requires extensive computing power Notoriously complex, especially for complex asset classes of the kind that the triggered the 'credit crunch Results in very high computational loads for RiskMetrics Daily risk analysis load equivalent to calculating risk on 4 trillion US Stocks Computational loads are characterised by high demand peaks and growth trend in calculation l complexity 2

3 Peak Load Characteristics 3

4 Growth trend in calculation complexity Maximum Complexity of Risk Analysis Processing Request Relative Eq quity Equiv valent Unit ts (Log Scale) Risk problem complexity has doubled every 6 months Processor power doubles every 2 years Moore s Law

5 Scaling out to the Cloud: Extending an existing architecture Data Intensive Compute Intensive Pricing Work Packet Generation Very Large Scale Market Data Sensitive Client Data Self-Contained Pricing Work Packets (contain no client data) 5

6 Scaling out to the Cloud: An Architectural Pattern for High Performance Computing Building large scale computation requires careful design Need to avoid the Von Neumann Bottleneck - embrace parallelism Increasing resources should result in proportional increase in performance/throughput h t Pattern focuses on Service Orientation (SOA) Decoupling components Horizontal Scaling Asynchronous interaction between on-premise and cloud, and within the Cloud Cloud centralized ed configuration o and control o Implement elasticity based on units of scalability Adaptive Load Generation in the context of a production service bridging g private & public clouds Operational Integration Outside-In Services Binding SLA Commitments Operational Commercialization 6

7 RiskBurst Architecture Operations Console ScaleUnitDaemon RiskBurstDaemon (Little (Big Daemon) Daemon) Blob Storage Log Analysis Servers Management Application Pulse Service Bus BigRedButton Service Bus Scale Unit Scale Unit Scale Unit Local Scale Unit Local Scale Unit Local Scale Unit Load Generator Blob Storage ScaleUnitDaemon ScaleUnitDaemon (Little (Little Daemon) Daemon) Worker Worker Role Role -- Worker Role - DealServer DealServer Table Storage 1.5m requests/hr Queue Storage Configuration Repository 7

8 Commercial Use of the Public Cloud All our usage of Windows Azure is billable We have Dev, QA, UAT and Prod Environments (all on Azure Production) Solution proven on 2,000 Azure compute instances; currently in UAT of initial commercial offering Cost Modelling Mapped Azure pricing i into our architectural t model Fixed monthly costs per environment Per Scale Unit Hour utilization charge Usage planning and budgeting Internal environment owners forecasting consumption and providing budget input RMG Account Owners monitoring billing Cloud provider notification of certain very large scale usage requirements TCO/ROI affected by development & test model Being an analytics provider, we have extensive QA through the development and test cycle Non-trivial use of Dev,,QA and UAT environments TCO analysis has to factor in all upstream consumption as well as actual production usage 8

9 RiskBurst Business Benefits Effective SOA+Cloud architectural pattern delivers key business benefits: Elastic scaling High reliability Enhanced services Improved agility Empowering innovation Dollars on the Cloud Innovation in operational, business and accounting processes to support per hour consumption of public cloud resources 9

10 Further Information 10