MathCloud: From Software Toolkit to Cloud Platform for Building Computing Services

Transcription

1 MathCloud: From Software Toolkit to Cloud Platform for Building Computing s O.V. Sukhoroslov Centre for Grid Technologies and Distributed Computing ISA RAS Moscow Institute for Physics and Technology

2 MathCloud Research project started in 2008 at CGTDC ISA RAS Investigate the use of distributed service-oriented environments for scientific research Develop an open platform for building, deployment, discovery and integration of computing services Create a testbed environment and use it for solving complex scientific problems

3 Problem

4 Problem

5 Problem

6 Problem

7 Problem Computing s Solve Specific Classes of Problems Hide Complex Computing Infrastructure Reusable and Composable

8 Requirements 1. Use computing environment as a service for solving a problem 2. Access services via convenient and familiar interface 3. Easily combine services with each other 4. Easily create services and share them with colleagues

9 -Oriented Environment Users Applications s B A C D Computing Resources

10 Approach

11 Approach Unified Programming Interface

12 Approach Web Interface Application Unified Programming Interface

13 Approach Web Interface Application Unified Programming Interface -Oriented Architecture -Oriented Science (Foster, 2005) Software as a Web 2.0 Representational State Transfer (REST)

14 Computing as a Web Function Identified by URI

15 Computing as a Web Function Input parameters (Problem Description) JSON Schema Identified by URI

16 Computing as a Web Function Input parameters (Problem Description) JSON Schema Identified by URI Output parameters (Problem Solution) JSON Schema

17 Computing as a Web Function HTTP Input parameters (Problem Description) Input JSON JSON Schema Identified by URI HTTP Output parameters (Problem Solution) JSON Schema Output JSON

18 Computing as a Web Function HTTP Input parameters (Problem Description) Input JSON Required Features Introspection Long Computations Passing Large Data JSON Schema Identified by URI HTTP Output parameters (Problem Solution) JSON Schema Output JSON Core Technologies HTTP + REST JSON JSON Schema

19 Unified REST API GET SERVICE_URI JOB_URI Get service description Get job status and results Get file data FILE_URI SERVER_URI Get list of services provided by server POST DELETE Submit new request Get JOB_URI Cancel job Delete job data

20 MathCloud Platform A set of tools for building, deployment, discovery and integration of computing services implementing the unified REST API Container (Everest) Catalogue Workflow Management System (WfMS) Client Interfaces Security Some components are publicly available Ultimate goal is to open source the whole platform

21 Container (Everest) Security Web UI Core REST API Manager command adapter Command-line application process java adapter Java application thread HTTP server Jersey JAX-RS REST API Job Manager cluster adapter Parallel application grid adapter Grid application cluster Client Extension Manager grid

22 "name": "POV-Ray", "description": "Runs POV-Ray raytracer", "inputs": { "scene": { "title": "Scene file", "type": "string", "format": "uri", "required": true }, "params": { "title": "Rendering params", "type": "string", "description": "Example: +W320 +H240" } },

23 "outputs": { "image": { "title": "Rendered image", "type": "string", "format": "uri", "required": true }, "log": { "title": "Runtime log", "type": "string", "format": "uri", "description": "This is the raw output of POV-Ray", "required": true } }, "config": { "input-files": { "scene": "scene.pov", }, "adapter": { "type": "command", "command": "povray +Iscene.pov +Oimage.png +FN $params$", }, "output-files" : { "image": "image.png", "log": "stderr" } }

24 Catalogue

25 Workflow Management System

26 Client Interfaces Accessing MathCloud services from applications Building complex workflows Java API Python API Command-line interface Any HTTP library or client (eg. curl) Easy access from JavaScript (eg. jquery)

27 Security

28 Security HTTPS any client

29 Security HTTPS any client HTTPS browser only Loginza.API

30 Security allow deny HTTPS any client HTTPS browser only Loginza.API

31 Security allow deny HTTPS any client HTTPS browser only Request on behalf of user X proxy.allow Loginza.API allow deny

32 Demo and Downloads Demo Testbed Everest + demo services WfMS Catalogue Downloads

33 MathCloud Platform as a Core s Catalogue WfMS Security Security

34 MathCloud Platform as a Core s Catalogue WfMS Security Security Hosting Everest NG (multiuser, dynamic deploy, clustered)

35 MathCloud Platform as a Core s Catalogue WfMS Security Security Hosting Everest NG (multiuser, dynamic deploy, clustered) Resource Management Accounting Computing Infrastructure CGTDC ISA RAS Partners Partners Partners Providers Partners Partners Clouds Partners Partners

36 We invite all interested parties to cooperate and join the project.