Parallel Data intensive applications in the cloud ---A data mining use case study in the life science

Transcription

1 Parallel Data intensive applications in the cloud ---A data mining use case study in the life science Liangxiu Han Co-authors: Tantana Saengngam and Jano van Hemert UK-eScience-2010, Cardiff

2 outline What is Cloud Computing? Cloud Computing models A data mining use case Parallel the data mining task in the Cloud Experimental evaluation

3 What Is Cloud Computing? (I) Cloud Metaphor A depiction of computer network diagram An abstraction of complex infrastructure it conceals

4 What Is Cloud Computing? (II) Definition Cloud Computing is a type of distributed computing paradigm augmented with a business model via a Service Level Agreement between providers and consumers. Technically, cloud computing offers resources, software applications, infrastructures over the Internet to users At a business level, cloud providers lease these resources immediately and temporarily to cloud consumers when required

5 What Is Cloud Computing? (III) Evolution Distributed Computing Parallel Computing Grid Computing Cloud Computing

6 outline What is Cloud Computing? Cloud Computing models A data mining use case Parallel the data mining task in the Cloud Experimental evaluation

7 Cloud Computing Models Service Model Deployment Model

8 Service Models of Cloud Computing (I) Software as a Service (SaaS) Platform as a Service (PaaS) Infrastructure as a Service (IaaS)

9 Deployment Models of Cloud Computing (I) Public Cloud Private Cloud Hybrid Cloud

10 outline What is Cloud Computing? Cloud Computing models A data mining use case Parallel the data mining task in the Cloud Experimental evaluation

11 Data Mining Use Case (I) The goal: automatically label images with terms from the anatomy ontology

12 Data Mining Use Case (II) Challenges 1500 anatomical terms in databases and 85,824 images in file systems Multi-gene patterns co-existing in an image Variable shape, location and orientation of images The number of images associated with a certain gene is uneven The dimensionality of each image is high (3kx4k pixels)

13 Data Mining Use Case (III) Framework

14 Data Mining Use Case (IV) Core methodologies Image Processing Wavelet Transform Fisher Ratio LDA(SVM, ANN, LSVM)

15 outline What is Cloud Computing? Cloud Computing models A data mining use case Parallel the data mining task in the Cloud Experimental evaluation

16 Parallel the Data Mining Task (I) Parallelisation is a sought after solution for speeding up an application, particularly for data intensive applications Three considerations for parallelising an application How to distribute workloads or decompose an algorithm into parts How to map the tasks onto various computing nodes and execute subtasks in parallel How to coordinate and communicate subtasks on those computing nodes.

17 Parallel the Data Mining Task (II) Data Parallelism: workload are distributed into different computing nodes and the same task can be executed on different subsets of the data simultaneously Task Parallelism: tasks are independent and can be executed purely in parallel Pipelining: an iteration of a task consisting of many stages, where each stage in the task is chained and executed in order and the output of one stage is the input of the next one.

18 Parallel the Data Mining Task (III).FG'F-+"H'' 234I6 Parallelising the data mining task (data and task parallelisms Parallel algorithms!"#$%$%&'(#)#*+)'?+#('>8#&+',$/+' 234E6 >8#&+'?+*0#/+ >8#&+ A+%1$*+234B6,+#)-"+' C+%+"#)$1%234D6,.I,.E,.7.#89/+.9/$)' 23476,+#)-"+'.+/+0)$1% 23456!+*)$%&'(#)#*+)?+#('>8#&+',$/+' 234E6 >8#&+'?+*0#/+ >8#&+ A+%1$*+234B6,+#)-"+' C+%+"#)$1%234D6!"#$%&"'"(()(*#+,"-"%&"'"(()(*#+,+#)-"+' 4:)"#0)$1%234;6,+#)-"+' 4:)"#0)$1%234;6 </#**$!+" 234=6 3"+($0)$1%' 4J#/-#)$1%234IK6

19 outline What is Cloud Computing? Cloud Computing models A data mining use case Parallel the data mining task in the Cloud Experimental evaluation

20 Experimental Evaluation (1)

21 Experimental Evaluation (II) Billing: The time for the whole task is less than 1 hour $0.085/per Instance $1.02/12 instances

22 Experiences A cheaper solution than buying your big machines when performing pilot studies With IaaS model, you need a precooked prototype Understand and balance the billing models of cloud providers

23 thank YOU!