Cloud Platforms, Challenges & Hadoop. Aditee Rele Karpagam Venkataraman Janani Ravi

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Cloud Platforms, Challenges & Hadoop. Aditee Rele Karpagam Venkataraman Janani Ravi"

Transcription

1 Cloud Platforms, Challenges & Hadoop Aditee Rele Karpagam Venkataraman Janani Ravi

2 Cloud Platform Models Aditee Rele Microsoft Corporation Dec 8, 2010

3

4 IT CAPACITY Provisioning IT Capacity Under-supply of capacities Allocated IT-capacities Load Forecast Waste of capacities Fixed cost of IT-capacities Barrier for innovations Actual Load TIME

5 The Cloud Platform Continuum On-Premises Servers Hosted Servers Cloud Platform Bring your own machines, connectivity, software, etc. Complete control Complete responsibility Static capabilities Upfront capital costs for the infrastructure Renting machines, connectivity, software Less control Fewer responsibilities Lower capital costs More flexible Pay for fixed capacity, even if idle Shared, multi-tenant infrastructure Virtualized and dynamic Scalable and available Abstracted from the infrastructure Higher-level services Pay as you go

6 Legacy vs. cloud computing Storage Scale & High Availability Computation & Multi-Tenancy Automated Service Management

7 Types of cloud services EC2 VM Role - Azure Azure Compute & Storage, AppFabric, SQL Google App Engine Salesforce.com SOFTWARE AS A SERVICE Google Apps INFRASTRUCTURE AS A SERVICE PLATFORM AS A SERVICE MS Online Services Facebook

8 You manage You manage Cloud Taxonomy You manage (On-Premises) Infrastructure (as a Service) Platform (as a Service) Software (as a Service) Applications Applications Applications Applications Data Data Data Data Runtime Middleware O/S Virtualization Servers Storage Networking Runtime Middleware O/S Virtualization Servers Storage Networking Managed by vendor Runtime Middleware O/S Virtualization Servers Storage Networking Managed by vendor Runtime Middleware O/S Virtualization Servers Storage Networking Managed by vendor

9 Traditional On-Premises Model Servers are dedicated to specific workloads Individual servers sized for peak or average capacity of a given workload Substantial idle/wasted capacity An application can t scale beyond the boundaries of boxes it resides on Provisioning new capacity takes time Private Cloud Model Servers are treated as a virtual pool of resources Apps consume from the pool rather than having dedicated resources Idle servers automatically shut down or put to sleep until needed Apps can scale to the available provisioned capacity in the pool Adding a new server adds capacity to the entire pool for all apps Dedicated infrastructure (i.e., Cloud resources are only accessible to your company, and not shared with others)

10 Microsoft Cloud Services

11 Challenges building cloud apps for Enterprise Janani Ravi Google Hyderabad

12 Traditional Enterprise applications Desktop base, typically single machine, single user Collaboration may not be a primary consideration Data stored within the Enterprise and owned by it Performance, scalability, security issues based on local data storage and access Discretionary upgrades based on Enterprise needs Enterprise responsible for backup, recovery, troubleshooting Localized failures and support issues, usually isolated to the Enterprise

13 Cloud-based applications Multi-user access, realtime collaboration, conflict resolution Offline access what if the user is not connected to the internet? User interface usually browser based getting all browsers to work Latency and scalability for users at different locations Build a developer universe

14 Building enterprise web applications Feedback, feature requests Configures, troubleshoots Application developer Enterprise web applications Designs, implements, operates Uses Services outage info, support Enterprise administrators End users Internal feedback

15 Challenges: Migration of existing data Enterprises usually have fully provisioned users and roles and existing applications like , calendar etc. o Tools for reliable data migration o Tools for interoperability with the older systems for partial migrations o Use single sign on or other methods to accept authentication from other systems

16 Challenges: Administrative tools Move to the cloud perceived as a loss of control o Requires good tools which allow enterprise admins to configure and manage services o Provide access control to manage different kinds of administrators o More transparency and monitoring tools for troubleshooting o Logs and audit reports to track activities

17 Challenges: Data location and ownership Organizations might care about where their data is stored, usually for legal reasons o Build controls which determine where data is located Organizations might care about which regions the data passes through "over the wire" o Much harder to address routing

18 Challenges: Data availability What happens if there is a major disaster? o Geographically distributed data centers How often has the system been down in the last few quarters? o Have a backup plan with multiple data centers Do you have scheduled downtimes? How do I access my data during downtimes? o Have good communications set up Provide a good offline story which is easy to use

19 Challenges: Data retrieval and tracking How do administrators track suspicious activity on an account? o Easy-to-use tools with logging and audit information to track this down o Meta logs with access and tracking information Report statistics and analytics to know how users use the applications Monitoring to track activity to determine patterns

20 Challenges: Upgrades and bug fixes Easier to fix bugs since explicit patches are not required. However easier to make inadvertent changes Enterprises often do not support frequent updates, need to have known rollout plans

21 In conclusion Administration, access, collaboration etc gets easier in the cloud Many hurdles to overcome before this becomes a reality for all enterprises

22 Things I ve worked on Offline capability on docs using Google Gears Data model and UI design on the next generation Google word processor Platform to manage policies for Enterprises And previously UI design and implementation for the IIS administrative tools

23 Cloud Platform Intro to Hadoop Karpagam Venkataraman Yahoo! Dec 8, 2010

24 Cloud Platform Cloud Platforms - foundations for building applications Loosely coupled Collection of services Semantics-free Broadly applicable Fault-tolerant over commodity hardware

25 What s in the Cloud Platform? Simple Web Service API s Cloud Platform Provisioning & Virtualization Analytical Data Storage & Processing Operational Storage & Processing Edge Content Services Other Services Messaging, Workflow, virtual DBs & Webserving ID & Account Management Security Metering, Billing, Accounting Monitoring & QoS Shared Infrastructure Fast Provisioning and Machine Virtualization Analytical Data Storage and Processing Operational Storage Edge Content Services Rest of this session

26 What is Hadoop? A scalable fault-tolerant cloud operating system for big data storage and processing A framework that provides distributed application services Operates on unstructured and structured data A large and active ecosystem Open source under the friendly Apache License

27 Hadoop Core Components Hadoop Distributed File System distributed storage MapReduce programming paradigm parallel applications

28 User Karishma Anand Karishma Sneha Anand Sneha Karishma Visits URL Example Data Analysis Application Find users who tend to visit good pages. Logic: Average page rank per user > 0.5 Time 8:00 8:05 10:00 10:15 12:00 12:02 12:30 Page_Visits User Karishma Anand Karishma Sneha Anand Sneha Karishma Pages URL URL Page Rank Page Rank Time 8:00 8:05 10:00 10:15 12:00 12:02 12:30

29 Map Reduce Divides the job into smaller tasks Location aware division of input Job Tracker - Schedules jobs across task tracker slaves Task Tracker runs data local computation task Each task is a map task or a reduce task. Language independent Data Definition Language Customizers Combiner, Partitioner, mapper (filename, file-contents): for each line in file-contents: fields = split(line, \t ) pg_rank = fields(3) emit (user, pg_rank) reducer (user, values): sum = 0 for each value in values: sum = sum + value avg_pg_rank = sum / sizeof(values) if avg_pg_rank > 0.5 emit (user, avg_pg_rank)

30 Hadoop - Data Flow What happens when we submit a job? Hadoop determines where the input data is located. Calulates number of splits required Split Size is computed as max(min(block_size, data/#maps), min_split_size) Creates tasks Copies necessary files to all nodes, and each slave node runs a task Once map tasks are over, starts reduce tasks, Collect output What user need to specify: Mapper class Reducer class Job configuration: job name, number of maps, reduces, any values required by the map and reduce classes etc. Build the code into a jar file and submit.

31 Split 1 Karishma\twww.myblog.com\t10:00 \t0.7 Sneha\twww.crap.com\t10:15\t0.2 Split i Karishma\twww.cnn.com\t8:00\t 0.9 Anand\twww.myblog.com\t8:07\ t0.7 Anand\twww.flickr.com\t12:00\t0.9 Sneha\twww.myblog.com\t12:02\t0. 7 Karishma\twww.crap.com\t12:30\t0. 2 Split M (docid, text) (docid, text) (docid, text) Example Application Data Flow mapper (filename, file-contents): for each line in file-contents: fields = split(line, \t ) pg_rank = fields(3) emit (user, pg_rank) Map 1 Map i Map M (user, pg_rank) Job Configuration: # of Maps = M # of Reducers = R Karishma, 0.7 (sorted users, pg_ranks) Shuffle reducer (user, values): sum = 0 for each value in values: sum = sum + value avg_pg_rank = sum / sizeof(values) if avg_pg_rank > 0.5 emit (user, avg_pg_rank) Reduce 1 Reduce i Reduce R (sorted users, avg_pg_ranks) Anand, 0.8 (sorted users, avg_pg_ranks) Karishma, 0.6 (sorted users, avg_pg_ranks) Output File 1 Output File i Output File R

32 Thank You! References Hadoop wiki Hadoop Tutorial at Yahoo! Amazon Elastic MapReduce Google MapReduce paper Microsoft Dryad

33 Appendix

34 HDFS Distributes data across nodes; Reliability through replication Rack aware; Load balancing across nodes Name Node manages the file system metadata Data Node - Stores and serves blocks of data

Introduction to Cloud Computing

Introduction to Cloud Computing Introduction to Cloud Computing Cloud Computing I (intro) 15 319, spring 2010 2 nd Lecture, Jan 14 th Majd F. Sakr Lecture Motivation General overview on cloud computing What is cloud computing Services

More information

Introduction to Cloud Computing. Srinath Beldona srinath_beldona@yahoo.com

Introduction to Cloud Computing. Srinath Beldona srinath_beldona@yahoo.com Introduction to Cloud Computing Srinath Beldona srinath_beldona@yahoo.com Agenda Pre-requisites Course objectives What you will learn in this tutorial? Brief history Is cloud computing new? Why cloud computing?

More information

Chapter 7. Using Hadoop Cluster and MapReduce

Chapter 7. Using Hadoop Cluster and MapReduce Chapter 7 Using Hadoop Cluster and MapReduce Modeling and Prototyping of RMS for QoS Oriented Grid Page 152 7. Using Hadoop Cluster and MapReduce for Big Data Problems The size of the databases used in

More information

Where We Are. References. Cloud Computing. Levels of Service. Cloud Computing History. Introduction to Data Management CSE 344

Where We Are. References. Cloud Computing. Levels of Service. Cloud Computing History. Introduction to Data Management CSE 344 Where We Are Introduction to Data Management CSE 344 Lecture 25: DBMS-as-a-service and NoSQL We learned quite a bit about data management see course calendar Three topics left: DBMS-as-a-service and NoSQL

More information

Cloud Computing Trends

Cloud Computing Trends UT DALLAS Erik Jonsson School of Engineering & Computer Science Cloud Computing Trends What is cloud computing? Cloud computing refers to the apps and services delivered over the internet. Software delivered

More information

The Cloud at Crawford. Evaluating the pros and cons of cloud computing and its use in claims management

The Cloud at Crawford. Evaluating the pros and cons of cloud computing and its use in claims management The Cloud at Crawford Evaluating the pros and cons of cloud computing and its use in claims management The Cloud at Crawford Wikipedia defines cloud computing as Internet-based computing, whereby shared

More information

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com ` CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS Review Business and Technology Series www.cumulux.com Table of Contents Cloud Computing Model...2 Impact on IT Management and

More information

Cloud Computing Backgrounder

Cloud Computing Backgrounder Cloud Computing Backgrounder No surprise: information technology (IT) is huge. Huge costs, huge number of buzz words, huge amount of jargon, and a huge competitive advantage for those who can effectively

More information

Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh

Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 1 Hadoop: A Framework for Data- Intensive Distributed Computing CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 2 What is Hadoop? Hadoop is a software framework for distributed processing of large datasets

More information

Apache Hadoop: Past, Present, and Future

Apache Hadoop: Past, Present, and Future The 4 th China Cloud Computing Conference May 25 th, 2012. Apache Hadoop: Past, Present, and Future Dr. Amr Awadallah Founder, Chief Technical Officer aaa@cloudera.com, twitter: @awadallah Hadoop Past

More information

Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud

Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud Use case Figure 1: Company C Architecture (Before Migration) Company C is an automobile insurance claim processing company with

More information

Cloud computing - Architecting in the cloud

Cloud computing - Architecting in the cloud Cloud computing - Architecting in the cloud anna.ruokonen@tut.fi 1 Outline Cloud computing What is? Levels of cloud computing: IaaS, PaaS, SaaS Moving to the cloud? Architecting in the cloud Best practices

More information

Cloud Courses Description

Cloud Courses Description Cloud Courses Description Cloud 101: Fundamental Cloud Computing and Architecture Cloud Computing Concepts and Models. Fundamental Cloud Architecture. Virtualization Basics. Cloud platforms: IaaS, PaaS,

More information

Architectural Implications of Cloud Computing

Architectural Implications of Cloud Computing Architectural Implications of Cloud Computing Grace Lewis Research, Technology and Systems Solutions (RTSS) Program Lewis is a senior member of the technical staff at the SEI in the Research, Technology,

More information

References. Introduction to Database Systems CSE 444. Motivation. Basic Features. Outline: Database in the Cloud. Outline

References. Introduction to Database Systems CSE 444. Motivation. Basic Features. Outline: Database in the Cloud. Outline References Introduction to Database Systems CSE 444 Lecture 24: Databases as a Service YongChul Kwon Amazon SimpleDB Website Part of the Amazon Web services Google App Engine Datastore Website Part of

More information

Introduction to Database Systems CSE 444

Introduction to Database Systems CSE 444 Introduction to Database Systems CSE 444 Lecture 24: Databases as a Service YongChul Kwon References Amazon SimpleDB Website Part of the Amazon Web services Google App Engine Datastore Website Part of

More information

Chapter 3. Database Architectures and the Web Transparencies

Chapter 3. Database Architectures and the Web Transparencies Chapter 3 Database Architectures and the Web Transparencies Database Environment - Objectives The meaning of the client server architecture and the advantages of this type of architecture for a DBMS. The

More information

C2030-280.Examcollection.Premium.Exam.34q

C2030-280.Examcollection.Premium.Exam.34q C2030-280.Examcollection.Premium.Exam.34q Number: C2030-280 Passing Score: 800 Time Limit: 120 min File Version: 32.2 http://www.gratisexam.com/ Exam Code: C2030-280 Exam Name: IBM Cloud Computing Infrastructure

More information

Cloud Computing. Cloud computing:

Cloud Computing. Cloud computing: Cloud computing: Cloud Computing A model of data processing in which high scalability IT solutions are delivered to multiple users: as a service, on a mass scale, on the Internet. Network services offering:

More information

Double-Take Cloud Migration Center (CMC) Tech Brief

Double-Take Cloud Migration Center (CMC) Tech Brief Double-Take Cloud Migration Center (CMC) Tech Brief Overview Double-Take Cloud Migration Center is an online service that enables migrations from VMware vsphere and Amazon Web Services EC2 to Microsoft

More information

The Fastest Path to the Cloud Building Your SaaS Company on Force.com

The Fastest Path to the Cloud Building Your SaaS Company on Force.com The Fastest Path to the Cloud Building Your SaaS Company on Force.com Kai Mäkelä salesforce.com kmakela@salesforce.com Safe Harbor Safe harbor statement under the Private Securities Litigation Reform Act

More information

Cloud Storage: More than a File System in the Sky

Cloud Storage: More than a File System in the Sky Cloud Storage: More than a File System in the Sky Erik Riedel, PhD Technology & Architecture Cloud Infrastructure Group EMC 1 Development vs. Deployment The previously separate roles of software developer

More information

24/11/14. During this course. Internet is everywhere. Frequency barrier hit. Management costs increase. Advanced Distributed Systems Cloud Computing

24/11/14. During this course. Internet is everywhere. Frequency barrier hit. Management costs increase. Advanced Distributed Systems Cloud Computing Advanced Distributed Systems Cristian Klein Department of Computing Science Umeå University During this course Treads in IT Towards a new data center What is Cloud computing? Types of Clouds Making applications

More information

APP DEVELOPMENT ON THE CLOUD MADE EASY WITH PAAS

APP DEVELOPMENT ON THE CLOUD MADE EASY WITH PAAS APP DEVELOPMENT ON THE CLOUD MADE EASY WITH PAAS This article looks into the benefits of using the Platform as a Service paradigm to develop applications on the cloud. It also compares a few top PaaS providers

More information

CLOUD DEVELOPMENT BEST PRACTICES & SUPPORT APPLICATIONS

CLOUD DEVELOPMENT BEST PRACTICES & SUPPORT APPLICATIONS whitepaper CLOUD DEVELOPMENT BEST PRACTICES & SUPPORT APPLICATIONS - Cloud Development Best Practices and Support Applications CLOUD DEVELOPMENT BEST PRACTICES 1 Cloud-based solutions are increasingly

More information

Cloud Computing using MapReduce, Hadoop, Spark

Cloud Computing using MapReduce, Hadoop, Spark Cloud Computing using MapReduce, Hadoop, Spark Benjamin Hindman benh@cs.berkeley.edu Why this talk? At some point, you ll have enough data to run your parallel algorithms on multiple computers SPMD (e.g.,

More information

Emerging Technology for the Next Decade

Emerging Technology for the Next Decade Emerging Technology for the Next Decade Cloud Computing Keynote Presented by Charles Liang, President & CEO Super Micro Computer, Inc. What is Cloud Computing? Cloud computing is Internet-based computing,

More information

Cloud Courses Description

Cloud Courses Description Courses Description 101: Fundamental Computing and Architecture Computing Concepts and Models. Data center architecture. Fundamental Architecture. Virtualization Basics. platforms: IaaS, PaaS, SaaS. deployment

More information

Google Apps Overview

Google Apps Overview Google Apps Overview Agenda Messaging & collaboration challenges Why existing solutions are costly & limited Google s innovative approach The time is now to switch to the cloud Google Apps Premier Edition

More information

Lecture 02a Cloud Computing I

Lecture 02a Cloud Computing I Mobile Cloud Computing Lecture 02a Cloud Computing I 吳 秀 陽 Shiow-yang Wu What is Cloud Computing? Computing with cloud? Mobile Cloud Computing Cloud Computing I 2 Note 1 What is Cloud Computing? Walking

More information

Course 20465C: Designing a Data Solution with Microsoft SQL Server

Course 20465C: Designing a Data Solution with Microsoft SQL Server Course 20465C: Designing a Data Solution with Microsoft SQL Server Module 1: Introduction to Enterprise Data Architecture As organizations grow to enterprise scale, their IT infrastructure requirements

More information

Cloud Service Model. Selecting a cloud service model. Different cloud service models within the enterprise

Cloud Service Model. Selecting a cloud service model. Different cloud service models within the enterprise Cloud Service Model Selecting a cloud service model Different cloud service models within the enterprise Single cloud provider AWS for IaaS Azure for PaaS Force fit all solutions into the cloud service

More information

Apache Hadoop. Alexandru Costan

Apache Hadoop. Alexandru Costan 1 Apache Hadoop Alexandru Costan Big Data Landscape No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard, except Hadoop 2 Outline What is Hadoop? Who uses it? Architecture HDFS MapReduce Open

More information

Overview. The Cloud. Characteristics and usage of the cloud Realities and risks of the cloud

Overview. The Cloud. Characteristics and usage of the cloud Realities and risks of the cloud Overview The purpose of this paper is to introduce the reader to the basics of cloud computing or the cloud with the aim of introducing the following aspects: Characteristics and usage of the cloud Realities

More information

Operationalize Policies. Take Action. Establish Policies. Opportunity to use same tools and practices from desktop management in server environment

Operationalize Policies. Take Action. Establish Policies. Opportunity to use same tools and practices from desktop management in server environment Microsoft IT Systems Strategy DATACENTER APPLICATIONS Process-Led, Model-Driven Unified and Virtualized USER-CENTRICITY ServiceEnabled Basic Standardize d Rationalized Dynamic Cost center Efficient cost

More information

Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum

Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum Trends and Research Opportunities in Spatial Big Data Analytics and Cloud Computing NCSU GeoSpatial Forum Siva Ravada Senior Director of Development Oracle Spatial and MapViewer 2 Evolving Technology Platforms

More information

Virtualizing Apache Hadoop. June, 2012

Virtualizing Apache Hadoop. June, 2012 June, 2012 Table of Contents EXECUTIVE SUMMARY... 3 INTRODUCTION... 3 VIRTUALIZING APACHE HADOOP... 4 INTRODUCTION TO VSPHERE TM... 4 USE CASES AND ADVANTAGES OF VIRTUALIZING HADOOP... 4 MYTHS ABOUT RUNNING

More information

Cloud Computing Now and the Future Development of the IaaS

Cloud Computing Now and the Future Development of the IaaS 2010 Cloud Computing Now and the Future Development of the IaaS Quanta Computer Division: CCASD Title: Project Manager Name: Chad Lin Agenda: What is Cloud Computing? Public, Private and Hybrid Cloud.

More information

Cloud Computing Paradigm

Cloud Computing Paradigm Cloud Computing Paradigm Julio Guijarro Automated Infrastructure Lab HP Labs Bristol, UK 2008 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

More information

BEDIFFERENT A C E 2 0 1 2 I N T E R N A T I O N A L

BEDIFFERENT A C E 2 0 1 2 I N T E R N A T I O N A L Copyright 2012 Aras. All Rights Reserved. BEDIFFERENT A C E 2 0 1 2 I N T E R N A T I O N A L Copyright 2012 Aras. All Rights Reserved. ACE 2012 I N TERNATIONAL Leveraging the Cloud Rob McAveney Director

More information

Hadoop Introduction. Olivier Renault Solution Engineer - Hortonworks

Hadoop Introduction. Olivier Renault Solution Engineer - Hortonworks Hadoop Introduction Olivier Renault Solution Engineer - Hortonworks Hortonworks A Brief History of Apache Hadoop Apache Project Established Yahoo! begins to Operate at scale Hortonworks Data Platform 2013

More information

Cloud Computing, Virtualization & Green IT

Cloud Computing, Virtualization & Green IT Cloud Computing, Virtualization & Green IT Cloud computing can change how IT supports business Consider the following: As much as 85% of computing capacity sits idle in distributed computing environments.

More information

Cloud Optimize Your IT

Cloud Optimize Your IT Cloud Optimize Your IT Windows Server 2012 The information contained in this presentation relates to a pre-release product which may be substantially modified before it is commercially released. This pre-release

More information

INTRODUCTION TO CLOUD COMPUTING CEN483 PARALLEL AND DISTRIBUTED SYSTEMS

INTRODUCTION TO CLOUD COMPUTING CEN483 PARALLEL AND DISTRIBUTED SYSTEMS INTRODUCTION TO CLOUD COMPUTING CEN483 PARALLEL AND DISTRIBUTED SYSTEMS CLOUD COMPUTING Cloud computing is a model for enabling convenient, ondemand network access to a shared pool of configurable computing

More information

Public Cloud Offerings and Private Cloud Options. Week 2 Lecture 4. M. Ali Babar

Public Cloud Offerings and Private Cloud Options. Week 2 Lecture 4. M. Ali Babar Public Cloud Offerings and Private Cloud Options Week 2 Lecture 4 M. Ali Babar Lecture Outline Public and private clouds Some key public cloud providers (More details in the lab) Private clouds Main Aspects

More information

MS 20465C: Designing a Data Solution with Microsoft SQL Server

MS 20465C: Designing a Data Solution with Microsoft SQL Server MS 20465C: Designing a Data Solution with Microsoft SQL Server Description: Note: Days: 5 Prerequisites: The focus of this five-day instructor-led course is on planning and implementing enterprise database

More information

BIG DATA TRENDS AND TECHNOLOGIES

BIG DATA TRENDS AND TECHNOLOGIES BIG DATA TRENDS AND TECHNOLOGIES THE WORLD OF DATA IS CHANGING Cloud WHAT IS BIG DATA? Big data are datasets that grow so large that they become awkward to work with using onhand database management tools.

More information

DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING. Carlos de Alfonso Andrés García Vicente Hernández

DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING. Carlos de Alfonso Andrés García Vicente Hernández DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING Carlos de Alfonso Andrés García Vicente Hernández 2 INDEX Introduction Our approach Platform design Storage Security

More information

Today: Data Centers & Cloud Computing" Data Centers"

Today: Data Centers & Cloud Computing Data Centers Today: Data Centers & Cloud Computing" Data Centers Cloud Computing Lecture 25, page 1 Data Centers" Large server and storage farms Used by enterprises to run server applications Used by Internet companies

More information

Sriram Krishnan, Ph.D. sriram@sdsc.edu

Sriram Krishnan, Ph.D. sriram@sdsc.edu Sriram Krishnan, Ph.D. sriram@sdsc.edu (Re-)Introduction to cloud computing Introduction to the MapReduce and Hadoop Distributed File System Programming model Examples of MapReduce Where/how to run MapReduce

More information

What Is It? Business Architecture Research Challenges Bibliography. Cloud Computing. Research Challenges Overview. Carlos Eduardo Moreira dos Santos

What Is It? Business Architecture Research Challenges Bibliography. Cloud Computing. Research Challenges Overview. Carlos Eduardo Moreira dos Santos Research Challenges Overview May 3, 2010 Table of Contents I 1 What Is It? Related Technologies Grid Computing Virtualization Utility Computing Autonomic Computing Is It New? Definition 2 Business Business

More information

Server Virtualization Cloud Partner Training Series

Server Virtualization Cloud Partner Training Series Server Virtualization Cloud Partner Training Series August 2015 Agenda What is it? Benefits Industry Landscape Evolve is Different How we can help you $ucceed Who is a fit? Who is a challenge? Key Questions

More information

Cloud Technologies and GIS Nathalie Smith

Cloud Technologies and GIS Nathalie Smith Cloud Technologies and GIS Nathalie Smith nsmith@esri.com Agenda What is Cloud Computing? How does it work? Cloud and GIS applications Esri Offerings Lots of hype Cloud computing remains the latest, most

More information

The Impact of PaaS on Business Transformation

The Impact of PaaS on Business Transformation The Impact of PaaS on Business Transformation September 2014 Chris McCarthy Sr. Vice President Information Technology 1 Legacy Technology Silos Opportunities Business units Infrastructure Provisioning

More information

Data movement for globally deployed Big Data Hadoop architectures

Data movement for globally deployed Big Data Hadoop architectures Data movement for globally deployed Big Data Hadoop architectures Scott Rudenstein VP Technical Services November 2015 WANdisco Background WANdisco: Wide Area Network Distributed Computing " Enterprise

More information

Harnessing the Power of the Microsoft Cloud for Deep Data Analytics

Harnessing the Power of the Microsoft Cloud for Deep Data Analytics 1 Harnessing the Power of the Microsoft Cloud for Deep Data Analytics Today's Focus How you can operate your business more efficiently and effectively by tapping into Cloud based data analytics solutions

More information

SPM rollouts in Large Ent erprise: different iat ing exist ing cloud architectures

SPM rollouts in Large Ent erprise: different iat ing exist ing cloud architectures SPM rollouts in Large Ent erprise: different iat ing exist ing cloud architectures 1 Table of contents Why this white paper?... 3 SPM for SMEs vs. SPM for LEs... 3 Why a multi-tenant and not single-tenant

More information

Making a Smooth Transition to a Hybrid Cloud with Microsoft Cloud OS

Making a Smooth Transition to a Hybrid Cloud with Microsoft Cloud OS Making a Smooth Transition to a Hybrid Cloud with Microsoft Cloud OS Transitioning from today s highly virtualized data center environments to a true cloud environment requires solutions that let companies

More information

Designing a Data Solution with Microsoft SQL Server

Designing a Data Solution with Microsoft SQL Server The focus of this five-day instructor-led course is on planning and implementing enterprise database infrastructure solutions by using SQL Server 2014 and other Microsoft technologies. It describes how

More information

IBM Spectrum Protect in the Cloud

IBM Spectrum Protect in the Cloud IBM Spectrum Protect in the Cloud. Disclaimer IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM s sole discretion. Information regarding

More information

Role of Cloud Computing in Big Data Analytics Using MapReduce Component of Hadoop

Role of Cloud Computing in Big Data Analytics Using MapReduce Component of Hadoop Role of Cloud Computing in Big Data Analytics Using MapReduce Component of Hadoop Kanchan A. Khedikar Department of Computer Science & Engineering Walchand Institute of Technoloy, Solapur, Maharashtra,

More information

Oracle Database - Engineered for Innovation. Sedat Zencirci Teknoloji Satış Danışmanlığı Direktörü Türkiye ve Orta Asya

Oracle Database - Engineered for Innovation. Sedat Zencirci Teknoloji Satış Danışmanlığı Direktörü Türkiye ve Orta Asya Oracle Database - Engineered for Innovation Sedat Zencirci Teknoloji Satış Danışmanlığı Direktörü Türkiye ve Orta Asya Oracle Database 11g Release 2 Shipping since September 2009 11.2.0.3 Patch Set now

More information

Designing a Data Solution with Microsoft SQL Server 2014

Designing a Data Solution with Microsoft SQL Server 2014 Page 1 of 8 Overview The focus of this five-day instructor-led course is on planning and implementing enterprise database infrastructure solutions by using SQL Server 2014 and other Microsoft technologies.

More information

Introduction to Cloud : Cloud and Cloud Storage. Lecture 2. Dr. Dalit Naor IBM Haifa Research Storage Systems. Dalit Naor, IBM Haifa Research

Introduction to Cloud : Cloud and Cloud Storage. Lecture 2. Dr. Dalit Naor IBM Haifa Research Storage Systems. Dalit Naor, IBM Haifa Research Introduction to Cloud : Cloud and Cloud Storage Lecture 2 Dr. Dalit Naor IBM Haifa Research Storage Systems 1 Advanced Topics in Storage Systems for Big Data - Spring 2014, Tel-Aviv University http://www.eng.tau.ac.il/semcom

More information

How cloud computing can transform your business landscape

How cloud computing can transform your business landscape How cloud computing can transform your business landscape Introduction It seems like everyone is talking about the cloud. Cloud computing and cloud services are the new buzz words for what s really a not

More information

Microsoft Private Cloud

Microsoft Private Cloud Microsoft Private Cloud Lorenz Wolf, Solution Specialist Datacenter, Microsoft SoftwareOne @ Au Premier Zürich - 22.03.2011 What is PRIVATE CLOUD Private Public Public Cloud Private Cloud shared resources.

More information

Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings

Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Solution Brief Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Introduction Accelerating time to market, increasing IT agility to enable business strategies, and improving

More information

Cloud Based Document Management

Cloud Based Document Management Cloud Based Document Management WHY IS THE CLOUD IMPORTANT? The Information Explosion It is the information that is setting up competitive differentiation, not specifically products and processes. It is

More information

Course 20465: Designing a Data Solution with Microsoft SQL Server

Course 20465: Designing a Data Solution with Microsoft SQL Server Course 20465: Designing a Data Solution with Microsoft SQL Server Overview About this course The focus of this five-day instructor-led course is on planning and implementing enterprise database infrastructure

More information

Introduction to Hadoop HDFS and Ecosystems. Slides credits: Cloudera Academic Partners Program & Prof. De Liu, MSBA 6330 Harvesting Big Data

Introduction to Hadoop HDFS and Ecosystems. Slides credits: Cloudera Academic Partners Program & Prof. De Liu, MSBA 6330 Harvesting Big Data Introduction to Hadoop HDFS and Ecosystems ANSHUL MITTAL Slides credits: Cloudera Academic Partners Program & Prof. De Liu, MSBA 6330 Harvesting Big Data Topics The goal of this presentation is to give

More information

Designing a Data Solution with Microsoft SQL Server

Designing a Data Solution with Microsoft SQL Server Course 20465C: Designing a Data Solution with Microsoft SQL Server Page 1 of 6 Designing a Data Solution with Microsoft SQL Server Course 20465C: 4 days; Instructor-Led Introduction The focus of this four-day

More information

Manage Your Data: Virtualization for Small Businesses

Manage Your Data: Virtualization for Small Businesses Manage Your Data: Virtualization for Small Businesses July 2012 Contents: Introduction 3 What is Virtualization 4 What Could Virtualization Mean for your Business 5 Is Server Virtualization Right for your

More information

20465: Designing a Data Solution with Microsoft SQL Server

20465: Designing a Data Solution with Microsoft SQL Server 20465: Designing a Data Solution with Microsoft SQL Server Microsoft - Base de Dados Nível: Avançado Duração: 30h Sobre o curso The focus of this five-day instructor-led course is on planning and implementing

More information

BMC Control-M for Cloud. BMC Control-M Workload Automation

BMC Control-M for Cloud. BMC Control-M Workload Automation BMC Control-M for Cloud BMC Control-M Workload Automation Virtualization & Cloud Computing Are Top Priorities Cloud Computing is a catalyst for improving IT maturity and moving virtualization to another

More information

Digital Signage in the Cloud

Digital Signage in the Cloud Hosted by: Digital Signage in the Cloud Doug Bannister Founder, CEO, Director of Software Development Omnivex Digital Signage Software dbannister@omnivex.com www.omnivex.com Copyright 2012 Omnivex Corporation.

More information

NEXT UP: John Sanderson, Windows Azure Specialist johnsand@microsoft.com 303-546-1397 (Denver) Page 1

NEXT UP: John Sanderson, Windows Azure Specialist johnsand@microsoft.com 303-546-1397 (Denver) Page 1 NEXT UP: John Sanderson, Windows Azure Specialist johnsand@microsoft.com 303-546-1397 (Denver) Page 1 The Cloud The Foundation for Private Cloud A Platform for ISV s Page 2 Cloud Computing Landscape Page

More information

Case Studies: Protecting Sensitive Data in

Case Studies: Protecting Sensitive Data in Case Studies: Protecting Sensitive Data in C.J. Radford Vice President, Cloud September 18, 2014 Contact: @cjrad; cradford@vormetric.com Agenda Data Security Challenges Top Considerations for Data Centric

More information

Cloud Computing and Big Data Analytics for Teaching & Research

Cloud Computing and Big Data Analytics for Teaching & Research Cloud Computing and Big Data Analytics for Teaching & Research Qusay H. Mahmoud, Ph.D., P.Eng. Professor & Chair Department of Electrical, Computer and Software Engineering University of Ontario Institute

More information

Data Centers and Cloud Computing. Data Centers. MGHPCC Data Center. Inside a Data Center

Data Centers and Cloud Computing. Data Centers. MGHPCC Data Center. Inside a Data Center Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises

More information

SaaS A Product Perspective

SaaS A Product Perspective SaaS A Product Perspective Software-as-a-Service (SaaS) is quickly gaining credibility and market share against traditional packaged software. This presents new opportunities for product groups and also

More information

Implement Hadoop jobs to extract business value from large and varied data sets

Implement Hadoop jobs to extract business value from large and varied data sets Hadoop Development for Big Data Solutions: Hands-On You Will Learn How To: Implement Hadoop jobs to extract business value from large and varied data sets Write, customize and deploy MapReduce jobs to

More information

TRANSFORMING DATA PROTECTION

TRANSFORMING DATA PROTECTION TRANSFORMING DATA PROTECTION Moving from Reactive to Proactive Mark Galpin 1 Our Protection Strategy: Best Of Breed Performance LEADER HIGH-END STORAGE VMAX Low Service Level LEADER SCALE-OUT NAS STORAGE

More information

Hadoop IST 734 SS CHUNG

Hadoop IST 734 SS CHUNG Hadoop IST 734 SS CHUNG Introduction What is Big Data?? Bulk Amount Unstructured Lots of Applications which need to handle huge amount of data (in terms of 500+ TB per day) If a regular machine need to

More information

ArcGIS for Server in the Amazon Cloud. Michele Lundeen Esri

ArcGIS for Server in the Amazon Cloud. Michele Lundeen Esri ArcGIS for Server in the Amazon Cloud Michele Lundeen Esri What we will cover ArcGIS for Server in the Amazon Cloud Why How Extras Why do you need ArcGIS Server? Some examples Publish - Dynamic Map Services

More information

Migrating SaaS Applications to Windows Azure

Migrating SaaS Applications to Windows Azure Migrating SaaS Applications to Windows Azure Lessons Learned 04.04.2012 Speaker Introduction Deepthi Raju Marketing Technology Services Deepthi joined Smartbridge in 2005 and has over twenty years of technology

More information

CLOUD STORAGE USING HADOOP AND PLAY

CLOUD STORAGE USING HADOOP AND PLAY 27 CLOUD STORAGE USING HADOOP AND PLAY Devateja G 1, Kashyap P V B 2, Suraj C 3, Harshavardhan C 4, Impana Appaji 5 1234 Computer Science & Engineering, Academy for Technical and Management Excellence

More information

Cloud computing an insight

Cloud computing an insight Cloud computing an insight Overview IT infrastructure is changing according the fast-paced world s needs. People in the world want to stay connected with Work / Family-Friends. The data needs to be available

More information

Simplified Private Cloud Management

Simplified Private Cloud Management BUSINESS PARTNER ClouTor Simplified Private Cloud Management ClouTor ON VSPEX by LOCUZ INTRODUCTION ClouTor on VSPEX for Enterprises provides an integrated software solution for extending your existing

More information

AVLOR SERVER CLOUD RECOVERY

AVLOR SERVER CLOUD RECOVERY AVLOR SERVER CLOUD RECOVERY WHITE PAPER 1 Table of Contents Abstract... 2 1. Introduction... 3 2. Server Cloud Recovery... 3 3. Amazon AWS Cloud... 4 a. What it is... 4 b. Why Use AWS?... 5 4. Difficulties

More information

Beyond Web Application Log Analysis using Apache TM Hadoop. A Whitepaper by Orzota, Inc.

Beyond Web Application Log Analysis using Apache TM Hadoop. A Whitepaper by Orzota, Inc. Beyond Web Application Log Analysis using Apache TM Hadoop A Whitepaper by Orzota, Inc. 1 Web Applications As more and more software moves to a Software as a Service (SaaS) model, the web application has

More information

INTRODUCTION TO CASSANDRA

INTRODUCTION TO CASSANDRA INTRODUCTION TO CASSANDRA This ebook provides a high level overview of Cassandra and describes some of its key strengths and applications. WHAT IS CASSANDRA? Apache Cassandra is a high performance, open

More information

9/26/2011. What is Virtualization? What are the different types of virtualization.

9/26/2011. What is Virtualization? What are the different types of virtualization. CSE 501 Monday, September 26, 2011 Kevin Cleary kpcleary@buffalo.edu What is Virtualization? What are the different types of virtualization. Practical Uses Popular virtualization products Demo Question,

More information

Microsoft Azure. DataSync. ModernBiz

Microsoft Azure. DataSync. ModernBiz Microsoft Azure DataSync ModernBiz Agenda ModernBiz: Your path to the Modern Business Microsoft Azure overview Key business solutions leveraging the cloud Back up your data Reduce unexpected downtime Host

More information

Introduction to Cloud Computing

Introduction to Cloud Computing 1 Introduction to Cloud Computing CERTIFICATION OBJECTIVES 1.01 Cloud Computing: Common Terms and Definitions 1.02 Cloud Computing and Virtualization 1.03 Early Examples of Cloud Computing 1.04 Cloud Computing

More information

Cloud Models and Platforms

Cloud Models and Platforms Cloud Models and Platforms Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF A Working Definition of Cloud Computing Cloud computing is a model

More information

Hadoop: Distributed Data Processing. Amr Awadallah Founder/CTO, Cloudera, Inc. ACM Data Mining SIG Thursday, January 25 th, 2010

Hadoop: Distributed Data Processing. Amr Awadallah Founder/CTO, Cloudera, Inc. ACM Data Mining SIG Thursday, January 25 th, 2010 Hadoop: Distributed Data Processing Amr Awadallah Founder/CTO, Cloudera, Inc. ACM Data Mining SIG Thursday, January 25 th, 2010 Outline Scaling for Large Data Processing What is Hadoop? HDFS and MapReduce

More information

Cloud Computing for SCADA

Cloud Computing for SCADA Cloud Computing for SCADA Moving all or part of SCADA applications to the cloud can cut costs significantly while dramatically increasing reliability and scalability. A White Paper from InduSoft Larry

More information

Oracle Database Cloud

Oracle Database Cloud Oracle Database Cloud Shakeeb Rahman Database Cloud Service Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may

More information

Data Centers and Cloud Computing. Data Centers

Data Centers and Cloud Computing. Data Centers Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing 1 Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises

More information

DATA SECURITY MODEL FOR CLOUD COMPUTING

DATA SECURITY MODEL FOR CLOUD COMPUTING DATA SECURITY MODEL FOR CLOUD COMPUTING POOJA DHAWAN Assistant Professor, Deptt of Computer Application and Science Hindu Girls College, Jagadhri 135 001 poojadhawan786@gmail.com ABSTRACT Cloud Computing

More information