Background on Elastic Compute Cloud (EC2) AMI s to choose from including servers hosted on different Linux distros

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Background on Elastic Compute Cloud (EC2) AMI s to choose from including servers hosted on different Linux distros"

Transcription

1 David Moses January 2014 Paper on Cloud Computing I Background on Tools and Technologies in Amazon Web Services (AWS) In this paper I will highlight the technologies from the AWS cloud which enable you to create a data warehouse. I will also discuss tools that can be used with AWS technologies for building a data warehouse. The technologies that I will discuss in this paper from AWS include Redshift, EC2, S3, EMR, and EBS. The tools that can be used with AWS that I will discuss in this paper include Tableau, D3.js, SQL workbench and Pentaho s Kettle software. I will also discuss the future directions of cloud computing. Background on Elastic Compute Cloud (EC2) The elastic compute cloud (EC2) holds computer servers hosted in Amazon s AWS cloud. An EC2 instance can be started by choosing an Amazon Machine Image (AMI) from Amazon s AWS website ( There are thousands of AMI s to choose from including servers hosted on different Linux distros (e.g. Ubuntu Linux) and on different versions of the Windows operating system. Most of the AMI s for EC2 have been configured with configurations for special purpose use. Amazon s EC2 is the core of AWS s cloud. Before you can start many of the features of EC2 it is necessary to setup security for your EC2 instance and choose an AMI for your EC2 instance.

2 Key pairs are one of the ways that Amazon handles security. A Key pair, which enable security of your EC2 instance, is the only way that you can access your EC2 instance. You can create a key pair for your AWS instance in the AWS console. Once you create a key pair you need to associate it with your EC2 instance. The private key needs to be downloaded and also needs to be stored safely to your local computer. After selecting the EC2 AMI and associating the AMI with your key pair you can now launch the EC2 instance. Background on Simple Storage Service (S3) S3 can be used for building a data warehouse by storing files in CSV format in an S3 bucket. An S3 bucket stores the files that you will use to load into a database (e.g. Redshift). The S3 bucket names must be unique across all S3 bucket names used in S3. S3 uses a REST service and you can send requests to Amazon S3 using its REST API. Every interaction with Amazon S3 is either authenticated or anonymous. Authentication is a process of verifying the identity of the requester trying to access the AWS product. Authenticated requests must include a signature value that authenticates the request sender. The signature value is, in part, generated from the requester's AWS access keys (access key ID and secret access key). If you are using the AWS SDK, the libraries compute the signature from the keys you provide. However, if you make direct REST API calls in your application, you must write the code to compute the signature and add it to the request. (Referenced:

3 Background on Elastic Map Reduce (EMR) Amazon s EMR simplifies running Hadoop and related big data applications on AWS. You can use it to manage and analyze vast amounts of data. For example, a cluster, can be configured to process terabytes or even petabytes of data. In order to develop and deploy custom Hadoop applications, you once needed to have access to a lot of hardware for your Hadoop programs. EMR makes it easy to spin up a set of EC2 instances as virtual servers to run your Hadoop cluster. Amazon s EMR allows you to easily deploy your production clusters. Once you are done with the development and testing phases of your project you can easily terminate unused testing clusters. By running your cluster on Amazon s EMR, you only pay for the server resources that you use. For example, if the amount of data you process in a daily cluster peaks on Monday, you can increase the number of servers to 25 in the cluster that day, and then scale back to 5 servers in the clusters that run on other days of the week. You won't have to pay to maintain any additional servers during the rest of the week as you would with physical servers. Amazon s EMR is integrated with other Amazon Web Services such as Amazon EC2, Amazon S3, Amazon DynamoDB, Amazon RDS, CloudWatch, and AWS Data Pipeline. Which means that you can easily access data stored in AWS from your cluster and you can make use of the functionality offered by other AWS services to manage your cluster and store the output of your cluster. (Referenced: dg.pdf)

4 Background on Code Deployment using Elastic Beanstalk (EBS) Amazon s Elastic Beanstalk allows you to deploy and scale services and applications developed with Java,.NET and other programming languages. To deploy a Java application using EBS you first create your application and package it as a web application archive file (WAR file). The next step is to upload the WAR file to Elastic Beanstalk using a web service API, the AWS management console, the AWS toolkit for Eclipse, or the command line interface. Once the WAR file has been uploaded it can then be deployed by EBS. EBS handles the deployment itself and automatically will provision server capacity, load balancing, auto- scaling, and application health monitoring. (Referenced: Functionality of EBS includes: (a) The ability to quickly and easily deploy new applications to a running environment. (b) Access to CloudWatch for monitoring the average CPU utilization, average latency and request count (c) notifications through Simple Notification Service (SNS) about changes to the health of an application or when application servers have been added or removed (d) The ability to quickly restart application servers on EC2 instances with a single command (e) Server log file access without needing to first login to the application servers. (Referenced: Background on ETL Tools Used in AWS To demonstrate an ETL tool that can be used with AWS, I will briefly describe the different methods by which Pentaho s Kettle ETL tool can be used to extract data

5 from different data sources, and then I will describe how that data can be transformed and loaded into an AWS database (e.g. Redshift). Kettle is packaged with several software tools including Spoon, Kitchen, and Pan. Spoon provides an IDE for doing transformations from one data type to another data type. For example, by using Spoon you can create a reusable process for transforming a CSV file into an XML file. Kettle s Spoon IDE provides an interface where ETL developers can create ETL jobs for their project. An ETL transformation, which is the primary ETL job, handles the manipulation of the rows or data of a dataset. (P. 25 of Pentaho Kettle s Solution). In Spoon the transformation steps are connected by transformation hops. A hop can best be described as a one- way channel that allows data to flow between the steps that are connected by the hop. Once a transformation has been created in Spoon you can then launch a Kettle job from the command line in Linux using Kettle tools such as Kitchen and Pan. Kitchen and Pan work by interpreting command- line parameters and invoking the Kettle engine to launch a job or transformation. (P. 322 of Pentaho Kettle Solutions). Kitchen can be started by using a shell script from the Kettle home directory in Linux. The scripts in Linux must be made executable first before they can be run. Scheduling a Kettle job with Kitchen can be done with the Cron utility. This can be done by simply adding entries for Cron jobs to the crontab file. (P.326 of Pentaho Kettle Solutions). Background on the Redshift Database

6 Redshift is a petabyte scale database cluster from AWS, which can be launched from within the Amazon Redshift management console. Before launching the Redshift database cluster you must select your parameter group, encryption option, VPC if you choose, as well as the availability zone. The Virtual private cloud (VPC) is a networking configuration that allows one to enable network isolation within the public portion of the cloud. After setting the configuration settings and completing the initialization for the cluster you can now launch the cluster. (Referenced: Getting Started with Amazon Redshift) The Redshift database uses PostgreSQL drivers and the core of the database is PostgreSQL. Although the Redshift database core is PostgreSQL, there are many features that have been removed for performance reasons. As with the design of other data warehouses it is not necessary to assign primary keys in Redshift. Assigning a primary key in Redshift will, however, be used by the optimizer to make good decisions about how to access the data. (Getting Started with Amazon Redshift) You can access Redshift from EC2 by running the connection string for Redshift as well as from the SQL Workbench IDE. To access Redshift from SQL Workbench you need the connection string that you used to connect to Redshift in EC2 and your Redshift credentials that you used when launching the Redshift cluster. You will also need to provide SQL Workbench with the PostgreSQL JDBC driver that is needed to connect to Redshift. The PostgreSQL JDBC can be downloaded from

7 Redshift is an example of a column oriented database or C- Store. A C- Store database differs most notably from relational databases in that it stores data in columns rather than in rows. In so doing it optimizes the database for reading data instead of writing data. (Referenced Store). The research and development of the first column oriented database, Vertica, was done by Professor Stonebraker and his colleagues at MIT in 2005 and led to the commercialization of the Vertica database by Stonebraker. Background on Business Intelligence (BI) Tools Used in AWS (A) Tableau BI Reports Business intelligence (BI) reporting software can be used with cloud based databases such as AWS s Redshift database. Using Tableau with Redshift you can create most of the types of BI reports included with the Tableau software. Tableau reports allow one to visualize trends and spot anomalies in the data. With big data sets it is very hard to do trend analysis and spot anomalies without BI reports that can visualize big data sets. Tableau reports have commonly been used in marketing and finance industries. An example of Tableau report that has been used in sales and marketing includes creating a Tableau dashboard to view Salesforce data of a sales team. In the finance industry Tableau has commonly been used for creating wealth management dashboards and in creating dashboards which analyze publicly traded company stock prices over a given time horizon. (Referenced URL:

8 (B) Custom BI Reports Custom BI reports can be created with Javascript frameworks including D3.js and ExtJS. Creating custom BI reports can be done by coding the design of the report with D3.js and consuming the JSON data from a Restful API. A web server such as Tomcat needs to be installed on the EC2 instance so that the report can be viewed on the internet. III Future Directions and opportunities for AWS and Cloud Computing The future of AWS and of cloud computing is quite bright. The interest level in hosting an application in the cloud has been and continues to be high for three reasons. First, the cost of hosting an application in the cloud is substantially less than the cost of buying the hardware and hosting the application yourself. Secondly, the technologies available to you in the cloud allow you to quickly and easily deploy a website, data warehouse, or other application. Third, the cloud computing space is evolving quite fast and companies including Amazon are constantly developing new tools and technologies in the space which is good for companies that wish to host an application with a cloud provider. To prove that AWS is constantly evolving this month alone AWS released the services Appstream, Amazon Workspaces, and Amazon Cloudtrail. Appsream, a media and interactive content streaming service, is oriented toward streaming content for applications that require high- definition video and features like interactivity and user authentication.

9 Amazon Workspaces is Amazon s entry into the virtual desktop market. Amazon Workspaces pricing structure is similar to that of other AWS products and services and you pay as you use it. The AWS virtual desktops are Windows 7- like instances and they can be accessed from a variety of devices: Mac OS and Windows desktops, as well as ipad, Kindle Fire, and Android tablets. Amazon Cloudtrail is a tracking service which logs all of your Amazon API traffic associated with a given account. Every single command line call, SDK call, and even calls initiated from within the Amazon management console are logged.

Chapter 9 PUBLIC CLOUD LABORATORY. Sucha Smanchat, PhD. Faculty of Information Technology. King Mongkut s University of Technology North Bangkok

Chapter 9 PUBLIC CLOUD LABORATORY. Sucha Smanchat, PhD. Faculty of Information Technology. King Mongkut s University of Technology North Bangkok CLOUD COMPUTING PRACTICE 82 Chapter 9 PUBLIC CLOUD LABORATORY Hand on laboratory based on AWS Sucha Smanchat, PhD Faculty of Information Technology King Mongkut s University of Technology North Bangkok

More information

19.10.11. Amazon Elastic Beanstalk

19.10.11. Amazon Elastic Beanstalk 19.10.11 Amazon Elastic Beanstalk A Short History of AWS Amazon started as an ECommerce startup Original architecture was restructured to be more scalable and easier to maintain Competitive pressure for

More information

Thing Big: How to Scale Your Own Internet of Things. Walter'Pernstecher'-'pernstec@amazon.de' Dr.'Markus'Schmidberger'-'schmidbe@amazon.

Thing Big: How to Scale Your Own Internet of Things. Walter'Pernstecher'-'pernstec@amazon.de' Dr.'Markus'Schmidberger'-'schmidbe@amazon. Thing Big: How to Scale Your Own Internet of Things Walter'Pernstecher'-'pernstec@amazon.de' Dr.'Markus'Schmidberger'-'schmidbe@amazon.de' Internet of Things is the network of physical objects or "things"

More information

Easy to begin and impossible to outgrow. An even easier way for developers to quickly deploy and manage applications in the AWS cloud.

Easy to begin and impossible to outgrow. An even easier way for developers to quickly deploy and manage applications in the AWS cloud. Agenda 1. What is AWS Elastic Beanstalk? 2. How do you use Elastic Beanstalk? 3. What does Elastic Beanstalk cost? 4. How does Elastic Beanstalk work? 5. Demo 6. Q&A 1. WHAT IS AWS ELASTIC BEANSTALK? Easy

More information

Every Silver Lining Has a Vault in the Cloud

Every Silver Lining Has a Vault in the Cloud Irvin Hayes Jr. Autodesk, Inc. PL6015-P Don t worry about acquiring hardware and additional personnel in order to manage your Vault software installation. Learn how to spin up a hosted server instance

More information

www.boost ur skills.com

www.boost ur skills.com www.boost ur skills.com AWS CLOUD COMPUTING WORKSHOP Write us at training@boosturskills.com BOOSTURSKILLS No 1736 1st Amrutha College Road Kasavanhalli,Off Sarjapur Road,Bangalore-35 1) Introduction &

More information

DLT Solutions and Amazon Web Services

DLT Solutions and Amazon Web Services DLT Solutions and Amazon Web Services For a seamless, cost-effective migration to the cloud PREMIER CONSULTING PARTNER DLT Solutions 2411 Dulles Corner Park, Suite 800 Herndon, VA 20171 Duane Thorpe Phone:

More information

Cloud Computing. Adam Barker

Cloud Computing. Adam Barker Cloud Computing Adam Barker 1 Overview Introduction to Cloud computing Enabling technologies Different types of cloud: IaaS, PaaS and SaaS Cloud terminology Interacting with a cloud: management consoles

More information

AIST Data Symposium. Ed Lenta. Managing Director, ANZ Amazon Web Services

AIST Data Symposium. Ed Lenta. Managing Director, ANZ Amazon Web Services AIST Data Symposium Ed Lenta Managing Director, ANZ Amazon Web Services Why are companies adopting cloud computing and AWS so quickly? #1: Agility The primary reason businesses are moving so quickly to

More information

Introduction to Amazon Web Services! Leo Zhadanovsky! @leozh leo@amazon.com! Senior Solutions Architect

Introduction to Amazon Web Services! Leo Zhadanovsky! @leozh leo@amazon.com! Senior Solutions Architect Introduction to Amazon Web Services! Leo Zhadanovsky! @leozh leo@amazon.com! Senior Solutions Architect AWS HISTORY About How didamazon Amazon Web Services! Deep experience in building and operating global

More information

Razvoj Java aplikacija u Amazon AWS Cloud: Praktična demonstracija

Razvoj Java aplikacija u Amazon AWS Cloud: Praktična demonstracija Razvoj Java aplikacija u Amazon AWS Cloud: Praktična demonstracija Robert Dukarić University of Ljubljana Faculty of Computer and Information Science Laboratory for information systems integration Competence

More information

Hadoop & Spark Using Amazon EMR

Hadoop & Spark Using Amazon EMR Hadoop & Spark Using Amazon EMR Michael Hanisch, AWS Solutions Architecture 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda Why did we build Amazon EMR? What is Amazon EMR?

More information

QLIKVIEW INTEGRATION TION WITH AMAZON REDSHIFT John Park Partner Engineering

QLIKVIEW INTEGRATION TION WITH AMAZON REDSHIFT John Park Partner Engineering QLIKVIEW INTEGRATION TION WITH AMAZON REDSHIFT John Park Partner Engineering June 2014 Page 1 Contents Introduction... 3 About Amazon Web Services (AWS)... 3 About Amazon Redshift... 3 QlikView on AWS...

More information

Alfresco Enterprise on AWS: Reference Architecture

Alfresco Enterprise on AWS: Reference Architecture Alfresco Enterprise on AWS: Reference Architecture October 2013 (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 13 Abstract Amazon Web Services (AWS)

More information

Sisense. Product Highlights. www.sisense.com

Sisense. Product Highlights. www.sisense.com Sisense Product Highlights Introduction Sisense is a business intelligence solution that simplifies analytics for complex data by offering an end-to-end platform that lets users easily prepare and analyze

More information

Zend Server Amazon AMI Quick Start Guide

Zend Server Amazon AMI Quick Start Guide Zend Server Amazon AMI Quick Start Guide By Zend Technologies www.zend.com Disclaimer This is the Quick Start Guide for The Zend Server Zend Server Amazon Machine Image The information in this document

More information

Lambda Architecture for Batch and Real- Time Processing on AWS with Spark Streaming and Spark SQL. May 2015

Lambda Architecture for Batch and Real- Time Processing on AWS with Spark Streaming and Spark SQL. May 2015 Lambda Architecture for Batch and Real- Time Processing on AWS with Spark Streaming and Spark SQL May 2015 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notices This document

More information

Introduction to Cloud Computing on Amazon Web Services (AWS) with focus on EC2 and S3. Horst Lueck

Introduction to Cloud Computing on Amazon Web Services (AWS) with focus on EC2 and S3. Horst Lueck Introduction to Cloud Computing on Amazon Web Services (AWS) with focus on EC2 and S3 Horst Lueck 2011-05-17 IT Pro Forum http://itproforum.org Thanks to Open Office Impress The Cloud the Name The 90s

More information

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect on AWS Services Overview Bernie Nallamotu Principle Solutions Architect \ So what is it? When your data sets become so large that you have to start innovating around how to collect, store, organize, analyze

More information

EEDC. Scalability Study of web apps in AWS. Execution Environments for Distributed Computing

EEDC. Scalability Study of web apps in AWS. Execution Environments for Distributed Computing EEDC Execution Environments for Distributed Computing 34330 Master in Computer Architecture, Networks and Systems - CANS Scalability Study of web apps in AWS Sergio Mendoza sergio.mendoza@est.fib.upc.edu

More information

Introduction to AWS in Higher Ed

Introduction to AWS in Higher Ed Introduction to AWS in Higher Ed Lori Clithero loricli@amazon.com 206.227.5054 University of Washington Cloud Day 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 2 Cloud democratizes

More information

Informatica Cloud & Redshift Getting Started User Guide

Informatica Cloud & Redshift Getting Started User Guide Informatica Cloud & Redshift Getting Started User Guide 2014 Informatica Corporation. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording

More information

APP DEVELOPMENT ON THE CLOUD MADE EASY WITH PAAS

APP DEVELOPMENT ON THE CLOUD MADE EASY WITH PAAS APP DEVELOPMENT ON THE CLOUD MADE EASY WITH PAAS This article looks into the benefits of using the Platform as a Service paradigm to develop applications on the cloud. It also compares a few top PaaS providers

More information

Getting Started with Amazon EC2 Management in Eclipse

Getting Started with Amazon EC2 Management in Eclipse Getting Started with Amazon EC2 Management in Eclipse Table of Contents Introduction... 4 Installation... 4 Prerequisites... 4 Installing the AWS Toolkit for Eclipse... 4 Retrieving your AWS Credentials...

More information

Scalability in the Cloud HPC Convergence with Big Data in Design, Engineering, Manufacturing

Scalability in the Cloud HPC Convergence with Big Data in Design, Engineering, Manufacturing Scalability in the Cloud HPC Convergence with Big Data in Design, Engineering, Manufacturing July 7, 2014 David Pellerin, Business Development Principal Amazon Web Services What Do We Hear From Customers?

More information

Amazon EC2 Product Details Page 1 of 5

Amazon EC2 Product Details Page 1 of 5 Amazon EC2 Product Details Page 1 of 5 Amazon EC2 Functionality Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to launch instances with a variety of

More information

Opsview in the Cloud. Monitoring with Amazon Web Services. Opsview Technical Overview

Opsview in the Cloud. Monitoring with Amazon Web Services. Opsview Technical Overview Opsview in the Cloud Monitoring with Amazon Web Services Opsview Technical Overview Page 2 Opsview In The Cloud: Monitoring with Amazon Web Services Contents Opsview in The Cloud... 3 Considerations...

More information

Innovative Geschäftsmodelle Ermöglicht durch die AWS Cloud

Innovative Geschäftsmodelle Ermöglicht durch die AWS Cloud Innovative Geschäftsmodelle Ermöglicht durch die AWS Cloud Rolf Kersten Business Development Manager Amazon Web Services Germany GmbH 2. Juli 2014 2014 Software AG. All rights reserved. Sechs Dinge, die

More information

Real Time Big Data Processing

Real Time Big Data Processing Real Time Big Data Processing Cloud Expo 2014 Ian Meyers Amazon Web Services Global Infrastructure Deployment & Administration App Services Analytics Compute Storage Database Networking AWS Global Infrastructure

More information

TECHNOLOGY WHITE PAPER Jan 2016

TECHNOLOGY WHITE PAPER Jan 2016 TECHNOLOGY WHITE PAPER Jan 2016 Technology Stack C# PHP Amazon Web Services (AWS) Route 53 Elastic Load Balancing (ELB) Elastic Compute Cloud (EC2) Amazon RDS Amazon S3 Elasticache CloudWatch Paypal Overview

More information

Managing Your Microsoft Windows Server Fleet with AWS Directory Service. May 2015

Managing Your Microsoft Windows Server Fleet with AWS Directory Service. May 2015 Managing Your Microsoft Windows Server Fleet with AWS Directory Service May 2015 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notices This document is provided for informational

More information

Crypto-Options on AWS. Bertram Dorn Specialized Solutions Architect Security/Compliance Network/Databases Amazon Web Services Germany GmbH

Crypto-Options on AWS. Bertram Dorn Specialized Solutions Architect Security/Compliance Network/Databases Amazon Web Services Germany GmbH Crypto-Options on AWS Bertram Dorn Specialized Solutions Architect Security/Compliance Network/Databases Amazon Web Services Germany GmbH Amazon.com, Inc. and its affiliates. All rights reserved. Agenda

More information

AWS Directory Service. Simple AD Administration Guide Version 1.0

AWS Directory Service. Simple AD Administration Guide Version 1.0 AWS Directory Service Simple AD Administration Guide AWS Directory Service: Simple AD Administration Guide Copyright 2015 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's

More information

Amazon WorkDocs. Administration Guide Version 1.0

Amazon WorkDocs. Administration Guide Version 1.0 Amazon WorkDocs Administration Guide Amazon WorkDocs: Administration Guide Copyright 2015 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not

More information

TECHNOLOGY WHITE PAPER Jun 2012

TECHNOLOGY WHITE PAPER Jun 2012 TECHNOLOGY WHITE PAPER Jun 2012 Technology Stack C# Windows Server 2008 PHP Amazon Web Services (AWS) Route 53 Elastic Load Balancing (ELB) Elastic Compute Cloud (EC2) Amazon RDS Amazon S3 Elasticache

More information

RDS Migration Tool Customer FAQ Updated 7/23/2015

RDS Migration Tool Customer FAQ Updated 7/23/2015 RDS Migration Tool Customer FAQ Updated 7/23/2015 Amazon Web Services is now offering the Amazon RDS Migration Tool a powerful utility for migrating data with minimal downtime from on-premise and EC2-based

More information

Introduction to DevOps on AWS

Introduction to DevOps on AWS Introduction to DevOps on AWS David Chapman December 2014 Contents Contents Abstract Introduction Agile Evolution to DevOps Infrastructure as Code AWS CloudFormation AWS AMI Continuous Deployment AWS CodeDeploy

More information

CLOUD COMPUTING FOR THE ENTERPRISE AND GLOBAL COMPANIES Steve Midgley Head of AWS EMEA

CLOUD COMPUTING FOR THE ENTERPRISE AND GLOBAL COMPANIES Steve Midgley Head of AWS EMEA CLOUD COMPUTING FOR THE ENTERPRISE AND GLOBAL COMPANIES Steve Midgley Head of AWS EMEA AWS Introduction Why are enterprises choosing AWS? What are enterprises using AWS for? How are enterprise getting

More information

A Comparison of Clouds: Amazon Web Services, Windows Azure, Google Cloud Platform, VMWare and Others (Fall 2012)

A Comparison of Clouds: Amazon Web Services, Windows Azure, Google Cloud Platform, VMWare and Others (Fall 2012) 1. Computation Amazon Web Services Amazon Elastic Compute Cloud (Amazon EC2) provides basic computation service in AWS. It presents a virtual computing environment and enables resizable compute capacity.

More information

AWS Toolkit for Visual Studio. User Guide Version v1.30

AWS Toolkit for Visual Studio. User Guide Version v1.30 AWS Toolkit for Visual Studio User Guide AWS Toolkit for Visual Studio: User Guide Copyright 2014 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. The following are trademarks of Amazon

More information

ArcGIS 10.3 Server on Amazon Web Services

ArcGIS 10.3 Server on Amazon Web Services ArcGIS 10.3 Server on Amazon Web Services Copyright 1995-2015 Esri. All rights reserved. Table of Contents Introduction What is ArcGIS Server on Amazon Web Services?............................... 5 Quick

More information

Amazon Web Services. Lawrence Berkeley LabTech Conference 9/10/15. Jamie Baker Federal Scientific Account Manager AWS WWPS bakjames@amazon.

Amazon Web Services. Lawrence Berkeley LabTech Conference 9/10/15. Jamie Baker Federal Scientific Account Manager AWS WWPS bakjames@amazon. Web Services Lawrence Berkeley LabTech Conference 9/10/15 Jamie Baker Federal Scientific Account Manager AWS WWPS bakjames@amazon.com 2015, Web Services, Inc. or its Affiliates. All rights reserved. AWS

More information

AWS Lambda. Developer Guide

AWS Lambda. Developer Guide AWS Lambda Developer Guide AWS Lambda: Developer Guide Copyright 2015 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection

More information

HADOOP BIG DATA DEVELOPER TRAINING AGENDA

HADOOP BIG DATA DEVELOPER TRAINING AGENDA HADOOP BIG DATA DEVELOPER TRAINING AGENDA About the Course This course is the most advanced course available to Software professionals This has been suitably designed to help Big Data Developers and experts

More information

Using ArcGIS for Server in the Amazon Cloud

Using ArcGIS for Server in the Amazon Cloud Using ArcGIS for Server in the Amazon Cloud Randall Williams, Esri Subrat Bora, Esri Esri UC 2014 Technical Workshop Agenda What is ArcGIS for Server on Amazon Web Services Sounds good! How much does it

More information

Amazon Web Services Primer. William Strickland COP 6938 Fall 2012 University of Central Florida

Amazon Web Services Primer. William Strickland COP 6938 Fall 2012 University of Central Florida Amazon Web Services Primer William Strickland COP 6938 Fall 2012 University of Central Florida AWS Overview Amazon Web Services (AWS) is a collection of varying remote computing provided by Amazon.com.

More information

CLOUD COMPUTING & DIGITAL CUSTOMER EXPERIENCE. Nicola Previati Territory Manager Italy

CLOUD COMPUTING & DIGITAL CUSTOMER EXPERIENCE. Nicola Previati Territory Manager Italy CLOUD COMPUTING & DIGITAL CUSTOMER EXPERIENCE Nicola Previati Territory Manager Italy 8 th Birthday Launched on March 14th, 2006 STARTUPS ON AWS ENTERPRISES ON AWS Hotels Music Magazines Storage 150,000

More information

Assignment # 1 (Cloud Computing Security)

Assignment # 1 (Cloud Computing Security) Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual

More information

Cloud Computing with Amazon Web Services and the DevOps Methodology. www.cloudreach.com

Cloud Computing with Amazon Web Services and the DevOps Methodology. www.cloudreach.com Cloud Computing with Amazon Web Services and the DevOps Methodology Who am I? Max Manders @maxmanders Systems Developer at Cloudreach @cloudreach Director / Co-Founder of Whisky Web @whiskyweb Who are

More information

Amazon AWS in.net. Presented by: Scott Reed scottr@develop.com

Amazon AWS in.net. Presented by: Scott Reed scottr@develop.com Amazon AWS in.net Presented by: Scott Reed scottr@develop.com Objectives Cloud Computing What Amazon provides Why Amazon Web Services? Q&A Instances Interacting with Instances Management Console Command

More information

Automated Data Ingestion. Bernhard Disselhoff Enterprise Sales Engineer

Automated Data Ingestion. Bernhard Disselhoff Enterprise Sales Engineer Automated Data Ingestion Bernhard Disselhoff Enterprise Sales Engineer Agenda Pentaho Overview Templated dynamic ETL workflows Pentaho Data Integration (PDI) Use Cases Pentaho Overview Overview What we

More information

ColdFusion 10 in the Amazon AWS Cloud. Sven Ramuschkat tecracer GmbH

ColdFusion 10 in the Amazon AWS Cloud. Sven Ramuschkat tecracer GmbH ColdFusion 10 in the Amazon AWS Cloud Sven Ramuschkat tecracer GmbH Sven Ramuschkat Focus on Web-Projects - Adobe ColdFusion - Adobe Flash Media Server - Adobe LiveCycle - Adobe Flex, Connect - Java, HTML5,

More information

BERLIN. 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

BERLIN. 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved BERLIN 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Build Your Mobile App Faster with AWS Mobile Services Jan Metzner AWS Solutions Architect @janmetzner Danilo Poccia AWS Technical

More information

How AWS Pricing Works

How AWS Pricing Works How AWS Pricing Works (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 15 Table of Contents Table of Contents... 2 Abstract... 3 Introduction... 3 Fundamental

More information

Using ArcGIS for Server in the Amazon Cloud

Using ArcGIS for Server in the Amazon Cloud Federal GIS Conference February 9 10, 2015 Washington, DC Using ArcGIS for Server in the Amazon Cloud Bonnie Stayer, Esri Amy Ramsdell, Blue Raster Session Outline AWS Overview ArcGIS in AWS Cloud Builder

More information

MATLAB on EC2 Instructions Guide

MATLAB on EC2 Instructions Guide MATLAB on EC2 Instructions Guide Contents Welcome to MATLAB on EC2...3 What You Need to Do...3 Requirements...3 1. MathWorks Account...4 1.1. Create a MathWorks Account...4 1.2. Associate License...4 2.

More information

Backup and Recovery of SAP Systems on Windows / SQL Server

Backup and Recovery of SAP Systems on Windows / SQL Server Backup and Recovery of SAP Systems on Windows / SQL Server Author: Version: Amazon Web Services sap- on- aws@amazon.com 1.1 May 2012 2 Contents About this Guide... 4 What is not included in this guide...

More information

Healthstone Monitoring System

Healthstone Monitoring System Healthstone Monitoring System Patrick Lambert v1.1.0 Healthstone Monitoring System 1 Contents 1 Introduction 2 2 Windows client 2 2.1 Installation.............................................. 2 2.2 Troubleshooting...........................................

More information

Amazon Glacier. Developer Guide API Version 2012-06-01

Amazon Glacier. Developer Guide API Version 2012-06-01 Amazon Glacier Developer Guide Amazon Glacier: Developer Guide Copyright 2016 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in

More information

AWS Development Essentials

AWS Development Essentials AWS Development Essentials Prabhakaran Kuppusamy Uchit Vyas Chapter No 1 "An Introduction to Amazon Web Services" In this package, you will find: The author s biography A preview chapter from the book,

More information

AWS Data Pipeline. Developer Guide API Version 2012-10-29

AWS Data Pipeline. Developer Guide API Version 2012-10-29 AWS Data Pipeline Developer Guide Amazon Web Services AWS Data Pipeline: Developer Guide Amazon Web Services What is AWS Data Pipeline?... 1 How Does AWS Data Pipeline Work?... 1 Pipeline Definition...

More information

SAS BIG DATA SOLUTIONS ON AWS SAS FORUM ESPAÑA, OCTOBER 16 TH, 2014 IAN MEYERS SOLUTIONS ARCHITECT / AMAZON WEB SERVICES

SAS BIG DATA SOLUTIONS ON AWS SAS FORUM ESPAÑA, OCTOBER 16 TH, 2014 IAN MEYERS SOLUTIONS ARCHITECT / AMAZON WEB SERVICES SAS BIG DATA SOLUTIONS ON AWS SAS FORUM ESPAÑA, OCTOBER 16 TH, 2014 IAN MEYERS SOLUTIONS ARCHITECT / AMAZON WEB SERVICES AWS GLOBAL INFRASTRUCTURE 10 Regions 25 Availability Zones 51 Edge locations WHAT

More information

Upgrading From PDI 4.0 to 4.1.0

Upgrading From PDI 4.0 to 4.1.0 Upgrading From PDI 4.0 to 4.1.0 This document is copyright 2011 Pentaho Corporation. No part may be reprinted without written permission from Pentaho Corporation. All trademarks are the property of their

More information

How AWS Pricing Works May 2015

How AWS Pricing Works May 2015 How AWS Pricing Works May 2015 (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 15 Table of Contents Table of Contents... 2 Abstract... 3 Introduction...

More information

AWS Toolkit for Visual Studio User Guide

AWS Toolkit for Visual Studio User Guide AWS Toolkit for Visual Studio User Guide Release 1.0 Amazon Web Services Dec 01, 2016 Contents 1 Using the Toolkit for Visual Studio 1 2 Setting Up the Toolkit for Visual Studio 5 3 Working with AWS Services

More information

Hadoop Setup. 1 Cluster

Hadoop Setup. 1 Cluster In order to use HadoopUnit (described in Sect. 3.3.3), a Hadoop cluster needs to be setup. This cluster can be setup manually with physical machines in a local environment, or in the cloud. Creating a

More information

Last time. Today. IaaS Providers. Amazon Web Services, overview

Last time. Today. IaaS Providers. Amazon Web Services, overview Last time General overview, motivation, expected outcomes, other formalities, etc. Please register for course Online (if possible), or talk to CS secretaries Cloud computing introduction General concepts

More information

CAPTURING & PROCESSING REAL-TIME DATA ON AWS

CAPTURING & PROCESSING REAL-TIME DATA ON AWS CAPTURING & PROCESSING REAL-TIME DATA ON AWS @ 2015 Amazon.com, Inc. and Its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent

More information

AWS Import/Export. Developer Guide API Version 2010-06-03

AWS Import/Export. Developer Guide API Version 2010-06-03 AWS Import/Export Developer Guide AWS Import/Export: Developer Guide Copyright 2014 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. The following are trademarks of Amazon Web Services,

More information

AWS Import/Export. Developer Guide API Version 2014-12-18

AWS Import/Export. Developer Guide API Version 2014-12-18 AWS Import/Export Developer Guide AWS Import/Export: Developer Guide Copyright 2015 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. The following are trademarks of Amazon Web Services,

More information

GeoCloud Project Report GEOSS Clearinghouse

GeoCloud Project Report GEOSS Clearinghouse GeoCloud Project Report GEOSS Clearinghouse Qunying Huang, Doug Nebert, Chaowei Yang, Kai Liu 2011.12.06 Description of Application GEOSS clearinghouse is a FGDC, GEO, and NASA project that connects directly

More information

Using SUSE Studio to Build and Deploy Applications on Amazon EC2. Guide. Solution Guide Cloud Computing. www.suse.com

Using SUSE Studio to Build and Deploy Applications on Amazon EC2. Guide. Solution Guide Cloud Computing. www.suse.com Using SUSE Studio to Build and Deploy Applications on Amazon EC2 Guide Solution Guide Cloud Computing Cloud Computing Solution Guide Using SUSE Studio to Build and Deploy Applications on Amazon EC2 Quickly

More information

Web Application Deployment in the Cloud Using Amazon Web Services From Infancy to Maturity

Web Application Deployment in the Cloud Using Amazon Web Services From Infancy to Maturity P3 InfoTech Solutions Pvt. Ltd http://www.p3infotech.in July 2013 Created by P3 InfoTech Solutions Pvt. Ltd., http://p3infotech.in 1 Web Application Deployment in the Cloud Using Amazon Web Services From

More information

IBM Campaign Version-independent Integration with IBM Engage Version 1 Release 3 April 8, 2016. Integration Guide IBM

IBM Campaign Version-independent Integration with IBM Engage Version 1 Release 3 April 8, 2016. Integration Guide IBM IBM Campaign Version-independent Integration with IBM Engage Version 1 Release 3 April 8, 2016 Integration Guide IBM Note Before using this information and the product it supports, read the information

More information

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013 Big Data Use Case How Rackspace is using Private Cloud for Big Data Bryan Thompson May 8th, 2013 Our Big Data Problem Consolidate all monitoring data for reporting and analytical purposes. Every device

More information

Last time. Today. IaaS Providers. Amazon Web Services, overview

Last time. Today. IaaS Providers. Amazon Web Services, overview Last time General overview, motivation, expected outcomes, other formalities, etc. Please register for course Online (if possible), or talk to Yvonne@CS Course evaluation forgotten Please assign one volunteer

More information

Building your Big Data Architecture on Amazon Web Services

Building your Big Data Architecture on Amazon Web Services Building your Big Data Architecture on Amazon Web Services Abhishek Sinha @abysinha sinhaar@amazon.com AWS Services Deployment & Administration Application Services Compute Storage Database Networking

More information

Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud

Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud Use case Figure 1: Company C Architecture (Before Migration) Company C is an automobile insurance claim processing company with

More information

Online Backup Guide for the Amazon Cloud: How to Setup your Online Backup Service using Vembu StoreGrid Backup Virtual Appliance on the Amazon Cloud

Online Backup Guide for the Amazon Cloud: How to Setup your Online Backup Service using Vembu StoreGrid Backup Virtual Appliance on the Amazon Cloud Online Backup Guide for the Amazon Cloud: How to Setup your Online Backup Service using Vembu StoreGrid Backup Virtual Appliance on the Amazon Cloud Here is a step-by-step set of instructions to get your

More information

Last time. Today. IaaS Providers. Amazon Web Services, overview

Last time. Today. IaaS Providers. Amazon Web Services, overview Last time General overview, motivation, expected outcomes, other formalities, etc. Please register for course Online (if possible), or talk to CS secretaries Course evaluation forgotten Please assign one

More information

StorReduce Technical White Paper Cloud-based Data Deduplication

StorReduce Technical White Paper Cloud-based Data Deduplication StorReduce Technical White Paper Cloud-based Data Deduplication See also at storreduce.com/docs StorReduce Quick Start Guide StorReduce FAQ StorReduce Solution Brief, and StorReduce Blog at storreduce.com/blog

More information

Netop Environment Security. Unified security to all Netop products while leveraging the benefits of cloud computing

Netop Environment Security. Unified security to all Netop products while leveraging the benefits of cloud computing Netop Environment Security Unified security to all Netop products while leveraging the benefits of cloud computing Contents Introduction... 2 AWS Infrastructure Security... 3 Standards - Compliancy...

More information

McAfee Public Cloud Server Security Suite

McAfee Public Cloud Server Security Suite Installation Guide McAfee Public Cloud Server Security Suite For use with McAfee epolicy Orchestrator COPYRIGHT Copyright 2015 McAfee, Inc., 2821 Mission College Boulevard, Santa Clara, CA 95054, 1.888.847.8766,

More information

MICROSTRATEGY ON AWS

MICROSTRATEGY ON AWS MICROSTRATEGY ON AWS Presented by: MicroStrategy World 2015 Tuesday, January 27th 3:30 4:30 PM Track 8 Session 3 WWW.IOLAP.COM 1 INTRODUCTIONS iolap Data Warehousing and Business Intelligence consultancy

More information

Research on AWS. Genomics Research Statistical Analysis Mathematical Modeling Computational Fluid Dynamics Grid/Cluster/High Performance Computing

Research on AWS. Genomics Research Statistical Analysis Mathematical Modeling Computational Fluid Dynamics Grid/Cluster/High Performance Computing Research on AWS Genomics Research Statistical Analysis Mathematical Modeling Computational Fluid Dynamics Grid/Cluster/High Performance Computing Research use cases on AWS (examples) Carat is an application

More information

Cloud Computing. AWS a practical example. Hugo Pérez UPC. Mayo 2012

Cloud Computing. AWS a practical example. Hugo Pérez UPC. Mayo 2012 Cloud Computing AWS a practical example Mayo 2012 Hugo Pérez UPC -2- Index Introduction Infraestructure Development and Results Conclusions Introduction In order to know deeper about AWS services, mapreduce

More information

Amazon Web Services. 2015 Annual ALGIM Conference. Tim Dacombe-Bird Regional Sales Manager Amazon Web Services New Zealand

Amazon Web Services. 2015 Annual ALGIM Conference. Tim Dacombe-Bird Regional Sales Manager Amazon Web Services New Zealand Amazon Web Services 2015 Annual ALGIM Conference Tim Dacombe-Bird Regional Sales Manager Amazon Web Services New Zealand 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda Who

More information

WE RUN SEVERAL ON AWS BECAUSE WE CRITICAL APPLICATIONS CAN SCALE AND USE THE INFRASTRUCTURE EFFICIENTLY.

WE RUN SEVERAL ON AWS BECAUSE WE CRITICAL APPLICATIONS CAN SCALE AND USE THE INFRASTRUCTURE EFFICIENTLY. WE RUN SEVERAL CRITICAL APPLICATIONS ON AWS BECAUSE WE CAN SCALE AND USE THE INFRASTRUCTURE EFFICIENTLY. - Murari Gopalan Director, Technology Expedia Expedia, a leading online travel company for leisure

More information

AdWhirl Open Source Server Setup Instructions

AdWhirl Open Source Server Setup Instructions AdWhirl Open Source Server Setup Instructions 11/09 AdWhirl Server Setup Instructions The server runs in Amazon s web cloud. To set up the server, you need an Amazon Web Services (AWS) account and the

More information

Lesson 1: Planning a Cloud Solution Architecture

Lesson 1: Planning a Cloud Solution Architecture Lesson 1: Planning a Cloud Solution Architecture Topic A: Determine Business Needs for Cloud Computing Solutions Determine the business needs for cloud computing solutions. Terminology Review Identify

More information

Creating a DUO MFA Service in AWS

Creating a DUO MFA Service in AWS Amazon AWS is a cloud based development environment with a goal to provide many options to companies wishing to leverage the power and convenience of cloud computing within their organisation. In 2013

More information

ST 810, Advanced computing

ST 810, Advanced computing ST 810, Advanced computing Eric B. Laber & Hua Zhou Department of Statistics, North Carolina State University January 30, 2013 Supercomputers are expensive. Eric B. Laber, 2011, while browsing the internet.

More information

INTRODUCTION & CONCEPTS. Definition of Cloud Computing Service Models Deployment Models... 23

INTRODUCTION & CONCEPTS. Definition of Cloud Computing Service Models Deployment Models... 23 Contents I INTRODUCTION & CONCEPTS 17 1 Introduction to Cloud Computing 19 11 Introduction 111 Definition of Cloud Computing 20 12 Characteristics of Cloud Computing 20 13 Cloud Models 22 131 132 Service

More information

Scalable Application. Mikalai Alimenkou http://xpinjection.com 11.05.2012

Scalable Application. Mikalai Alimenkou http://xpinjection.com 11.05.2012 Scalable Application Development on AWS Mikalai Alimenkou http://xpinjection.com 11.05.2012 Background Java Technical Lead/Scrum Master at Zoral Labs 7+ years in software development 5+ years of working

More information

ArcGIS for Server: Administrative Scripting and Automation

ArcGIS for Server: Administrative Scripting and Automation ArcGIS for Server: Administrative Scripting and Automation Shreyas Shinde Ranjit Iyer Esri UC 2014 Technical Workshop Agenda Introduction to server administration Command line tools ArcGIS Server Manager

More information

AWS IaaS Services. Methods Digital GCloud Service Definition

AWS IaaS Services. Methods Digital GCloud Service Definition Methods Digital GCloud Service Definition HEAD OFFICE: 125 Shaftesbury Avenue, London WC2H 8AD Scottish Office: Exchange Place 2, 5 Semple Street, Edinburgh, EH3 8BL Midlands Office: Pure Offices, Lake

More information

Amazon Elastic Compute Cloud Getting Started Guide. My experience

Amazon Elastic Compute Cloud Getting Started Guide. My experience Amazon Elastic Compute Cloud Getting Started Guide My experience Prepare Cell Phone Credit Card Register & Activate Pricing(Singapore) Region Amazon EC2 running Linux(SUSE Linux Windows Windows with SQL

More information

Running a Program on an AVD

Running a Program on an AVD Running a Program on an AVD Now that you have a project that builds an application, and an AVD with a system image compatible with the application s build target and API level requirements, you can run

More information