Amazon Web Services Building in the Cloud

Transcription

1

2 Amazon Web Services Building in the Cloud

3 Amazon has Three Parts

4 AWS Principles Easy to use Fast Elastic Highly available Secure Pay as you go

5 The Utility Model

6 AWS Bandwidth Growth

7 AWS Storage Growth

8 Services Overview Building block services that allow developers to innovate and make money Infrastructure As a Service Amazon Simple Storage Service Amazon Elastic Compute Cloud Amazon Simple Queue Service Amazon SimpleDB Commerce As a Service Amazon Flexible Payments Service Fulfillment Web Service People As a Service Amazon Mechanical Turk Alexa Web Services Alexa Web Information Service Alexa Top Sites Alexa Site Thumbnail Alexa Web Search Platform

9 Amazon S3 Simple Storage Service Highly scalable data storage in-the-cloud Programmatic access via web services API Simple to get going, simple to use Highly available and durable Pay-as-you-go: Storage: $0.15 / GB / month Data Transfer: starts at $0.17 / GB Requests: nominal charges Content Delivery Network

10 Amazon EC2 Elastic Compute Cloud Resizable compute capacity in the cloud Obtain and boot new server instances in minutes Quickly scale capacity, up or down, as your computing requirements change Full root access to a blank Linux or Windows machine Simple Web service management interface Changes the economics of computing

11 Amazon EC2 Features Elastic Block Store Persistent storage, up to 1 TB Backed by S3 Snapshots Availability Zones Separate Data Centers Increased Reliability Elastic IP Addresses Fixed IP can be quickly attached to an instance Don't need to wait for DNS propagation delays

12 Predictions Cost Money

13 Amazon Machine Images Public AMIs: Use pre-configured, template AMIs to get up and running immediately. Choose from Fedora, Ubuntu, Windows configurations and more Private AMIs: Create an Amazon Machine Image (AMI) containing your applications, libraries, data and associated configuration settings Paid AMIs: Set a price for your AMI and let others purchase and use it (Single payment and/or per hour)

14 Instance Types Standard High CPU Small Large Extra Large Medium Extra Large Bits RAM 1.7 GB 7.5 GB 15 GB 1.7 GB 7 GB Disk 160 GB 850 GB 1690 GB 350 GB 1690 GB EC2 Compute Units I/O Performa nce Medium High High High High Firewall Yes Yes Yes Yes Yes Per Hour $0.10 $0.40 $0.80 $0.20 $0.80

15 Amazon SQS Simple Queue Service Scalable Messaging - many application threads/servers can be used without worrying about overloading the messaging system Unlimited - any number of queues, any number of messages per queue Supports reader fail-over -messages become invisible to other readers when read the first time. Adjustable time before message is again visible to other readers

16 Amazon SimpleDB Simple to use - implements most common / simple database features behind a set of web service calls Flexible - doesn't require structured definition of data which allows you to add new data easily Scalable - scales easily as your application grows. Simply add more domains or data within your domains Fast - quick efficient storage and retrieval of your data Reliable - uses Amazon's high-availability data centers to provide strong and consistent performance Inexpensive - you pay only for the resources you consume, so you avoid significant up-front costs. Eventual Consistency!

17 Using Amazon SimpleDB item description color 123 Sweater Blue, Red 456 Dress shirt White, Blue 789 Shoes Black material Leather PUT (item, 123), (description, Sweater), (color, Blue), (color, Red) PUT (item, 456), (description, Dress shirt), (color, White), (color, Blue) PUT (item, 789), (description, Shoes), (color, Black), (material, Leather) Query Domain = MyStore [ description = Sweater ]

18 Leveraging AWS Application hosting - Using EC2 to provided an elastic web tier Batch processing - Using SQS, EC2 and S3 to process large amounts of data Searchable Object Store - Combine S3 and SimpleDB to store metadata data about objects and run searches Cloud-Bursting - Leveraging EC2 for excess capacity to augment your own data center

19 New York Times and AWS

20 24 hours later

21 Animoto scales

22 50 to 3500 servers in 3 days!

23 Using AWS with Java Web services provide language independence WSDL can be used to build an access layer Popular Open Source Java libraries include; Jets3t - Amazon S3 access typica - Amazon EC2, SQS, SimpleDB, FPS, DevPay

24 Jets3t An open source library for interacting with Amazon S3 and objects stored there. Supports both SOAP and REST interfaces. Well tested and very robust.

25 S3 API package org.jets3t.service; Class to know about; S3Service S3Bucket createbucket(string name) S3Object getobject(s3bucket bucket, String key) S3Bucket setacl(accesscontrollist acl) S3Object setcontenttype(string type) setdatainputstream(inputstream istr)

26 S3 Code Sample // code from a servlet that proxies data from S3 s3 = new RestS3Service(new AWSCredentials(awsAccessId, awssecretkey)); // get the object details from S3 S3Object obj = new S3Object(objectId); S3Bucket bucket = new S3Bucket(bucketName); obj = s3.getobject(bucket, objectid); // use the stream from the S3Object to move data to the servlet response stream OutputStream ostr = response.getoutputstream(); response.setcontenttype(obj.getcontenttype()); InputStream in = obj.getdatainputstream(); Copier.copyStreams(in, ostr);

27 Typica Typica was designed initially to support SQS via the REST interface in a very reliable and thread-safe way. The initial single digit downloads were inspiring enough to get EC2 support added. Incorporates patches from users and supports many popular services. Current popularity means hundreds of downloads for each release.

28 SQS API package com.xerox.amazonws.sqs; Classes to know about; QueueService messagequeue getorcreatemessagequeue(queuename) List<MessageQueue> listmessagequeues MessageQueue sendmessage(msg) String receivemessage() deletemessage(msgid) deletequeue(force) Message String getmessagebody()

29 SQS Sample Code QueueService qs = new QueueService(<accessId>, <secretkey>); MessageQueue queue = qs.getorcreatemessagequeue("testqueue"); queue.sendmessage("test Message"); Message msg = queue.receivemessage(); logger.debug("message received : "+msg.getmessagebody());

30 EC2 API package com.xerox.amazonws.ec2; Classes to know about; Jec2 List<ImageDescription> describeimages(imageids) List<ReservationDescription> describeinstances (instanceids) ReservationDescription runinstances(imageid,...) ConsoleOutput getconsoleoutput(instanceid) ReservationDescription List<Instance> getinstances();

31 EC2 Sample Code Jec2 ec2 = new Jec2(<accessId>, <secretkey>); ReservationDescription res = ec2.runinstances("ami ", 1, 1, null, null, "dak-keypair"); String instanceid = res.getinstances().get(0).getinstanceid(); while (!res.getinstances().get(0).getstate().equals("running")) { try { Thread.sleep(5000); } catch (InterruptedException ex) {} res = ec2.describeinstances(instanceid); } logger.debug("instance running : "+instanceid); ConsoleOutput cons = ec2.getconsoleoutput(instanceid); logger.debug("console output = "+cons.getoutput()); ec2.terminateinstances(new String [] {instanceid});

32 SimpleDB API package com.xerox.amazonws.sdb; Classes to know about; SimpleDB ListDomainsResult listdomains() Domain createdomain(string name) Domain QueryResult listitems(string query) QueryWithAttributesResult listitemswithattributes(string query) Item List<ItemAttribute> getattributes() SDBResult putattributes(list<itemattribute> attrs)

33 SimpleDB Sample Code SimpleDB sdb = new SimpleDB(<accessId>, <secretkey>); Domain dom = sdb.getdomain("testdomain"); logger.info("items:"); String nexttoken = ""; while (nexttoken!= null) { ListDomainsResult result = dom.listitems(query, nexttoken, 10); List<Item> items = result.getitemlist(); for (Item item : items) { logger.info(item.getidentifier()); } nexttoken = result.getnexttoken(); }

34 Useful Links