How SoundCloud scales

Size: px
Start display at page:

Download "How SoundCloud scales"

Transcription

1 How SoundCloud scales Alexander

2 Youtube for Audio

3 Fakten

4 Fakten Alexa Ranking (233)

5 Fakten Alexa Ranking (233) Employees (190) - 80 engineers

6 Fakten Alexa Ranking (233) Employees (190) - 80 engineers 4 office locations (Berlin, London, San Francisco, Sofia)

7 Fakten Alexa Ranking (233) Employees (190) - 80 engineers 4 office locations (Berlin, London, San Francisco, Sofia) >35 million registered users

8 Fakten Alexa Ranking (233) Employees (190) - 80 engineers 4 office locations (Berlin, London, San Francisco, Sofia) >35 million registered users partners (BBC, CNN)

9 Fakten Alexa Ranking (233) Employees (190) - 80 engineers 4 office locations (Berlin, London, San Francisco, Sofia) >35 million registered users partners (BBC, CNN) > Apps

10

11 How SoundCloud scales

12 ?

13 Organization Technology

14 What is the foundation?

15

16 Principles

17 Principles Top-Down vs. Bottom-Up

18 Principles Top-Down vs. Bottom-Up Trust

19 Principles Top-Down vs. Bottom-Up Trust How to react to mistakes

20 Principles Top-Down vs. Bottom-Up Trust How to react to mistakes Continuous Improvement

21 Principles Top-Down vs. Bottom-Up Trust How to react to mistakes Continuous Improvement Innovation

22 Top-Down vs. Bottom-Up Flat Hierarchies? Everybody has them... Engineering teams responsible for architecture (not architecture teams) Influence on the product

23 Trust Deployment Change Management Process Time for technical debt

24 Reaction to mistakes What will be changed if a mistake happens?

25 Innovation R&D Department? Only Product?

26 SoundCloud s basic principles

27 SoundCloud s basic principles No dedicated QA roles

28 SoundCloud s basic principles No dedicated QA roles You build it - you run it

29 SoundCloud s basic principles No dedicated QA roles You build it - you run it Team Building (Rotation, constant changes)

30 SoundCloud s basic principles No dedicated QA roles You build it - you run it Team Building (Rotation, constant changes) Hacker Time

31 SoundCloud s basic principles No dedicated QA roles You build it - you run it Team Building (Rotation, constant changes) Hacker Time Continuous Improvement

32 SoundCloud s basic principles No dedicated QA roles You build it - you run it Team Building (Rotation, constant changes) Hacker Time Continuous Improvement Move fast and break things

33 No QA Does that mean we don t test?

34 You build it - you run it

35 Flat Hierarchy

36 Innovation Hacker Time

37

38 Continuous Improvement Retrospectives Post Mortems (Projekte)

39 Mistakes...

40 Scrum? Kanban? Process?

41 What is the most important thing you have to do so that this works?

42

43 ???

44 ??? Initial Culture

45 ??? Initial Culture Hiring!!!

46

47 SoundCloud Hiring Process

48 How does it look like in reality?

49 Organisation

50 HTML5 Mobile (native) other V1 API Search T&S Payment Partner Tools Data Activ. Delivery Platform System Engineering (Berlin / SF)

51 How do we structure teams? cross functional?

52 Do teams stay the same? Change!

53 Technology

54 Ruby/Scala/Go/Clojure/ Javascript

55 Scale Horizontally/ Vertically?

56 Rails->Mysql RabbitMQ->Cassandra evolution-of-soundclouds-architecture/

57 MySQL 7 Master, 9 slaves, 900GB data Additional slaves in EC2 Master Peak is 15K IOPS writes, 45k reads one slave lags intentionally 1 hour behind Je nach Konsistenzanforderung wird Slave oder Master gewählt (für reads - unterstützt read-only mode)

58 Cassandra 3 Cluster 88 Nodes 15 TB of data up to 70K reads/sec

59 Hadoop Elastic Search Proprietary Spam Detection

60 ElasticSearch 80 Mio Docs (gaining 1 Mio every 6 days) 175 QPS (gaining 15 QPS every month)

61 Hosted in S3/EC2 and physical DCs

62 Outages

63

64 Post Mortems

65 Summary Move fast and break things Don t get slow because you fear to make mistakes

66 backstage.soundcloud.com

67 Alexander Grosse We are hiring!

Scalable Architecture on Amazon AWS Cloud

Scalable Architecture on Amazon AWS Cloud Scalable Architecture on Amazon AWS Cloud Kalpak Shah Founder & CEO, Clogeny Technologies [email protected] 1 * http://www.rightscale.com/products/cloud-computing-uses/scalable-website.php 2 Architect

More information

What is Cloud Computing? Tackling the Challenges of Big Data. Tackling The Challenges of Big Data. Matei Zaharia. Matei Zaharia. Big Data Collection

What is Cloud Computing? Tackling the Challenges of Big Data. Tackling The Challenges of Big Data. Matei Zaharia. Matei Zaharia. Big Data Collection Introduction What is Cloud Computing? Cloud computing means computing resources available on demand Resources can include storage, compute cycles, or software built on top (e.g. database as a service)

More information

HYPER-CONVERGED INFRASTRUCTURE STRATEGIES

HYPER-CONVERGED INFRASTRUCTURE STRATEGIES 1 HYPER-CONVERGED INFRASTRUCTURE STRATEGIES MYTH BUSTING & THE FUTURE OF WEB SCALE IT 2 ROADMAP INFORMATION DISCLAIMER EMC makes no representation and undertakes no obligations with regard to product planning

More information

Understanding AWS Storage Options

Understanding AWS Storage Options Understanding AWS Storage Options Ian Massingham, Technical Evangelist @IanMmmm [email protected] 30 April 2014 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified,

More information

Ground up Introduction to In-Memory Data (Grids)

Ground up Introduction to In-Memory Data (Grids) Ground up Introduction to In-Memory Data (Grids) QCON 2015 NEW YORK, NY 2014 Hazelcast Inc. Why you here? 2014 Hazelcast Inc. Java Developer on a quest for scalability frameworks Architect on low-latency

More information

Tech Radar - May 2015

Tech Radar - May 2015 Tech Radar - May 2015 Or how Obecto is staying fresh and current with new technologies and tools, while maintaining its focus on the industry standards. This is our May 15 edition of the Obecto Tech Radar.

More information

So What s the Big Deal?

So What s the Big Deal? So What s the Big Deal? Presentation Agenda Introduction What is Big Data? So What is the Big Deal? Big Data Technologies Identifying Big Data Opportunities Conducting a Big Data Proof of Concept Big Data

More information

In Memory Accelerator for MongoDB

In Memory Accelerator for MongoDB In Memory Accelerator for MongoDB Yakov Zhdanov, Director R&D GridGain Systems GridGain: In Memory Computing Leader 5 years in production 100s of customers & users Starts every 10 secs worldwide Over 15,000,000

More information

MyISAM Default Storage Engine before MySQL 5.5 Table level locking Small footprint on disk Read Only during backups GIS and FTS indexing Copyright 2014, Oracle and/or its affiliates. All rights reserved.

More information

SCALABILITY. Hodicska Gergely. email: [email protected] twitter: @felhobacsi. Web Engineering Manager as Ustream. May 7, 2012

SCALABILITY. Hodicska Gergely. email: felho@ustream.tv twitter: @felhobacsi. Web Engineering Manager as Ustream. May 7, 2012 SCALABILITY Hodicska Gergely Web Engineering Manager as Ustream email: [email protected] twitter: @felhobacsi SCALABILITY BME 1 DEFINING SCALABILITY It is not: Performance Easier to scale HA It is the ability

More information

Hadoop: Distributed Data Processing. Amr Awadallah Founder/CTO, Cloudera, Inc. ACM Data Mining SIG Thursday, January 25 th, 2010

Hadoop: Distributed Data Processing. Amr Awadallah Founder/CTO, Cloudera, Inc. ACM Data Mining SIG Thursday, January 25 th, 2010 Hadoop: Distributed Data Processing Amr Awadallah Founder/CTO, Cloudera, Inc. ACM Data Mining SIG Thursday, January 25 th, 2010 Outline Scaling for Large Data Processing What is Hadoop? HDFS and MapReduce

More information

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013 Big Data Use Case How Rackspace is using Private Cloud for Big Data Bryan Thompson May 8th, 2013 Our Big Data Problem Consolidate all monitoring data for reporting and analytical purposes. Every device

More information

Performance and Scalability Overview

Performance and Scalability Overview Performance and Scalability Overview This guide provides an overview of some of the performance and scalability capabilities of the Pentaho Business Analytics Platform. Contents Pentaho Scalability and

More information

Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases. Lecture 14

Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases. Lecture 14 Department of Computer Science University of Cyprus EPL646 Advanced Topics in Databases Lecture 14 Big Data Management IV: Big-data Infrastructures (Background, IO, From NFS to HFDS) Chapter 14-15: Abideboul

More information

Cloud Computing and Amazon Web Services. CJUG March, 2009 Tom Malaher

Cloud Computing and Amazon Web Services. CJUG March, 2009 Tom Malaher Cloud Computing and Amazon Web Services CJUG March, 2009 Tom Malaher Agenda What is Cloud Computing? Amazon Web Services (AWS) Other Offerings Composing AWS Services Use Cases Ecosystem Reality Check Pros&Cons

More information

High Availability Using MySQL in the Cloud:

High Availability Using MySQL in the Cloud: High Availability Using MySQL in the Cloud: Today, Tomorrow and Keys to Success Jason Stamper, Analyst, 451 Research Michael Coburn, Senior Architect, Percona June 10, 2015 Scaling MySQL: no longer a nice-

More information

EMC BACKUP MEETS BIG DATA

EMC BACKUP MEETS BIG DATA EMC BACKUP MEETS BIG DATA Strategies To Protect Greenplum, Isilon And Teradata Systems 1 Agenda Big Data: Overview, Backup and Recovery EMC Big Data Backup Strategy EMC Backup and Recovery Solutions for

More information

Managing MySQL Scale Through Consolidation

Managing MySQL Scale Through Consolidation Hello Managing MySQL Scale Through Consolidation Percona Live 04/15/15 Chris Merz, @merzdba DB Systems Architect, SolidFire Enterprise Scale MySQL Challenges Many MySQL instances (10s-100s-1000s) Often

More information

Welcome to Virtual Developer Day MySQL!

Welcome to Virtual Developer Day MySQL! Welcome to Virtual Developer Day MySQL! Keynote: Developer and DBA Guide to What s New in MySQL Andrew Morgan - MySQL Product Management @andrewmorgan www.clusterdb.com 1 Program Agenda 1:00 PM Keynote:

More information

Scalable Web Application

Scalable Web Application Scalable Web Applications Reference Architectures and Best Practices Brian Adler, PS Architect 1 Scalable Web Application 2 1 Scalable Web Application What? An application built on an architecture that

More information

MakeMyTrip CUSTOMER SUCCESS STORY

MakeMyTrip CUSTOMER SUCCESS STORY MakeMyTrip CUSTOMER SUCCESS STORY MakeMyTrip is the leading travel site in India that is running two ClustrixDB clusters as multi-master in two regions. It removed single point of failure. MakeMyTrip frequently

More information

The Flash Transformed Data Center & the Unlimited Future of Flash John Scaramuzzo Sr. Vice President & General Manager, Enterprise Storage Solutions

The Flash Transformed Data Center & the Unlimited Future of Flash John Scaramuzzo Sr. Vice President & General Manager, Enterprise Storage Solutions The Flash Transformed Data Center & the Unlimited Future of Flash John Scaramuzzo Sr. Vice President & General Manager, Enterprise Storage Solutions Flash Memory Summit 5-7 August 2014 1 Forward-Looking

More information

Intro to AWS: Storage Services

Intro to AWS: Storage Services Intro to AWS: Storage Services Matt McClean, AWS Solutions Architect 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved AWS storage options Scalable object storage Inexpensive archive

More information

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud)

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud) Open Cloud System (Integration of Eucalyptus, Hadoop and into deployment of University Private Cloud) Thinn Thu Naing University of Computer Studies, Yangon 25 th October 2011 Open Cloud System University

More information

WOLKEN KOSTEN GELD GUSTAVO ALONSO SYSTEMS GROUP ETH ZURICH WWW.SYSTEMS.ETHZ.CH

WOLKEN KOSTEN GELD GUSTAVO ALONSO SYSTEMS GROUP ETH ZURICH WWW.SYSTEMS.ETHZ.CH WOLKEN KOSTEN GELD GUSTAVO ALONSO SYSTEMS GROUP ETH ZURICH WWW.SYSTEMS.ETHZ.CH ELCA Update June 16, 2010, Gustavo Alonso About the speaker Professor of Computer Science at ETH Zurich Areas of interest:

More information

MongoDB and Couchbase

MongoDB and Couchbase Benchmarking MongoDB and Couchbase No-SQL Databases Alex Voss Chris Choi University of St Andrews TOP 2 Questions Should a social scientist buy MORE or UPGRADE computers? Which DATABASE(s)? Document Oriented

More information

Can Flash help you ride the Big Data Wave? Steve Fingerhut Vice President, Marketing Enterprise Storage Solutions Corporation

Can Flash help you ride the Big Data Wave? Steve Fingerhut Vice President, Marketing Enterprise Storage Solutions Corporation Can Flash help you ride the Big Data Wave? Steve Fingerhut Vice President, Marketing Enterprise Storage Solutions Corporation Forward-Looking Statements During our meeting today we may make forward-looking

More information

High Performance MySQL Choices in Amazon Web Services: Beyond RDS. Andrew Shieh, SmugMug Operations shandrew @ smugmug.

High Performance MySQL Choices in Amazon Web Services: Beyond RDS. Andrew Shieh, SmugMug Operations shandrew @ smugmug. High Performance MySQL Choices in Amazon Web Services: Beyond RDS Andrew Shieh, SmugMug Operations shandrew @ smugmug.com April 15, 2015 Agenda 2 All about AWS Current RDS alternatives Cloud failures ->

More information

The Power of Pentaho and Hadoop in Action. Demonstrating MapReduce Performance at Scale

The Power of Pentaho and Hadoop in Action. Demonstrating MapReduce Performance at Scale The Power of Pentaho and Hadoop in Action Demonstrating MapReduce Performance at Scale Introduction Over the last few years, Big Data has gone from a tech buzzword to a value generator for many organizations.

More information

Introduction to Polyglot Persistence. Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace

Introduction to Polyglot Persistence. Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace Introduction to Polyglot Persistence Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace FOSSCOMM 2016 Background - 14 years in databases and system engineering - NoSQL DBA @ ObjectRocket

More information

High Availability of VistA EHR in Cloud. ViSolve Inc. White Paper February 2015. www.visolve.com

High Availability of VistA EHR in Cloud. ViSolve Inc. White Paper February 2015. www.visolve.com High Availability of VistA EHR in Cloud ViSolve Inc. White Paper February 2015 1 Abstract Inspite of the accelerating migration to cloud computing in the Healthcare Industry, high availability and uptime

More information

The Flash-Transformed Financial Data Center. Jean S. Bozman Enterprise Solutions Manager, Enterprise Storage Solutions Corporation August 6, 2014

The Flash-Transformed Financial Data Center. Jean S. Bozman Enterprise Solutions Manager, Enterprise Storage Solutions Corporation August 6, 2014 The Flash-Transformed Financial Data Center Jean S. Bozman Enterprise Solutions Manager, Enterprise Storage Solutions Corporation August 6, 2014 Forward-Looking Statements During our meeting today we will

More information

Performance and Scalability Overview

Performance and Scalability Overview Performance and Scalability Overview This guide provides an overview of some of the performance and scalability capabilities of the Pentaho Business Analytics platform. PENTAHO PERFORMANCE ENGINEERING

More information

Ø Teaching Evaluations. q Open March 3 through 16. Ø Final Exam. q Thursday, March 19, 4-7PM. Ø 2 flavors: q Public Cloud, available to public

Ø Teaching Evaluations. q Open March 3 through 16. Ø Final Exam. q Thursday, March 19, 4-7PM. Ø 2 flavors: q Public Cloud, available to public Announcements TIM 50 Teaching Evaluations Open March 3 through 16 Final Exam Thursday, March 19, 4-7PM Lecture 19 20 March 12, 2015 Cloud Computing Cloud Computing: refers to both applications delivered

More information

Big data blue print for cloud architecture

Big data blue print for cloud architecture Big data blue print for cloud architecture -COGNIZANT Image Area Prabhu Inbarajan Srinivasan Thiruvengadathan Muralicharan Gurumoorthy Praveen Codur 2012, Cognizant Next 30 minutes Big Data / Cloud challenges

More information

Open source large scale distributed data management with Google s MapReduce and Bigtable

Open source large scale distributed data management with Google s MapReduce and Bigtable Open source large scale distributed data management with Google s MapReduce and Bigtable Ioannis Konstantinou Email: [email protected] Web: http://www.cslab.ntua.gr/~ikons Computing Systems Laboratory

More information

Using MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam [email protected]

Using MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam sastry.vedantam@oracle.com Using MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam [email protected] Agenda The rise of Big Data & Hadoop MySQL in the Big Data Lifecycle MySQL Solutions for Big Data Q&A

More information

OTM in the Cloud. Ryan Haney

OTM in the Cloud. Ryan Haney OTM in the Cloud Ryan Haney The Cloud The Cloud is a set of services and technologies that delivers real-time and ondemand computing resources Software as a Service (SaaS) delivers preconfigured applications,

More information

www.expaway.com Offerte del 10 maggio 2013

www.expaway.com Offerte del 10 maggio 2013 www.expaway.com Offerte del 10 maggio 2013 On behalf of our client, a leading Internet Company in Berlin, we are currently accepting resumes for the following professional opportunity: PS007D SENIOR BACKEND

More information

Building. Applications. in the Cloud. Concepts, Patterns, and Projects. AAddison-Wesley. Christopher M. Mo^ar. Cape Town Sydney.

Building. Applications. in the Cloud. Concepts, Patterns, and Projects. AAddison-Wesley. Christopher M. Mo^ar. Cape Town Sydney. Building Applications in the Cloud Concepts, Patterns, and Projects Christopher M. Mo^ar Upper Saddle River, NJ Boston AAddison-Wesley New York 'Toronto Montreal London Munich Indianapolis San Francisco

More information

Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh

Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 1 Hadoop: A Framework for Data- Intensive Distributed Computing CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 2 What is Hadoop? Hadoop is a software framework for distributed processing of large datasets

More information

STeP-IN SUMMIT 2013. June 18 21, 2013 at Bangalore, INDIA. Performance Testing of an IAAS Cloud Software (A CloudStack Use Case)

STeP-IN SUMMIT 2013. June 18 21, 2013 at Bangalore, INDIA. Performance Testing of an IAAS Cloud Software (A CloudStack Use Case) 10 th International Conference on Software Testing June 18 21, 2013 at Bangalore, INDIA by Sowmya Krishnan, Senior Software QA Engineer, Citrix Copyright: STeP-IN Forum and Quality Solutions for Information

More information

CiteSeer x in the Cloud

CiteSeer x in the Cloud Published in the 2nd USENIX Workshop on Hot Topics in Cloud Computing 2010 CiteSeer x in the Cloud Pradeep B. Teregowda Pennsylvania State University C. Lee Giles Pennsylvania State University Bhuvan Urgaonkar

More information

Building Your Big Data Team

Building Your Big Data Team Building Your Big Data Team With all the buzz around Big Data, many companies have decided they need some sort of Big Data initiative in place to stay current with modern data management requirements.

More information

MOVING THE ELEPHANT IN THE ROOM Data Migration at Scale

MOVING THE ELEPHANT IN THE ROOM Data Migration at Scale MOVING THE ELEPHANT IN THE ROOM Data Migration at Scale WHO AM I? BDPA Los Angeles Chapter 4 year HSCC participant Columbia University, CC 14 Conductor, Inc. linkedin.com/in/calltyrone 2 CONDUCTOR, INC.

More information

Secrets to Backups and Disaster Recovery for Architects. How to Prevent Losing Your Data And Losing Your Business

Secrets to Backups and Disaster Recovery for Architects. How to Prevent Losing Your Data And Losing Your Business Secrets to Backups and Disaster Recovery for Architects How to Prevent Losing Your Data And Losing Your Business Secrets to Backups & Disaster Recovery 1 About ArcSource 2 Backup Stats and Stories 3 Backup

More information

Microservices on AWS

Microservices on AWS Microservices on AWS AWS Summit Berlin 2016 Matthias Jung, Solutions Architect Julien Simon, Evangelist April, 12 th, 2016 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda

More information

Open source Google-style large scale data analysis with Hadoop

Open source Google-style large scale data analysis with Hadoop Open source Google-style large scale data analysis with Hadoop Ioannis Konstantinou Email: [email protected] Web: http://www.cslab.ntua.gr/~ikons Computing Systems Laboratory School of Electrical

More information

ebay Storage, From Good to Great

ebay Storage, From Good to Great ebay Storage, From Good to Great Farid Yavari Sr. Storage Architect - Global Platform & Infrastructure September 11,2014 ebay Journey from Good to Great 2009 to 2011 TURNAROUND 2011 to 2013 POSITIONING

More information

Hadoop IST 734 SS CHUNG

Hadoop IST 734 SS CHUNG Hadoop IST 734 SS CHUNG Introduction What is Big Data?? Bulk Amount Unstructured Lots of Applications which need to handle huge amount of data (in terms of 500+ TB per day) If a regular machine need to

More information

Big Data Web Analytics Platform on AWS for Yottaa

Big Data Web Analytics Platform on AWS for Yottaa Big Data Web Analytics Platform on AWS for Yottaa Background Yottaa is a young, innovative company, providing a website acceleration platform to optimize Web and mobile applications and maximize user experience,

More information

Savanna Hadoop on. OpenStack. Savanna Technical Lead

Savanna Hadoop on. OpenStack. Savanna Technical Lead Savanna Hadoop on OpenStack Sergey Lukjanov Savanna Technical Lead Mirantis, 2013 Agenda Savanna Overview Savanna Use Cases Roadmap & Current Status Architecture & Features Overview Hadoop vs. Virtualization

More information

Planning an OpenStack PoC

Planning an OpenStack PoC Accelerating the adoption of Cloud Computing Planning an OpenStack PoC Webinar April 10, 2014 Speakers Today Brad Vaughan Seth Fox 20 years as architect of infrastructure solutions for the enterprise Experience

More information

Real Time Big Data Processing

Real Time Big Data Processing Real Time Big Data Processing Cloud Expo 2014 Ian Meyers Amazon Web Services Global Infrastructure Deployment & Administration App Services Analytics Compute Storage Database Networking AWS Global Infrastructure

More information

How, What, and Where of Data Warehouses for MySQL

How, What, and Where of Data Warehouses for MySQL How, What, and Where of Data Warehouses for MySQL Robert Hodges CEO, Continuent. Introducing Continuent The leading provider of clustering and replication for open source DBMS Our Product: Continuent Tungsten

More information

Web Application Deployment in the Cloud Using Amazon Web Services From Infancy to Maturity

Web Application Deployment in the Cloud Using Amazon Web Services From Infancy to Maturity P3 InfoTech Solutions Pvt. Ltd http://www.p3infotech.in July 2013 Created by P3 InfoTech Solutions Pvt. Ltd., http://p3infotech.in 1 Web Application Deployment in the Cloud Using Amazon Web Services From

More information

Building HTML5 and hybrid mobile apps using cloud services. Andrei Glazunov

Building HTML5 and hybrid mobile apps using cloud services. Andrei Glazunov Building HTML5 and hybrid mobile apps using cloud services Andrei Glazunov About Exadel Exadel is a global software engineering company. Founded in 1998, headquarters in San Francisco Bay Area 7 development

More information

YARN, the Apache Hadoop Platform for Streaming, Realtime and Batch Processing

YARN, the Apache Hadoop Platform for Streaming, Realtime and Batch Processing YARN, the Apache Hadoop Platform for Streaming, Realtime and Batch Processing Eric Charles [http://echarles.net] @echarles Datalayer [http://datalayer.io] @datalayerio FOSDEM 02 Feb 2014 NoSQL DevRoom

More information

Data Pipeline with Kafka

Data Pipeline with Kafka Data Pipeline with Kafka Peerapat Asoktummarungsri AGODA Senior Software Engineer Agoda.com Contributor Thai Java User Group (THJUG.com) Contributor Agile66 AGENDA Big Data & Data Pipeline Kafka Introduction

More information

Web Application Hosting in the AWS Cloud Best Practices

Web Application Hosting in the AWS Cloud Best Practices Web Application Hosting in the AWS Cloud Best Practices September 2012 Matt Tavis, Philip Fitzsimons Page 1 of 14 Abstract Highly available and scalable web hosting can be a complex and expensive proposition.

More information

Hadoop on OpenStack Cloud. Dmitry Mescheryakov Software Engineer, @MirantisIT

Hadoop on OpenStack Cloud. Dmitry Mescheryakov Software Engineer, @MirantisIT Hadoop on OpenStack Cloud Dmitry Mescheryakov Software Engineer, @MirantisIT Agenda OpenStack Sahara Demo Hadoop Performance on Cloud Conclusion OpenStack Open source cloud computing platform 17,209 commits

More information

OPTIMIZING APPLICATION MONITORING

OPTIMIZING APPLICATION MONITORING OPTIMIZING APPLICATION MONITORING INTRODUCTION As the use of n-tier computing architectures become ubiquitous, the availability and performance of software applications is ever more dependent upon complex

More information

Embracing Open Source: Practice and Experience from Alibaba. Wensong Zhang The 11 th Northeast Asia OSS Promotion Forum 2012.11.13

Embracing Open Source: Practice and Experience from Alibaba. Wensong Zhang The 11 th Northeast Asia OSS Promotion Forum 2012.11.13 Embracing Open Source: Practice and Experience from Alibaba Wensong Zhang The 11 th Northeast Asia OSS Promotion Forum 2012.11.13 1 Agenda 1. Overview of Alibaba 2. Taobao Software Infrastructure 3. Cases:

More information

Power Tools for Pivotal Tracker

Power Tools for Pivotal Tracker Power Tools for Pivotal Tracker Pivotal Labs Dezmon Fernandez Victoria Kay Eric Dattore June 16th, 2015 Power Tools for Pivotal Tracker 1 Client Description Pivotal Labs is an agile software development

More information

Scaling Pinterest. Yash Nelapati Ascii Artist. Pinterest Engineering. Saturday, August 31, 13

Scaling Pinterest. Yash Nelapati Ascii Artist. Pinterest Engineering. Saturday, August 31, 13 Scaling Pinterest Yash Nelapati Ascii Artist Pinterest is... An online pinboard to organize and share what inspires you. Growth March 2010 Page views per day Mar 2010 Jan 2011 Jan 2012 May 2012 Growth

More information

Time-to-live. Adrian Colyer CTO SpringSource

Time-to-live. Adrian Colyer CTO SpringSource Time-to-live Adrian Colyer CTO SpringSource Time-to-live Time to (going) live the time from initial project / feature / service conception to live software Slide 2 Time-to-live Time to value the time from

More information

Search Big Data with MySQL and Sphinx. Mindaugas Žukas www.ivinco.com

Search Big Data with MySQL and Sphinx. Mindaugas Žukas www.ivinco.com Search Big Data with MySQL and Sphinx Mindaugas Žukas www.ivinco.com Agenda Big Data Architecture Factors and Technologies MySQL and Big Data Sphinx Search Server overview Case study: building a Big Data

More information

Improve performance and availability of Banking Portal with HADOOP

Improve performance and availability of Banking Portal with HADOOP Improve performance and availability of Banking Portal with HADOOP Our client is a leading U.S. company providing information management services in Finance Investment, and Banking. This company has a

More information

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES Relational vs. Non-Relational Architecture Relational Non-Relational Rational Predictable Traditional Agile Flexible Modern 2 Agenda Big Data

More information

Database Virtualization and the Cloud

Database Virtualization and the Cloud Database Virtualization and the Cloud How database virtualization, cloud computing and other advances will reshape the database landscape by Mike Hogan, CEO ScaleDB Inc. December 10, 2009 Introduction

More information

Are You Ready for the Holiday Rush?

Are You Ready for the Holiday Rush? Are You Ready for the Holiday Rush? Five Survival Tips Written by Joseph Palumbo, Cloud Usability Team Leader Are You Ready for the Holiday Rush? Five Survival Tips Cover Table of Contents 1. Vertical

More information

From SOA to MOA: Transitioning from Service to Mobile Oriented Architectures

From SOA to MOA: Transitioning from Service to Mobile Oriented Architectures From SOA to MOA: Transitioning from Service to Mobile Oriented Architectures OVERVIEW The evolution of enterprise mobility has brought with it new challenges to organizations attempting to enable the right

More information

Time series IoT data ingestion into Cassandra using Kaa

Time series IoT data ingestion into Cassandra using Kaa Time series IoT data ingestion into Cassandra using Kaa Andrew Shvayka [email protected] Agenda Data ingestion challenges Why Kaa? Why Cassandra? Reference architecture overview Hands-on Sandbox

More information

Certified Cloud Computing Professional VS-1067

Certified Cloud Computing Professional VS-1067 Certified Cloud Computing Professional VS-1067 Certified Cloud Computing Professional Certification Code VS-1067 Vskills Cloud Computing Professional assesses the candidate for a company s cloud computing

More information

BUILDING THE IXONOS ELASTIC CLOUD WITH RED HAT. Ixonos Plc

BUILDING THE IXONOS ELASTIC CLOUD WITH RED HAT. Ixonos Plc BUILDING THE IXONOS ELASTIC CLOUD WITH RED HAT IXONOS IN BRIEF We create wireless technologies, connected devices as well as multichannel online services and mobile apps. We enhance the competitiveness

More information

bla bla OPEN-XCHANGE Open-Xchange Hardware Needs

bla bla OPEN-XCHANGE Open-Xchange Hardware Needs bla bla OPEN-XCHANGE Open-Xchange Hardware Needs OPEN-XCHANGE: Open-Xchange Hardware Needs Publication date Wednesday, 8 January version. . Hardware Needs with Open-Xchange.. Overview The purpose of this

More information

How is a rational (big) data deployment approach like optimizing the generation mix of a power company?

How is a rational (big) data deployment approach like optimizing the generation mix of a power company? How is a rational (big) data deployment approach like optimizing the generation mix of a power company? John Akred & Stephen O Sullivan @SVDataScience John Akred @BigDataAnalysis Puppy Daddy PIF Husband

More information

TECHNOLOGY WHITE PAPER Jun 2012

TECHNOLOGY WHITE PAPER Jun 2012 TECHNOLOGY WHITE PAPER Jun 2012 Technology Stack C# Windows Server 2008 PHP Amazon Web Services (AWS) Route 53 Elastic Load Balancing (ELB) Elastic Compute Cloud (EC2) Amazon RDS Amazon S3 Elasticache

More information

Protect Data... in the Cloud

Protect Data... in the Cloud QUASICOM Private Cloud Backups with ExaGrid Deduplication Disk Arrays Martin Lui Senior Solution Consultant Quasicom Systems Limited Protect Data...... in the Cloud 1 Mobile Computing Users work with their

More information

Elasticsearch on Cisco Unified Computing System: Optimizing your UCS infrastructure for Elasticsearch s analytics software stack

Elasticsearch on Cisco Unified Computing System: Optimizing your UCS infrastructure for Elasticsearch s analytics software stack Elasticsearch on Cisco Unified Computing System: Optimizing your UCS infrastructure for Elasticsearch s analytics software stack HIGHLIGHTS Real-Time Results Elasticsearch on Cisco UCS enables a deeper

More information

Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings

Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Solution Brief Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Introduction Accelerating time to market, increasing IT agility to enable business strategies, and improving

More information

C a r l G o e t h a l s T e r r e m a r k E u r o p e. C a r l. g o e t h a l s @ t e r r e m a r k. c o m

C a r l G o e t h a l s T e r r e m a r k E u r o p e. C a r l. g o e t h a l s @ t e r r e m a r k. c o m Cloud Computing for Government & Enterprise C a r l G o e t h a l s T e r r e m a r k E u r o p e C a r l. g o e t h a l s @ t e r r e m a r k. c o m Agenda Some history on Terremark State of Cloud Public

More information

MySQL és Hadoop mint Big Data platform (SQL + NoSQL = MySQL Cluster?!)

MySQL és Hadoop mint Big Data platform (SQL + NoSQL = MySQL Cluster?!) MySQL és Hadoop mint Big Data platform (SQL + NoSQL = MySQL Cluster?!) Erdélyi Ernő, Component Soft Kft. [email protected] www.component.hu 2013 (c) Component Soft Ltd Leading Hadoop Vendor Copyright 2013,

More information

Alexander Rubin Principle Architect, Percona April 18, 2015. Using Hadoop Together with MySQL for Data Analysis

Alexander Rubin Principle Architect, Percona April 18, 2015. Using Hadoop Together with MySQL for Data Analysis Alexander Rubin Principle Architect, Percona April 18, 2015 Using Hadoop Together with MySQL for Data Analysis About Me Alexander Rubin, Principal Consultant, Percona Working with MySQL for over 10 years

More information

Modern IT Operations Management. Why a New Approach is Required, and How Boundary Delivers

Modern IT Operations Management. Why a New Approach is Required, and How Boundary Delivers Modern IT Operations Management Why a New Approach is Required, and How Boundary Delivers TABLE OF CONTENTS EXECUTIVE SUMMARY 3 INTRODUCTION: CHANGING NATURE OF IT 3 WHY TRADITIONAL APPROACHES ARE FAILING

More information

Flash Technology Update from Micron and Intel

Flash Technology Update from Micron and Intel Flash Technology Update from Micron and Intel 3D NAND Technology Announcement Brian Shirley, Vice President, Memory and Technology Solutions, Micron Technology Scott DeBoer, Vice President, Research and

More information

Winning Against All Odds: Big Data for the Budget Travel Industry. Silviu Preoteasa Head of Marketing Technology

Winning Against All Odds: Big Data for the Budget Travel Industry. Silviu Preoteasa Head of Marketing Technology Winning Against All Odds: Big Data for the Budget Travel Industry Silviu Preoteasa Head of Marketing Technology ABOUT Launched in 1999 6M+ visitors / mo 1M+ pages indexed in Google 30,000+ properties listed

More information

The Pros and Cons of Erasure Coding & Replication vs. RAID in Next-Gen Storage Platforms. Abhijith Shenoy Engineer, Hedvig Inc.

The Pros and Cons of Erasure Coding & Replication vs. RAID in Next-Gen Storage Platforms. Abhijith Shenoy Engineer, Hedvig Inc. The Pros and Cons of Erasure Coding & Replication vs. RAID in Next-Gen Storage Platforms Abhijith Shenoy Engineer, Hedvig Inc. @hedviginc The need for new architectures Business innovation Time-to-market

More information

Middleware- Driven Mobile Applications

Middleware- Driven Mobile Applications Middleware- Driven Mobile Applications A motwin White Paper When Launching New Mobile Services, Middleware Offers the Fastest, Most Flexible Development Path for Sophisticated Apps 1 Executive Summary

More information