Pavlo Baron. Big Data and CDN

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Pavlo Baron. Big Data and CDN"

Transcription

1 Pavlo Baron Big Data and CDN

2 Pavlo Baron

3

4

5 What is Big Data

6 Big Data describes datasets that grow so large that they become awkward to work with using on-hand database management tools (Wikipedia)

7 Huh?

8 Somewhere a mosquito coughs

9 and somewhere else a data center gets flooded with data

10 Huh???

11 More than 30 billion pieces of content (web links, news stories, blog posts, notes, photo albums, etc.) get shared each month on Facebook

12 Aha

13 Twitter users are, in total, tweeting an average of 55 million tweets a day, also including links etc.

14 OMG!

15 But there is even much more: cameras, sensors, RFID, logs, geolocation, GIS and so on

16 kk

17 There are several perspectives at Big Data

18 Data storage and archiving

19 Data preparation

20 Live data provisioning

21 Data analysis / analytics

22 Real-time event and stream processing

23 Data visualization

24 Where does Big Data come from

25 Uncontrolled human activities in the world wide web, or Web 2.0 if you like

26 Huh?

27 Das Bild kann nicht angezeigt werden. Dieser Computer verfügt möglicherweise über zu wenig Arbeitsspeicher, um das Bild zu öffnen, oder das Bild ist beschädigt. Starten Sie den Computer neu, und öffnen Sie dann erneut die Datei. Wenn weiterhin das rote x angezeigt wird, müssen Sie das Bild möglicherweise löschen und dann erneut einfügen. Every human leaves a vast number of data marks on the web every day: intentionally, accidentally and unknowingly

28 Huh???

29 Intentionally: we blog, tweet, upload, flatter, link, etc

30 And: the web has become an industry of its own. With us in the thick of it

31 Accidentally: we are humans and we make mistakes

32 Unknowingly: we get tricked, misled, controlled, logged etc

33 The vast number of data marks we leave on the web every day gets copied, duplicated. Data explodes.

34 Panic!

35 Wait! There s even more!

36 Huh?

37 Data flowing on streams at a very high rate from many actors

38 Huh??

39 The amount of data flying over the air has become enormous, and it s growing unpredictably

40 Aha

41 It s not only nuclear reactors anymore having hi-tech sensors and generating tons of data

42 Aha

43 And our physically huge globe

44 has become a tiny electronic ball. It s completely wired. Data needs just seconds to circumnavigate the world

45 OMG!

46 But there s even more!

47 Huh?

48 Laws and regulations force us to store and archive all sorts of data, and it s getting more and more

49 Human knowledge grows extremely fast. It s far too gigantic for one single brain

50 Oh no

51 And there s still more!

52 Huh?

53 Big Brother Big Data. We get observed, filmed, recorded, logged, geolocated etc.

54 Panic!

55 Don t panic. Get over it. Brace yourself for the battle.

56 First of all, some major changes have happened

57 Instead of huge expensive cabinets

58 we can use lots of cheap commodity hardware

59 Physics hit the wall

60 and we need to think parallel

61 Our physically huge globe

62 s has become a tiny electronic ball. It s completely wired

63 Spontaneous requirements

64 can be covered by the fog (aka cloud)

65 And what are my weapons

66 Cut your data in smaller pieces

67 Make those pieces bitesize (manageable)

68 Bring the data closer to those who need it

69 Bring the data closer to where it s physically accessed

70 Give up relations where you don t need them

71 Give up actuality where you don t need it

72 Find optimal and effective replication mechanisms

73 Consider latency an adjustment screw if you can

74 Consider availability an adjustment screw if you can

75 Be prepared to deal with unlimited amount of data depending on the perspective

76 Know your data

77 Know your volumes

78 Know your scenarios

79 Consider it what it is: a science

80 Right tool for the job

81 kk

82 And how does this technically work

83 Live data provisioning

84 What s the problem

85 Your users are widely spread, maybe all over the world

86 And you own Big Data, which has many facets geographic, financial etc.

87 And your classic silo architecture could break under the weight of such data

88 And why would I need that

89 You start and want to be one of those. Aha, ok

90 You simply grew up to a level

91 Now you need to segment your users and thus to be faster and more reliable at locations,

92 to keep your servers free of load and thus to avoid bottle necks,

93 to cut your big data in smaller, better manageable chunks

94 What are my weapons

95 If your content is static in web terms, you are already well prepared

96 In many cases, you can make your dynamic data static (precompute content)

97 Huh?

98 Let s take a look at an online bookstore

99 Hey, the online bookstore is completely dynamic (except images) it s a shop system!

100 Really?

101 Book description page: even when you modify the prices and offer Web 2.0 features such as rating you still can pre-compute the page at some time you don t need to compute the content while the page is getting accessed

102 Browser mode: this is a classic use case for static content precomputation. There is often simply no need to navigate through dynamically built paths

103 Book search: even this ultimately dynamic sounding feature can be (partially) dedynamized. Consider the index as static content, not necessarily the data itself

104 You see: many parts of an online bookstore seem dynamic, but can be actually pre-computed and delivered as static content in web terms. It s all about the frequency of change and the big data pain

105 Owning big data doesn t necessarily mean owning 100% dynamic data in terms of web

106 Aha

107 And now distribute it with CDN content delivery network

108 Huh?

109 Akamai web traffic dominance

110 Akamai web traffic monitoring

111 Akamai EdgePlatform

112 73,000 servers 70 countries 1,000 networks 30% of world s web traffic (OMG, is the rest Google?)

113 There are several CDN providers offering (world wide) such infrastructures

114 And now let s get a little insane

115

116

117

118 Huh???

119 Yeah, something s going on behind the scenes

120 How does this technically work

121 CDN is like a deputy. You make a contract, and it takes over parts of your platform. From here, it delivers to your users the content you tell it to deliver, but being much closer to them and much more intelligent than you when it comes to managing the load

122 Huh?

123 CDN has its infrastructure including actual nodes directly at the backbones, offering web caching, server load balancing, request routing and, based upon these techniques, content delivery services

124 Aha

125 What you have seen earlier: based on the IP address of the machine (origin) which made the DNS A query, the DNS server of the CDN has each time decided to return a different IP address e.g. one from the same geographic region

126 Aha

127 What you now can expect is that the returned IP address leads you to a load balancer your gate to a whole subinfrastructure of the CDN which balances between web caches or web servers or similar

128 Aha

129 CDN uses different algorithms to decide where it routes user requests to: based upon current load, cost, location etc.

130 Aha

131 But in the end, your content gets delivered to the user. If it expires, CDN refreshes it from your servers in the background

132 Often, you have to offer the last mile the very last database access, e.g. the last item view or similar. Here, the user hits your server

133 Huh?

134 cache access A query inter-cache updates caches cache refresh caches your servers

135 kk

136 How can I benefit from this having big data

137 When you have e.g. images as your big data, you can consider this data as static and thus push down-/uploads to CDN. So, you segment your users and keep your own servers free of load. What you might lose, is consistency between segments

138 Or you pre-compute static content out of your dynamic big data a sort of snapshots, and push them to CDN. So, you keep you database servers free of load and scale only through the web servers. Complexity comes with the snapshot management

139 Or you can even push some functional parts of your platform to CDN such as searches etc. You win a lot dealing with big data, but you are more dependent from the CDN provider, and your overall architecture is weaker

140 Or if you really want to experiment, you can even try to push whole executed database queries to CDN like you would do it with memory caches. That s really cool, but even much more complex and unreliable than a clusterdistributed memory cache

141 If you use CDN to collect your new data, you might need some complex replication mechanisms

142 Anyway, with all that in mind: you can have a lot of your big data out there with CDN

143 Thank you

144

145 Most images were licensed from istockphoto.com Several images were taken from corresponding Wikipedia articles, product pages and open sources

Smart moulds intelligente Formen

Smart moulds intelligente Formen 1 Positioning Factory Automation Enterprise Layer SOLVE IT! Control Layer I/O-Layer BUS IT! Connectivity Layer CONNECT IT! Sensor Layer SENSE IT! 2 What is an intelligent chocolate mould? Data can be written

More information

Arbitrage-free Volatility Surface Interpolation. Author: Dr. Kay Moritzen (B&C) Dr. Ulrich Leiner (B&C)

Arbitrage-free Volatility Surface Interpolation. Author: Dr. Kay Moritzen (B&C) Dr. Ulrich Leiner (B&C) Arbitrage-free Volatility Surface Interpolation Author: Dr. Kay Moritzen (B&C) Dr. Ulrich Leiner (B&C) Challenges we see Market prices of options reflect both, a modelinduced shape of the volatility surface

More information

The use of Vegetation for Social Housing Renovations: a cases study in the city of Palermo

The use of Vegetation for Social Housing Renovations: a cases study in the city of Palermo The use of Vegetation for Social Housing Renovations: a cases study in the city of Palermo Luisa Pastore Università degli Studi di Palermo Italy Rossella Corrao Università degli Studi di Palermo Italy

More information

USING MOBILE MONEY TO PREPAY FOR HEALTHCARE IN KENYA

USING MOBILE MONEY TO PREPAY FOR HEALTHCARE IN KENYA USING MOBILE MONEY TO PREPAY FOR HEALTHCARE IN KENYA Mexico City, November 2014 0 PHARMACCESS GROUP 1 Connecting people to quality healthcare Kenya s health system challenges, opportunities & strategies

More information

The Energy Turnaround in Europe and its Consequences for Renewable Generation, Energy Infrastructure and end-use

The Energy Turnaround in Europe and its Consequences for Renewable Generation, Energy Infrastructure and end-use Das Bild kann nicht angezeigt werden. Dieser Computer verfügt möglicherweise über zu wenig Arbeitsspeicher, um das Bild zu öffnen, oder das Bild ist beschädigt. Starten Sie den Computer neu, und öffnen

More information

Traffic delivery evolution in the Internet ENOG 4 Moscow 23 rd October 2012

Traffic delivery evolution in the Internet ENOG 4 Moscow 23 rd October 2012 Traffic delivery evolution in the Internet ENOG 4 Moscow 23 rd October 2012 January 29th, 2008 Christian Kaufmann Director Network Architecture Akamai Technologies, Inc. way-back machine Web 1998 way-back

More information

Metering PDU Manual DN-95601 DN-95602

Metering PDU Manual DN-95601 DN-95602 Metering PDU Manual DN-95601 DN-95602 Safety Advice The device must be installed only by qualified personnel according to the following installation and operating instructions. The manufacturer does not

More information

From Internet Data Centers to Data Centers in the Cloud

From Internet Data Centers to Data Centers in the Cloud From Internet Data Centers to Data Centers in the Cloud This case study is a short extract from a keynote address given to the Doctoral Symposium at Middleware 2009 by Lucy Cherkasova of HP Research Labs

More information

The EuroSDR Performance Test for Digital Aerial Camera Systems

The EuroSDR Performance Test for Digital Aerial Camera Systems Institut für Photogrammetrie Digital Airborne Camera Performance The DGPF test Overview and results Michael Cramer Photogrammetric Week 2009 Stuttgart September 8, 2009 Institut für Photogrammetrie The

More information

Combining Laser Range Measurements and a Dual-IMU IPNS for Precise Indoor SLAM

Combining Laser Range Measurements and a Dual-IMU IPNS for Precise Indoor SLAM Institute of Systems Optimization Combining Laser Range Measurements and a Dual-IMU IPNS for Precise Indoor SLAM Dipl.-Ing. Christian Ascher Dipl.-Ing. Christoph Keßler Prof. Gert Trommer Institute of

More information

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next

More information

Testing & Assuring Mobile End User Experience Before Production. Neotys

Testing & Assuring Mobile End User Experience Before Production. Neotys Testing & Assuring Mobile End User Experience Before Production Neotys Agenda Introduction The challenges Best practices NeoLoad mobile capabilities Mobile devices are used more and more At Home In 2014,

More information

Solution IT Architectures Key Elements 2 Quality and Constraints

Solution IT Architectures Key Elements 2 Quality and Constraints Kai Schwidder Distinguished IT Architect Enterprise IT Architectures Solution IT Architectures Key Elements 2 Quality and Constraints 2014 Hans-Peter Hoidn Agenda (Part I - today) Enterprise IT Architectures

More information

Content Delivery Networks (CDN) Dr. Yingwu Zhu

Content Delivery Networks (CDN) Dr. Yingwu Zhu Content Delivery Networks (CDN) Dr. Yingwu Zhu Web Cache Architecure Local ISP cache cdn Reverse Reverse Proxy Reverse Proxy Reverse Proxy Proxy L4 Switch Content Content Content Server Content Server

More information

Measuring CDN Performance. Hooman Beheshti, VP Technology

Measuring CDN Performance. Hooman Beheshti, VP Technology Measuring CDN Performance Hooman Beheshti, VP Technology Why this matters Performance is one of the main reasons we use a CDN Seems easy to measure, but isn t Performance is an easy way to comparison shop

More information

Web Caching and CDNs. Aditya Akella

Web Caching and CDNs. Aditya Akella Web Caching and CDNs Aditya Akella 1 Where can bottlenecks occur? First mile: client to its ISPs Last mile: server to its ISP Server: compute/memory limitations ISP interconnections/peerings: congestion

More information

The big data revolution

The big data revolution The big data revolution Friso van Vollenhoven (Xebia) Enterprise NoSQL Recently, there has been a lot of buzz about the NoSQL movement, a collection of related technologies mostly concerned with storing

More information

Lambda Architecture. Near Real-Time Big Data Analytics Using Hadoop. January 2015. Email: bdg@qburst.com Website: www.qburst.com

Lambda Architecture. Near Real-Time Big Data Analytics Using Hadoop. January 2015. Email: bdg@qburst.com Website: www.qburst.com Lambda Architecture Near Real-Time Big Data Analytics Using Hadoop January 2015 Contents Overview... 3 Lambda Architecture: A Quick Introduction... 4 Batch Layer... 4 Serving Layer... 4 Speed Layer...

More information

Eric Ledu, The Createch Group, a BELL company

Eric Ledu, The Createch Group, a BELL company Eric Ledu, The Createch Group, a BELL company Intelligence Analytics maturity Past Present Future Predictive Modeling Optimization What is the best that could happen? Raw Data Cleaned Data Standard Reports

More information

5 - Low Cost Ways to Increase Your

5 - Low Cost Ways to Increase Your - 5 - Low Cost Ways to Increase Your DIGITAL MARKETING Presence Contents Introduction Social Media Email Marketing Blogging Video Marketing Website Optimization Final Note 3 4 7 9 11 12 14 2 Taking a Digital

More information

Teridion. Rethinking Network Performance. The Internet. Lightning Fast. Technical White Paper July, 2015 www.teridion.com

Teridion. Rethinking Network Performance. The Internet. Lightning Fast. Technical White Paper July, 2015 www.teridion.com Teridion The Internet. Lightning Fast. Rethinking Network Performance Technical White Paper July, 2015 www.teridion.com Executive summary Online services face the growing dual challenge of supporting many

More information

WINDOWS AZURE DATA MANAGEMENT

WINDOWS AZURE DATA MANAGEMENT David Chappell October 2012 WINDOWS AZURE DATA MANAGEMENT CHOOSING THE RIGHT TECHNOLOGY Sponsored by Microsoft Corporation Copyright 2012 Chappell & Associates Contents Windows Azure Data Management: A

More information

Cloud Computing and Big Data. What s the Big Deal?

Cloud Computing and Big Data. What s the Big Deal? Cloud Computing and Big Data. What s the Big Deal? Arlene Minkiewicz, Chief Scientist PRICE Systems, LLC arlene.minkiewicz@pricesystems.com 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management

More information

The old Internet. Software in the Network: Outline. Traditional Design. 1) Basic Caching. The Arrival of Software (in the network)

The old Internet. Software in the Network: Outline. Traditional Design. 1) Basic Caching. The Arrival of Software (in the network) The old Software in the Network: What Happened and Where to Go Prof. Eric A. Brewer UC Berkeley Inktomi Corporation Local networks with local names and switches IP creates global namespace and links the

More information

Request Routing, Load-Balancing and Fault- Tolerance Solution - MediaDNS

Request Routing, Load-Balancing and Fault- Tolerance Solution - MediaDNS White paper Request Routing, Load-Balancing and Fault- Tolerance Solution - MediaDNS June 2001 Response in Global Environment Simply by connecting to the Internet, local businesses transform themselves

More information

Your Web Site Parts - Domain Names, URLs, Web Hosts, DNS - What They Are, and What You Need

Your Web Site Parts - Domain Names, URLs, Web Hosts, DNS - What They Are, and What You Need Your Web Site Parts - Domain Names, URLs, Web Hosts, DNS - What They Are, and What You Need What s all this domain name, web host, URL stuff? When you first look into getting a web site, you may find an

More information

Cloud Computing For Bioinformatics

Cloud Computing For Bioinformatics Cloud Computing For Bioinformatics Cloud Computing: what is it? Cloud Computing is a distributed infrastructure where resources, software, and data are provided in an on-demand fashion. Cloud Computing

More information

CDN and Traffic-structure

CDN and Traffic-structure CDN and Traffic-structure Outline Basics CDN Traffic Analysis 2 Outline Basics CDN Building Blocks Services Evolution Traffic Analysis 3 A Centralized Web! Slow content must traverse multiple backbones

More information

1. Comments on reviews a. Need to avoid just summarizing web page asks you for:

1. Comments on reviews a. Need to avoid just summarizing web page asks you for: 1. Comments on reviews a. Need to avoid just summarizing web page asks you for: i. A one or two sentence summary of the paper ii. A description of the problem they were trying to solve iii. A summary of

More information

Data Refinery with Big Data Aspects

Data Refinery with Big Data Aspects International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data

More information

Tableau Server 7.0 scalability

Tableau Server 7.0 scalability Tableau Server 7.0 scalability February 2012 p2 Executive summary In January 2012, we performed scalability tests on Tableau Server to help our customers plan for large deployments. We tested three different

More information

The Value of a Content Delivery Network

The Value of a Content Delivery Network September 2010 White Paper The Value of a Content Delivery Network Table of Contents Introduction... 3 Performance... 3 The Second Generation of CDNs... 6 Conclusion... 7 About NTT America... 8 Introduction

More information

Busin i ess I n I t n e t ll l i l g i e g nce c T r T e r nds For 2013

Busin i ess I n I t n e t ll l i l g i e g nce c T r T e r nds For 2013 Business Intelligence Trends For 2013 10 Trends The last few years the change in Business Intelligence seems to accelerate under the pressure of increased business demand and technology innovations. Here

More information

We are Big Data A Sonian Whitepaper

We are Big Data A Sonian Whitepaper EXECUTIVE SUMMARY Big Data is not an uncommon term in the technology industry anymore. It s of big interest to many leading IT providers and archiving companies. But what is Big Data? While many have formed

More information

Assignment # 1 (Cloud Computing Security)

Assignment # 1 (Cloud Computing Security) Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual

More information

Internet Content Distribution

Internet Content Distribution Internet Content Distribution Chapter 2: Server-Side Techniques (TUD Student Use Only) Chapter Outline Server-side techniques for content distribution Goals Mirrors Server farms Surrogates DNS load balancing

More information

2012 AKAMAI FASTER FORWARD TM

2012 AKAMAI FASTER FORWARD TM Every Second Counts Ravi Maira, Akamai Bryan Einwalter, Grainger Brad Ledford, Build.com How fast is fast enough? We were pretty sure speed mattered Origin Akamaized! and then we knew for sure. Real users

More information

Web Email DNS Peer-to-peer systems (file sharing, CDNs, cycle sharing)

Web Email DNS Peer-to-peer systems (file sharing, CDNs, cycle sharing) 1 1 Distributed Systems What are distributed systems? How would you characterize them? Components of the system are located at networked computers Cooperate to provide some service No shared memory Communication

More information

The future of connectivity and consumer rights: exponential digital transformation!? @gleonhard

The future of connectivity and consumer rights: exponential digital transformation!? @gleonhard The future of connectivity and consumer rights: exponential digital transformation!? @gleonhard Don t be fooled because technology was mostly pretty daft until now via mashable.com Twitter: @gleonhard

More information

Web Hosting 101. with Patrick McNeil

Web Hosting 101. with Patrick McNeil Web Hosting 101 with Patrick McNeil Alphabet soup Why learn the technical side? To help your clients solve problems To help you work better with your tech team To better understand how the web works To

More information

HIGH-SPEED BRIDGE TO CLOUD STORAGE

HIGH-SPEED BRIDGE TO CLOUD STORAGE HIGH-SPEED BRIDGE TO CLOUD STORAGE Addressing throughput bottlenecks with Signiant s SkyDrop 2 The heart of the Internet is a pulsing movement of data circulating among billions of devices worldwide between

More information

Big Data Scoring. April 2014

Big Data Scoring. April 2014 Big Data Scoring April 2014 There was 5 exabytes of information created between the dawn of civilization through 2003 that much information is now created every 2 days - Eric Schmidt, Google CEO 2 Why

More information

BIG DATA FUNDAMENTALS

BIG DATA FUNDAMENTALS BIG DATA FUNDAMENTALS Timeframe Minimum of 30 hours Use the concepts of volume, velocity, variety, veracity and value to define big data Learning outcomes Critically evaluate the need for big data management

More information

Big Data. Donald Kossmann & Nesime Tatbul Systems Group ETH Zurich

Big Data. Donald Kossmann & Nesime Tatbul Systems Group ETH Zurich Big Data Donald Kossmann & Nesime Tatbul Systems Group ETH Zurich Goal of Today What is Big Data? introduce all major buzz words What is not Big Data? get a feeling for opportunities & limitations Answering

More information

Overview. 15-441 15-441 Computer Networking 15-641. Lecture 18: Delivering Content Content Delivery Networks Peter Steenkiste

Overview. 15-441 15-441 Computer Networking 15-641. Lecture 18: Delivering Content Content Delivery Networks Peter Steenkiste Overview 5-44 5-44 Computer Networking 5-64 Lecture 8: Delivering Content Content Delivery Networks Peter Steenkiste Fall 04 www.cs.cmu.edu/~prs/5-44-f4 Web Consistent hashing Peer-to-peer CDN Motivation

More information

LARGE SCALE INTERNET SERVICES

LARGE SCALE INTERNET SERVICES 1 LARGE SCALE INTERNET SERVICES 2110414 Large Scale Computing Systems Natawut Nupairoj, Ph.D. Outline 2 Overview Background Knowledge Architectural Case Studies Real-World Case Study 3 Overview Overview

More information

Intelligent Content Delivery Network (CDN) The New Generation of High-Quality Network

Intelligent Content Delivery Network (CDN) The New Generation of High-Quality Network White paper Intelligent Content Delivery Network (CDN) The New Generation of High-Quality Network July 2001 Executive Summary Rich media content like audio and video streaming over the Internet is becoming

More information

The Flash-Transformed Financial Data Center. Jean S. Bozman Enterprise Solutions Manager, Enterprise Storage Solutions Corporation August 6, 2014

The Flash-Transformed Financial Data Center. Jean S. Bozman Enterprise Solutions Manager, Enterprise Storage Solutions Corporation August 6, 2014 The Flash-Transformed Financial Data Center Jean S. Bozman Enterprise Solutions Manager, Enterprise Storage Solutions Corporation August 6, 2014 Forward-Looking Statements During our meeting today we will

More information

FAMILY. Reference Guide. Pogoplug Family. Reference Guide. 2012 Cloud Engines, Inc. All Rights Reserved.

FAMILY. Reference Guide. Pogoplug Family. Reference Guide. 2012 Cloud Engines, Inc. All Rights Reserved. Reference Guide Pogoplug Family Reference Guide Table of Contents Table of Contents 1. What s Included 2. Setting Up Your Pogoplug Device 3. Back Up a. Backing Up Your Computers b. Backing Up Your Mobile

More information

Embedded inside the database. No need for Hadoop or customcode. True real-time analytics done per transaction and in aggregate. On-the-fly linking IP

Embedded inside the database. No need for Hadoop or customcode. True real-time analytics done per transaction and in aggregate. On-the-fly linking IP Operates more like a search engine than a database Scoring and ranking IP allows for fuzzy searching Best-result candidate sets returned Contextual analytics to correctly disambiguate entities Embedded

More information

Converged, Real-time Analytics Enabling Faster Decision Making and New Business Opportunities

Converged, Real-time Analytics Enabling Faster Decision Making and New Business Opportunities Technology Insight Paper Converged, Real-time Analytics Enabling Faster Decision Making and New Business Opportunities By John Webster February 2015 Enabling you to make the best technology decisions Enabling

More information

Azure Media Service Cloud Video Delivery KILROY HUGHES MICROSOFT AZURE MEDIA 2015.08.20

Azure Media Service Cloud Video Delivery KILROY HUGHES MICROSOFT AZURE MEDIA 2015.08.20 Azure Media Service Cloud Video Delivery KILROY HUGHES MICROSOFT AZURE MEDIA 2015.08.20 Azure Cloud Topology Public cloud providers such as Amazon Web Service, Google, IBM, Rackspace, etc. have similar

More information

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Agenda» Overview» What is Big Data?» Accelerates advances in computer & technologies» Revolutionizes data measurement»

More information

Object Storage: A Growing Opportunity for Service Providers. White Paper. Prepared for: 2012 Neovise, LLC. All Rights Reserved.

Object Storage: A Growing Opportunity for Service Providers. White Paper. Prepared for: 2012 Neovise, LLC. All Rights Reserved. Object Storage: A Growing Opportunity for Service Providers Prepared for: White Paper 2012 Neovise, LLC. All Rights Reserved. Introduction For service providers, the rise of cloud computing is both a threat

More information

BASICS OF SCALING: LOAD BALANCERS

BASICS OF SCALING: LOAD BALANCERS BASICS OF SCALING: LOAD BALANCERS Lately, I ve been doing a lot of work on systems that require a high degree of scalability to handle large traffic spikes. This has led to a lot of questions from friends

More information

2D Image Processing. Edge and Corner Detection. Prof. Didier Stricker Dr. Alain Pagani

2D Image Processing. Edge and Corner Detection. Prof. Didier Stricker Dr. Alain Pagani 2D Image Processing Edge and Corner Detection Prof. Didier Stricker Dr. Alain Pagani Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de

More information

XpoLog Center Suite Log Management & Analysis platform

XpoLog Center Suite Log Management & Analysis platform XpoLog Center Suite Log Management & Analysis platform Summary: 1. End to End data management collects and indexes data in any format from any machine / device in the environment. 2. Logs Monitoring -

More information

Internet Traffic Evolution 2007-2011

Internet Traffic Evolution 2007-2011 Internet Traffic Evolution 2007-2011 Craig Labovitz April 6, 2011 Talk Outline Four-year ongoing inter-domain traffic study Review of 2010 results (NANOG / IETF / SIGCOMM) Methodology Changing carrier

More information

Synapse s SNAP Network Operating System

Synapse s SNAP Network Operating System Synapse s SNAP Network Operating System by David Ewing, Chief Technology Officer, Synapse Wireless Today we are surrounded by tiny embedded machines electro-mechanical systems that monitor the environment

More information

AKAMAI WHITE PAPER. Delivering Dynamic Web Content in Cloud Computing Applications: HTTP resource download performance modelling

AKAMAI WHITE PAPER. Delivering Dynamic Web Content in Cloud Computing Applications: HTTP resource download performance modelling AKAMAI WHITE PAPER Delivering Dynamic Web Content in Cloud Computing Applications: HTTP resource download performance modelling Delivering Dynamic Web Content in Cloud Computing Applications 1 Overview

More information

Big Data With Hadoop

Big Data With Hadoop With Saurabh Singh singh.903@osu.edu The Ohio State University February 11, 2016 Overview 1 2 3 Requirements Ecosystem Resilient Distributed Datasets (RDDs) Example Code vs Mapreduce 4 5 Source: [Tutorials

More information

The Big Data Paradigm Shift. Insight Through Automation

The Big Data Paradigm Shift. Insight Through Automation The Big Data Paradigm Shift Insight Through Automation Agenda The Problem Emcien s Solution: Algorithms solve data related business problems How Does the Technology Work? Case Studies 2013 Emcien, Inc.

More information

Apache HBase. Crazy dances on the elephant back

Apache HBase. Crazy dances on the elephant back Apache HBase Crazy dances on the elephant back Roman Nikitchenko, 16.10.2014 YARN 2 FIRST EVER DATA OS 10.000 nodes computer Recent technology changes are focused on higher scale. Better resource usage

More information

CONTENT DELIVERY WHITE PAPER 2014. www.keycdn.com. proinity GmbH 1

CONTENT DELIVERY WHITE PAPER 2014. www.keycdn.com. proinity GmbH 1 CONTENT DELIVERY WHITE PAPER 2014 www.keycdn.com proinity GmbH 1 KeyCDN White Paper 2014 CONTENT 01. INTRODUCTION 03 02. FEATURES 04 03. BENEFITS 06 04. NETWORK 08 05. PRICING 09 06. ABOUT US 11 2 proinity

More information

Scalability of web applications. CSCI 470: Web Science Keith Vertanen

Scalability of web applications. CSCI 470: Web Science Keith Vertanen Scalability of web applications CSCI 470: Web Science Keith Vertanen Scalability questions Overview What's important in order to build scalable web sites? High availability vs. load balancing Approaches

More information

Big Data and Analytics: Getting Started with ArcGIS. Mike Park Erik Hoel

Big Data and Analytics: Getting Started with ArcGIS. Mike Park Erik Hoel Big Data and Analytics: Getting Started with ArcGIS Mike Park Erik Hoel Agenda Overview of big data Distributed computation User experience Data management Big data What is it? Big Data is a loosely defined

More information

Ensighten Tag Delivery Network. Advanced Infrastructure for Enterprise-Class Tag Management

Ensighten Tag Delivery Network. Advanced Infrastructure for Enterprise-Class Tag Management Ensighten Tag Delivery Network Advanced Infrastructure for Enterprise-Class Tag Management Limitless scalability for the enterprise, with vastly improved page loading results and digital touch-point performance.

More information

Digital Collections as Big Data. Leslie Johnston, Library of Congress Digital Preservation 2012

Digital Collections as Big Data. Leslie Johnston, Library of Congress Digital Preservation 2012 Digital Collections as Big Data Leslie Johnston, Library of Congress Digital Preservation 2012 Data is not just generated by satellites, identified during experiments, or collected during surveys. Datasets

More information

Open source Google-style large scale data analysis with Hadoop

Open source Google-style large scale data analysis with Hadoop Open source Google-style large scale data analysis with Hadoop Ioannis Konstantinou Email: ikons@cslab.ece.ntua.gr Web: http://www.cslab.ntua.gr/~ikons Computing Systems Laboratory School of Electrical

More information

Overlay Networks. Slides adopted from Prof. Böszörményi, Distributed Systems, Summer 2004.

Overlay Networks. Slides adopted from Prof. Böszörményi, Distributed Systems, Summer 2004. Overlay Networks An overlay is a logical network on top of the physical network Routing Overlays The simplest kind of overlay Virtual Private Networks (VPN), supported by the routers If no router support

More information

Introduction to Big Data the four V's

Introduction to Big Data the four V's Chapter 1: Introduction to Big Data the four V's This chapter is mainly based on the Big Data script by Donald Kossmann and Nesime Tatbul (ETH Zürich) Big Data Management and Analytics 15 Goal of Today

More information

Are You Ready for the Holiday Rush?

Are You Ready for the Holiday Rush? Are You Ready for the Holiday Rush? Five Survival Tips Written by Joseph Palumbo, Cloud Usability Team Leader Are You Ready for the Holiday Rush? Five Survival Tips Cover Table of Contents 1. Vertical

More information

Backup architectures in the modern data center. Author: Edmond van As edmond@competa.com Competa IT b.v.

Backup architectures in the modern data center. Author: Edmond van As edmond@competa.com Competa IT b.v. Backup architectures in the modern data center. Author: Edmond van As edmond@competa.com Competa IT b.v. Existing backup methods Most companies see an explosive growth in the amount of data that they have

More information

Learning Management Redefined. Acadox Infrastructure & Architecture

Learning Management Redefined. Acadox Infrastructure & Architecture Learning Management Redefined Acadox Infrastructure & Architecture w w w. a c a d o x. c o m Outline Overview Application Servers Databases Storage Network Content Delivery Network (CDN) & Caching Queuing

More information

Tutorial: Big Data Algorithms and Applications Under Hadoop KUNPENG ZHANG SIDDHARTHA BHATTACHARYYA

Tutorial: Big Data Algorithms and Applications Under Hadoop KUNPENG ZHANG SIDDHARTHA BHATTACHARYYA Tutorial: Big Data Algorithms and Applications Under Hadoop KUNPENG ZHANG SIDDHARTHA BHATTACHARYYA http://kzhang6.people.uic.edu/tutorial/amcis2014.html August 7, 2014 Schedule I. Introduction to big data

More information

nomorerack CUSTOMER SUCCESS STORY RELIABILITY AND AVAILABILITY WITH FAST GROWTH IN THE CLOUD

nomorerack CUSTOMER SUCCESS STORY RELIABILITY AND AVAILABILITY WITH FAST GROWTH IN THE CLOUD nomorerack RELIABILITY AND AVAILABILITY WITH FAST GROWTH IN THE CLOUD CUSTOMER SUCCESS STORY Nomorerack is one of the fastest growing e-commerce companies in the US with 1023% growth in revenue and 15-20x

More information

Big Table in Plain Language

Big Table in Plain Language Big Table in Plain Language Some people remember exactly where they were when JFK was shot. Other people remember exactly where they were when Neil Armstrong stepped on the moon. I remember exactly where

More information

Cloud Computing with Microsoft Azure

Cloud Computing with Microsoft Azure Cloud Computing with Microsoft Azure Michael Stiefel www.reliablesoftware.com development@reliablesoftware.com http://www.reliablesoftware.com/dasblog/default.aspx Azure's Three Flavors Azure Operating

More information

BM 465E Distributed Systems

BM 465E Distributed Systems BM 465E Distributed Systems Lecture 4 Networking (cont.) Mehmet Demirci Today Overlay networks Data centers Content delivery networks Overlay Network A virtual network built on top of another network Overlay

More information

www.coremedia.com The Content Distribution Network (CDN) Challenge A Hybrid Approach to Dynamic, High Performance Web Caching

www.coremedia.com The Content Distribution Network (CDN) Challenge A Hybrid Approach to Dynamic, High Performance Web Caching www.coremedia.com The Content Distribution Network (CDN) Challenge A Hybrid Approach to Dynamic, High Performance Web Caching Content Distribution Networks (CDNs) are a popular and effective means of increasing

More information

Integrated Physical Security and Incident Management

Integrated Physical Security and Incident Management IT Enterprise Services Integrated Physical Security and Incident Management Every organisation needs to be confident about its physical security and its ability to respond to unexpected incidents. Protecting

More information

development : Canadian and international best practices

development : Canadian and international best practices Social media for economic development : Canadian and international best practices APDEQ & EDAC Conference 2010 Quebec City, QC Active In Regional Economic Development Strategies Since 1998 We apply: Business

More information

Big Data Tools: Game Changer for Mainstream Enterprises

Big Data Tools: Game Changer for Mainstream Enterprises Big Data Tools: Game Changer for Mainstream Enterprises Aashish Chandra DVP, Application Modernization, Sears Holdings & GM, Big Data / Legacy Modernization, MetaScale LLC Inside the Modern Consumer s

More information

Content Delivery Networks. Shaxun Chen April 21, 2009

Content Delivery Networks. Shaxun Chen April 21, 2009 Content Delivery Networks Shaxun Chen April 21, 2009 Outline Introduction to CDN An Industry Example: Akamai A Research Example: CDN over Mobile Networks Conclusion Outline Introduction to CDN An Industry

More information

http://bradhedlund.com/?p=3108

http://bradhedlund.com/?p=3108 http://bradhedlund.com/?p=3108 This article is Part 1 in series that will take a closer look at the architecture and methods of a Hadoop cluster, and how it relates to the network and server infrastructure.

More information

Ubuntu: helping drive business insight from Big Data

Ubuntu: helping drive business insight from Big Data WHITE PAPER Ubuntu: helping drive business insight from Big Data February 2012 Copyright Canonical 2012 www.canonical.com Executive introduction For years, web giants such as Facebook, Google and ebay

More information

ANATOMY OF A DDoS ATTACK AGAINST THE DNS INFRASTRUCTURE

ANATOMY OF A DDoS ATTACK AGAINST THE DNS INFRASTRUCTURE ANATOMY OF A DDoS ATTACK AGAINST THE DNS INFRASTRUCTURE ANATOMY OF A DDOS ATTACK AGAINST THE DNS INFRASTRUCTURE The Domain Name System (DNS) is part of the functional infrastructure of the Internet and

More information

Lecture 3: Scaling by Load Balancing 1. Comments on reviews i. 2. Topic 1: Scalability a. QUESTION: What are problems? i. These papers look at

Lecture 3: Scaling by Load Balancing 1. Comments on reviews i. 2. Topic 1: Scalability a. QUESTION: What are problems? i. These papers look at Lecture 3: Scaling by Load Balancing 1. Comments on reviews i. 2. Topic 1: Scalability a. QUESTION: What are problems? i. These papers look at distributing load b. QUESTION: What is the context? i. How

More information

Hadoop and its Usage at Facebook. Dhruba Borthakur dhruba@apache.org, June 22 rd, 2009

Hadoop and its Usage at Facebook. Dhruba Borthakur dhruba@apache.org, June 22 rd, 2009 Hadoop and its Usage at Facebook Dhruba Borthakur dhruba@apache.org, June 22 rd, 2009 Who Am I? Hadoop Developer Core contributor since Hadoop s infancy Focussed on Hadoop Distributed File System Facebook

More information

ZCorum s Ask a Broadband Expert Series:

ZCorum s Ask a Broadband Expert Series: s Ask a Broadband Expert Series: The Advantages of Network Virtualization An Interview with Peter Olivia, Director of Systems Engineering ZCorum 1.800.909.9441 4501 North Point Parkway, Suite 125 Alpharetta,

More information

Data Driven Success. Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA)

Data Driven Success. Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA) Data Driven Success Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA) In business, data is everything. Regardless of the products or services you sell or the systems you support,

More information

Big Data Buzzwords From A to Z. By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012

Big Data Buzzwords From A to Z. By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012 Big Data Buzzwords From A to Z By Rick Whiting, CRN 4:00 PM ET Wed. Nov. 28, 2012 Big Data Buzzwords Big data is one of the, well, biggest trends in IT today, and it has spawned a whole new generation

More information

Hybrid Software Architectures for Big Data. Laurence.Hubert@hurence.com @hurence http://www.hurence.com

Hybrid Software Architectures for Big Data. Laurence.Hubert@hurence.com @hurence http://www.hurence.com Hybrid Software Architectures for Big Data Laurence.Hubert@hurence.com @hurence http://www.hurence.com Headquarters : Grenoble Pure player Expert level consulting Training R&D Big Data X-data hot-line

More information

Fog Computing and the Internet of Things: Extend the Cloud to Where the Things Are

Fog Computing and the Internet of Things: Extend the Cloud to Where the Things Are White Paper Fog Computing and the Internet of Things: Extend the Cloud to Where the Things Are What You Will Learn The Internet of Things (IoT) is generating an unprecedented volume and variety of data.

More information

Big Data a threat or a chance?

Big Data a threat or a chance? Big Data a threat or a chance? Helwig Hauser University of Bergen, Dept. of Informatics Big Data What is Big Data? well, lots of data, right? we come back to this in a moment. certainly, a buzz-word but

More information

The Requirement for a New Type of Cloud Based CDN

The Requirement for a New Type of Cloud Based CDN The Requirement for a New Type of Cloud Based CDN Executive Summary The growing use of SaaS-based applications has highlighted some of the fundamental weaknesses of the Internet that significantly impact

More information

Exploring Big Data in Social Networks

Exploring Big Data in Social Networks Exploring Big Data in Social Networks virgilio@dcc.ufmg.br (meira@dcc.ufmg.br) INWEB National Science and Technology Institute for Web Federal University of Minas Gerais - UFMG May 2013 Some thoughts about

More information

As companies emerge from challenging economic times and turn the corner to face tremendous opportunities, the CIO's role is more and more about

As companies emerge from challenging economic times and turn the corner to face tremendous opportunities, the CIO's role is more and more about As companies emerge from challenging economic times and turn the corner to face tremendous opportunities, the CIO's role is more and more about strategy and optimizing business results. Today s complex

More information

Rapid Visualization with Big Data Analytics. Ravi Chalaka VP, Solution and Social Innovation Marketing

Rapid Visualization with Big Data Analytics. Ravi Chalaka VP, Solution and Social Innovation Marketing Rapid Visualization with Big Data Analytics Ravi Chalaka VP, Solution and Social Innovation Marketing Imagine the Future Innovative cities that dramatically enhance the wellbeing of its citizens Safer

More information