Distributed Caching Algorithms for Content Distribution Networks

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Distributed Caching Algorithms for Content Distribution Networks"

Transcription

1 Distributed Caching Algorithms for Content Distribution Networks Sem Borst, Varun Gupta, Anwar Walid Alcatel-Lucent Bell Labs, CMU BCAM Seminar Bilbao, September 30, 2010

2 Introduction Scope: personalized/on-demand delivery of high-definition video through service provider CatchUp TV / PauseLive TV features NPVR (Network Personal Video Recorder) capabilities Movie libraries / VoD (Video-on-Demand) User-generated content Unicast nature defies conventional broadcast TV paradigm 1

3 Caching strategies Focus: hierarchical network architecture Store popular content closer to network edge to reduce traffic load, capital expense and performance bottlenecks VHO 40% of content stored 5% of traffic served IO 30% stored 20% served CO 20% stored 30% served DSLAM STB 9% stored 35% served 1% stored 10% served 2

4 Caching strategies (cont d) Typically there are caches installed at only one or two levels VHO 60% of content stored (or 100%) 25% of traffic served IO CO 30% stored 30% served DSLAM 10% stored 45% served STB 3

5 Caching strategies (cont d) Two interrelated problems Design: optimal cache locations and sizes (joint work with Marty Reiman) Operation: items efficient (dynamic) placement of content 4

6 Popularity statistics Cache effectiveness strongly depends on locality / commonality in user requests request frequencies N content item ranks 5

7 Popularity statistics (cont d) Empirical data suggests that rank statistics resemble Zipf- Mandelbrot distribution Relative frequency of n-th most popular item is with p n = H (q + n) α, n = 1,..., N, α 0: shape parameter q 0: shift parameter H = [ Nn=1 1 (q+n) α ] 1 normalization constant Ideal hit ratio for cache of size B N is R = B n=1 H (q + n) α 6

8 Popularity statistics (cont d) Shape parameter α varies with content type, and strongly impacts cache effectiveness 1 B = 100 B = 500 B = 1000 Hit ratio R Shape parameter alpha Hit ratio as function of shape parameter α for various cache sizes B and population of N = 10, 000 content items 7

9 Popularity statistics (cont d) Zipf-Mandelbrot distribution is inherently static, and difficult to reconcile with dynamic phenomena Dynamic content ingestion and removal Time-varying popularity, request-at-most-once Both adverse and favorable implications Requires agile caching strategies policies and (implicit) popularity estimation, negatively affecting caching performance Causes popularity distribution to be steeper (higher α values over shorter time intervals), improving potential caching effectiveness 8

10 Optimal content placement Consider symmetric scenario (cache sizes, popularity distributions) For now, assume strictly hierarchical topology: content can only be requested from parent node Caches should be filled with most popular content items from lowest level up VHO IO CO DSLAM STB 9

11 Greedy content placement strategy Whenever node receives request for item, its local popularity estimate for that item is updated If requested item is not stored in local cache, then Request is forwarded to parent node Popularity estimate for requested item is compared with that for marginal item, which may then be evicted and replaced Provable convergence to optimal content placement 10

12 Optimal content placement (cont d) Relies on two strong (though reasonable) assumptions Symmetric popularity distributions and cache sizes Strictly hierarchical topology What if popularity distributions are spatially heterogeneous? Or what if content can be requested from peers as well? 11

13 Optimal content placement (cont d) Assume there are caches installed at only two levels VHO IO CO DSLAM STB 12

14 Optimal content placement (cont d) Consider cluster of M nodes at same level in hierarchy Cluster nodes are either directly connected or indirectly via common parent node root node parent node leaf nodes 1 2 M 13

15 Optimal content placement (cont d) Some notation c 0 : transfer cost from root node - 1 to parent node 0 c i : transfer cost from parent node 0 to node i c ij < c 0 +c i : transfer cost from leaf node j to leaf node i Then f ij := c 0 + c i j = i c 0 j = 0 0 j = 1 c 0 + c i c ij > 0 j 1, 0, i represents transport cost savings achieved by transferring data to leaf node i from node j instead of root node 14

16 Optimal content placement (cont d) Problem of maximizing cost savings may be formulated as max sub M N i=1 n=1 N n=1 s n d in M j=0 f ij x jin (1) s n x in B i, i = 0, 1,..., M (2) x jin x jn, M j=0 i = 1,..., M, j = 0, 1,..., M, n = 1,..., N(3) x jin 1, i = 1,..., M, n = 1,..., N, (4) with B i denoting cache size of i-th node, s n size of n-th item, d in demand for n-th item at i-th node 15

17 Inter-level cache cooperation Allow for heterogeneous demands, but assume c ij =, i.e., content can only be fetched from parent node and not from peers For compactness, denote c min := min i=1,...,m c i Proposition For arbitrary demands, greedy content placement strategy is guaranteed to achieve at least fraction (M 1)c min + Mc 0 (M 1)c min + (2M 1)c 0 M 2M 1 of maximum achievable cost savings 16

18 Intra-level cache collaboration Now suppose content can be requested from peers as well Intra-level connectivity allows distributed caches to cooperate and act as single logical cache, and makes caching at lower levels more cost-effective Greedy optimization of local hit rate will lead to complete replication of cache content Cache cooperation improves aggregate hit rate across cache cluster, at expense of lower local hit rate Optimal trade-off and degree of replication depends on cost of intra-level transfers relative to transfers from parent or root node 17

19 Intra-level cache cooperation (cont d) Assume symmetric transport cost, cache sizes and demands: B i B, c i c, c ij c, and d in d n For compactness, denote c := M(c + c 0 ) (M 1)c > c Problem (1) (4) may be simplified to max sub N n=1 N n=1 N n=1 s n d n (c p n + (M 1)c q n + Mc 0 x 0n ) (5) s n x 0n B 0 (6) s n (p n + (M 1)q n) MB (7) p n + x 0n 1, n = 1,..., N (8) q n + x 0n 1, n = 1,..., N (9) Knapsack problem type structure 18

20 Intra-level cache collaboration (cont d) Optimal solution of content placement problem has relatively simple structure Distinguish between two cases Mc (M 1)c : more advantageous to store un-replicated content in leaf nodes than in parent node Mc (M 1)c : more attractive to store un-replicated content in parent node than in leaf nodes with c cost between parent and leaf node and c cost between two leaf nodes 19

21 Case Mc (M 1)c root node parent node leaf nodes 1 2 M Four popularity tiers Highly popular (red): replicated in all leafs p n = 1, q n = 1 Fairly popular (pink): stored in single leaf p n = 1 Mildly popular (yellow): stored in parent node x 0n = 1 Hardly popular (green): stored in root node only 20

22 Case Mc (M 1)c (cont d) n 2 n 2 n 2 1 n 1 x 0n n 1 x0n n 1 x 0n n 1 n 1 1 n 1 n 0 1 n 0 n 0 1 n 0 n 0 1 q n p n qn 0 = 0 q n p n1 = 0 p n q n p n = 0 x 0n2 21

23 Case Mc (M 1)c root node parent node leaf nodes 1 2 M Four popularity tiers Highly popular (red): replicated in all leafs p n = 1, q n = 1 Fairly popular (pink): stored in common parent x 0n = 1 Mildly popular (yellow): stored in single leaf p n = 1 Hardly popular (green): stored in root node only 22

24 Case Mc (M 1)c (cont d) n 1 n 1 n 1 1 n 0 x 0n n x 0n n 0 x 0 n n 2 n 2 1 n 2 n 1 n 0 1 n 1 n 0 n 0 1 n 1 n 0 n 0 1 q n p n qn 0 = 0 q n p n p n 2 = 0 q n p n x 0n 1 = 0 23

25 Local-Greedy algorithm For convenience, assume B 0 = 0, s n = 1 for all n = 1,... N If requested item is not stored in local cache, then Item is fetched from peer if cached elsewhere in cluster and otherwise from root node Value of requested item is compared with marginal cache value, i.e., value provided by marginal item in local cache, which may then be evicted and replaced Value of item n = { c d n if stored elsewhere in cluster c d n otherwise 24

26 Local-Greedy algorithm (cont d) May get stuck in suboptimal configuration globally optimal configuration local optimum Duplicating red item less valuable than single yellow item Duplicating yellow item less valuable then single green item 25

27 Local-Greedy algorithm (cont d) Performance guarantees (competitive ratios) Symmetric demands: within factor 4/3 from optimal Arbitrary demands: within factor 2 from optimal 26

28 Numerical experiments M = 10 leaf nodes, each with cache of size B = 1 TB Unit transport cost c 0 = 2, c = 1, c = 1 N = 10, 000 content items, with common size S = 2 GB Each leaf node can store K = B/S content items 27

29 Numerical experiments (cont d) Each leaf receives average of 1 request every 160 sec, i.e., total request rate per leaf is ν = sec 1 Zipf-Mandelbrot popularity distribution with shape parameter α and shift parameter q, i.e., p n = with normalization constant H (q + n) α, n = 1,..., N, H = N n=1 1 (q + n) α 1 Request rate for n-th item at each leaf node is d n = p n ν 28

30 Gains from cooperative caching Compare minimum bandwidth cost with that in two other scenarios Full replication: each leaf node stores K most popular items No replication: only single copy of M K most popular items is stored in one of leaf nodes Without caching, bandwidth cost would be MνS(c + c 0 ) = = GBps = 3 Gbps 29

31 Bandwidth cost as function of shape parameter α for various scenarios and cache sizes 30

32 Some observations Caching effectiveness improves as popularity distribution gets steeper: bandwidth costs markedly decrease with increasing values of α Even when collective cache space can only hold 10% of total content, bandwidth costs reduce to fraction of those without caching, as long as value of α is not too low Best between either full or zero replication is often not much worse than optimal content placement; however, neither full nor zero replication performs well across entire range of α values Critical to adjust degree of replication to steepness of popularity distribution; Local-Greedy algorithm does just that 31

33 Performance of Local-Greedy algorithm Various leaf nodes receive requests over time, from Zipf-Mandelbrot popularity distribution sampled If requested item is not presently stored, node decides whether to cache it and if so, which currently stored item to evict Distinguish three scenarios for initial placement Full replication: each leaf node stores 500 top items No replication: only single copy of 5000 top items is stored in one of leaf nodes Random: each leaf stores 500 randomly selected items In optimal placement, items 1 through 165 fully replicated, and single copies of items 166 through 3515 stored 32

34 Performance ratio as function of number of requests, with static or dynamic popularities 33

35 Bandwidth savings as function of number of requests, with inaccurate popularity estimates 34

36 Some observations Local-Greedy algorithm gets progressively closer to optimum as system receives more requests and replaces items over time After only 3000 requests (out of total number 10,000 items) Local-Greedy algorithm has come to within 1% of optimum, and stays there Performs markedly better than worst-case ratio of 3/4 might suggest While algorithm seems to converge for all three initial states, scenario with no replication appears to be most favorable one, due to fact that in optimal placement only items 1 through 165 are fully replicated 35

Design and Implementation of Smart Camera Network for Efficient Wide Area Surveillance

Design and Implementation of Smart Camera Network for Efficient Wide Area Surveillance Vol.5, Issue (04), pp.-8 http://dx.doi.org/0.457/ijeic.04.5..0 Design and Implementation of Smart Camera Network for Efficient Wide Area Surveillance Soomi Yang Department of Information Security, The

More information

Content Delivery Networks. Shaxun Chen April 21, 2009

Content Delivery Networks. Shaxun Chen April 21, 2009 Content Delivery Networks Shaxun Chen April 21, 2009 Outline Introduction to CDN An Industry Example: Akamai A Research Example: CDN over Mobile Networks Conclusion Outline Introduction to CDN An Industry

More information

Bloom Filter based Inter-domain Name Resolution: A Feasibility Study

Bloom Filter based Inter-domain Name Resolution: A Feasibility Study Bloom Filter based Inter-domain Name Resolution: A Feasibility Study Konstantinos V. Katsaros, Wei Koong Chai and George Pavlou University College London, UK Outline Inter-domain name resolution in ICN

More information

Efficient and Robust Allocation Algorithms in Clouds under Memory Constraints

Efficient and Robust Allocation Algorithms in Clouds under Memory Constraints Efficient and Robust Allocation Algorithms in Clouds under Memory Constraints Olivier Beaumont,, Paul Renaud-Goud Inria & University of Bordeaux Bordeaux, France 9th Scheduling for Large Scale Systems

More information

Modeling and Optimization of Resource Allocation in Cloud

Modeling and Optimization of Resource Allocation in Cloud 1 / 40 Modeling and Optimization of Resource Allocation in Cloud PhD Thesis Proposal Atakan Aral Thesis Advisor: Asst. Prof. Dr. Tolga Ovatman Istanbul Technical University Department of Computer Engineering

More information

Multimedia Caching Strategies for Heterogeneous Application and Server Environments

Multimedia Caching Strategies for Heterogeneous Application and Server Environments Multimedia Tools and Applications 4, 279 312 (1997) c 1997 Kluwer Academic Publishers. Manufactured in The Netherlands. Multimedia Caching Strategies for Heterogeneous Application and Server Environments

More information

BELL LABS METRO NETWORK TRAFFIC GROWTH: AN ARCHITECTURE IMPACT STUDY

BELL LABS METRO NETWORK TRAFFIC GROWTH: AN ARCHITECTURE IMPACT STUDY BELL LABS METRO NETWORK TRAFFIC GROWTH: AN ARCHITECTURE IMPACT STUDY STRATEGIC WHITE PAPER Like many studies before it, the Alcatel-Lucent Bell Labs Metro Network Traffic Growth Study provides evidence

More information

ACHIEVING HIGHER RESOLUTION STORAGE FABRICS THROUGH VIRTUALIZATION

ACHIEVING HIGHER RESOLUTION STORAGE FABRICS THROUGH VIRTUALIZATION ACHIEVING HIGHER RESOLUTION STORAGE FABRICS THROUGH VIRTUALIZATION Nevoa Networks storage virtualization suite, Nevoa Storage System (NSS), enables the deployment of high-resolution storage fabrics while

More information

Proxy-Assisted Periodic Broadcast for Video Streaming with Multiple Servers

Proxy-Assisted Periodic Broadcast for Video Streaming with Multiple Servers 1 Proxy-Assisted Periodic Broadcast for Video Streaming with Multiple Servers Ewa Kusmierek and David H.C. Du Digital Technology Center and Department of Computer Science and Engineering University of

More information

The introduction of storage presales tool-edesigner. Storage edesigner Project Team

The introduction of storage presales tool-edesigner. Storage edesigner Project Team The introduction of storage presales tool-edesigner Storage edesigner Project Team 0 Can t finish product evaluation or product configuration in several hours? Can t guarantee the accuracy? Master complicate

More information

COSHH: A Classification and Optimization based Scheduler for Heterogeneous Hadoop Systems

COSHH: A Classification and Optimization based Scheduler for Heterogeneous Hadoop Systems COSHH: A Classification and Optimization based Scheduler for Heterogeneous Hadoop Systems Aysan Rasooli a, Douglas G. Down a a Department of Computing and Software, McMaster University, L8S 4K1, Canada

More information

Load Balancing in Distributed Web Server Systems With Partial Document Replication

Load Balancing in Distributed Web Server Systems With Partial Document Replication Load Balancing in Distributed Web Server Systems With Partial Document Replication Ling Zhuo, Cho-Li Wang and Francis C. M. Lau Department of Computer Science and Information Systems The University of

More information

The Benefits of Purpose Built Super Efficient Video Servers

The Benefits of Purpose Built Super Efficient Video Servers Whitepaper Deploying Future Proof On Demand TV and Video Services: The Benefits of Purpose Built Super Efficient Video Servers The Edgeware Storage System / Whitepaper / Edgeware AB 2010 / Version1 A4

More information

14.1 Rent-or-buy problem

14.1 Rent-or-buy problem CS787: Advanced Algorithms Lecture 14: Online algorithms We now shift focus to a different kind of algorithmic problem where we need to perform some optimization without knowing the input in advance. Algorithms

More information

Concept of Cache in web proxies

Concept of Cache in web proxies Concept of Cache in web proxies Chan Kit Wai and Somasundaram Meiyappan 1. Introduction Caching is an effective performance enhancing technique that has been used in computer systems for decades. However,

More information

Opt p ic i al a l Netw t or o k r s k Bas a ic i Concepts (Part 1)

Opt p ic i al a l Netw t or o k r s k Bas a ic i Concepts (Part 1) Optical Networks Basic Concepts (Part 1) Introduction What is an optical network? Optical devices and components Basic concepts in optical networking Optimization of optical network design How to handle

More information

Video-on-Demand Networks: Design Approaches and Future Challenges

Video-on-Demand Networks: Design Approaches and Future Challenges Video-on-Demand Networks: Design Approaches and Future Challenges Frederic Thouin and Mark Coates, McGill University Abstract IP network based deployments of interactive video-on-demand (VoD) systems are

More information

Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand

Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand P. Balaji, K. Vaidyanathan, S. Narravula, K. Savitha, H. W. Jin D. K. Panda Network Based

More information

All-Flash Arrays Weren t Built for Dynamic Environments. Here s Why... This whitepaper is based on content originally posted at www.frankdenneman.

All-Flash Arrays Weren t Built for Dynamic Environments. Here s Why... This whitepaper is based on content originally posted at www.frankdenneman. WHITE PAPER All-Flash Arrays Weren t Built for Dynamic Environments. Here s Why... This whitepaper is based on content originally posted at www.frankdenneman.nl 1 Monolithic shared storage architectures

More information

Scaling 10Gb/s Clustering at Wire-Speed

Scaling 10Gb/s Clustering at Wire-Speed Scaling 10Gb/s Clustering at Wire-Speed InfiniBand offers cost-effective wire-speed scaling with deterministic performance Mellanox Technologies Inc. 2900 Stender Way, Santa Clara, CA 95054 Tel: 408-970-3400

More information

Simulating a File-Sharing P2P Network

Simulating a File-Sharing P2P Network Simulating a File-Sharing P2P Network Mario T. Schlosser, Tyson E. Condie, and Sepandar D. Kamvar Department of Computer Science Stanford University, Stanford, CA 94305, USA Abstract. Assessing the performance

More information

Storage Systems Autumn 2009. Chapter 6: Distributed Hash Tables and their Applications André Brinkmann

Storage Systems Autumn 2009. Chapter 6: Distributed Hash Tables and their Applications André Brinkmann Storage Systems Autumn 2009 Chapter 6: Distributed Hash Tables and their Applications André Brinkmann Scaling RAID architectures Using traditional RAID architecture does not scale Adding news disk implies

More information

Maginatics Cloud Storage Platform for Elastic NAS Workloads

Maginatics Cloud Storage Platform for Elastic NAS Workloads Maginatics Cloud Storage Platform for Elastic NAS Workloads Optimized for Cloud Maginatics Cloud Storage Platform () is the first solution optimized for the cloud. It provides lower cost, easier administration,

More information

Part 2: Community Detection

Part 2: Community Detection Chapter 8: Graph Data Part 2: Community Detection Based on Leskovec, Rajaraman, Ullman 2014: Mining of Massive Datasets Big Data Management and Analytics Outline Community Detection - Social networks -

More information

Placement of Storage Capacity in Distributed Video Servers

Placement of Storage Capacity in Distributed Video Servers Placement of Storage Capacity in Distributed Video s Juan Segarra and Vicent Cholvi Departament de Llenguatges i Sistemes Informàtics Universitat Jaume I 171 Castelló, Spain Abstract In this paper, we

More information

2015:07. Tree Approximation for hp- Adaptivity. Peter Binev

2015:07. Tree Approximation for hp- Adaptivity. Peter Binev INTERDISCIPLINARY MATHEMATICS INSTITUTE 205:07 Tree Approximation for hp- Adaptivity Peter Binev IMI PREPRINT SERIES COLLEGE OF ARTS AND SCIENCES UNIVERSITY OF SOUTH CAROLINA Tree Approximation for hp-adaptivity

More information

Applications. Network Application Performance Analysis. Laboratory. Objective. Overview

Applications. Network Application Performance Analysis. Laboratory. Objective. Overview Laboratory 12 Applications Network Application Performance Analysis Objective The objective of this lab is to analyze the performance of an Internet application protocol and its relation to the underlying

More information

Load Balancing. Load Balancing 1 / 24

Load Balancing. Load Balancing 1 / 24 Load Balancing Backtracking, branch & bound and alpha-beta pruning: how to assign work to idle processes without much communication? Additionally for alpha-beta pruning: implementing the young-brothers-wait

More information

Big Data & Scripting storage networks and distributed file systems

Big Data & Scripting storage networks and distributed file systems Big Data & Scripting storage networks and distributed file systems 1, 2, in the remainder we use networks of computing nodes to enable computations on even larger datasets for a computation, each node

More information

EMC DATA DOMAIN SISL SCALING ARCHITECTURE

EMC DATA DOMAIN SISL SCALING ARCHITECTURE EMC DATA DOMAIN SISL SCALING ARCHITECTURE A Detailed Review ABSTRACT While tape has been the dominant storage medium for data protection for decades because of its low cost, it is steadily losing ground

More information

Big Graph Processing: Some Background

Big Graph Processing: Some Background Big Graph Processing: Some Background Bo Wu Colorado School of Mines Part of slides from: Paul Burkhardt (National Security Agency) and Carlos Guestrin (Washington University) Mines CSCI-580, Bo Wu Graphs

More information

Interactive Analytical Processing in Big Data Systems,BDGS: AMay Scalable 23, 2014 Big Data1 Generat / 20

Interactive Analytical Processing in Big Data Systems,BDGS: AMay Scalable 23, 2014 Big Data1 Generat / 20 Interactive Analytical Processing in Big Data Systems,BDGS: A Scalable Big Data Generator Suite in Big Data Benchmarking,Study about DataSet May 23, 2014 Interactive Analytical Processing in Big Data Systems,BDGS:

More information

Optimized content distribution in a push-vod scenario

Optimized content distribution in a push-vod scenario Optimized content distribution in a push-vod scenario Sandford Bessler Telecommunications Research Center Vienna (ftw.) Donau-City 1, A-122 Vienna Austria Abstract This paper discusses a novel approach

More information

CS 31: Intro to Systems Virtual Memory. Kevin Webb Swarthmore College April 7, 2015

CS 31: Intro to Systems Virtual Memory. Kevin Webb Swarthmore College April 7, 2015 CS 31: Intro to Systems Virtual Memory Kevin Webb Swarthmore College April 7, 2015 Reading Quiz Memory Abstraction goal: make every process think it has the same memory layout. MUCH simpler for compiler

More information

8 Conclusion and Future Work

8 Conclusion and Future Work 8 Conclusion and Future Work This chapter concludes this thesis and provides an outlook on future work in the area of mobile ad hoc networks and peer-to-peer overlay networks 8.1 Conclusion Due to the

More information

Stability of QOS. Avinash Varadarajan, Subhransu Maji {avinash,smaji}@cs.berkeley.edu

Stability of QOS. Avinash Varadarajan, Subhransu Maji {avinash,smaji}@cs.berkeley.edu Stability of QOS Avinash Varadarajan, Subhransu Maji {avinash,smaji}@cs.berkeley.edu Abstract Given a choice between two services, rest of the things being equal, it is natural to prefer the one with more

More information

2004 Networks UK Publishers. Reprinted with permission.

2004 Networks UK Publishers. Reprinted with permission. Riikka Susitaival and Samuli Aalto. Adaptive load balancing with OSPF. In Proceedings of the Second International Working Conference on Performance Modelling and Evaluation of Heterogeneous Networks (HET

More information

Linear Programming. March 14, 2014

Linear Programming. March 14, 2014 Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

More information

%WILEY TELEVISION, MOBILE AND CONVERGED NETWORKS DIGITAL VIDEO DISTRIBUTION IN BROADBAND, TRENDS, CHALLENGES AND SOLUTIONS. Sanjoy Paul, Ph.

%WILEY TELEVISION, MOBILE AND CONVERGED NETWORKS DIGITAL VIDEO DISTRIBUTION IN BROADBAND, TRENDS, CHALLENGES AND SOLUTIONS. Sanjoy Paul, Ph. DIGITAL VIDEO DISTRIBUTION IN BROADBAND, TELEVISION, MOBILE AND CONVERGED NETWORKS TRENDS, CHALLENGES AND SOLUTIONS Sanjoy Paul, Ph.D Formerly of Bell Labs and WINLAB, Rutgers University, USA, Limited,

More information

Network Architecture and Topology

Network Architecture and Topology 1. Introduction 2. Fundamentals and design principles 3. Network architecture and topology 4. Network control and signalling 5. Network components 5.1 links 5.2 switches and routers 6. End systems 7. End-to-end

More information

MagFS: The Ideal File System for the Cloud

MagFS: The Ideal File System for the Cloud : The Ideal File System for the Cloud is the first true file system for the cloud. It provides lower cost, easier administration, and better scalability and performance than any alternative in-cloud file

More information

Optimizing a ëcontent-aware" Load Balancing Strategy for Shared Web Hosting Service Ludmila Cherkasova Hewlett-Packard Laboratories 1501 Page Mill Road, Palo Alto, CA 94303 cherkasova@hpl.hp.com Shankar

More information

IN recent years, user demand for the delivery of broadcast

IN recent years, user demand for the delivery of broadcast Studying the Performance of Cooperative Delivery Techniques to Support Video-on-Demand Service in IPTV Networks Aytac Azgin, Member, IEEE,, Ghassan AlRegib, Senior Member, IEEE, Yucel Altunbasak, Fellow,

More information

Method To Solve Linear, Polynomial, or Absolute Value Inequalities:

Method To Solve Linear, Polynomial, or Absolute Value Inequalities: Solving Inequalities An inequality is the result of replacing the = sign in an equation with ,, or. For example, 3x 2 < 7 is a linear inequality. We call it linear because if the < were replaced with

More information

Multimedia Communications. Huffman Coding

Multimedia Communications. Huffman Coding Multimedia Communications Huffman Coding Optimal codes Suppose that a i -> w i C + is an encoding scheme for a source alphabet A={a 1,, a N }. Suppose that the source letter a 1,, a N occur with relative

More information

Scalability of Control Planes for Software Defined Networks:Modeling and Evaluation

Scalability of Control Planes for Software Defined Networks:Modeling and Evaluation of Control Planes for Software Defined Networks:Modeling and Evaluation Jie Hu, Chuang Lin, Xiangyang Li, Jiwei Huang Department of Computer Science and Technology, Tsinghua University Department of Computer

More information

MaxDeploy Ready. Hyper- Converged Virtualization Solution. With SanDisk Fusion iomemory products

MaxDeploy Ready. Hyper- Converged Virtualization Solution. With SanDisk Fusion iomemory products MaxDeploy Ready Hyper- Converged Virtualization Solution With SanDisk Fusion iomemory products MaxDeploy Ready products are configured and tested for support with Maxta software- defined storage and with

More information

OPTIMIZING VIDEO STORAGE AT THE EDGE OF THE NETWORK

OPTIMIZING VIDEO STORAGE AT THE EDGE OF THE NETWORK White Paper OPTIMIZING VIDEO STORAGE AT THE EDGE OF THE NETWORK Leveraging Intelligent Content Distribution Software, Off-the-Shelf Hardware and MLC Flash to Deploy Scalable and Economical Pay-As-You-Grow

More information

Essentials Guide CONSIDERATIONS FOR SELECTING ALL-FLASH STORAGE ARRAYS

Essentials Guide CONSIDERATIONS FOR SELECTING ALL-FLASH STORAGE ARRAYS Essentials Guide CONSIDERATIONS FOR SELECTING ALL-FLASH STORAGE ARRAYS M ost storage vendors now offer all-flash storage arrays, and many modern organizations recognize the need for these highperformance

More information

Clustered Content Replication for Hierarchical Content Delivery Networks

Clustered Content Replication for Hierarchical Content Delivery Networks Clustered Content Replication for Hierarchical Content Delivery Networks Lazaros Gkatzikis Huawei, France. lazaros.gkatzikis@huawei.com Vasilis Sourlas UCL, UK. v.sourlas@ucl.ac.uk Carlo Fischione KTH,

More information

On the Interaction and Competition among Internet Service Providers

On the Interaction and Competition among Internet Service Providers On the Interaction and Competition among Internet Service Providers Sam C.M. Lee John C.S. Lui + Abstract The current Internet architecture comprises of different privately owned Internet service providers

More information

Collaborative Hierarchical Caching with Dynamic Request Routing for Massive Content Distribution

Collaborative Hierarchical Caching with Dynamic Request Routing for Massive Content Distribution Collaborative Hierarchical Caching with Dynamic Request Routing for Massive Content Distribution Jie Dai, Zhan Hu, Bo Li, Jiangchuan Liu, Baochun Li Hong Kong University of Science and Technology Simon

More information

Hardware Configuration Guide

Hardware Configuration Guide Hardware Configuration Guide Contents Contents... 1 Annotation... 1 Factors to consider... 2 Machine Count... 2 Data Size... 2 Data Size Total... 2 Daily Backup Data Size... 2 Unique Data Percentage...

More information

A NOVEL RESOURCE EFFICIENT DMMS APPROACH

A NOVEL RESOURCE EFFICIENT DMMS APPROACH A NOVEL RESOURCE EFFICIENT DMMS APPROACH FOR NETWORK MONITORING AND CONTROLLING FUNCTIONS Golam R. Khan 1, Sharmistha Khan 2, Dhadesugoor R. Vaman 3, and Suxia Cui 4 Department of Electrical and Computer

More information

6.852: Distributed Algorithms Fall, 2009. Class 2

6.852: Distributed Algorithms Fall, 2009. Class 2 .8: Distributed Algorithms Fall, 009 Class Today s plan Leader election in a synchronous ring: Lower bound for comparison-based algorithms. Basic computation in general synchronous networks: Leader election

More information

CDN/CLOUD/SDN/NFV FOR MULTIMEDIA SERVICES

CDN/CLOUD/SDN/NFV FOR MULTIMEDIA SERVICES CDN/CLOUD/SDN/NFV FOR MULTIMEDIA SERVICES Nakjung Choi, Technical Director, Bell Labs Seoul KRnet 2014, June 24 th, 2014 CONTENT Background on CDN/Cloud/SDN/NFV Video Delivery in the Existing Infrastructure

More information

Approximation Algorithms

Approximation Algorithms Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

More information

How to Conduct a Hypothesis Test

How to Conduct a Hypothesis Test How to Conduct a Hypothesis Test The idea of hypothesis testing is relatively straightforward. In various studies we observe certain events. We must ask, is the event due to chance alone, or is there some

More information

Distributed Data Management Part 2 - Data Broadcasting in Mobile Networks

Distributed Data Management Part 2 - Data Broadcasting in Mobile Networks Distributed Data Management Part 2 - Data Broadcasting in Mobile Networks 2006/7, Karl Aberer, EPFL-IC, Laboratoire de systèmes d'informations répartis Mobile Data Management - 1 1 Today's Questions 1.

More information

Online Storage and Content Distribution System at a Large-scale: Peer-assistance and Beyond

Online Storage and Content Distribution System at a Large-scale: Peer-assistance and Beyond Online Storage and Content Distribution System at a Large-scale: Peer-assistance and Beyond Bo Li Email: bli@cse.ust.hk Department of Computer Science and Engineering Hong Kong University of Science &

More information

Web Search Engines: Solutions

Web Search Engines: Solutions Web Search Engines: Solutions Problem 1: A. How can the owner of a web site design a spider trap? Answer: He can set up his web server so that, whenever a client requests a URL in a particular directory

More information

Video-on-Demand Networks: Design Approaches and Future Challenges

Video-on-Demand Networks: Design Approaches and Future Challenges Video-on-Demand Networks: Design Approaches and Future Challenges Abstract IP network-based deployments of Interactive Video-on-Demand (VoD) systems are today very limited in scope, but there is a strong

More information

Near Optimal Solutions

Near Optimal Solutions Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

More information

CASH FLOW MATCHING PROBLEM WITH CVaR CONSTRAINTS: A CASE STUDY WITH PORTFOLIO SAFEGUARD. Danjue Shang and Stan Uryasev

CASH FLOW MATCHING PROBLEM WITH CVaR CONSTRAINTS: A CASE STUDY WITH PORTFOLIO SAFEGUARD. Danjue Shang and Stan Uryasev CASH FLOW MATCHING PROBLEM WITH CVaR CONSTRAINTS: A CASE STUDY WITH PORTFOLIO SAFEGUARD Danjue Shang and Stan Uryasev PROJECT REPORT #2011-1 Risk Management and Financial Engineering Lab Department of

More information

Trends in Data Protection and Restoration Technologies. Mike Fishman, EMC 2 Corporation (Author and Presenter)

Trends in Data Protection and Restoration Technologies. Mike Fishman, EMC 2 Corporation (Author and Presenter) Trends in Data Protection and Restoration Technologies Mike Fishman, EMC 2 Corporation (Author and Presenter) SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless

More information

Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

More information

On the effect of forwarding table size on SDN network utilization

On the effect of forwarding table size on SDN network utilization IBM Haifa Research Lab On the effect of forwarding table size on SDN network utilization Rami Cohen IBM Haifa Research Lab Liane Lewin Eytan Yahoo Research, Haifa Seffi Naor CS Technion, Israel Danny Raz

More information

diversifeye Application Note

diversifeye Application Note diversifeye Application Note Test Performance of IGMP based Multicast Services with emulated IPTV STBs Shenick Network Systems Test Performance of IGMP based Multicast Services with emulated IPTV STBs

More information

Condusiv s V-locity 4 Boosts Virtual Machine Performance Over 50% Without Additional Hardware

Condusiv s V-locity 4 Boosts Virtual Machine Performance Over 50% Without Additional Hardware openbench Labs Executive Briefing: December 5, 2012 Condusiv s V-locity 4 Boosts Virtual Machine Performance Over 50% Without Additional Hardware Improving Storage Efficiency by Increasing I/O Throughput

More information

Towards running complex models on big data

Towards running complex models on big data Towards running complex models on big data Working with all the genomes in the world without changing the model (too much) Daniel Lawson Heilbronn Institute, University of Bristol 2013 1 / 17 Motivation

More information

Analysis of Algorithms I: Optimal Binary Search Trees

Analysis of Algorithms I: Optimal Binary Search Trees Analysis of Algorithms I: Optimal Binary Search Trees Xi Chen Columbia University Given a set of n keys K = {k 1,..., k n } in sorted order: k 1 < k 2 < < k n we wish to build an optimal binary search

More information

Cooperative Caching with Return on Investment

Cooperative Caching with Return on Investment Cooperative Caching with Return on Investment Gala Yadgar Computer Science Department, Technion Haifa, Israel Email: gala@cs.technion.ac.il Michael Factor IBM Research Haifa, Israel Email: factor@il.ibm.com

More information

A Fault Tolerant Video Server Using Combined Raid 5 and Mirroring

A Fault Tolerant Video Server Using Combined Raid 5 and Mirroring Proceedings of Multimedia Computing and Networking 1997 (MMCN97), San Jose, CA, February 1997 A Fault Tolerant Video Server Using Combined Raid 5 and Mirroring Ernst W. BIERSACK, Christoph BERNHARDT Institut

More information

PowerVault MD3 SSD Cache Overview. White Paper

PowerVault MD3 SSD Cache Overview. White Paper PowerVault MD3 SSD Cache Overview White Paper 2012 Dell Inc. All Rights Reserved. PowerVault is a trademark of Dell Inc. 2 Dell PowerVault MD3 SSD Cache Overview Table of contents 1 Overview... 4 2 Architecture...

More information

Mass Storage Structure

Mass Storage Structure Mass Storage Structure 12 CHAPTER Practice Exercises 12.1 The accelerating seek described in Exercise 12.3 is typical of hard-disk drives. By contrast, floppy disks (and many hard disks manufactured before

More information

External Sorting. Why Sort? 2-Way Sort: Requires 3 Buffers. Chapter 13

External Sorting. Why Sort? 2-Way Sort: Requires 3 Buffers. Chapter 13 External Sorting Chapter 13 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing

More information

Web Caching. Chapter 11 από: Web Protocols and Caching, B. Krishnamurthy, J. Rexford

Web Caching. Chapter 11 από: Web Protocols and Caching, B. Krishnamurthy, J. Rexford Web Caching Chapter 11 από: Web Protocols and Caching, B. Krishnamurthy, J. Rexford Introduction The rapid growth of WWW increased network traffic Increased user-perceived latency WWW applications in contrast

More information

PARALLELS CLOUD STORAGE

PARALLELS CLOUD STORAGE PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...

More information

Routing and Peering in a Competitive Internet

Routing and Peering in a Competitive Internet Routing and Peering in a Competitive Internet Ramesh Johari Stanford University rjohari@stanford.edu John N. Tsitsiklis MIT jnt@mit.edu Abstract Today s Internet is a loose federation of independent network

More information

Application Centric Infrastructure Overview: Implement a Robust Transport Network for Dynamic Workloads

Application Centric Infrastructure Overview: Implement a Robust Transport Network for Dynamic Workloads White Paper Application Centric Infrastructure Overview: Implement a Robust Transport Network for Dynamic Workloads What You Will Learn Application centric infrastructure (ACI) provides a robust transport

More information

(Refer Slide Time: 01:35)

(Refer Slide Time: 01:35) Artificial Intelligence Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture - 6 Informed Search -2 Today we start our 6 th lecture. This

More information

Data Caching in Networks with Reading, Writing and Storage Costs

Data Caching in Networks with Reading, Writing and Storage Costs Data Caching in Networks with Reading, 1 Writing and Storage Costs Himanshu Gupta Computer Science Department Stony Brook University Stony Brook, NY 11790 Email: hgupta@cs.sunysb.edu Bin Tang Computer

More information

Scala Storage Scale-Out Clustered Storage White Paper

Scala Storage Scale-Out Clustered Storage White Paper White Paper Scala Storage Scale-Out Clustered Storage White Paper Chapter 1 Introduction... 3 Capacity - Explosive Growth of Unstructured Data... 3 Performance - Cluster Computing... 3 Chapter 2 Current

More information

A Load Balanced PC-Cluster for Video-On-Demand Server Systems

A Load Balanced PC-Cluster for Video-On-Demand Server Systems International Journal of Grid and Distributed Computing 63 A Load Balanced PC-Cluster for Video-On-Demand Server Systems Liang-Teh Lee 1, Hung-Yuan Chang 1,2, Der-Fu Tao 2, and Siang-Lin Yang 1 1 Dept.

More information

A Theory of Cloud Bandwidth Pricing for Video-on-Demand Providers

A Theory of Cloud Bandwidth Pricing for Video-on-Demand Providers A Theory of Cloud Bandwidth Pricing for Video-on-Demand Providers Di Niu, Chen Feng, Baochun Li Department of Electrical and Computer Engineering University of Toronto IEEE INFOCOM 2012, Cloud/Grid computing

More information

Object Storage: A Growing Opportunity for Service Providers. White Paper. Prepared for: 2012 Neovise, LLC. All Rights Reserved.

Object Storage: A Growing Opportunity for Service Providers. White Paper. Prepared for: 2012 Neovise, LLC. All Rights Reserved. Object Storage: A Growing Opportunity for Service Providers Prepared for: White Paper 2012 Neovise, LLC. All Rights Reserved. Introduction For service providers, the rise of cloud computing is both a threat

More information

Hadoop Distributed File System. T-111.5550 Seminar On Multimedia 2009-11-11 Eero Kurkela

Hadoop Distributed File System. T-111.5550 Seminar On Multimedia 2009-11-11 Eero Kurkela Hadoop Distributed File System T-111.5550 Seminar On Multimedia 2009-11-11 Eero Kurkela Agenda Introduction Flesh and bones of HDFS Architecture Accessing data Data replication strategy Fault tolerance

More information

Hyacinth An IEEE 802.11-based Multi-channel Wireless Mesh Network

Hyacinth An IEEE 802.11-based Multi-channel Wireless Mesh Network Hyacinth An IEEE 802.11-based Multi-channel Wireless Mesh Network 1 Gliederung Einführung Vergleich und Problemstellung Algorithmen Evaluation 2 Aspects Backbone Last mile access stationary commodity equipment

More information

The Future of Archive Storage

The Future of Archive Storage Long Term Preservation: The Future of Archive Storage Chris Powers Vice President, HP Storage Change is Constant data growth technologies High Performance long term preservation Near-Line Archive Flash

More information

Big Data: Conor Duffy: Enterprise Strategist. Ziad Najjar: National Director, ESG Specialist team April 2014

Big Data: Conor Duffy: Enterprise Strategist. Ziad Najjar: National Director, ESG Specialist team April 2014 Big Data: Conor Duffy: Enterprise Strategist. Ziad Najjar: National Director, ESG Specialist team April 2014 Mega Trends impacting IT Big Data Cloud 69% of CXOs expect to see a significant or complete

More information

Fault-Tolerant Application Placement in Heterogeneous Cloud Environments. Bart Spinnewyn, prof. Steven Latré

Fault-Tolerant Application Placement in Heterogeneous Cloud Environments. Bart Spinnewyn, prof. Steven Latré Fault-Tolerant Application Placement in Heterogeneous Cloud Environments Bart Spinnewyn, prof. Steven Latré Cloud Application Placement Problem (CAPP) Application Placement admission control: decide on

More information

A Practical Scheme for Wireless Network Operation

A Practical Scheme for Wireless Network Operation A Practical Scheme for Wireless Network Operation Radhika Gowaikar, Amir F. Dana, Babak Hassibi, Michelle Effros June 21, 2004 Abstract In many problems in wireline networks, it is known that achieving

More information

Information Science 1

Information Science 1 Topics covered Information Science 1 Terms and concepts from Week 05 The software development process Program'Development /Algorithms'and'Problem'Solving/'' ' software software life cycle life-cycle models

More information

Diablo and VMware TM powering SQL Server TM in Virtual SAN TM. A Diablo Technologies Whitepaper. May 2015

Diablo and VMware TM powering SQL Server TM in Virtual SAN TM. A Diablo Technologies Whitepaper. May 2015 A Diablo Technologies Whitepaper Diablo and VMware TM powering SQL Server TM in Virtual SAN TM May 2015 Ricky Trigalo, Director for Virtualization Solutions Architecture, Diablo Technologies Daniel Beveridge,

More information

DROBO PERFORMANCE STATS

DROBO PERFORMANCE STATS DROBO PERFORMANCE STATS By StorageNerve http://storagenerve.com 11/30/2009-1 - Drobo Performance Stats Some discussions on twitter last week triggered the topic of performance relating to the Drobo s.

More information

Clustering UE 141 Spring 2013

Clustering UE 141 Spring 2013 Clustering UE 141 Spring 013 Jing Gao SUNY Buffalo 1 Definition of Clustering Finding groups of obects such that the obects in a group will be similar (or related) to one another and different from (or

More information

Latency on a Switched Ethernet Network

Latency on a Switched Ethernet Network Application Note 8 Latency on a Switched Ethernet Network Introduction: This document serves to explain the sources of latency on a switched Ethernet network and describe how to calculate cumulative latency

More information

The assignment of chunk size according to the target data characteristics in deduplication backup system

The assignment of chunk size according to the target data characteristics in deduplication backup system The assignment of chunk size according to the target data characteristics in deduplication backup system Mikito Ogata Norihisa Komoda Hitachi Information and Telecommunication Engineering, Ltd. 781 Sakai,

More information

Using Multicast Call Admission Control for IPTV Bandwidth Management

Using Multicast Call Admission Control for IPTV Bandwidth Management Application Note Using Multicast Call Admission Control for IPTV Bandwidth Management Managing Multicast Bandwidth in IPTV Networks Using Multicast Call Admission Control in the Edge Router Juniper Networks,

More information

Managing Trust in Self-organized Mobile Ad Hoc Networks

Managing Trust in Self-organized Mobile Ad Hoc Networks Managing Trust in Self-organized Mobile Ad Hoc Networks John S. Baras and Tao Jiang Electrical and Computer Engineering Department Computer Science Department and the Institute for Systems Research University

More information