The Cost Models For VLDB 2009 Summer School Shanghai Architecture

Size: px
Start display at page:

Download "The Cost Models For VLDB 2009 Summer School Shanghai Architecture"

Transcription

1 Cost models for Main-Memory database systems Peter Boncz Stefan Manegold CWI (Amsterdam)

2 Contents Generic Database Cost Models for Hierarchical Memory VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 2

3 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 3

4 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 4

5 vz Generic Database Cost Models for Hierarchical Memory VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 5

6 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 6

7 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 7

8 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 8

9 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 9

10 Region size VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 10

11 Skippy Access VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 11

12 Full sequential access VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 12

13 Random access: compulsory misses + data misses VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 13

14 Sequential memory access Beware automatic prefetching! Sequential access is much cheaper -5GB/s vs 500MB/s Workaround: use amoritzed latency for sequential accesses Random memory access VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 14

15 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 15

16 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 16

17 Perfect reuse VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 17

18 No reuse VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 18

19 LRU reuse VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 19

20 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 20

21 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 21

22 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 22

23 keys buckets 0 1 collision-list BAT mask link VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 23

24 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 24

25 Writes are non-blocking Ł Latency = 0 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 25

26 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 26

27 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 27

28 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 28

29 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 29

30 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 30

31 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 31

32 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 32

33 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 33

34 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 34

35 VLDB 2009 Summer School Shanghai Architecture-Conscious Database Techniques 35

FPGA-based Multithreading for In-Memory Hash Joins

FPGA-based Multithreading for In-Memory Hash Joins FPGA-based Multithreading for In-Memory Hash Joins Robert J. Halstead, Ildar Absalyamov, Walid A. Najjar, Vassilis J. Tsotras University of California, Riverside Outline Background What are FPGAs Multithreaded

More information

Storage in Database Systems. CMPSCI 445 Fall 2010

Storage in Database Systems. CMPSCI 445 Fall 2010 Storage in Database Systems CMPSCI 445 Fall 2010 1 Storage Topics Architecture and Overview Disks Buffer management Files of records 2 DBMS Architecture Query Parser Query Rewriter Query Optimizer Query

More information

COSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters

COSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters COSC 6374 Parallel Computation Parallel I/O (I) I/O basics Spring 2008 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network

More information

Operating Systems. Virtual Memory

Operating Systems. Virtual Memory Operating Systems Virtual Memory Virtual Memory Topics. Memory Hierarchy. Why Virtual Memory. Virtual Memory Issues. Virtual Memory Solutions. Locality of Reference. Virtual Memory with Segmentation. Page

More information

Deliverable 2.1.4. 150 Billion Triple dataset hosted on the LOD2 Knowledge Store Cluster. LOD2 Creating Knowledge out of Interlinked Data

Deliverable 2.1.4. 150 Billion Triple dataset hosted on the LOD2 Knowledge Store Cluster. LOD2 Creating Knowledge out of Interlinked Data Collaborative Project LOD2 Creating Knowledge out of Interlinked Data Project Number: 257943 Start Date of Project: 01/09/2010 Duration: 48 months Deliverable 2.1.4 150 Billion Triple dataset hosted on

More information

SSD Server Hard Drives for IBM

SSD Server Hard Drives for IBM New for 2011! Axiom is introducing a new line of SSD Hot-Swap Server Hard Drives. Our high performance SSD hard drives have been paired with system specific Hot-Swap trays to deliver seamless integration

More information

The Use of Flash in Large-Scale Storage Systems. Nathan.Rutman@Seagate.com

The Use of Flash in Large-Scale Storage Systems. Nathan.Rutman@Seagate.com The Use of Flash in Large-Scale Storage Systems Nathan.Rutman@Seagate.com 1 Seagate s Flash! Seagate acquired LSI s Flash Components division May 2014 Selling multiple formats / capacities today Nytro

More information

System Architecture. CS143: Disks and Files. Magnetic disk vs SSD. Structure of a Platter CPU. Disk Controller...

System Architecture. CS143: Disks and Files. Magnetic disk vs SSD. Structure of a Platter CPU. Disk Controller... System Architecture CS143: Disks and Files CPU Word (1B 64B) ~ 10 GB/sec Main Memory System Bus Disk Controller... Block (512B 50KB) ~ 100 MB/sec Disk 1 2 Magnetic disk vs SSD Magnetic Disk Stores data

More information

DSS. High performance storage pools for LHC. Data & Storage Services. Łukasz Janyst. on behalf of the CERN IT-DSS group

DSS. High performance storage pools for LHC. Data & Storage Services. Łukasz Janyst. on behalf of the CERN IT-DSS group DSS High performance storage pools for LHC Łukasz Janyst on behalf of the CERN IT-DSS group CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it Introduction The goal of EOS is to provide a

More information

ICP. Cache Hierarchies. Squid. Squid Cache ICP Use. Squid. Squid

ICP. Cache Hierarchies. Squid. Squid Cache ICP Use. Squid. Squid Caching & CDN s 15-44: Computer Networking L-21: Caching and CDNs HTTP APIs Assigned reading [FCAB9] Summary Cache: A Scalable Wide- Area Cache Sharing Protocol [Cla00] Freenet: A Distributed Anonymous

More information

Systems Infrastructure for Data Science. Web Science Group Uni Freiburg WS 2014/15

Systems Infrastructure for Data Science. Web Science Group Uni Freiburg WS 2014/15 Systems Infrastructure for Data Science Web Science Group Uni Freiburg WS 2014/15 Lecture I: Storage Storage Part I of this course Uni Freiburg, WS 2014/15 Systems Infrastructure for Data Science 3 The

More information

Database Management Systems

Database Management Systems 4411 Database Management Systems Acknowledgements and copyrights: these slides are a result of combination of notes and slides with contributions from: Michael Kiffer, Arthur Bernstein, Philip Lewis, Anestis

More information

Database Architecture Optimized for the new Bottleneck: Memory Access

Database Architecture Optimized for the new Bottleneck: Memory Access Database Architecture Optimized for the new Bottleneck: Memory Access Peter Boncz Stefan Manegold Martin L. Kersten Data Distilleries B.V. CWI Amsterdam The Netherlands Amsterdam The Netherlands P.Boncz@ddi.nl

More information

Cloud Gateway. Agenda. Cloud concepts Gateway concepts My work. Monica Stebbins

Cloud Gateway. Agenda. Cloud concepts Gateway concepts My work. Monica Stebbins Approved for Public Release; Distribution Unlimited. Case Number 15 0196 Cloud Gateway Monica Stebbins Agenda 2 Cloud concepts Gateway concepts My work 3 Cloud concepts What is Cloud 4 Similar to hosted

More information

The Quest for Speed - Memory. Cache Memory. A Solution: Memory Hierarchy. Memory Hierarchy

The Quest for Speed - Memory. Cache Memory. A Solution: Memory Hierarchy. Memory Hierarchy The Quest for Speed - Memory Cache Memory CSE 4, Spring 25 Computer Systems http://www.cs.washington.edu/4 If all memory accesses (IF/lw/sw) accessed main memory, programs would run 20 times slower And

More information

Extensible Data Model with Applications for Trading Systems

Extensible Data Model with Applications for Trading Systems , October 24-26, 2012, San Francisco, USA Extensible Data Model with Applications for Trading Systems Iosif Ziman Abstract An extensible main-memory data model is presented with applications in writing

More information

Scaling from Datacenter to Client

Scaling from Datacenter to Client Scaling from Datacenter to Client KeunSoo Jo Sr. Manager Memory Product Planning Samsung Semiconductor Audio-Visual Sponsor Outline SSD Market Overview & Trends - Enterprise What brought us to NVMe Technology

More information

1 Storage Devices Summary

1 Storage Devices Summary Chapter 1 Storage Devices Summary Dependability is vital Suitable measures Latency how long to the first bit arrives Bandwidth/throughput how fast does stuff come through after the latency period Obvious

More information

Chapter 6. 6.1 Introduction. Storage and Other I/O Topics. p. 570( 頁 585) Fig. 6.1. I/O devices can be characterized by. I/O bus connections

Chapter 6. 6.1 Introduction. Storage and Other I/O Topics. p. 570( 頁 585) Fig. 6.1. I/O devices can be characterized by. I/O bus connections Chapter 6 Storage and Other I/O Topics 6.1 Introduction I/O devices can be characterized by Behavior: input, output, storage Partner: human or machine Data rate: bytes/sec, transfers/sec I/O bus connections

More information

Bright Idea: GE s Storage Performance Best Practices Brian W. Walker

Bright Idea: GE s Storage Performance Best Practices Brian W. Walker Bright Idea: GE s Storage Performance Best Practices Brian W. Walker Principal Architect, Cloud Solutions 1 Speaker Introduction Brian Walker Principal Architect, Cloud Solutions Brian brings more than

More information

COSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters

COSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters COSC 6374 Parallel I/O (I) I/O basics Fall 2012 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network card 1 Network card

More information

Optics Technology Trends in Data Centers

Optics Technology Trends in Data Centers Optics Technology Trends in Data Centers Marc A. Taubenblatt IBM OFC Market Watch 2014 Outline/Summary Data movement/aggregate BW increasing and increasingly important Historically, per channel data rate

More information

Concept of Cache in web proxies

Concept of Cache in web proxies Concept of Cache in web proxies Chan Kit Wai and Somasundaram Meiyappan 1. Introduction Caching is an effective performance enhancing technique that has been used in computer systems for decades. However,

More information

Software-defined Storage Architecture for Analytics Computing

Software-defined Storage Architecture for Analytics Computing Software-defined Storage Architecture for Analytics Computing Arati Joshi Performance Engineering Colin Eldridge File System Engineering Carlos Carrero Product Management June 2015 Reference Architecture

More information

Azure VM Performance Considerations Running SQL Server

Azure VM Performance Considerations Running SQL Server Azure VM Performance Considerations Running SQL Server Your company logo here Vinod Kumar M @vinodk_sql http://blogs.extremeexperts.com Session Objectives And Takeaways Session Objective(s): Learn the

More information

A NOR Emulation Strategy over NAND Flash Memory

A NOR Emulation Strategy over NAND Flash Memory A NOR Emulation Strategy over NAND Flash Memory Jian-Hong Lin, Yuan-Hao Chang, Jen-Wei Hsieh, and Tei-Wei Kuo Embedded Systems and Wireless Networking Laboratory Dept. of Computer Science and Information

More information

Sun 8Gb/s Fibre Channel HBA Performance Advantages for Oracle Database

Sun 8Gb/s Fibre Channel HBA Performance Advantages for Oracle Database Performance Advantages for Oracle Database At a Glance This Technical Brief illustrates that even for smaller online transaction processing (OLTP) databases, the Sun 8Gb/s Fibre Channel Host Bus Adapter

More information

Hadoop: Embracing future hardware

Hadoop: Embracing future hardware Hadoop: Embracing future hardware Suresh Srinivas @suresh_m_s Page 1 About Me Architect & Founder at Hortonworks Long time Apache Hadoop committer and PMC member Designed and developed many key Hadoop

More information

RAID. RAID 0 No redundancy ( AID?) Just stripe data over multiple disks But it does improve performance. Chapter 6 Storage and Other I/O Topics 29

RAID. RAID 0 No redundancy ( AID?) Just stripe data over multiple disks But it does improve performance. Chapter 6 Storage and Other I/O Topics 29 RAID Redundant Array of Inexpensive (Independent) Disks Use multiple smaller disks (c.f. one large disk) Parallelism improves performance Plus extra disk(s) for redundant data storage Provides fault tolerant

More information

1.1 The Nature of Software... Object-Oriented Software Engineering Practical Software Development using UML and Java. The Nature of Software...

1.1 The Nature of Software... Object-Oriented Software Engineering Practical Software Development using UML and Java. The Nature of Software... 1.1 The Nature of Software... Object-Oriented Software Engineering Practical Software Development using UML and Java Chapter 1: Software and Software Engineering Software is intangible Hard to understand

More information

Multimedia Caching Strategies for Heterogeneous Application and Server Environments

Multimedia Caching Strategies for Heterogeneous Application and Server Environments Multimedia Tools and Applications 4, 279 312 (1997) c 1997 Kluwer Academic Publishers. Manufactured in The Netherlands. Multimedia Caching Strategies for Heterogeneous Application and Server Environments

More information

ICRI-CI Retreat Architecture track

ICRI-CI Retreat Architecture track ICRI-CI Retreat Architecture track Uri Weiser June 5 th 2015 - Funnel: Memory Traffic Reduction for Big Data & Machine Learning (Uri) - Accelerators for Big Data & Machine Learning (Ran) - Machine Learning

More information

Price/performance Modern Memory Hierarchy

Price/performance Modern Memory Hierarchy Lecture 21: Storage Administration Take QUIZ 15 over P&H 6.1-4, 6.8-9 before 11:59pm today Project: Cache Simulator, Due April 29, 2010 NEW OFFICE HOUR TIME: Tuesday 1-2, McKinley Last Time Exam discussion

More information

WITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE

WITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE WITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE 1 W W W. F U S I ON I O.COM Table of Contents Table of Contents... 2 Executive Summary... 3 Introduction: In-Memory Meets iomemory... 4 What

More information

Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle

Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle Agenda Introduction Database Architecture Direct NFS Client NFS Server

More information

Generating code for holistic query evaluation Konstantinos Krikellas 1, Stratis D. Viglas 2, Marcelo Cintra 3

Generating code for holistic query evaluation Konstantinos Krikellas 1, Stratis D. Viglas 2, Marcelo Cintra 3 Generating code for holistic query evaluation Konstantinos Krikellas 1, Stratis D. Viglas 2, Marcelo Cintra 3 School of Informatics, University of Edinburgh, UK 1 k.krikellas@sms.ed.ac.uk 2 sviglas@inf.ed.ac.uk

More information

Speeding Up Cloud/Server Applications Using Flash Memory

Speeding Up Cloud/Server Applications Using Flash Memory Speeding Up Cloud/Server Applications Using Flash Memory Sudipta Sengupta Microsoft Research, Redmond, WA, USA Contains work that is joint with B. Debnath (Univ. of Minnesota) and J. Li (Microsoft Research,

More information

Outline. Principles of Database Management Systems. Memory Hierarchy: Capacities and access times. CPU vs. Disk Speed ... ...

Outline. Principles of Database Management Systems. Memory Hierarchy: Capacities and access times. CPU vs. Disk Speed ... ... Outline Principles of Database Management Systems Pekka Kilpeläinen (after Stanford CS245 slide originals by Hector Garcia-Molina, Jeff Ullman and Jennifer Widom) Hardware: Disks Access Times Example -

More information

Hardware-Oblivious Parallelism for In-Memory Column-Stores

Hardware-Oblivious Parallelism for In-Memory Column-Stores Hardware-Oblivious Parallelism for In-Memory Column-Stores Max Heimel Technische Universität Berlin max.heimel@tu-berlin.de Stefan Manegold CWI Amsterdam stefan.manegold@cwi.nl Michael Saecker ParStream

More information

How NAND Flash Threatens DRAM

How NAND Flash Threatens DRAM How NAND Flash Threatens DRAM Jim Handy OBJECTIVE ANALYSIS Outline Why even think about DRAM vs. NAND? The memory/storage hierarchy What benchmarks tell us What about 3D XPoint memory? The system of the

More information

1 / 25. CS 137: File Systems. Persistent Solid-State Storage

1 / 25. CS 137: File Systems. Persistent Solid-State Storage 1 / 25 CS 137: File Systems Persistent Solid-State Storage Technology Change is Coming Introduction Disks are cheaper than any solid-state memory Likely to be true for many years But SSDs are now cheap

More information

Configuring Apache Derby for Performance and Durability Olav Sandstå

Configuring Apache Derby for Performance and Durability Olav Sandstå Configuring Apache Derby for Performance and Durability Olav Sandstå Database Technology Group Sun Microsystems Trondheim, Norway Overview Background > Transactions, Failure Classes, Derby Architecture

More information

Evaluating the Effect of Online Data Compression on the Disk Cache of a Mass Storage System

Evaluating the Effect of Online Data Compression on the Disk Cache of a Mass Storage System Evaluating the Effect of Online Data Compression on the Disk Cache of a Mass Storage System Odysseas I. Pentakalos and Yelena Yesha Computer Science Department University of Maryland Baltimore County Baltimore,

More information

Hybrid Storage Management for Database Systems

Hybrid Storage Management for Database Systems Hybrid Storage Management for Database Systems Xin Liu University of Waterloo, Canada x39liu@uwaterloo.ca Kenneth Salem University of Waterloo, Canada ksalem@uwaterloo.ca ABSTRACT The use of flash-based

More information

Multi-level Hybrid Cache: Impact and Feasibility

Multi-level Hybrid Cache: Impact and Feasibility Multi-level Hybrid Cache: Impact and Feasibility ZheZhang,YoungjaeKim,XiaosongMa,GalenShipman,YuanyuanZhou OakRidgeNationalLaboratory, NorthCarolinaStateUniversity, UniversityofCaliforniaSanDiego Abstract

More information

SQL Clusters in Virtualized Environments April 10 th, 2013

SQL Clusters in Virtualized Environments April 10 th, 2013 SQL Clusters in Virtualized Environments April 10 th, 2013 Session Overview This session s primary objective is to help you gain insight into where clustering fits in the overall picture of HA/DR as it

More information

COS 318: Operating Systems. Virtual Memory and Address Translation

COS 318: Operating Systems. Virtual Memory and Address Translation COS 318: Operating Systems Virtual Memory and Address Translation Today s Topics Midterm Results Virtual Memory Virtualization Protection Address Translation Base and bound Segmentation Paging Translation

More information

Main Memory & Backing Store. Main memory backing storage devices

Main Memory & Backing Store. Main memory backing storage devices Main Memory & Backing Store Main memory backing storage devices 1 Introduction computers store programs & data in two different ways: nmain memory ntemporarily stores programs & data that are being processed

More information

Disk Storage & Dependability

Disk Storage & Dependability Disk Storage & Dependability Computer Organization Architectures for Embedded Computing Wednesday 19 November 14 Many slides adapted from: Computer Organization and Design, Patterson & Hennessy 4th Edition,

More information

Flash for Databases. September 22, 2015 Peter Zaitsev Percona

Flash for Databases. September 22, 2015 Peter Zaitsev Percona Flash for Databases September 22, 2015 Peter Zaitsev Percona In this Presentation Flash technology overview Review some of the available technology What does this mean for databases? Specific opportunities

More information

Hybrid In-Memory and On-Disk Tables for Speeding-Up Table Accesses

Hybrid In-Memory and On-Disk Tables for Speeding-Up Table Accesses Hybrid In-Memory and On-Disk Tables for Speeding-Up Table Accesses Joan Guisado-Gámez 1,AntoniWolski 2, Calisto Zuzarte 3, Josep-Lluís Larriba-Pey 1,andVictorMuntés-Mulero 1 1 DAMA-UPC, Departament d Arquitectura

More information

F600Q 8Gb FC Storage Performance Report Date: 2012/10/30

F600Q 8Gb FC Storage Performance Report Date: 2012/10/30 F600Q 8Gb FC Storage Performance Report Date: 2012/10/30 Table of Content IO Feature Highlights Test Configurations Maximum IOPS & Best Throughput Maximum Sequential IOPS Test Configurations Random IO

More information

Jae Woo Choi, Dong In Shin, Young Jin Yu, Hyunsang Eom, Heon Young Yeom. Seoul National Univ. TAEJIN INFOTECH

Jae Woo Choi, Dong In Shin, Young Jin Yu, Hyunsang Eom, Heon Young Yeom. Seoul National Univ. TAEJIN INFOTECH Jae Woo Choi, Dong In Shin, Young Jin Yu, Hyunsang Eom, Heon Young Yeom Seoul National Univ. TAEJIN INFOTECH Host Computer (initiator) SAN with High-Speed Network + Fast Storage = Fast SAN Environment???

More information

Scalable Cache Miss Handling For High MLP

Scalable Cache Miss Handling For High MLP Scalable Cache Miss Handling For High MLP James Tuck, Luis Ceze, and Josep Torrellas University of Illinois at Urbana-Champaign http://iacoma.cs.uiuc.edu Introduction Checkpointed processors are promising

More information

SATA II 3Gb/s SSD. SSD630 Benefits. Enhanced Performance. Applications

SATA II 3Gb/s SSD. SSD630 Benefits. Enhanced Performance. Applications SATA II 3Gb/s SSD SSD630 Benefits Super slim thickness of 7mm Fully compatible with devices and OS that support the SATA II 3Gb/s standard Non-volatile Flash Memory for outstanding data retention Built-in

More information

6. Storage and File Structures

6. Storage and File Structures ECS-165A WQ 11 110 6. Storage and File Structures Goals Understand the basic concepts underlying different storage media, buffer management, files structures, and organization of records in files. Contents

More information

CSE 120 Principles of Operating Systems

CSE 120 Principles of Operating Systems CSE 120 Principles of Operating Systems Fall 2004 Lecture 13: FFS, LFS, RAID Geoffrey M. Voelker Overview We ve looked at disks and file systems generically Now we re going to look at some example file

More information

The Classical Architecture. Storage 1 / 36

The Classical Architecture. Storage 1 / 36 1 / 36 The Problem Application Data? Filesystem Logical Drive Physical Drive 2 / 36 Requirements There are different classes of requirements: Data Independence application is shielded from physical storage

More information

Upgrading Small Business Client and Server Infrastructure E-LEET Solutions. E-LEET Solutions is an information technology consulting firm

Upgrading Small Business Client and Server Infrastructure E-LEET Solutions. E-LEET Solutions is an information technology consulting firm Thank you for considering E-LEET Solutions! E-LEET Solutions is an information technology consulting firm that specializes in low-cost high-performance computing solutions. This document was written as

More information

Architecture of Hitachi SR-8000

Architecture of Hitachi SR-8000 Architecture of Hitachi SR-8000 University of Stuttgart High-Performance Computing-Center Stuttgart (HLRS) www.hlrs.de Slide 1 Most of the slides from Hitachi Slide 2 the problem modern computer are data

More information

An Overview of Flash Storage for Databases

An Overview of Flash Storage for Databases An Overview of Flash Storage for Databases Vadim Tkachenko Morgan Tocker http://percona.com MySQL CE Apr 2010 -2- Introduction Vadim Tkachenko Percona Inc, CTO and Lead of Development Morgan Tocker Percona

More information

IO latency tracking. Daniel Lezcano (dlezcano) Linaro Power Management Team

IO latency tracking. Daniel Lezcano (dlezcano) Linaro Power Management Team IO latency tracking Daniel Lezcano (dlezcano) Linaro Power Management Team CPUIDLE CPUidle is a framework divided into three parts: The generic framework The back-end driver: low level calls or hidden

More information

Outline. CS 245: Database System Principles. Notes 02: Hardware. Hardware DBMS ... ... Data Storage

Outline. CS 245: Database System Principles. Notes 02: Hardware. Hardware DBMS ... ... Data Storage CS 245: Database System Principles Notes 02: Hardware Hector Garcia-Molina Outline Hardware: Disks Access Times Solid State Drives Optimizations Other Topics: Storage costs Using secondary storage Disk

More information

Secondary Storage. Any modern computer system will incorporate (at least) two levels of storage: magnetic disk/optical devices/tape systems

Secondary Storage. Any modern computer system will incorporate (at least) two levels of storage: magnetic disk/optical devices/tape systems 1 Any modern computer system will incorporate (at least) two levels of storage: primary storage: typical capacity cost per MB $3. typical access time burst transfer rate?? secondary storage: typical capacity

More information

Will SSD replace HDD?

Will SSD replace HDD? Will SSD replace HDD? SSD vs HDD HDD the workhorse of data storage The battle for storage has been going on for decades with hard disk drives (HDD) dominating because of cost and capacity Tier 0 is Flash

More information

Data and Storage Services

Data and Storage Services Data and Storage Services G. Cancio, D. Duellmann, J. Iven, M. Lamanna, A. Pace, A.J. Peters, R.Toebbicke CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it CERN IT Department CH-1211 Genève

More information

Can High-Performance Interconnects Benefit Memcached and Hadoop?

Can High-Performance Interconnects Benefit Memcached and Hadoop? Can High-Performance Interconnects Benefit Memcached and Hadoop? D. K. Panda and Sayantan Sur Network-Based Computing Laboratory Department of Computer Science and Engineering The Ohio State University,

More information

Improve Business Productivity and User Experience with a SanDisk Powered SQL Server 2014 In-Memory OLTP Database

Improve Business Productivity and User Experience with a SanDisk Powered SQL Server 2014 In-Memory OLTP Database WHITE PAPER Improve Business Productivity and User Experience with a SanDisk Powered SQL Server 2014 In-Memory OLTP Database 951 SanDisk Drive, Milpitas, CA 95035 www.sandisk.com Table of Contents Executive

More information

Microkernels & Database OSs. Recovery Management in QuickSilver. DB folks: Stonebraker81. Very different philosophies

Microkernels & Database OSs. Recovery Management in QuickSilver. DB folks: Stonebraker81. Very different philosophies Microkernels & Database OSs Recovery Management in QuickSilver. Haskin88: Roger Haskin, Yoni Malachi, Wayne Sawdon, Gregory Chan, ACM Trans. On Computer Systems, vol 6, no 1, Feb 1988. Stonebraker81 OS/FS

More information

How it can benefit your enterprise. Dejan Kocic Hitachi Data Systems (HDS)

How it can benefit your enterprise. Dejan Kocic Hitachi Data Systems (HDS) PRESENTATION Case for flash TITLE GOES storage HERE How it can benefit your enterprise Dejan Kocic Hitachi Data Systems (HDS) SNIA Legal Notice The material contained in this tutorial is copyrighted by

More information

COS 318: Operating Systems. Storage Devices. Kai Li and Andy Bavier Computer Science Department Princeton University

COS 318: Operating Systems. Storage Devices. Kai Li and Andy Bavier Computer Science Department Princeton University COS 318: Operating Systems Storage Devices Kai Li and Andy Bavier Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall13/cos318/ Today s Topics! Magnetic disks!

More information

Sequential Performance Analysis with Callgrind and KCachegrind

Sequential Performance Analysis with Callgrind and KCachegrind Sequential Performance Analysis with Callgrind and KCachegrind 2 nd Parallel Tools Workshop, HLRS, Stuttgart, July 7/8, 2008 Josef Weidendorfer Lehrstuhl für Rechnertechnik und Rechnerorganisation Institut

More information

Liteon True Speed White Paper SPEED UP NETWORK STORAGE PERFORMANCE WITH LITEON TRUE SPEED TECHNOLOGY

Liteon True Speed White Paper SPEED UP NETWORK STORAGE PERFORMANCE WITH LITEON TRUE SPEED TECHNOLOGY SPEED UP NETWORK STORAGE PERFORMANCE WITH LITEON TRUE SPEED TECHNOLOGY 1 Contents Introduction... Migrating from Hard Disk Drive to Solid State Disk... It s all in the IOPS... Liteon True Speed... Reliable

More information

Storage Class Memory Aware Data Management

Storage Class Memory Aware Data Management Storage Class Memory Aware Data Management Bishwaranjan Bhattacharjee IBM T. J. Watson Research Center bhatta@us.ibm.com George A. Mihaila IBM T. J. Watson Research Center mihaila@us.ibm.com Mustafa Canim

More information

Objectivity Data Migration

Objectivity Data Migration Objectivity Data Migration M. Nowak, K. Nienartowicz, A. Valassi, M. Lübeck, D. Geppert CERN, CH-1211 Geneva 23, Switzerland In this article we describe the migration of event data collected by the COMPASS

More information

An Evaluation of Checkpoint Recovery for Massively Multiplayer Online Games

An Evaluation of Checkpoint Recovery for Massively Multiplayer Online Games An Evaluation of Checkpoint Recovery for Massively Multiplayer Online Games Marcos Vaz Salles, Tuan Cao, Benjamin Sowell, Alan Demers, Johannes Gehrke, Christoph Koch, Walker White Cornell University Ithaca,

More information

LEVERAGING FLASH MEMORY in ENTERPRISE STORAGE. Matt Kixmoeller, Pure Storage

LEVERAGING FLASH MEMORY in ENTERPRISE STORAGE. Matt Kixmoeller, Pure Storage LEVERAGING FLASH MEMORY in ENTERPRISE STORAGE Matt Kixmoeller, Pure Storage SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies

More information

Conjunctive Selection Conditions in Main Memory

Conjunctive Selection Conditions in Main Memory Conjunctive Selection Conditions in Main Memory Kenneth A. Ross Columbia University kar@cs.columbia.edu ABSTRACT We consider the fundamental operation of applying a conjunction of selection conditions

More information

Table 1 SDR to DDR Quick Reference

Table 1 SDR to DDR Quick Reference TECHNICAL NOTE TN-6-05 GENERAL DDR SDRAM FUNCTIONALITY INTRODUCTION The migration from single rate synchronous DRAM (SDR) to double rate synchronous DRAM (DDR) memory is upon us. Although there are many

More information

A Review of Customized Dynamic Load Balancing for a Network of Workstations

A Review of Customized Dynamic Load Balancing for a Network of Workstations A Review of Customized Dynamic Load Balancing for a Network of Workstations Taken from work done by: Mohammed Javeed Zaki, Wei Li, Srinivasan Parthasarathy Computer Science Department, University of Rochester

More information

PDSA Special Report. Custom Software vs Off the Shelf

PDSA Special Report. Custom Software vs Off the Shelf PDSA Special Report Deciding Between Custom and Off the Shelf Your legacy system is getting tired and old and you need to begin looking for an alternative solution. Do you buy off the shelf or do you develop

More information

idedup: Latency-aware, inline data deduplication for primary storage

idedup: Latency-aware, inline data deduplication for primary storage idedup: Latency-aware, inline data deduplication for primary storage Kiran Srinivasan, Tim Bisson, Garth Goodson, Kaladhar Voruganti NetApp, Inc. {skiran, tbisson, goodson, kaladhar}@netapp.com Abstract

More information

Samsung Portable SSD. Branded Product Marketing Team, Memory Business

Samsung Portable SSD. Branded Product Marketing Team, Memory Business Samsung Portable SSD Branded Product Marketing Team, Memory Business New concept of external storage = PORTABLE SSD Target Segment of Market External storage has two main market segments, PC-connected

More information

Deep Dive: Maximizing EC2 & EBS Performance

Deep Dive: Maximizing EC2 & EBS Performance Deep Dive: Maximizing EC2 & EBS Performance Tom Maddox, Solutions Architect 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved What we ll cover Amazon EBS overview Volumes Snapshots

More information

Storing Data: Disks and Files. Disks and Files. Why Not Store Everything in Main Memory? Chapter 7

Storing Data: Disks and Files. Disks and Files. Why Not Store Everything in Main Memory? Chapter 7 Storing : Disks and Files Chapter 7 Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Disks and

More information

White paper. QNAP Turbo NAS with SSD Cache

White paper. QNAP Turbo NAS with SSD Cache White paper QNAP Turbo NAS with SSD Cache 1 Table of Contents Introduction... 3 Audience... 3 Terminology... 3 SSD cache technology... 4 Applications and benefits... 5 Limitations... 6 Performance Test...

More information

Parallel Algorithm Engineering

Parallel Algorithm Engineering Parallel Algorithm Engineering Kenneth S. Bøgh PhD Fellow Based on slides by Darius Sidlauskas Outline Background Current multicore architectures UMA vs NUMA The openmp framework Examples Software crisis

More information

Fast Iterative Graph Computation with Resource Aware Graph Parallel Abstraction

Fast Iterative Graph Computation with Resource Aware Graph Parallel Abstraction Human connectome. Gerhard et al., Frontiers in Neuroinformatics 5(3), 2011 2 NA = 6.022 1023 mol 1 Paul Burkhardt, Chris Waring An NSA Big Graph experiment Fast Iterative Graph Computation with Resource

More information

Operating Systems CSE 410, Spring 2004. File Management. Stephen Wagner Michigan State University

Operating Systems CSE 410, Spring 2004. File Management. Stephen Wagner Michigan State University Operating Systems CSE 410, Spring 2004 File Management Stephen Wagner Michigan State University File Management File management system has traditionally been considered part of the operating system. Applications

More information

Writing Assignment #2 due Today (5:00pm) - Post on your CSC101 webpage - Ask if you have questions! Lab #2 Today. Quiz #1 Tomorrow (Lectures 1-7)

Writing Assignment #2 due Today (5:00pm) - Post on your CSC101 webpage - Ask if you have questions! Lab #2 Today. Quiz #1 Tomorrow (Lectures 1-7) Overview of Computer Science CSC 101 Summer 2011 Main Memory vs. Auxiliary Storage Lecture 7 July 14, 2011 Announcements Writing Assignment #2 due Today (5:00pm) - Post on your CSC101 webpage - Ask if

More information

ChunkStash: Speeding up Inline Storage Deduplication using Flash Memory

ChunkStash: Speeding up Inline Storage Deduplication using Flash Memory ChunkStash: Speeding up Inline Storage Deduplication using Flash Memory Biplob Debnath Sudipta Sengupta Jin Li Microsoft Research, Redmond, WA, USA University of Minnesota, Twin Cities, USA Abstract Storage

More information

High Performance Tier Implementation Guideline

High Performance Tier Implementation Guideline High Performance Tier Implementation Guideline A Dell Technical White Paper PowerVault MD32 and MD32i Storage Arrays THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS

More information

ECLIPSE Performance Benchmarks and Profiling. January 2009

ECLIPSE Performance Benchmarks and Profiling. January 2009 ECLIPSE Performance Benchmarks and Profiling January 2009 Note The following research was performed under the HPC Advisory Council activities AMD, Dell, Mellanox, Schlumberger HPC Advisory Council Cluster

More information

The Piranha computer algebra system. introduction and implementation details

The Piranha computer algebra system. introduction and implementation details : introduction and implementation details Advanced Concepts Team European Space Agency (ESTEC) Course on Differential Equations and Computer Algebra Estella, Spain October 29-30, 2010 Outline A Brief Overview

More information

NUMA obliviousness through memory mapping

NUMA obliviousness through memory mapping NUMA obliviousness through memory mapping Mrunal Gawade CWI, Amsterdam mrunal.gawade@cwi.nl Martin Kersten CWI, Amsterdam martin.kersten@cwi.nl ABSTRACT With the rise of multi-socket multi-core CPUs a

More information

This Unit: Caches. CIS 501 Introduction to Computer Architecture. Motivation. Types of Memory

This Unit: Caches. CIS 501 Introduction to Computer Architecture. Motivation. Types of Memory This Unit: Caches CIS 5 Introduction to Computer Architecture Unit 3: Storage Hierarchy I: Caches Application OS Compiler Firmware CPU I/O Memory Digital Circuits Gates & Transistors Memory hierarchy concepts

More information

KVM & Memory Management Updates

KVM & Memory Management Updates KVM & Memory Management Updates KVM Forum 2012 Rik van Riel Red Hat, Inc. KVM & Memory Management Updates EPT Accessed & Dirty Bits 1GB hugepages Balloon vs. Transparent Huge Pages Automatic NUMA Placement

More information

Warming up Storage-Level Caches with Bonfire

Warming up Storage-Level Caches with Bonfire Warming up Storage-Level Caches with Bonfire Yiying Zhang, Gokul Soundararajan, Mark W. Storer, Lakshmi N. Bairavasundaram, Sethuraman Subbiah, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau University

More information

NetApp FAS Hybrid Array Flash Efficiency. Silverton Consulting, Inc. StorInt Briefing

NetApp FAS Hybrid Array Flash Efficiency. Silverton Consulting, Inc. StorInt Briefing NetApp FAS Hybrid Array Flash Efficiency Silverton Consulting, Inc. StorInt Briefing PAGE 2 OF 7 Introduction Hybrid storage arrays (storage systems with both disk and flash capacity) have become commonplace

More information

Demonstration of Electronic Health Records (EHR) on Java Card 3.0 Technology-Based Devices

Demonstration of Electronic Health Records (EHR) on Java Card 3.0 Technology-Based Devices BOF-4576 Demonstration of Electronic Health Records (EHR) on Java Card 3.0 Technology-Based Devices Nicolas Anciaux Jean-Jacques Vandewalle Overview Show a real deployment of Java Card 3 devices in the

More information