Spatial Data Management over Flash Memory

Save this PDF as:

Size: px
Start display at page:

Download "Spatial Data Management over Flash Memory"

Transcription

1 Spatial Data Management over Flash Memory Ioannis Koltsidas 1 and Stratis D. Viglas 2 1 IBM Research, Zurich, Switzerland 2 School of Informatics, University of Edinburgh, UK Abstract. We present desiderata for improved i/o performance of spatial data structures over flash memory and hybrid flash-magnetic storage configurations. We target the organization of the data structures and the management of the in-memory buffer pool when dealing with spatial data. Our proposals are based on the fundamentals of flash memory, thereby increasing the likelihood of being immune to future trends. 1 Introduction Flash memory has moved from being used for short-term, low-volume storage and data transfer, to becoming the primary alternative to magnetic disks for longterm, high-volume and persistent storage. Appearing in systems as disparate as pdas, handheld tablets, laptops and desktops, enterprise servers and clusters, it is one of the most ubiquitous storage media on the market. At the same time, the proliferation of location-based services means that spatial data management techniques acquire increasing traction. As a result, access to and processing of spatial data moves from highly customized application scenarios to commodity ones. The combination of these two trends necessitates revisiting spatial data management for flash memory both at the server and at the client levels. In this paper we will present ideas that we posit are imperatives when it comes to high-performing techniques for flash-resident spatial data. Solid state drives, orssds, are arrays of flash memory chips packaged in a single enclosure with a controller. An ssd is presented to the operating system as a single storage device using the same interface as traditional hdds (e.g., sata). The similarities between the two types of storage medium, however, stop here. The key characteristics of ssds are(a) the lack of mechanical moving parts; (b) the asymmetry between their read and write latencies; and (c) theirerasebefore-write limitation. The first characteristic means that there is no difference between sequential and random access latencies. erformance is not penalized by being dependent on the seek time or the rotational delay, as is the case for hdds. The second characteristic has to do with the physical characteristics of flash memory, which make reading the value of a flash memory cell faster than changing it; therefore, reads are in general faster than writes. The discrepancy between read and write latencies is further influenced by the underlying technology of flash memory and, in particular, by how many bits are stored in each D. foser et al. (Eds.): SSTD 2011, NCS 6849, pp , c Springer-Verlag Berlin Heidelberg 2011

2 450 I. Koltsidas and S.D. Viglas cell: a single-level cell (slc) device can store a single bit, while a multi-level cell (mlc) can store two bits. slc devices inherently have higher performance but lower density; on the other hand, mlc devices have higher density but lower performance. Finally, the third characteristic stems for the inherent properties of ssds: to update an already written flash page, the controller must first erase it and then overwrite it. Erasures are performed at an erase unit granularity, where each erase unit is a flash block, i.e., a number of contiguous flash pages. A garbage collection mechanism is required to reclaim flash blocks as pages become invalid due to user overwrites. This is especially detrimental to the performance of random write workloads: random writes result in continuously erasing blocks and moving data at the flash level. It is not uncommon for the random write throughput to be five times less than the random read throughput for slc devices; or even up to two orders of magnitude less for mlc devices. Though approaches for efficient spatial data structures have been proposed [1,5], clearly, data management over flash memory requires rethinking our priorities and not improving performance in specific cases. To complicate the situation, existing hdd-based hardware is not going to disappear any time soon; the trend is towards augmenting hdd storage with flash memory. It is therefore imperative to design towards hybrid storage configurations employing both ssds and hdds. We propose ways to tackle spatial data management in flash and hybrid setups for the applications of the foreseeable future. We cannot claim that these ideas will be future-proof; however, we solidly ground them on the inherent characteristics of flash memory, thereby increasing their potential for applicability. We do not require any flash features be exposed apart from a standard i/o interface. Our proposals revolve around two axes: the organization of spatial data structures in ssd-only as well as in hybrid configurations, and the management of pages belonging to spatial data structures once the pages have been brought in main memory. In what follows we will present each axis in turn. 2 Data Structure Organization One of the key goals of secondary storage data structures is performance guarantees. For instance, the best performing spatial data structures are balanced trees. Their guarantee is that all paths from root to leaves are of equal length. This is achieved through bottom-up management algorithms: insertions and deletions cause splits and merges at the leaves; these splits and merges are recursively applied in the bottom-up traversal and propagate higher up in the tree; resulting in the tree having its height increased or decreased. Such algorithms make perfect sense for structures over hdds where the read and write costs are uniform. Consider, however, the read/write discrepancy of ssds and an insertion like the one of Fig. 1. Typically, as shown in the left part, an insertion to a full leaf will cause a split of the leaf into itself and a sibling S; and an update at the original leaf s parent with the new bounding box for ; and the insertion of the bounding box for S. To balance the tree we must perform three disk writes. An alternative, instead of splitting into and S, is to allow to overflow into

3 Spatial Data Management over Flash Memory 451 balanced insertion unbalanced insertion S original setup S Fig. 1. Balanced vs. unbalanced insertions S and we not update,since s bounding box has not changed (right part of Fig. 1). Thus, instead of performing three writes, we perform one: only for S. Assume now that the ratio between write and read is x, i.e., reading is x times faster than writing. Depending on the application, the cost associated with each type of operation may be any combination of the latency and throughput of the ssd for that type of operation. Saving two writes during insertion and causing the imbalance means that we have saved 2x time units: any operation that will read would have read it anyway; any operation having to access S too, would have accessed it anyway, but going through a direct child pointer from.thus, the structure overall has x time units to spare with respect to its optimal form (i.e., the fully balanced one). Then, we allow this imbalance and maintain a counter c at that measures how many times the overflow pointer to S has been traversed. As long as c<x, the data structure still is more efficient; once c = x we can rebalance it by propagating the bounding box from S to,thereby expensing the write we saved in the first place. This idea can work across all levels of the tree, though one must be careful to avoid long overflow chains. In such cases rebalancing can be prioritized using global rather than local metrics. In a typical scenario, the savings will be substantial. As the tree structure grows, the likelihood of a single leaf being bombarded with read or write requests decreases for all typical scenarios. Thus, the likelihood of the tree s imbalance paying off increases. Though generally applicable to all tree structures, catering for imbalance is more likely to pay off for spatial data and location-based applications: the update rate of the spatial data is low, while the point queries themselves are highly volatile as users move about the area. Observations like the one that lead to unbalanced structures can be generalized for hybrid configurations. In more detail, and for a tree structure, the likelihood of having to write data increases as we descend the tree. One might then consider placing the different parts of the tree structure on different media. The top levels of the tree have a read-intensive workload, used to direct searches. Whereas the bottom levels of the tree might have an update-intensive workload; moreover, the likelihood of an update being propagated upstream decreases with the height of the tree. It might then be conducive to place the write-intensive leavesofthetreeonthehdd and the read-intensive index nodes on the ssd [3].

4 452 I. Koltsidas and S.D. Viglas Coupled with the previous idea of keeping the flash-resident portion of the tree unbalanced, this will result in further performance boosts. 3 Buffer Management The next issue to focus on is buffering spatial data from the ssd into main memory. As is usually the case, we assume demand paging: data pages can only be processed in main memory and they are brought from the ssd into main memory only on reference. Main memory is of limited capacity and smaller than the capacity of the ssd; thus, only a subset of the flash-resident data pages can be kept in main memory at any point in time. Once the memory is full, data pages will be evicted to make room for new ones. There has been a substantial body of research targeting general buffer management for ssds [2,4]. Tailoring the approaches to spatial data, in conjunction with the i/o asymmetry of ssds, offers further room for improvement. Consider a location-based application which continuously accesses a spatial data index for information (e.g., points of interest close to the user s location). Such an application will generate multiple hot paths in the index, tracking the user s motion. As the user moves about, the paths followed from the root of the tree to the leaves will be constantly changing. Therefore, using simple reference counts as the eviction decision metric, though correct, can be further improved. By predicting the motion we can make more informed decisions regarding which pages to keep in the buffer pool and which have a lower probability of being referenced in the future. By selectively caching the parts of the indexed space that are more likely to receive updates, a lot of flash writes, and thereby, erasures can be avoided. Coupled with cost-based replacement policies [3], such an approach can improve i/o performance. One might even envision horizontal or vertical tree partitioning and placement based on access frequency and storage capabilities. 4 Conclusions and Outlook While ssds are attracting more traction from the research community and industry alike, surprisingly enough, one of the areas that is under-represented is spatial data management over flash memory. We have presented some initial ideas on improving the performance of flash-resident spatial data structures. One of the aspects we did not address is endurance and wear-leveling. More precisely, each flash cell can only be written to a fixed number of times. Thus, the controller spreads writes evenly throughout the flash chips and maintains a mapping between logical block identifiers and physical ones, through a software layer called the flash translation layer, orftl. While ftl algorithms are typically inaccessible to the user, some ssd manufacturers are now pushing more of that functionality to the os driver level; in addition, ssd-specific commands have lately been introduced, giving direct control to the filesystem for some flash operations (e.g., the trim command for explicitly erasing blocks). In light of that, the ideas presented in the previous sections can be more efficiently coupled with the ssd andresultinfurtheri/o savings.

5 Spatial Data Management over Flash Memory 453 References 1. Emrich, T., et al.: On the impact of flash SSDs on spatial indexing. In: DAMON 2010 (2010) 2. Kim, H., Ahn, S.: BRU: a buffer management scheme for improving random writes in flash storage. In: FAST (2008) 3. Koltsidas, I., Viglas, S.D.: Flashing up the storage layer. roc. VDB Endow. 1(1), (2008) 4. ark, S.-Y., et al.: CFRU: a replacement algorithm for flash memory. In: CASES (2006) 5. Wu, C.-H., et al.: An efficient R-tree implementation over flash-memory storage systems. In: GIS (2003)

Flash-optimized Data Progression

Flash-optimized Data Progression A Dell white paper Howard Shoobe, Storage Enterprise Technologist John Shirley, Product Management Dan Bock, Product Management Table of contents Executive summary... 3 What is different about Dell Compellent

More information

DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION

DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION A DIABLO WHITE PAPER AUGUST 2014 Ricky Trigalo Director of Business Development Virtualization, Diablo Technologies

More information

89 Fifth Avenue, 7th Floor. New York, NY 10003. www.theedison.com 212.367.7400. White Paper. HP 3PAR Adaptive Flash Cache: A Competitive Comparison

89 Fifth Avenue, 7th Floor. New York, NY 10003. www.theedison.com 212.367.7400. White Paper. HP 3PAR Adaptive Flash Cache: A Competitive Comparison 89 Fifth Avenue, 7th Floor New York, NY 10003 www.theedison.com 212.367.7400 White Paper HP 3PAR Adaptive Flash Cache: A Competitive Comparison Printed in the United States of America Copyright 2014 Edison

More information

Understanding endurance and performance characteristics of HP solid state drives

Understanding endurance and performance characteristics of HP solid state drives Understanding endurance and performance characteristics of HP solid state drives Technology brief Introduction... 2 SSD endurance... 2 An introduction to endurance... 2 NAND organization... 2 SLC versus

More information

Solid State Drive (SSD) FAQ

Solid State Drive (SSD) FAQ Solid State Drive (SSD) FAQ Santosh Kumar Rajesh Vijayaraghavan O c t o b e r 2 0 1 1 List of Questions Why SSD? Why Dell SSD? What are the types of SSDs? What are the best Use cases & applications for

More information

Lab Evaluation of NetApp Hybrid Array with Flash Pool Technology

Lab Evaluation of NetApp Hybrid Array with Flash Pool Technology Lab Evaluation of NetApp Hybrid Array with Flash Pool Technology Evaluation report prepared under contract with NetApp Introduction As flash storage options proliferate and become accepted in the enterprise,

More information

The Data Placement Challenge

The Data Placement Challenge The Data Placement Challenge Entire Dataset Applications Active Data Lowest $/IOP Highest throughput Lowest latency 10-20% Right Place Right Cost Right Time 100% 2 2 What s Driving the AST Discussion?

More information

File System Management

File System Management Lecture 7: Storage Management File System Management Contents Non volatile memory Tape, HDD, SSD Files & File System Interface Directories & their Organization File System Implementation Disk Space Allocation

More information

High-Performance SSD-Based RAID Storage. Madhukar Gunjan Chakhaiyar Product Test Architect

High-Performance SSD-Based RAID Storage. Madhukar Gunjan Chakhaiyar Product Test Architect High-Performance SSD-Based RAID Storage Madhukar Gunjan Chakhaiyar Product Test Architect 1 Agenda HDD based RAID Performance-HDD based RAID Storage Dynamics driving to SSD based RAID Storage Evolution

More information

Flash for Databases. September 22, 2015 Peter Zaitsev Percona

Flash for Databases. September 22, 2015 Peter Zaitsev Percona Flash for Databases September 22, 2015 Peter Zaitsev Percona In this Presentation Flash technology overview Review some of the available technology What does this mean for databases? Specific opportunities

More information

Implementation of Buffer Cache Simulator for Hybrid Main Memory and Flash Memory Storages

Implementation of Buffer Cache Simulator for Hybrid Main Memory and Flash Memory Storages Implementation of Buffer Cache Simulator for Hybrid Main Memory and Flash Memory Storages Soohyun Yang and Yeonseung Ryu Department of Computer Engineering, Myongji University Yongin, Gyeonggi-do, Korea

More information

Benefits of Solid-State Storage

Benefits of Solid-State Storage This Dell technical white paper describes the different types of solid-state storage and the benefits of each. Jeff Armstrong Gary Kotzur Rahul Deshmukh Contents Introduction... 3 PCIe-SSS... 3 Differences

More information

CAVE: Channel-Aware Buffer Management Scheme for Solid State Disk

CAVE: Channel-Aware Buffer Management Scheme for Solid State Disk CAVE: Channel-Aware Buffer Management Scheme for Solid State Disk Sung Kyu Park, Youngwoo Park, Gyudong Shim, and Kyu Ho Park Korea Advanced Institute of Science and Technology (KAIST) 305-701, Guseong-dong,

More information

Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software

Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software WHITEPAPER Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software SanDisk ZetaScale software unlocks the full benefits of flash for In-Memory Compute and NoSQL applications

More information

Using MLC NAND in Datacenters (a.k.a. Using Client SSD Technology in Datacenters) Tony Roug, Intel Principal Engineer

Using MLC NAND in Datacenters (a.k.a. Using Client SSD Technology in Datacenters) Tony Roug, Intel Principal Engineer Using MLC NAND in Datacenters (a.k.a. Using Client SSD Technology in Datacenters) Tony Roug, Intel Principal Engineer SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA.

More information

The Economics of Intelligent Hybrid Storage. An Enmotus White Paper Sep 2014

The Economics of Intelligent Hybrid Storage. An Enmotus White Paper Sep 2014 The Economics of Intelligent Hybrid Storage An Enmotus White Paper Sep 2014 SUMMARY Solid State Storage is no longer the storage of the future. It can be found in high- end data centers, in the servers

More information

NetApp FAS Hybrid Array Flash Efficiency. Silverton Consulting, Inc. StorInt Briefing

NetApp FAS Hybrid Array Flash Efficiency. Silverton Consulting, Inc. StorInt Briefing NetApp FAS Hybrid Array Flash Efficiency Silverton Consulting, Inc. StorInt Briefing PAGE 2 OF 7 Introduction Hybrid storage arrays (storage systems with both disk and flash capacity) have become commonplace

More information

Indexing on Solid State Drives based on Flash Memory

Indexing on Solid State Drives based on Flash Memory Indexing on Solid State Drives based on Flash Memory Florian Keusch MASTER S THESIS Systems Group Department of Computer Science ETH Zurich http://www.systems.ethz.ch/ September 2008 - March 2009 Supervised

More information

Disks and RAID. Profs. Bracy and Van Renesse. based on slides by Prof. Sirer

Disks and RAID. Profs. Bracy and Van Renesse. based on slides by Prof. Sirer Disks and RAID Profs. Bracy and Van Renesse based on slides by Prof. Sirer 50 Years Old! 13th September 1956 The IBM RAMAC 350 Stored less than 5 MByte Reading from a Disk Must specify: cylinder # (distance

More information

Automated Data-Aware Tiering

Automated Data-Aware Tiering Automated Data-Aware Tiering White Paper Drobo s revolutionary new breakthrough technology automates the provisioning, deployment, and performance acceleration for a fast tier of SSD storage in the Drobo

More information

Reduce Latency and Increase Application Performance Up to 44x with Adaptec maxcache 3.0 SSD Read and Write Caching Solutions

Reduce Latency and Increase Application Performance Up to 44x with Adaptec maxcache 3.0 SSD Read and Write Caching Solutions MAXCACHE 3. WHITEPAPER Reduce Latency and Increase Application Performance Up to 44x with Adaptec maxcache 3. SSD Read and Write Caching Solutions Executive Summary Today s data centers and cloud computing

More information

Managing Storage Space in a Flash and Disk Hybrid Storage System

Managing Storage Space in a Flash and Disk Hybrid Storage System Managing Storage Space in a Flash and Disk Hybrid Storage System Xiaojian Wu, and A. L. Narasimha Reddy Dept. of Electrical and Computer Engineering Texas A&M University IEEE International Symposium on

More information

Nasir Memon Polytechnic Institute of NYU

Nasir Memon Polytechnic Institute of NYU Nasir Memon Polytechnic Institute of NYU SSD Drive Technology Overview SSD Drive Components NAND FLASH Microcontroller SSD Drive Forensics Challenges Overview SSD s are fairly new to the market Whereas

More information

Physical Data Organization

Physical Data Organization Physical Data Organization Database design using logical model of the database - appropriate level for users to focus on - user independence from implementation details Performance - other major factor

More information

SOLID STATE DRIVES AND PARALLEL STORAGE

SOLID STATE DRIVES AND PARALLEL STORAGE SOLID STATE DRIVES AND PARALLEL STORAGE White paper JANUARY 2013 1.888.PANASAS www.panasas.com Overview Solid State Drives (SSDs) have been touted for some time as a disruptive technology in the storage

More information

Technologies Supporting Evolution of SSDs

Technologies Supporting Evolution of SSDs Technologies Supporting Evolution of SSDs By TSUCHIYA Kenji Notebook PCs equipped with solid-state drives (SSDs), featuring shock and vibration durability due to the lack of moving parts, appeared on the

More information

WHITE PAPER. Drobo TM Hybrid Storage TM

WHITE PAPER. Drobo TM Hybrid Storage TM WHITE PAPER Drobo TM Hybrid Storage TM Table of Contents Introduction...3 What is Hybrid Storage?...4 SSDs Enable Hybrid Storage...4 One Pool, Multiple Tiers...5 Fully Automated Tiering...5 Tiering Without

More information

Accelerating Server Storage Performance on Lenovo ThinkServer

Accelerating Server Storage Performance on Lenovo ThinkServer Accelerating Server Storage Performance on Lenovo ThinkServer Lenovo Enterprise Product Group April 214 Copyright Lenovo 214 LENOVO PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER

More information

Flash In The Enterprise

Flash In The Enterprise Flash In The Enterprise Technology and Market Overview Chris M Evans, Langton Blue Ltd Architecting IT January 2014 Doc ID: AI1401-01S Table of Contents The Need for Flash Storage... 3 IOPS Density...

More information

SH-Sim: A Flexible Simulation Platform for Hybrid Storage Systems

SH-Sim: A Flexible Simulation Platform for Hybrid Storage Systems , pp.61-70 http://dx.doi.org/10.14257/ijgdc.2014.7.3.07 SH-Sim: A Flexible Simulation Platform for Hybrid Storage Systems Puyuan Yang 1, Peiquan Jin 1,2 and Lihua Yue 1,2 1 School of Computer Science and

More information

Data Distribution Algorithms for Reliable. Reliable Parallel Storage on Flash Memories

Data Distribution Algorithms for Reliable. Reliable Parallel Storage on Flash Memories Data Distribution Algorithms for Reliable Parallel Storage on Flash Memories Zuse Institute Berlin November 2008, MEMICS Workshop Motivation Nonvolatile storage Flash memory - Invented by Dr. Fujio Masuoka

More information

The Shortcut Guide to Balancing Storage Costs and Performance with Hybrid Storage

The Shortcut Guide to Balancing Storage Costs and Performance with Hybrid Storage The Shortcut Guide to Balancing Storage Costs and Performance with Hybrid Storage sponsored by Dan Sullivan Chapter 1: Advantages of Hybrid Storage... 1 Overview of Flash Deployment in Hybrid Storage Systems...

More information

SSD Performance Tips: Avoid The Write Cliff

SSD Performance Tips: Avoid The Write Cliff ebook 100% KBs/sec 12% GBs Written SSD Performance Tips: Avoid The Write Cliff An Inexpensive and Highly Effective Method to Keep SSD Performance at 100% Through Content Locality Caching Share this ebook

More information

Offline Deduplication for Solid State Disk Using a Lightweight Hash Algorithm

Offline Deduplication for Solid State Disk Using a Lightweight Hash Algorithm JOURNAL OF SEMICONDUCTOR TECHNOLOGY AND SCIENCE, VOL.15, NO.5, OCTOBER, 2015 ISSN(Print) 1598-1657 http://dx.doi.org/10.5573/jsts.2015.15.5.539 ISSN(Online) 2233-4866 Offline Deduplication for Solid State

More information

21 st Century Storage What s New and What s Changing

21 st Century Storage What s New and What s Changing 21 st Century Storage What s New and What s Changing Randy Kerns Senior Strategist Evaluator Group Overview New technologies in storage - Continued evolution - Each has great economic value - Differing

More information

Flash Memory Technology in Enterprise Storage

Flash Memory Technology in Enterprise Storage NETAPP WHITE PAPER Flash Memory Technology in Enterprise Storage Flexible Choices to Optimize Performance Mark Woods and Amit Shah, NetApp November 2008 WP-7061-1008 EXECUTIVE SUMMARY Solid state drives

More information

The Pitfalls of Deploying Solid-State Drive RAIDs

The Pitfalls of Deploying Solid-State Drive RAIDs The Pitfalls of Deploying Solid-State Drive RAIDs Nikolaus Jeremic 1, Gero Mühl 1, Anselm Busse 2 and Jan Richling 2 Architecture of Application Systems Group 1 Faculty of Computer Science and Electrical

More information

Choosing the Right NAND Flash Memory Technology

Choosing the Right NAND Flash Memory Technology Choosing the Right NAND Flash Memory Technology A Basic Introduction to NAND Flash Offerings Dean Klein Vice President of System Memory Development Micron Technology, Inc. Executive Summary A 75% increase

More information

FLASH GAINS GROUND AS ENTERPRISE STORAGE OPTION

FLASH GAINS GROUND AS ENTERPRISE STORAGE OPTION FLASH GAINS GROUND AS ENTERPRISE STORAGE OPTION With new management functions placing it closer to parity with hard drives, as well as new economies, flash is gaining traction as a standard media for mainstream

More information

Impact of Stripe Unit Size on Performance and Endurance of SSD-Based RAID Arrays

Impact of Stripe Unit Size on Performance and Endurance of SSD-Based RAID Arrays 1 Impact of Stripe Unit Size on Performance and Endurance of SSD-Based RAID Arrays Farzaneh Rajaei Salmasi Hossein Asadi Majid GhasemiGol rajaei@ce.sharif.edu asadi@sharif.edu ghasemigol@ce.sharif.edu

More information

A Data De-duplication Access Framework for Solid State Drives

A Data De-duplication Access Framework for Solid State Drives JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 28, 941-954 (2012) A Data De-duplication Access Framework for Solid State Drives Department of Electronic Engineering National Taiwan University of Science

More information

An Exploration of Hybrid Hard Disk Designs Using an Extensible Simulator

An Exploration of Hybrid Hard Disk Designs Using an Extensible Simulator An Exploration of Hybrid Hard Disk Designs Using an Extensible Simulator Pavan Konanki Thesis submitted to the Faculty of the Virginia Polytechnic Institute and State University in partial fulfillment

More information

Oracle Aware Flash: Maximizing Performance and Availability for your Database

Oracle Aware Flash: Maximizing Performance and Availability for your Database Oracle Aware Flash: Maximizing Performance and Availability for your Database Gurmeet Goindi Principal Product Manager Oracle Kirby McCord Database Architect US Cellular Kodi Umamageswaran Vice President,

More information

HHB+tree Index for Functional Enhancement of NAND Flash Memory-Based Database

HHB+tree Index for Functional Enhancement of NAND Flash Memory-Based Database , pp. 289-294 http://dx.doi.org/10.14257/ijseia.2015.9.9.25 HHB+tree Index for Functional Enhancement of NAND Flash Memory-Based Database Huijeong Ju 1 and Sungje Cho 2 1,2 Department of Education Dongbang

More information

Maximizing the Benefits from Flash In Your SAN Storage Systems

Maximizing the Benefits from Flash In Your SAN Storage Systems Maximizing the Benefits from Flash In Your SAN Storage Systems Dot Hill Storage Hybrid Arrays integrate flash and HDDs in Optimal Configurations Maximizing the Benefits of Flash WP 1 INTRODUCTION EXECUTIVE

More information

Solid State Technology What s New?

Solid State Technology What s New? Solid State Technology What s New? Dennis Martin, President, Demartek www.storagedecisions.com Agenda: Solid State Technology What s New? Demartek About Us Solid-state storage overview Types of NAND flash

More information

Calsoft Webinar - Debunking QA myths for Flash- Based Arrays

Calsoft Webinar - Debunking QA myths for Flash- Based Arrays Most Trusted Names in Data Centre Products Rely on Calsoft! September 2015 Calsoft Webinar - Debunking QA myths for Flash- Based Arrays Agenda Introduction to Types of Flash-Based Arrays Challenges in

More information

Comparison of NAND Flash Technologies Used in Solid- State Storage

Comparison of NAND Flash Technologies Used in Solid- State Storage An explanation and comparison of SLC and MLC NAND technologies August 2010 Comparison of NAND Flash Technologies Used in Solid- State Storage By Shaluka Perera IBM Systems and Technology Group Bill Bornstein

More information

Getting the Most Out of Flash Storage

Getting the Most Out of Flash Storage Business in a Flash. Getting the Most Out of Flash Storage Introduction, Usability, Optimization May 2015 David Lin Solutions Architect dlin@vmem.com 1 The copyright for images, icons, and logos used belong

More information

Using Synology SSD Technology to Enhance System Performance Synology Inc.

Using Synology SSD Technology to Enhance System Performance Synology Inc. Using Synology SSD Technology to Enhance System Performance Synology Inc. Synology_SSD_Cache_WP_ 20140512 Table of Contents Chapter 1: Enterprise Challenges and SSD Cache as Solution Enterprise Challenges...

More information

COS 318: Operating Systems. Storage Devices. Kai Li Computer Science Department Princeton University. (http://www.cs.princeton.edu/courses/cos318/)

COS 318: Operating Systems. Storage Devices. Kai Li Computer Science Department Princeton University. (http://www.cs.princeton.edu/courses/cos318/) COS 318: Operating Systems Storage Devices Kai Li Computer Science Department Princeton University (http://www.cs.princeton.edu/courses/cos318/) Today s Topics Magnetic disks Magnetic disk performance

More information

FAWN - a Fast Array of Wimpy Nodes

FAWN - a Fast Array of Wimpy Nodes University of Warsaw January 12, 2011 Outline Introduction 1 Introduction 2 3 4 5 Key issues Introduction Growing CPU vs. I/O gap Contemporary systems must serve millions of users Electricity consumed

More information

A Balanced Approach to Optimizing Storage Performance in the Data Center

A Balanced Approach to Optimizing Storage Performance in the Data Center A Balanced Approach to Optimizing Storage Performance in the Data How a mix of HDDs and SSDs can maximize the performance of your workloads October 19, 2015 Mark T. Chapman Lenovo.com/systems Executive

More information

OPTIMIZING VIDEO STORAGE AT THE EDGE OF THE NETWORK

OPTIMIZING VIDEO STORAGE AT THE EDGE OF THE NETWORK White Paper OPTIMIZING VIDEO STORAGE AT THE EDGE OF THE NETWORK Leveraging Intelligent Content Distribution Software, Off-the-Shelf Hardware and MLC Flash to Deploy Scalable and Economical Pay-As-You-Grow

More information

p-oftl: An Object-based Semantic-aware Parallel Flash Translation Layer

p-oftl: An Object-based Semantic-aware Parallel Flash Translation Layer p-oftl: An Object-based Semantic-aware Parallel Flash Translation Layer Wei Wang, Youyou Lu, and Jiwu Shu Department of Computer Science and Technology, Tsinghua University, Beijing, China Tsinghua National

More information

Flash Memory Arrays Enabling the Virtualized Data Center. July 2010

Flash Memory Arrays Enabling the Virtualized Data Center. July 2010 Flash Memory Arrays Enabling the Virtualized Data Center July 2010 2 Flash Memory Arrays Enabling the Virtualized Data Center This White Paper describes a new product category, the flash Memory Array,

More information

Efficient Updates for OLAP Range Queries on Flash Memory

Efficient Updates for OLAP Range Queries on Flash Memory Efficient Updates for OLAP Range Queries on Flash Memory Mitzi McCarthy and Zhen He Department of Computer Science and Computer Engineering, La Trobe University, VIC 3086, Australia Email: m.mccarthy@latrobe.edu.au;

More information

The Classical Architecture. Storage 1 / 36

The Classical Architecture. Storage 1 / 36 1 / 36 The Problem Application Data? Filesystem Logical Drive Physical Drive 2 / 36 Requirements There are different classes of requirements: Data Independence application is shielded from physical storage

More information

The search engine you can see. Connects people to information and services

The search engine you can see. Connects people to information and services The search engine you can see Connects people to information and services The search engine you cannot see Total data: ~1EB Processing data : ~100PB/day Total web pages: ~1000 Billion Web pages updated:

More information

Speeding Up Cloud/Server Applications Using Flash Memory

Speeding Up Cloud/Server Applications Using Flash Memory Speeding Up Cloud/Server Applications Using Flash Memory Sudipta Sengupta Microsoft Research, Redmond, WA, USA Contains work that is joint with B. Debnath (Univ. of Minnesota) and J. Li (Microsoft Research,

More information

Using Synology SSD Technology to Enhance System Performance. Based on DSM 5.2

Using Synology SSD Technology to Enhance System Performance. Based on DSM 5.2 Using Synology SSD Technology to Enhance System Performance Based on DSM 5.2 Table of Contents Chapter 1: Enterprise Challenges and SSD Cache as Solution Enterprise Challenges... 3 SSD Cache as Solution...

More information

File System & Device Drive. Overview of Mass Storage Structure. Moving head Disk Mechanism. HDD Pictures 11/13/2014. CS341: Operating System

File System & Device Drive. Overview of Mass Storage Structure. Moving head Disk Mechanism. HDD Pictures 11/13/2014. CS341: Operating System CS341: Operating System Lect 36: 1 st Nov 2014 Dr. A. Sahu Dept of Comp. Sc. & Engg. Indian Institute of Technology Guwahati File System & Device Drive Mass Storage Disk Structure Disk Arm Scheduling RAID

More information

SALSA Flash-Optimized Software-Defined Storage

SALSA Flash-Optimized Software-Defined Storage Flash-Optimized Software-Defined Storage Nikolas Ioannou, Ioannis Koltsidas, Roman Pletka, Sasa Tomic,Thomas Weigold IBM Research Zurich 1 New Market Category of Big Data Flash Multiple workloads don t

More information

Fine-grained Updates in Database Management Systems for Flash Memory

Fine-grained Updates in Database Management Systems for Flash Memory Fine-grained Updates in Database Management Systems for Flash Memory Zhen He and Prakash Veeraraghavan Department of Computer Science and Computer Engineering La Trobe University VIC 3086 Australia {z.he,

More information

Operating Systems CSE 410, Spring 2004. File Management. Stephen Wagner Michigan State University

Operating Systems CSE 410, Spring 2004. File Management. Stephen Wagner Michigan State University Operating Systems CSE 410, Spring 2004 File Management Stephen Wagner Michigan State University File Management File management system has traditionally been considered part of the operating system. Applications

More information

All-Flash Storage Solution for SAP HANA:

All-Flash Storage Solution for SAP HANA: All-Flash Storage Solution for SAP HANA: Storage Considerations using SanDisk Solid State Devices WHITE PAPER 951 SanDisk Drive, Milpitas, CA 95035 www.sandisk.com Table of Contents Preface 3 Why SanDisk?

More information

Chapter 13. Disk Storage, Basic File Structures, and Hashing

Chapter 13. Disk Storage, Basic File Structures, and Hashing Chapter 13 Disk Storage, Basic File Structures, and Hashing Chapter Outline Disk Storage Devices Files of Records Operations on Files Unordered Files Ordered Files Hashed Files Dynamic and Extendible Hashing

More information

COS 318: Operating Systems. Storage Devices. Kai Li and Andy Bavier Computer Science Department Princeton University

COS 318: Operating Systems. Storage Devices. Kai Li and Andy Bavier Computer Science Department Princeton University COS 318: Operating Systems Storage Devices Kai Li and Andy Bavier Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall13/cos318/ Today s Topics! Magnetic disks!

More information

SLIDE 1 www.bitmicro.com. Previous Next Exit

SLIDE 1 www.bitmicro.com. Previous Next Exit SLIDE 1 MAXio All Flash Storage Array Popular Applications MAXio N1A6 SLIDE 2 MAXio All Flash Storage Array Use Cases High speed centralized storage for IO intensive applications email, OLTP, databases

More information

WHITE PAPER FUJITSU PRIMERGY SERVER BASICS OF DISK I/O PERFORMANCE

WHITE PAPER FUJITSU PRIMERGY SERVER BASICS OF DISK I/O PERFORMANCE WHITE PAPER BASICS OF DISK I/O PERFORMANCE WHITE PAPER FUJITSU PRIMERGY SERVER BASICS OF DISK I/O PERFORMANCE This technical documentation is aimed at the persons responsible for the disk I/O performance

More information

In-Block Level Redundancy Management for Flash Storage System

In-Block Level Redundancy Management for Flash Storage System , pp.309-318 http://dx.doi.org/10.14257/ijmue.2015.10.9.32 In-Block Level Redundancy Management for Flash Storage System Seung-Ho Lim Division of Computer and Electronic Systems Engineering Hankuk University

More information

Solid State Drive Architecture

Solid State Drive Architecture Solid State Drive Architecture A comparison and evaluation of data storage mediums Tyler Thierolf Justin Uriarte Outline Introduction Storage Device as Limiting Factor Terminology Internals Interface Architecture

More information

Ensuring Robust Data Integrity

Ensuring Robust Data Integrity TECHNICAL WHITE PAPER Ensuring Robust Data Integrity www.tintri.com Executive Summary Purpose-built storage appliances serving enterprise workloads typically use commodity hardware with value-added OS

More information

Flash Memory. Jian-Jia Chen (Slides are based on Yuan-Hao Chang) TU Dortmund Informatik 12 Germany 2015 年 01 月 27 日. technische universität dortmund

Flash Memory. Jian-Jia Chen (Slides are based on Yuan-Hao Chang) TU Dortmund Informatik 12 Germany 2015 年 01 月 27 日. technische universität dortmund 12 Flash Memory Jian-Jia Chen (Slides are based on Yuan-Hao Chang) TU Dortmund Informatik 12 Germany 2015 年 01 月 27 日 These slides use Microsoft clip arts Microsoft copyright restrictions apply Springer,

More information

Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 13-1

Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 13-1 Slide 13-1 Chapter 13 Disk Storage, Basic File Structures, and Hashing Chapter Outline Disk Storage Devices Files of Records Operations on Files Unordered Files Ordered Files Hashed Files Dynamic and Extendible

More information

Advantages of Intel SSDs for Data Centres

Advantages of Intel SSDs for Data Centres Advantages of Intel SSDs for Data Centres Executive Summary Most businesses depend on at least one data centre infrastructure to be successful. A highly functioning data centre has several requirements,

More information

Sistemas Operativos: Input/Output Disks

Sistemas Operativos: Input/Output Disks Sistemas Operativos: Input/Output Disks Pedro F. Souto (pfs@fe.up.pt) April 28, 2012 Topics Magnetic Disks RAID Solid State Disks Topics Magnetic Disks RAID Solid State Disks Magnetic Disk Construction

More information

An Overview of Flash Storage for Databases

An Overview of Flash Storage for Databases An Overview of Flash Storage for Databases Vadim Tkachenko Morgan Tocker http://percona.com MySQL CE Apr 2010 -2- Introduction Vadim Tkachenko Percona Inc, CTO and Lead of Development Morgan Tocker Percona

More information

Bigdata High Availability (HA) Architecture

Bigdata High Availability (HA) Architecture Bigdata High Availability (HA) Architecture Introduction This whitepaper describes an HA architecture based on a shared nothing design. Each node uses commodity hardware and has its own local resources

More information

New economies of storage with the Compellent Flash-optimized solutions. Flash at the price of disk for I/O-intensive applications

New economies of storage with the Compellent Flash-optimized solutions. Flash at the price of disk for I/O-intensive applications New economies of storage with the Compellent Flash-optimized solutions Flash at the price of disk for I/O-intensive applications A Dell Business White Paper September 2013 THIS WHITE PAPER IS FOR INFORMATIONAL

More information

Seeking Fast, Durable Data Management: A Database System and Persistent Storage Benchmark

Seeking Fast, Durable Data Management: A Database System and Persistent Storage Benchmark Seeking Fast, Durable Data Management: A Database System and Persistent Storage Benchmark In-memory database systems (IMDSs) eliminate much of the performance latency associated with traditional on-disk

More information

Benchmarking Cassandra on Violin

Benchmarking Cassandra on Violin Technical White Paper Report Technical Report Benchmarking Cassandra on Violin Accelerating Cassandra Performance and Reducing Read Latency With Violin Memory Flash-based Storage Arrays Version 1.0 Abstract

More information

Solid State Storage in Massive Data Environments Erik Eyberg

Solid State Storage in Massive Data Environments Erik Eyberg Solid State Storage in Massive Data Environments Erik Eyberg Senior Analyst Texas Memory Systems, Inc. Agenda Taxonomy Performance Considerations Reliability Considerations Q&A Solid State Storage Taxonomy

More information

Differential RAID: Rethinking RAID for SSD Reliability

Differential RAID: Rethinking RAID for SSD Reliability Differential RAID: Rethinking RAID for SSD Reliability ABSTRACT Asim Kadav University of Wisconsin Madison, WI kadav@cs.wisc.edu Vijayan Prabhakaran vijayanp@microsoft.com Deployment of SSDs in enterprise

More information

Lecture 1: Data Storage & Index

Lecture 1: Data Storage & Index Lecture 1: Data Storage & Index R&G Chapter 8-11 Concurrency control Query Execution and Optimization Relational Operators File & Access Methods Buffer Management Disk Space Management Recovery Manager

More information

FAST 11. Yongseok Oh <ysoh@uos.ac.kr> University of Seoul. Mobile Embedded System Laboratory

FAST 11. Yongseok Oh <ysoh@uos.ac.kr> University of Seoul. Mobile Embedded System Laboratory CAFTL: A Content-Aware Flash Translation Layer Enhancing the Lifespan of flash Memory based Solid State Drives FAST 11 Yongseok Oh University of Seoul Mobile Embedded System Laboratory

More information

Chapter 13 Disk Storage, Basic File Structures, and Hashing.

Chapter 13 Disk Storage, Basic File Structures, and Hashing. Chapter 13 Disk Storage, Basic File Structures, and Hashing. Copyright 2004 Pearson Education, Inc. Chapter Outline Disk Storage Devices Files of Records Operations on Files Unordered Files Ordered Files

More information

The IntelliMagic White Paper: Storage Performance Analysis for an IBM Storwize V7000

The IntelliMagic White Paper: Storage Performance Analysis for an IBM Storwize V7000 The IntelliMagic White Paper: Storage Performance Analysis for an IBM Storwize V7000 Summary: This document describes how to analyze performance on an IBM Storwize V7000. IntelliMagic 2012 Page 1 This

More information

White Paper. Apacer and Diskeeper Corporation. The Best Gets Better

White Paper. Apacer and Diskeeper Corporation. The Best Gets Better White Paper Apacer and Diskeeper Corporation The Best Gets Better Apacer and Diskeeper Corporation The Best Gets Better Table of Contents Overview 1 The Problem 1 Problem Analysis 1 The Software Solution:

More information

Flash Controller Architecture for All Flash Arrays

Flash Controller Architecture for All Flash Arrays Flash Controller Architecture for All Flash Arrays Andy Walls Distinguished Engineer, IBM Systems and Technology Group CTO, Flash Systems and Technology Santa Clara, CA August 2013 1 Once upon a Time,.....

More information

The Technologies & Architectures. President, Demartek

The Technologies & Architectures. President, Demartek Deep Dive on Solid State t Storage The Technologies & Architectures Dennis Martin Dennis Martin President, Demartek Demartek Company Overview Industry analysis with on-site test lab Lab includes servers,

More information

Architecture Enterprise Storage Performance: It s All About The Interface.

Architecture Enterprise Storage Performance: It s All About The Interface. Architecture Enterprise Storage Performance: It s All About The Interface. A DIABLO WHITE PAPER APRIL 214 diablo-technologies.com Diablo_Tech Enterprise Storage Performance: It s All About The Architecture.

More information

Flash at the price of disk Redefining the Economics of Storage. Kris Van Haverbeke Enterprise Marketing Manager Dell Belux

Flash at the price of disk Redefining the Economics of Storage. Kris Van Haverbeke Enterprise Marketing Manager Dell Belux Flash at the price of disk Redefining the Economics of Storage Kris Van Haverbeke Enterprise Marketing Manager Dell Belux We believe that Redefining the Economics of Storage helps businesses achieve their

More information

SSD Write Performance IOPS Confusion Due to Poor Benchmarking Techniques

SSD Write Performance IOPS Confusion Due to Poor Benchmarking Techniques SSD Write Performance IOPS Confusion Due to Poor Benchmarking Techniques Dominique A. Heger dheger@dhtusa.com www.dhtusa.com Introduction Over the last few years, the proliferation of solid state disk

More information

Database!Fatal!Flash!Flaws!No!One! Talks!About!!!

Database!Fatal!Flash!Flaws!No!One! Talks!About!!! MarcStaimer,President&CDSDragonSlayerConsulting W h i t e P A P E R DatabaseFatalFlashFlawsNoOne TalksAbout AndHowtoAvoidThem WHITEPAPER DatabaseFatalFlashFlawsNoOneTalksAbout AndHowtoAvoidThem DatabaseFatalFlashFlawsNoOneTalksAbout

More information

ioscale: The Holy Grail for Hyperscale

ioscale: The Holy Grail for Hyperscale ioscale: The Holy Grail for Hyperscale The New World of Hyperscale Hyperscale describes new cloud computing deployments where hundreds or thousands of distributed servers support millions of remote, often

More information

SanDisk SSD Boot Storm Testing for Virtual Desktop Infrastructure (VDI)

SanDisk SSD Boot Storm Testing for Virtual Desktop Infrastructure (VDI) WHITE PAPER SanDisk SSD Boot Storm Testing for Virtual Desktop Infrastructure (VDI) August 214 951 SanDisk Drive, Milpitas, CA 9535 214 SanDIsk Corporation. All rights reserved www.sandisk.com 2 Table

More information

Using Synology SSD Technology to Enhance System Performance Synology Inc.

Using Synology SSD Technology to Enhance System Performance Synology Inc. Using Synology SSD Technology to Enhance System Performance Synology Inc. Synology_WP_ 20121112 Table of Contents Chapter 1: Enterprise Challenges and SSD Cache as Solution Enterprise Challenges... 3 SSD

More information

ProTrack: A Simple Provenance-tracking Filesystem

ProTrack: A Simple Provenance-tracking Filesystem ProTrack: A Simple Provenance-tracking Filesystem Somak Das Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology das@mit.edu Abstract Provenance describes a file

More information

Flash Performance in Storage Systems. Bill Moore Chief Engineer, Storage Systems Sun Microsystems

Flash Performance in Storage Systems. Bill Moore Chief Engineer, Storage Systems Sun Microsystems Flash Performance in Storage Systems Bill Moore Chief Engineer, Storage Systems Sun Microsystems 1 Disk to CPU Discontinuity Moore s Law is out-stripping disk drive performance (rotational speed) As a

More information