SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery
|
|
- Jessie Small
- 8 years ago
- Views:
Transcription
1 SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery Ismail Oukid*, Daniel Booss, Wolfgang Lehner*, Peter Bumbulis, and Thomas Willhalm + *Dresden University of Technology SAP AG + Intel GmbH DaMoN 2014, Snowbird, Utah, USA, June 23, 2014 Prof. Dr.-Ing. Wolfgang Lehner Diese Zeile ersetzt man über: Einfügen > Kopf- und
2 What is Storage Class Memory? Non-Volatile Memory Byteaddressable (load/store semantics) Latency close to DRAM s, with writes slower than reads More energy efficient than DRAM Storage Class Memory Denser than DRAM (i.e. better scaling) e.g. PCM, STTRAM, MRAM and Memristors Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 2
3 SCM Compared to Other Technologies Latency Bandwidth Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 3
4 SCM Compared to Other Technologies Latency HDD Bandwidth Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 4
5 SCM Compared to Other Technologies Latency HDD DRAM Bandwidth Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 5
6 SCM Compared to Other Technologies Latency HDD Bridging the gap between memory and storage DRAM Bandwidth Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 6
7 SCM Compared to Other Technologies Latency HDD SSD Bridging the gap between memory and storage DRAM Bandwidth Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 7
8 SCM Compared to Other Technologies Latency HDD SSD Bridging the gap between memory and storage SCM DRAM Bandwidth Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 8
9 SCM Compared to Other Technologies Latency HDD SSD Bridging the gap between memory and storage SCM DRAM Bandwidth Storage Class Memory is a merging point between memory and storage Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 9
10 Main Memory DB Restart Time Availability guarantees are an important part of many SLAs Availability of 99% downtime of 3.65 days/year! Many database crash conditions are transient A reasonable approach to recovery: restarting the database Database restart time impacts directly database availability Long restart times: a shortcoming of main memory databases Improve database restart time using SCM Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 10
11 Overview of SOFORT (1) SOFORT: A hybrid SCM-DRAM storage engine for fast data recovery Requirements: Fast data recovery Mixed OLTP and OLAP A hybrid SCM-DRAM memory environment Architecture: Single-level column-store Dictionary encoded Multi version concurrency control (MVCC) No transactional log Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 11
12 Overview of SOFORT (2) SOFORT Table MVCC Entry CTS DTS Appendonly TX Array Tables MVCC Array Columns Column On DRAM to enable better performance Dictionary Encoded Persisted on SCM Volatile in DRAM Dict. Index Dict. Index (optional) Value IDs Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 12
13 Persisting Data on the Fly SCM accessed via the CPU cache: Writes not guaranteed to have reached SCM Need to flush data from cache Code reordered at compilation time & Complex CPU out-of-order execution Need to order memory operations Persistence primitives: flushing instructions (CLFLUSH), memory barriers (MFENCE, SFENCE, LFENCE) and non-temporal stores () Data may still be held in memory controller buffers Need to flush memory controller buffers on power failures Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 13
14 Persisting Data on the Fly: Example Int var = 0; Bool persisted = false; var = 1; Flush var; persisted = true; Flush persisted; Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 14
15 Persisting Data on the Fly: Example Int var = 0; Bool persisted = false; var = 1; Flush var; persisted = true; Cache SCM var 0 0 persisted False False Flush persisted; Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 15
16 Persisting Data on the Fly: Example Int var = 0; Bool persisted = false; var = 1; Flush var; persisted = true; Cache SCM var 0 0 persisted True True Flush persisted; Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 16
17 Persisting Data on the Fly: Example Int var = 0; Bool persisted = false; var = 1; Flush var; persisted = true; Cache SCM var 1 0 persisted True True Flush persisted; Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 17
18 Persisting Data on the Fly: Example Int var = 0; Bool persisted = false; var = 1; Flush var; MFENCE; persisted = true; Cache SCM var 0 0 persisted False False Flush persisted; Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 18
19 Persisting Data on the Fly: Example Int var = 0; Bool persisted = false; var = 1; Flush var; MFENCE; persisted = true; Cache SCM var 0 0 persisted False False Flush persisted; Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 19
20 Persisting Data on the Fly: Example Int var = 0; Bool persisted = false; var = 1; MFENCE; Flush var; MFENCE; persisted = true; Cache SCM var 0 0 persisted False False Flush persisted; Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 20
21 Persisting Data on the Fly: Example Int var = 0; Bool persisted = false; var = 1; MFENCE; Flush var; MFENCE; persisted = true; MFENCE; Flush persisted; MFENCE; Cache SCM var 0 0 persisted False False Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 21
22 Persistent Memory Allocator Persistent Memory File System (PMFS): SCM-aware file system No buffering in DRAM on mmap direct access to SCM PMAllocator: Huge PMFS files as memory pages Pages cut into segments for allocation Persistent counter of memory pages Mapping from persistent memory to VM PMFS file = Memory page Segment File name = Unique page ID Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 22
23 Persistent Memory Pointers Regular pointers are bound to the program s address space Cannot be used for recovery We propose Persistent Memory Pointers (PMPtrs): Struct PMPtr { int64_t m_baseid; ptrdiff_t m_offset; } Persistent memory page ID Offset indicating the start of the allocated block Can be converted (swizzled) to regular pointers PMPtrs stay valid across failures Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 23
24 Recovery Mechanism PM Entry Point Reload PMAllocator Reload SOFORT A PMPtr to the PMAllocator A PMPtr to the storage engine Reload memory pages Reconstruct mapping to VM Rollback unfinished transactions Allocate ressources for recovery Reload Table M Reload Table 1 Data structures sanity check Initiate columns recovery Reload column N Reload column 1 Data structures sanity check Reconstruct dictionary indexes Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 24
25 Evaluation: Setup Hardware-based SCM simulation: Intel Xeon 20MB L3 cache, 8 physical cores/socket Avoiding NUMA effects: benchmark run on a single socket Limitation: symmetric instead of asymmetric read/write latency Socket 0 Socket 1 QPI Link CPU (8 cores) CPU (8 cores) Memory Bus Memory Bus DRAM SCM (PMFS mount) DRAM Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 25
26 Evaluation: Setup TATP benchmark: Simple but realistic OLTP benchmark (20% write, 80% reads) Simulates a telecommunication application Initial population of 1 million subscribers (~2GB data size) Shore-MT: A storage manager designed for scalability in multicores Run on Ramdisk (tempfs) Restart time: Single, 4 column table SCM latency set to 200ns Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 26
27 Mio. Tx/s Evaluation: Throughput 2 1,8 1,6 GetSubData Throughput (Read Only) Realistic case Upper bound 1,4 1,2 shore-read-shm 1 0,8 0,6 0,4 0,2 User contention. Solved with PLP. Worst case sofort-read-shm sofort-read-200ns sofort-read-300ns sofort-read-700ns # users Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 27
28 Mio. Tx/s Evaluation: Throughput 1,2 1 0,8 0,6 0,4 0,2 TATP Throughput (80% Read, 20% Write) ~900K Tx/s ~550K Tx/s ~250K Tx/s Upper bound Worst case Realistic case shore-shm sofort-shm sofort-200ns sofort-300ns sofort-700ns # users SOFORT outperforms Shore-MT even in a high latency SCM environment Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 28
29 Mio Tx/s Mio Tx/s Restart Time (s) Restart Time (s) Evaluation: Restart Time Dict. Size Fixed Mio. Rows/Table SOFORT on SCM ~2sec Time (s) Restart time is independent of data size time-dbfixedhashmap time-dictfixedskiplist time-dictfixedhashmap SOFORT recovers nearly instantly Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June Data Size Fixed Mio. Distinct Values Restart time is independent of instance size and depends only on dictionaries size SOFORT on DISK ~18sec Time (s) Volatile structures drive restart time time-dbfixedskiplist No persistence guarantees
30 Conclusion and Future Work We showed that SCM can help getting rid of the transactional log and significantly improving database restart times Latency study based on hardware simulation showed that SOFORT exhibits competitive performance even in a high latency SCM environment Future Work: Extend SOFORT to support long transactions Explore new recovery schemes Design persistent index structures Experiment with write-through caching policy Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery DaMoN 14, June 23 30
31 Takeaway: SCM is a game changer in the database industry Make the change happen! Thank You! Questions? Comments? Ismail Oukid SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery Ismail Oukid, Daniel Booss, Wolfgang Lehner, Peter Bumbulis, and Thomas Willhalm + DaMoN 2014, Snowbird, Utah, USA, June 23, 2014 Prof. Dr.-Ing. Wolfgang Lehner Diese Zeile ersetzt man über: Einfügen > Kopf- und
Instant Recovery for Main-Memory Databases
Persistent Storage Non-Volatile Main Memory Transient Main Memory Transient Main Memory runtime data moving the persistency bar Instant Recovery for Main-Memory Databases Ismail Oukid Technische Universität
More informationA Study of Application Performance with Non-Volatile Main Memory
A Study of Application Performance with Non-Volatile Main Memory Yiying Zhang, Steven Swanson 2 Memory Storage Fast Slow Volatile In bytes Persistent In blocks Next-Generation Non-Volatile Memory (NVM)
More informationNVRAM-aware Logging in Transaction Systems. Jian Huang
NVRAM-aware Logging in Transaction Systems Jian Huang Karsten Schwan Moinuddin K. Qureshi Logging Support for Transactions 2 Logging Support for Transactions ARIES: Disk-Based Approach (TODS 92) Write-ahead
More informationComputer Engineering and Systems Group Electrical and Computer Engineering SCMFS: A File System for Storage Class Memory
SCMFS: A File System for Storage Class Memory Xiaojian Wu, Narasimha Reddy Texas A&M University What is SCM? Storage Class Memory Byte-addressable, like DRAM Non-volatile, persistent storage Example: Phase
More informationNVRAM-aware Logging in Transaction Systems
NVRAM-aware Logging in Transaction Systems Jian Huang jhuang95@cc.gatech.edu Karsten Schwan schwan@cc.gatech.edu Moinuddin K. Qureshi moin@ece.gatech.edu Georgia Institute of Technology ABSTRACT Emerging
More informationDIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION
DIABLO TECHNOLOGIES MEMORY CHANNEL STORAGE AND VMWARE VIRTUAL SAN : VDI ACCELERATION A DIABLO WHITE PAPER AUGUST 2014 Ricky Trigalo Director of Business Development Virtualization, Diablo Technologies
More informationNV-DIMM: Fastest Tier in Your Storage Strategy
NV-DIMM: Fastest Tier in Your Storage Strategy Introducing ArxCis-NV, a Non-Volatile DIMM Author: Adrian Proctor, Viking Technology [email: adrian.proctor@vikingtechnology.com] This paper reviews how Non-Volatile
More informationTips and Tricks for Using Oracle TimesTen In-Memory Database in the Application Tier
Tips and Tricks for Using Oracle TimesTen In-Memory Database in the Application Tier Simon Law TimesTen Product Manager, Oracle Meet The Experts: Andy Yao TimesTen Product Manager, Oracle Gagan Singh Senior
More informationSAP HANA - Main Memory Technology: A Challenge for Development of Business Applications. Jürgen Primsch, SAP AG July 2011
SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications Jürgen Primsch, SAP AG July 2011 Why In-Memory? Information at the Speed of Thought Imagine access to business data,
More informationData Growth. Von 2000 bis 2002 sind mehr Daten generiert worden als in den 40.000 Jahren davor
Prof. Dr.-Ing. Wolfgang Lehner 200. Datenbankstammtisch Data Growth One Minute of Internet Von 2000 bis 2002 sind mehr Daten generiert worden als in den 40.000 Jahren davor Von 2004-2005 hat sich diese
More informationParallel Processing and Software Performance. Lukáš Marek
Parallel Processing and Software Performance Lukáš Marek DISTRIBUTED SYSTEMS RESEARCH GROUP http://dsrg.mff.cuni.cz CHARLES UNIVERSITY PRAGUE Faculty of Mathematics and Physics Benchmarking in parallel
More informationnvm malloc: Memory Allocation for NVRAM
nvm malloc: Memory Allocation for NVRAM David Schwalb, Tim Berning, Martin Faust, Markus Dreseler, Hasso Plattner Hasso-Plattner-Institute, Germany SAP SE, Walldorf, Germany Contact: david.schwalb@hpi.de
More informationIntel Xeon Processor 5560 (Nehalem EP)
SAP NetWeaver Mobile 7.1 Intel Xeon Processor 5560 (Nehalem EP) Prove performance to synchronize 10,000 devices in ~60 mins Intel SAP NetWeaver Solution Management Intel + SAP Success comes from maintaining
More informationAccelerating Server Storage Performance on Lenovo ThinkServer
Accelerating Server Storage Performance on Lenovo ThinkServer Lenovo Enterprise Product Group April 214 Copyright Lenovo 214 LENOVO PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER
More informationA client side persistent block cache for the data center. Vault Boston 2015 - Luis Pabón - Red Hat
PBLCACHE A client side persistent block cache for the data center Vault Boston 2015 - Luis Pabón - Red Hat ABOUT ME LUIS PABÓN Principal Software Engineer, Red Hat Storage IRC, GitHub: lpabon QUESTIONS:
More informationWhy Hybrid Storage Strategies Give the Best Bang for the Buck
JANUARY 28, 2014, SAN JOSE, CA Tom Coughlin, Coughlin Associates & Jim Handy, Objective Analysis PRESENTATION TITLE GOES HERE Why Hybrid Storage Strategies Give the Best Bang for the Buck 1 Outline Different
More informationPSAM, NEC PCIe SSD Appliance for Microsoft SQL Server (Reference Architecture) September 11 th, 2014 NEC Corporation
PSAM, NEC PCIe SSD Appliance for Microsoft SQL Server (Reference Architecture) September 11 th, 2014 NEC Corporation 1. Overview of NEC PCIe SSD Appliance for Microsoft SQL Server Page 2 NEC Corporation
More informationBlurred Persistence in Transactional Persistent Memory
Blurred Persistence in Transactional Youyou Lu, Jiwu Shu, Long Sun Department of Computer Science and Technology, Tsinghua University, Beijing, China luyouyou@tsinghua.edu.cn, shujw@tsinghua.edu.cn, sun-l12@mails.tsinghua.edu.cn
More informationEmerging storage and HPC technologies to accelerate big data analytics Jerome Gaysse JG Consulting
Emerging storage and HPC technologies to accelerate big data analytics Jerome Gaysse JG Consulting Introduction Big Data Analytics needs: Low latency data access Fast computing Power efficiency Latest
More informationIn-Memory Data Management for Enterprise Applications
In-Memory Data Management for Enterprise Applications Jens Krueger Senior Researcher and Chair Representative Research Group of Prof. Hasso Plattner Hasso Plattner Institute for Software Engineering University
More informationImplications of Storage Class Memories (SCM) on Software Architectures
Implications of Storage Class Memories (SCM) on Software Architectures C. Mohan, IBM Almaden Research Center, San Jose mohan@almaden.ibm.com http://www.almaden.ibm.com/u/mohan Suparna Bhattacharya, IBM
More informationNon-Volatile Memory. Non-Volatile Memory & its use in Enterprise Applications. Contents
Non-Volatile Memory Non-Volatile Memory & its use in Enterprise Applications Author: Adrian Proctor, Viking Technology [email: adrian.proctor@vikingtechnology.com] This paper reviews different memory technologies,
More informationCaching Mechanisms for Mobile and IOT Devices
Caching Mechanisms for Mobile and IOT Devices Masafumi Takahashi Toshiba Corporation JEDEC Mobile & IOT Technology Forum Copyright 2016 Toshiba Corporation Background. Unified concept. Outline The first
More informationNon-Volatile Memory and Its Use in Enterprise Applications
Non-Volatile Memory and Its Use in Enterprise Applications Contributor: Viking Technology January 2014 About the SNIA The Storage Networking Industry Association (SNIA) is a not for profit global organization,
More informationBenchmarking Cassandra on Violin
Technical White Paper Report Technical Report Benchmarking Cassandra on Violin Accelerating Cassandra Performance and Reducing Read Latency With Violin Memory Flash-based Storage Arrays Version 1.0 Abstract
More informationLet s Talk About Storage & Recovery Methods for Non-Volatile Memory Database Systems
Let s Talk About Storage & Recovery Methods for Non-Volatile Database Systems Joy Arulraj jarulraj@cs.cmu.edu Andrew Pavlo pavlo@cs.cmu.edu Subramanya R. Dulloor subramanya.r.dulloor@intel.com Carnegie
More informationIS IN-MEMORY COMPUTING MAKING THE MOVE TO PRIME TIME?
IS IN-MEMORY COMPUTING MAKING THE MOVE TO PRIME TIME? EMC and Intel work with multiple in-memory solutions to make your databases fly Thanks to cheaper random access memory (RAM) and improved technology,
More informationSpeed and Persistence for Real-Time Transactions
Speed and Persistence for Real-Time Transactions by TimesTen and Solid Data Systems July 2002 Table of Contents Abstract 1 Who Needs Speed and Persistence 2 The Reference Architecture 3 Benchmark Results
More informationScaling Database Performance in Azure
Scaling Database Performance in Azure Results of Microsoft-funded Testing Q1 2015 2015 2014 ScaleArc. All Rights Reserved. 1 Test Goals and Background Info Test Goals and Setup Test goals Microsoft commissioned
More informationHP ProLiant BL660c Gen9 and Microsoft SQL Server 2014 technical brief
Technical white paper HP ProLiant BL660c Gen9 and Microsoft SQL Server 2014 technical brief Scale-up your Microsoft SQL Server environment to new heights Table of contents Executive summary... 2 Introduction...
More informationDistributed Architecture of Oracle Database In-memory
Distributed Architecture of Oracle Database In-memory Niloy Mukherjee, Shasank Chavan, Maria Colgan, Dinesh Das, Mike Gleeson, Sanket Hase, Allison Holloway, Hui Jin, Jesse Kamp, Kartik Kulkarni, Tirthankar
More informationBoosting Database Batch workloads using Flash Memory SSDs
Boosting Database Batch workloads using Flash Memory SSDs Won-Gill Oh and Sang-Won Lee School of Information and Communication Engineering SungKyunKwan University, 27334 2066, Seobu-Ro, Jangan-Gu, Suwon-Si,
More informationRethinking SIMD Vectorization for In-Memory Databases
SIGMOD 215, Melbourne, Victoria, Australia Rethinking SIMD Vectorization for In-Memory Databases Orestis Polychroniou Columbia University Arun Raghavan Oracle Labs Kenneth A. Ross Columbia University Latest
More informationbla bla OPEN-XCHANGE Open-Xchange Hardware Needs
bla bla OPEN-XCHANGE Open-Xchange Hardware Needs OPEN-XCHANGE: Open-Xchange Hardware Needs Publication date Wednesday, 8 January version. . Hardware Needs with Open-Xchange.. Overview The purpose of this
More informationAssessing the Performance of OpenMP Programs on the Intel Xeon Phi
Assessing the Performance of OpenMP Programs on the Intel Xeon Phi Dirk Schmidl, Tim Cramer, Sandra Wienke, Christian Terboven, and Matthias S. Müller schmidl@rz.rwth-aachen.de Rechen- und Kommunikationszentrum
More informationNV-Heaps: Making Persistent Objects Fast and Safe with Next-Generation, Non-Volatile Memories Joel Coburn
NV-Heaps: Making Persistent Objects Fast and Safe with Next-Generation, Non-Volatile Memories Joel Coburn Work done at UCSD with Adrian M. Caulfield, Ameen Akel, Laura M. Grupp, Rajesh K. Gupta, Ranjit
More information<Insert Picture Here> Oracle In-Memory Database Cache Overview
Oracle In-Memory Database Cache Overview Simon Law Product Manager The following is intended to outline our general product direction. It is intended for information purposes only,
More informationParallel Programming Survey
Christian Terboven 02.09.2014 / Aachen, Germany Stand: 26.08.2014 Version 2.3 IT Center der RWTH Aachen University Agenda Overview: Processor Microarchitecture Shared-Memory
More informationVERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i. Performance Report
VERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i Performance Report V E R I T A S W H I T E P A P E R Table of Contents Introduction.................................................................................1
More informationThe Methodology Behind the Dell SQL Server Advisor Tool
The Methodology Behind the Dell SQL Server Advisor Tool Database Solutions Engineering By Phani MV Dell Product Group October 2009 Executive Summary The Dell SQL Server Advisor is intended to perform capacity
More informationMulti-Threading Performance on Commodity Multi-Core Processors
Multi-Threading Performance on Commodity Multi-Core Processors Jie Chen and William Watson III Scientific Computing Group Jefferson Lab 12000 Jefferson Ave. Newport News, VA 23606 Organization Introduction
More informationIn-memory database systems, NVDIMMs and data durability
In-memory database systems, NVDIMMs and data durability Steve Graves - July 23, 2014 Database management system (DBMS) software is increasingly common in electronics, spurred by growing data management
More informationMeasuring Cache and Memory Latency and CPU to Memory Bandwidth
White Paper Joshua Ruggiero Computer Systems Engineer Intel Corporation Measuring Cache and Memory Latency and CPU to Memory Bandwidth For use with Intel Architecture December 2008 1 321074 Executive Summary
More informationThyNVM: Enabling Software-Transparent Crash Consistency in Persistent Memory Systems
ThyNVM: Enabling Software-Transparent Crash Consistency in Persistent Memory Systems Jinglei Ren Jishen Zhao Samira Khan Jongmoo Choi + Yongwei Wu Onur Mutlu Carnegie Mellon University Tsinghua University
More informationWITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE
WITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE 1 W W W. F U S I ON I O.COM Table of Contents Table of Contents... 2 Executive Summary... 3 Introduction: In-Memory Meets iomemory... 4 What
More informationData Warehousing & Data Mining
Data Warehousing & Data Mining Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 8. Real-Time DW 8. Real-Time Data Warehouses
More informationC. Mohan, IBM Almaden Research Center, San Jose, CA 95120 mohan@almaden.ibm.com http://www.almaden.ibm.com/u/mohan
Implications of Storage Class Memories (SCM) on Software Architectures C. Mohan, IBM Almaden Research Center, San Jose, CA 95120 mohan@almaden.ibm.com http://www.almaden.ibm.com/u/mohan New England Database
More informationVirtuoso and Database Scalability
Virtuoso and Database Scalability By Orri Erling Table of Contents Abstract Metrics Results Transaction Throughput Initializing 40 warehouses Serial Read Test Conditions Analysis Working Set Effect of
More informationAccelerate SQL Server 2014 AlwaysOn Availability Groups with Seagate. Nytro Flash Accelerator Cards
Accelerate SQL Server 2014 AlwaysOn Availability Groups with Seagate Nytro Flash Accelerator Cards Technology Paper Authored by: Mark Pokorny, Database Engineer, Seagate Overview SQL Server 2014 provides
More informationOracle Database Reliability, Performance and scalability on Intel Xeon platforms Mitch Shults, Intel Corporation October 2011
Oracle Database Reliability, Performance and scalability on Intel platforms Mitch Shults, Intel Corporation October 2011 1 Intel Processor E7-8800/4800/2800 Product Families Up to 10 s and 20 Threads 30MB
More informationSystem Requirements Table of contents
Table of contents 1 Introduction... 2 2 Knoa Agent... 2 2.1 System Requirements...2 2.2 Environment Requirements...4 3 Knoa Server Architecture...4 3.1 Knoa Server Components... 4 3.2 Server Hardware Setup...5
More informationThe Data Placement Challenge
The Data Placement Challenge Entire Dataset Applications Active Data Lowest $/IOP Highest throughput Lowest latency 10-20% Right Place Right Cost Right Time 100% 2 2 What s Driving the AST Discussion?
More informationCOS 318: Operating Systems
COS 318: Operating Systems File Performance and Reliability Andy Bavier Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall10/cos318/ Topics File buffer cache
More informationDiablo and VMware TM powering SQL Server TM in Virtual SAN TM. A Diablo Technologies Whitepaper. May 2015
A Diablo Technologies Whitepaper Diablo and VMware TM powering SQL Server TM in Virtual SAN TM May 2015 Ricky Trigalo, Director for Virtualization Solutions Architecture, Diablo Technologies Daniel Beveridge,
More informationIn-Memory Databases Algorithms and Data Structures on Modern Hardware. Martin Faust David Schwalb Jens Krüger Jürgen Müller
In-Memory Databases Algorithms and Data Structures on Modern Hardware Martin Faust David Schwalb Jens Krüger Jürgen Müller The Free Lunch Is Over 2 Number of transistors per CPU increases Clock frequency
More informationAn Approach for Hybrid-Memory Scaling Columnar In-Memory Databases
An Approach for Hybrid-Memory Scaling Columnar In-Memory Databases Bernhard Höppner SAP SE Potsdam, Germany bernhard.hoeppner@ sap.com Ahmadshah Waizy Fujitsu Technology Solutions GmbH Paderborn, Germany
More informationRAID 5 rebuild performance in ProLiant
RAID 5 rebuild performance in ProLiant technology brief Abstract... 2 Overview of the RAID 5 rebuild process... 2 Estimating the mean-time-to-failure (MTTF)... 3 Factors affecting RAID 5 array rebuild
More informationRAMCloud and the Low- Latency Datacenter. John Ousterhout Stanford University
RAMCloud and the Low- Latency Datacenter John Ousterhout Stanford University Most important driver for innovation in computer systems: Rise of the datacenter Phase 1: large scale Phase 2: low latency Introduction
More informationThe SAP HANA Database An Architecture Overview
The SAP HANA Database An Architecture Overview Franz Färber and Norman May and Wolfgang Lehner and Philipp Große and Ingo Müller and Hannes Rauhe and Jonathan Dees Abstract Requirements of enterprise applications
More informationAn Overview of Flash Storage for Databases
An Overview of Flash Storage for Databases Vadim Tkachenko Morgan Tocker http://percona.com MySQL CE Apr 2010 -2- Introduction Vadim Tkachenko Percona Inc, CTO and Lead of Development Morgan Tocker Percona
More informationSolid State Drive Architecture
Solid State Drive Architecture A comparison and evaluation of data storage mediums Tyler Thierolf Justin Uriarte Outline Introduction Storage Device as Limiting Factor Terminology Internals Interface Architecture
More informationLeveraging EMC Fully Automated Storage Tiering (FAST) and FAST Cache for SQL Server Enterprise Deployments
Leveraging EMC Fully Automated Storage Tiering (FAST) and FAST Cache for SQL Server Enterprise Deployments Applied Technology Abstract This white paper introduces EMC s latest groundbreaking technologies,
More informationBigdata High Availability (HA) Architecture
Bigdata High Availability (HA) Architecture Introduction This whitepaper describes an HA architecture based on a shared nothing design. Each node uses commodity hardware and has its own local resources
More informationFlash for Databases. September 22, 2015 Peter Zaitsev Percona
Flash for Databases September 22, 2015 Peter Zaitsev Percona In this Presentation Flash technology overview Review some of the available technology What does this mean for databases? Specific opportunities
More informationAccelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software
WHITEPAPER Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software SanDisk ZetaScale software unlocks the full benefits of flash for In-Memory Compute and NoSQL applications
More informationIn-Memory Databases MemSQL
IT4BI - Université Libre de Bruxelles In-Memory Databases MemSQL Gabby Nikolova Thao Ha Contents I. In-memory Databases...4 1. Concept:...4 2. Indexing:...4 a. b. c. d. AVL Tree:...4 B-Tree and B+ Tree:...5
More informationA High-Throughput In-Memory Index, Durable on Flash-based SSD
A High-Throughput In-Memory Index, Durable on Flash-based SSD Insights into the Winning Solution of the SIGMOD Programming Contest 2011 Thomas Kissinger, Benjamin Schlegel, Matthias Boehm, Dirk Habich,
More informationMicrosoft SQL Server 2014 Fast Track
Microsoft SQL Server 2014 Fast Track 34-TB Certified Data Warehouse 103-TB Maximum User Data Tegile Systems Solution Review 2U Design: Featuring Tegile T3800 All-Flash Storage Array http:// www.tegile.com/solutiuons/sql
More informationCapacity Management for Oracle Database Machine Exadata v2
Capacity Management for Oracle Database Machine Exadata v2 Dr. Boris Zibitsker, BEZ Systems NOCOUG 21 Boris Zibitsker Predictive Analytics for IT 1 About Author Dr. Boris Zibitsker, Chairman, CTO, BEZ
More informationNUMA obliviousness through memory mapping
NUMA obliviousness through memory mapping Mrunal Gawade CWI, Amsterdam mrunal.gawade@cwi.nl Martin Kersten CWI, Amsterdam martin.kersten@cwi.nl ABSTRACT With the rise of multi-socket multi-core CPUs a
More informationSafe Harbor Statement
Safe Harbor Statement "Safe Harbor" Statement: Statements in this presentation relating to Oracle's future plans, expectations, beliefs, intentions and prospects are "forward-looking statements" and are
More informationHardware Configuration Guide
Hardware Configuration Guide Contents Contents... 1 Annotation... 1 Factors to consider... 2 Machine Count... 2 Data Size... 2 Data Size Total... 2 Daily Backup Data Size... 2 Unique Data Percentage...
More informationSoftware Support for Atomicity and Persistence in Non-volatile Memory
Software Support for Atomicity and Persistence in Non-volatile Memory Ellis Giles Rice University erg@rice.edu Kshitij Doshi Intel Corp. kshitij.a.doshi@intel.com Peter Varman Rice University pjv@rice.edu
More informationSystem Architecture. In-Memory Database
System Architecture for Are SSDs Ready for Enterprise Storage Systems In-Memory Database Anil Vasudeva, President & Chief Analyst, Research 2007-13 Research All Rights Reserved Copying Prohibited Contact
More informationIN-MEMORY DATABASE SYSTEMS. Prof. Dr. Uta Störl Big Data Technologies: In-Memory DBMS - SoSe 2015 1
IN-MEMORY DATABASE SYSTEMS Prof. Dr. Uta Störl Big Data Technologies: In-Memory DBMS - SoSe 2015 1 Analytical Processing Today Separation of OLTP and OLAP Motivation Online Transaction Processing (OLTP)
More information3. Memory Persistency Goals. 2. Background
Memory Persistency Steven Pelley Peter M. Chen Thomas F. Wenisch University of Michigan {spelley,pmchen,twenisch}@umich.edu Abstract Emerging nonvolatile memory technologies (NVRAM) promise the performance
More informationStorage Management in the NVRAM Era
Storage Management in the NVRAM Era ABSTRACT Steven Pelley University of Michigan spelley@umich.edu Brian T. Gold Oracle Corporation brian.t.gold@gmail.com Emerging nonvolatile memory technologies (NVRAM)
More informationCONSOLIDATING MICROSOFT SQL SERVER OLTP WORKLOADS ON THE EMC XtremIO ALL FLASH ARRAY
Reference Architecture CONSOLIDATING MICROSOFT SQL SERVER OLTP WORKLOADS ON THE EMC XtremIO ALL FLASH ARRAY An XtremIO Reference Architecture Abstract This Reference architecture examines the storage efficiencies
More informationThe Benefits of Virtualizing
T E C H N I C A L B R I E F The Benefits of Virtualizing Aciduisismodo Microsoft SQL Dolore Server Eolore in Dionseq Hitachi Storage Uatummy Environments Odolorem Vel Leveraging Microsoft Hyper-V By Heidi
More informationSt eps t o w ards HW/SW- DB- Co De sign. Wolfgang Lehner
St eps t o w ards HW/SW- DB- Co De sign Wolfgang Lehner Modern Hardware All over the place 2 A Look at Hardware Trends 201x 3 A Look at Hardware Trends - 2015 Extreme NUMA Effects System Level 2 Component
More informationResolving Journaling of Journal Anomaly via Weaving Recovery Information into DB Page. Beomseok Nam
NVRAMOS 14 10.30. 2014 Resolving Journaling of Journal Anomaly via Weaving Recovery Information into DB Page Beomseok Nam UNIST Outline Motivation Journaling of Journal Anomaly How to resolve Journaling
More informationCloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com
Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...
More informationIncrease Database Performance by Implementing Cirrus Data Solutions DCS SAN Caching Appliance With the Seagate Nytro Flash Accelerator Card
Implementing Cirrus Data Solutions DCS SAN Caching Appliance With the Seagate Nytro Technology Paper Authored by Rick Stehno, Principal Database Engineer, Seagate Introduction Supporting high transaction
More informationPrice/performance Modern Memory Hierarchy
Lecture 21: Storage Administration Take QUIZ 15 over P&H 6.1-4, 6.8-9 before 11:59pm today Project: Cache Simulator, Due April 29, 2010 NEW OFFICE HOUR TIME: Tuesday 1-2, McKinley Last Time Exam discussion
More informationScaling Objectivity Database Performance with Panasas Scale-Out NAS Storage
White Paper Scaling Objectivity Database Performance with Panasas Scale-Out NAS Storage A Benchmark Report August 211 Background Objectivity/DB uses a powerful distributed processing architecture to manage
More informationManaging Storage Space in a Flash and Disk Hybrid Storage System
Managing Storage Space in a Flash and Disk Hybrid Storage System Xiaojian Wu, and A. L. Narasimha Reddy Dept. of Electrical and Computer Engineering Texas A&M University IEEE International Symposium on
More informationBinary search tree with SIMD bandwidth optimization using SSE
Binary search tree with SIMD bandwidth optimization using SSE Bowen Zhang, Xinwei Li 1.ABSTRACT In-memory tree structured index search is a fundamental database operation. Modern processors provide tremendous
More informationThe Technologies & Architectures. President, Demartek
Deep Dive on Solid State t Storage The Technologies & Architectures Dennis Martin Dennis Martin President, Demartek Demartek Company Overview Industry analysis with on-site test lab Lab includes servers,
More informationCOS 318: Operating Systems. Virtual Memory and Address Translation
COS 318: Operating Systems Virtual Memory and Address Translation Today s Topics Midterm Results Virtual Memory Virtualization Protection Address Translation Base and bound Segmentation Paging Translation
More informationApplication-Tier In-Memory Analytics Best Practices and Use Cases
Application-Tier In-Memory Analytics Best Practices and Use Cases Susan Cheung Vice President Product Management Oracle, Server Technologies Oct 01, 2014 Guest Speaker: Kiran Tailor Senior Oracle DBA and
More informationAnalysis of VDI Storage Performance During Bootstorm
Analysis of VDI Storage Performance During Bootstorm Introduction Virtual desktops are gaining popularity as a more cost effective and more easily serviceable solution. The most resource-dependent process
More informationSWISSBOX REVISITING THE DATA PROCESSING SOFTWARE STACK
3/2/2011 SWISSBOX REVISITING THE DATA PROCESSING SOFTWARE STACK Systems Group Dept. of Computer Science ETH Zürich, Switzerland SwissBox Humboldt University Dec. 2010 Systems Group = www.systems.ethz.ch
More informationAchieving a High Performance OLTP Database using SQL Server and Dell PowerEdge R720 with Internal PCIe SSD Storage
Achieving a High Performance OLTP Database using SQL Server and Dell PowerEdge R720 with This Dell Technical White Paper discusses the OLTP performance benefit achieved on a SQL Server database using a
More informationAzure VM Performance Considerations Running SQL Server
Azure VM Performance Considerations Running SQL Server Your company logo here Vinod Kumar M @vinodk_sql http://blogs.extremeexperts.com Session Objectives And Takeaways Session Objective(s): Learn the
More informationEnhancing Recovery Using an SSD Buffer Pool Extension
Enhancing Recovery Using an SSD Buffer Pool Extension Bishwaranjan Bhattacharjee IBM T.J.Watson Research Center bhatta@us.ibm.com Christian Lang* Acelot Inc. clang@acelot.com George A Mihaila* Google Inc.
More informationMemory Channel Storage ( M C S ) Demystified. Jerome McFarland
ory nel Storage ( M C S ) Demystified Jerome McFarland Principal Product Marketer AGENDA + INTRO AND ARCHITECTURE + PRODUCT DETAILS + APPLICATIONS THE COMPUTE-STORAGE DISCONNECT + Compute And Data Have
More informationNon Volatile Memory Invades the Memory Bus: Performance and Versatility is the Result
Non Volatile Memory Invades the Memory Bus: Performance and Versatility is the Result A discussion on why and how Non Volatile Memory is moving onto the DDR bus and what it means for the industry, as well
More informationTushar Joshi Turtle Networks Ltd
MySQL Database for High Availability Web Applications Tushar Joshi Turtle Networks Ltd www.turtle.net Overview What is High Availability? Web/Network Architecture Applications MySQL Replication MySQL Clustering
More informationSolid State Technology What s New?
Solid State Technology What s New? Dennis Martin, President, Demartek www.storagedecisions.com Agenda: Solid State Technology What s New? Demartek About Us Solid-state storage overview Types of NAND flash
More informationRemoving Performance Bottlenecks in Databases with Red Hat Enterprise Linux and Violin Memory Flash Storage Arrays. Red Hat Performance Engineering
Removing Performance Bottlenecks in Databases with Red Hat Enterprise Linux and Violin Memory Flash Storage Arrays Red Hat Performance Engineering Version 1.0 August 2013 1801 Varsity Drive Raleigh NC
More information