Big Data Storage: Excuse me, but

Transcription

1 Big Data Storage: Excuse me, but Where can I keep my bits...?!? Michael Carey University of California, Irvine 0

2 DB History: Shared What? Shared- everything Shared- nothing Shared- disk (1980 s) 1

3 AsterixDB (Cartoon View) 2

4 Distributed Storage in AsterixDB Hash- parmmoned, shared- nothing, local drives ParMMoning based on primary key (hashing) Secondary indexes local to, and consistent with, corresponding primary parmmons (all LSM- based) Also support external datasets (for HDFS) MulMple (Hive) formats, secondary index support Index parmmons co- located with data (if possible) Developed to save space and offer IT comfort 3

5 AsterixDB Data ReplicaMon (WIP) Chained Declustering Log-Based Replication (synchronous, recovery-only copies kept) 4

6 Where Should My Bits Go...? CompuMng may be transient and/or elasmc but accumulated data is not...!!! NaMve storage à hard to expand and contract! Calls for an SD approach based on HPC (or cloud) storage facilimes Obviously workload- dependent (e.g., queries and/or analymcs, Big ML, Big Science,...) Serious experimentamon is needed (IMO) E.g., SAN- based HPC architectures? E.g., Google persistent disks (in Google Cloud)? Performance implicamons interesmng to explore... 5

7 Where Should My Bits Go...? Ex: SDSC s Gordon cluster... Dedicated cluster with 1024 compute nodes and 64 I/O nodes. Each compute node contains two 8- core 2.6 GHz Intel EM64T Xeon E5 (Sandy Bridge) processors and 64 GB of DDR memory. Each I/O node contains two 6- core 2.67 GHz Intel X5650 (Westmere) processors, 48 GB of DDR memory, and sixteen 300 GB Intel 710 solid state drives. Network is a 4x4x4 3D torus with adjacent switches connected by three 4x QDR InfiniBand links (120 Gbit/s). Compute nodes (16 per switch) and I/O nodes (1 per switch) are connected to the switches by 4x QDR (40 Gbit/s). TheoreMcal peak performance is 341 TFlop/s. 6

8 Hedging Our AsterixDB Bets Currently pormng our LSM- based storage to also work on top of HDFS (and YARN) Might somehow feel more comformng (and/or environmentally friendly ) to Big Data IT shops A different path to replicamon & high availability InteresMng experiments lie ahead! Revisit Stonebraker- like OS issues (today s version) Bake- off: Distributed record management vs. DFS, local versus remotely amached storage,... E.g., how well does HDFS do w.r.t. locality of writes? 7

9 This research was sponsored by U.S. Air Force Office of Scientific Research Grant 78- referenced 3596, U.S. Army by Research down performance. Office Grant COMPUTING DAAG29-76-G-0245, Naval Electronics Systems Command Contract N G-0013, and PRACTICES National COMPUTING Science Foundation Grant MCS A01. PRACTICES Key words and phrases: database management, operating systems, buffer management, file systems, scheduling, 6.2 interprocess Buffering communication CR Categories: 3.50, 3.70, 4.22, 4.33, 4.34, 4.35 Author's address: M. Stonebraker, Dept. of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA ACM /81/ $ e files F1 and y are program ly, a user never eads or writes; e paging facilie his file blocks memory. Here, e problems inh. Operating System Support Section 2 concerning buffering provide (e.g., a main memory cache for prefetch, non-lru management, for for Database the Management file system. Figure 1 illustrates and selected force out) this exist service. in a In brief, UNIX provides paged virtual memory Michael Stonebraker context. a buffer How pool whose size is set when University Michael of California, Stonebraker Berkeley mory scheme hich 412 are large ing hardware of 4 bytes per nsequently, a ve an overhead he page table. ory is decreast be reasonable e table of this nt in primary here is th e posoperation will ts: one for the page table for nd one on the e second fault, a large page 1. Introduction Database management systems (DBMS) provide higher level user support than conventional operating systems. The DBMS designer must work in the context of the OS he/she is faced with. Different operating systems are designed for different use. In this paper we examine several popular operating system services and indicate whether they are appropriate for support of database management functions. Often we will see that the wrong service is provided or that severe performance problems exist. When possible, we offer some 1. Introduction Database management systems (DBMS) provide higher level user support than conventional operating systems. The DBMS designer must work in the context of the OS he/she is faced with. Different operating systems are designed for different use. In this paper we examine several popular operating system services and indicate whether they are appropriate for support of database man- Permission to copy without fee all or part of agement this functions. material is granted Often provided we will that the see copies are not made or distributed for direct that the wrong commercial service advantage, is the provided ACM copyright or that severe notice performance and the title of the publication problems and its "date appear, and notice is given that copying exist. When is by permission possible, of the we Association offer some for Computing Machinery. To copy otherwise, or to Permission republish, to copy requires without a fee and/or all or specific part of perthis material mission. granted provided that the cop- This research was sponsored by U.S. Air ies are not made or distributed for direct relational database system [19, 20] open, this may substantially slow which was designed for use with UNIX. Most of the points made for It is an open question whether or this environment have general applinot novel paging organizations can cability to other operating systems assist in solving the problems mentioned in this section. Is History and RepeaMng data managers. Itself? 2. Buffer Pool Management All of the problems discussed in they can be cleanly handled in this context is another unanswered question. University of California, Berkeley Communications of the ACM 7. Conclusions The bottom line is that operating system services in many existing systems are either too slow or inappropilate. Current DBMSs usually provide their own suggestions and make concerning little improve- or no use of those offered ments. In the by next the several operating sections we look at the services provided by system. It is buffer important pool management; that the future file system; scheduling, process manageoperating system ment, and designers interprocess communica- become tion; and consistency control. We more sensitive to DBMS needs. then conclude with a discussion of A DBMS the would merits of including prefer all a files small in a paged virtual memory. efficient operating The examples system in this with paper only are drawn primarily from the UNIX opdesired services. erating Of system those [17] and the currently INGRES available, the relational so-called database real-time system [19, op- 20] SUMMARY: Several operating system services are examined with a view toward their applicability to support of database management functions. These services include buffer pool management; the file system; scheduling, process management, and interprocess communication; and consistency control. SUMMARY: Several operating system services are examined with a view toward their applicability to support of database management functions. These services include buffer pool management; the file system; scheduling, process management, and interprocess communication; and consistency control. suggestions concerning improvements. In the next several sections we look at the services provided by buffer pool management; the file system; scheduling, process management, and interprocess communication; and consistency control. We then conclude with a discussion of Many modern operating systems the operating system is compiled. Then, all file I/O is handled through this cache. A file read (e.g., read X in Figure 1) returns data directly from a block in the cache, if possible; otherwise, it causes a block to be "pushed" to disk and replaced by the desired block. In Figure 1 we show block Y being pushed to make room for block X. A file write simply moves data into the cache; at some later time the buffer manager writes the block to the disk. The UNIX buffer manager used the popular the operating system is compiled. Then, all file I/O is handled through this cache. A file read (e.g., read X in Figure 1) returns data directly from a block in the cache, if possible; otherwise, it causes a block to be "pushed" to disk and replaced by the desired block. In Figure 1 we show buffer manager used LRU [15] replacemen nally, when UNIX dete access to a file, it pref before they are request Conceptually, this sirable because block there is so-called locali [15, 18] will remain in repeated reads and wri the problems enumera lowing subsections aris service for database m July 1981 Volume 24 Number 7 8