How To Write A Database Program

Transcription

1 SQL, NoSQL, and Next Generation DBMSs Shahram Ghandeharizadeh Director of the USC Database Lab

2 Outline A brief history of DBMSs. OSs SQL NoSQL 1960/

3 Before Computers Database DBMS/Data Store

4 Digital Era Database File System/ Data Store

5 Before DBMSs: 1960/70s Developer 1 Application programs Data Developer 2 Application programs Data

6 After DBMSs Developer 1 Application programs DBMS Application programs Developer 2 Physical Data Independence. SQL as a what -oriented language.

7 SQL Data Stores Manage records/tuples A record/tuple is a row in a table where attribute names are pre-defined in a schema. Alternative physical designs: Column-store versus Row-store. Transactions with ACID properties

8

9 SQL IS OVERHYPED

10 Why? Marketing campaigns have become too exaggerated! Relational vendors claim RDBMS is the answer to all data management needs. What are some counter examples? Seltzer. Beyond Relational Databases. Communications of the ACM, July 2008.

11 Web Search Semi-structured data HTML pages instead of raw data. Queries are keyword lookups and the desired response is a sorted list of possible answers. Need for efficient inverted indices. Bulk updates, read mostly. Need for nontraditional indexing.

12 Directory Services International organizations with distributed resources and personnel. Requirement: fast lookup of entities arranged in a hierarchical structure that corresponds to a hierarchy of the organization. LDAP standard. Core of identification and authentication system from a number of vendors, e.g., IBM Tivoli, Microsoft Active Directory Server, SUN ONE Directory Server. Bulk updates similar to data warehousing. Multi-valued attributes. Queries are single-row retrieval or lookups based on attribute values.

13 Other Examples Mobile device caching Your cell phone s directory as a transient cache of a global directory. Stream management Real-time filtering of streams for interesting patterns. Example: identify hotly traded stock, or a stock that is not traded as heavily as expected. Filters look like SQL selection predicates, causing developers to mistake a RDBMS as the right choice. XML management

14 Summary Relational DBMS have been designed for transaction processing and workloads consisting of ad hoc queries and significant amount of updates. 25 years ago, One market for DBMS: Business data processing. This has changed to include different applications with different requirements. Example applications are read-dominated: No need for transactional guarantees. SQL is the wrong choice for stream processing. One software architecture will not support the diverse needs of these applications. Possible solutions: 1) each application re-builds its own storage manager from scratch, 2) provide a flexible solution that can be tailored to the needs of a particular application.

15 Past 25 Years Two trends: 1. Bloated systems. Need for a specialist, a trained DBA, to keep a system and its applications running. 2. Few applications need all the features available in today s RDBMSs. The application must pay for all the features even though it requires a small subset.

16 NOSQL DATA STORES

17 NoSQL Data Stores Scale horizontally for simple operations using many servers. Replicate and distribute (partition) data across many servers. Provide a simple call level interface or protocol. A weaker concurrency model than ACID: Basically Available, Soft state, Eventually consistent (BASE). Efficient use of distributed indexes and DRAM for data storage. Ability to dynamically add new attributes to data records. Cattell. Scalable SQL and NoSQL Data Stores. SIGMOD Record 39(4), Ghandeharizadeh, Boghrati, and Barahmand. An Evaluation of Graph Data Models. TPCTC 2014.

18 NoSQL Data Model A key-value store: A distributed hash table, A key/value may be an arbitrary sequence of bytes, E.g., memcached, Voldemort, Riak, Redis, Tokyo Cabinet, Membase, Membrain. A document store: A value may be a scalar, lists, nested documents, Attribute names might be dynamically defined at runtime, E.g., SimpleDB, CouchDB, MongoDB, Terrastore. An Extensible record store: A hybrid between a SQL store and a document store, Families of attributes are defined in a schema and new attributes can be added, Attributes may be list-valued, E.g., BigTable, HBase, HyperTable, Cassandra, PNUTs.

19 MIDDLEWARE: CACHE AUGMENTED DATA STORES

20 Simple Operations Operations that read and write a small amount of data. Challenge: High volume of requests with a low latency requirement. Person-to-person service providers in 1 Minute: 100M queries 7K user visits 147K page views 347K Tweets Facebook, Google, Twitter, Wikipedia,

21 How? Look up query result instead of query processing. Ideal for applications with workloads that exhibit a high read to write ratio. Key-value store as the cache manager. Query result caching: Key: query string, Value: result set Trillions of cached key-value pairs.

22 Cache Augmented DBMSs 1. Value = Get (Key) 2. If Value is found, go to Step SQL queries 4. Query results Application constructs Value using the results 5. Put(Key, Value) 6. Use Value to generate HTML result page 4 RDBMS Server Cache Server (KVS, e.g., memcached)

23 CADBMS: Update 1. SQL DML Command: Insert, Delete, Update 2. Invalidate keyvalue pairs: Delete 1 2 Alternatives to invalidate include Refill/Refresh and incremental update RDBMS Server Cache Server (KVS, e.g., memcached)

24 CADBMS Today Developer 1 Stale Application programs In-memory Copy of Data memcached Cache Server Developer 2 Application programs Persistent Data Data Store

25 Future CADBMSs Developer 1 Application programs Key Value Cache Server Application programs CADBMS Developer 2 Physical Data Independence. A what -oriented language. Data Store

26 KOSAR Developer 1 Application programs Key Value Cache Server Application programs KOSAR Developer 2 RDBMS Physical Data Independence. SQL as a what -oriented language. Ghandeharizadeh et. al. A Demonstration of KOSAR. Middleware 2014.

27 Architecture A database driven application: Application Data Store Client Data Store Server

28 Architecture: Example An RDBMS driven application authored using Java: Application JDBC SQL Result Set MySQL Server

29 KOSAR: Transparent Caching Simply replace the client component of your application with KOSAR and see it run much faster. Application Data Store Client Data Store Server Ghandeharizadeh, Yap, and Nguyen. Strong Consistency in Cache Augmented SQL Systems. Middleware Ghandeharizadeh, Irani, Lam, Yap. CAMP: A Multi-Queue Eviction Policy for Key-Value Stores. Middleware 2014.

30 How? 1. Lookup query result instead of query processing. Application Data Store Client Data Store Server memcached Servers Ideal for workloads that exhibit a high read to write ratio.

31 Client-Server Architecture SoAR (Actions/Second) CADBMS CADBMS SQL-X SQL-X 0.1% Write 10% Write SLA: 95% of actions to observe a response time faster than 100 msec. Barahmand and Ghandeharizadeh. BG: A Social Networking Benchmark. CIDR Barahmand and Ghandeharizadeh. Expedited Benchmarking of Social Network Actions. CIKM 2013.

32 BG Benchmark, BG is a macro benchmark for interactive social networking actions. BG quantifies the Social Action Rating (SoAR) of a data store: For a given workload, the maximum number of simultaneous actions performed by a data store while satisfying a pre-specified SLA. Ph.D. Fellowship Barahmand and Ghandeharizadeh. BG: A Social Networking Benchmark. CIDR Barahmand and Ghandeharizadeh. D-Zipfian: A Decentralized Implementation of Zipfian. SIGMOD DBTest Barahmand and Ghandeharizadeh. Expedited Benchmarking of Social Network Actions. CIKM Alabdulkarim, Barahmand and Ghandeharizadeh. A Scalable Benchmark for Interactive Social Networking Actions.

33 Client-Server Architecture SoAR (Actions/Second) CADBMS CADBMS SQL-X SQL-X 0.1% Write 10% Write SLA: 95% of actions to observe a response time faster than 100 msec.

34 Shared Address Space 1. Avoid overhead of serialization and network communication Application Data Store Client Data Store Server

35 Shared Address Space SoAR (Actions/Second) CADBMS CADBMS SQL-X 0.1% Write SQL-X 10% Write SLA: 95% of actions to observe a response time faster than 100 msec.

36 Shared Address Space SoAR (Actions/Second) CADBMS CADBMS SQL-X 0.1% Write SQL-X 10% Write SLA: 95% of actions to observe a response time faster than 100 msec.

37 Why? 1. CPU overhead of query processing is more than 85% [1, 2]. Application Data Store Client Data Store Server Cache Servers Harizopoulos et. al. OLTP: Through the Looking Glass and What We Found There. SIGMOD Stonebraker and Cattell. 10 Rules for Scalable Performance in Simple Operation Datastores. CACM 2011.

38 Architectures Client-Server, Shared-Address Space, and Hybrids. Client-Server Shared-Address Space Ghandeharizadeh, and Yap. Cache Augmented Data Stores. SIGMOD DBSocial 2013.

39 NON VOLATILE MEMORY

40 Non Volatile Memory Flash CPU CPU DRAM HDD NVM Flash CPU DRAM HDD Flash CPU DRAM HDD Traditional DRAM (late 2016)

41 Non-Volatile Memory Byte-addressable Time to rewrite the key-value stores & database engine! Configurable: DRAM CPU CPU Emulated Flash Emulated HDD Emulated DRAM Emulated Flash Emulated HDD NVM Time to re-design algorithms NVM

42 Digital Era Database File System/ Data Store

43 Future (Biological) Computers Database DBMS/Data Store