CISC 432/CMPE 432/CISC 832 Advanced Database Systems

Transcription

1 CISC 432/CMPE 432/CISC 832 Advanced Database Systems

2 Course Info Instructor: Patrick Martin Goodwin Hall Office Hours: Wednesday 11:00 1:00 or by appointment Schedule: Slot 22 (Monday 3:30 4:30, Wednesday 2:30 3:30 and Thursday 4:30 5:30) Botterell Hall B129 Prerequisites: CISC 332* or permission of the instructor. 432/832 2

3 Assumed Background Previous course in DBMS or equivalent experience Relational model, SQL, relational algebra, schema design File structures, indexes (tree and hash) 432/832 3

4 Reference Materials Textbook: Database System Concepts(6 th Edition) by A. Silbershatz, H. Korth and S. Sudarshan, McGraw-Hill. Research papers 432/832 4

5 Learning Outcomes Apply optimization algorithms to SQL queries to produce efficient query plans. Apply concurrency control and recovery algorithms to sample transaction workloads to ensure ACID properties are maintained. Assess the use of relational DBMSs and NoSQL systems for different types of data and applications. Apply a NoSQL system to the creation of a sample database. Apply the MapReduce framework to a sample big data problem. Evaluate the use of a big data approach to a sample application area or problem. 432/832 5

6 Marking Schemes Undergraduate students (432) 4 assignments (60 %). Due dates: Oct 10, Oct 24, Nov 14, Nov 28 2 term tests (30%). Class participation (10%). 432/832 6

7 Marking Schemes (Cont.) Graduate students (832) 4 assignments (40%). Due dates: Oct 10, Oct 24, Nov 14, Nov 28 2 term tests (20%). Class participation (10%). Term project (30 %). Proposal due Oct. 17, Report due Dec /832 7

8 Marking Schemes (Cont.) Grading Method In this course, some components will be graded using numerical percentage marks. Other components will receive letter grades, which for purposes of calculating your course average will be translated into numerical equivalents using the Faculty of Arts and Science approved scale (see below). Your course average will then be converted to a final letter grade according to Queen s Official Grade Conversion Scale. Late Policy Assignments should be handed in by 4:00 pm on the day they are due. Late assignments are subject to a 10% per day late penalty, with weekends counted as one day. Late assignments will not be accepted beyond 5 days past the date due. 432/832 8

9 Class Participation Rubric 9-10 marks 7 8 marks 5 6 marks < 5 marks Frequency and Quality Attends class regularly and always contributes to the discussion by raising thoughtful questions, analyzing relevant issues, building on others ideas, synthesizing across readings and discussions, expanding the class perspective. Attends class regularly and sometimes contributes to the discussion in the aforementioned ways. Attends class regularly but rarely contributes to the discussion in the aforementioned ways. Attends class regularly but never contributes to the discussion in the aforementioned ways. Readings and Surveys Reads all of the assigned readings, demonstrates understanding of the material, completes all required surveys. Reads most of the assigned readings, demonstrates some understanding of the material, completes all required surveys. Reads some of the assigned readings, completes all required surveys Reads some of the assigned readings, completes some required surveys 432/832 9

10 Academic Integrity Academic integrity is constituted by the five core fundamental values of honesty, trust, fairness, respect and responsibility (see These values are central to the building, nurturing and sustaining of an academic community in which all members of the community will thrive. Adherence to the values expressed through academic integrity forms a foundation for the "freedom of inquiry and exchange of ideas" essential to the intellectual life of the University (see the Senate Report on Principles and Priorities) Students are responsible for familiarizing themselves with the regulations concerning academic integrity and for ensuring that their assignments conform to the principles of academic integrity. Information on academic integrity is available in the Arts and Science Calendar (see Academic Regulation 1), on the Arts and Science website (see and from the instructor of this course. Departures from academic integrity include plagiarism, use of unauthorized materials, facilitation, forgery and falsification, and are antithetical to the development of an academic community at Queen's. Given the seriousness of these matters, actions which contravene the regulation on academic integrity carry sanctions that can range from a warning or the loss of grades on an assignment to the failure of a course to a requirement to withdraw from the university. 432/832 10

11 Course Content Big Data Structured data Unstructured data Streaming data RDBMS distributed parallel column-oriented multi-tenant NewSQL NoSQL key-value document graph column-oriented DSMS (data model, query performance, consistency, scalability, availability) 432/832 11

12 Course Schedule Structured Data (weeks 1 7) Relational DBMSs Query processing, query optimization, transaction management, concurrency control, recovery, BI & data warehousing, column-oriented storage, database architectures, SaaS & multi-tenant DBs NewSQL systems 432/832 12

13 Course Outline (cont) Unstructured Data (weeks 8 11) NoSQL database systems, MapReduce technology for analytics Streaming Data (week 12) Data stream management systems 432/832 13

14 Focus Issues Data model: language and logical constructs that determine the logical structure of a database and consequently how data is stored, organized, and manipulated. 432/832 14

15 Focus Issues (cont) Query performance Efficiency of a DBMS typically expressed with metrics like query response time and/or query throughput Consistency Guarantee concerning state of a data item when accessed Eg ACID, BASE 432/832 15

16 Focus Issues (cont) Scalability Ability of a system to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth Availability Proportion of time that requests received by a system receive a response. 432/832 16

17 What is Big Data? 7 definitions of big data ( 3 V s: Volume, Velocity, Variety New scale-out technologies like Hadoop and NoSQL Data distinctions: generated from transactions versus interactions (eg click on web page) and observations (eg sensors) 432/832 17

18 What is Big Data? 7 definitions (cont) Signals: data is real-time and facilitates realtime business decisions Opportunity: analyzing data that was previously ignored because of technology limitations Metaphor: process of helping planet grow a nervous system and we are just another sensor New term for old stuff 432/832 18

19 Related Buzzwords Data analytics (Business Intelligence) Analytics is the process of obtaining an optimal and realistic decision based on existing data. Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information. 432/832 19

20 Related Buzzwords Cloud computing A networking solution in which everything from computing power to computing infrastructure, applications, business processes to personal collaboration is delivered as a service wherever and whenever you need. 432/832 20

21 Related Buzzwords NoSQL (Not Only SQL) NoSQL encompasses a wide range of technologies and architectures not based on the relational model that seeks to solve the scalability and big data performance issues of relational databases. Eg Cassandra, BigTable, SimpleDB, MongoDB, Voldemort 432/832 21

22 Related Buzzwords NewSQL Encompasses solutions aimed at bringing to the relational model the benefits of horizontal scalability and fault tolerance provided by NoSQL solutions Eg. Google Spanner, VoltDB 432/832 22