Data Models and Database Management Systems (DBMSs) Dr. Philip Cannata

Transcription

1 Data Models and Database Management Systems (DBMSs) Dr. Philip Cannata 1

2 Data Models in the 1960s, 1970s, and 1980s Hierarchical Network (Graph) Relational Schema (Model) - first 1956 Vern Watts was IMS's chief architect for many years. Watts joined IBM in 1956 and worked at IBM's Silicon Valley development labs until his death on April 4, IBM began the designed of IMS with Rockwell and Caterpillar for the Apollo program, where it was used to inventory the very large bill of materials (BOM) for the Saturn V moon rocket and Apollo space vehicle The first "IMS READY" message appeared on an IBM 2740 terminal in Downey, California, on 14 August CODASYL (Conference / Committee on Data Systems Languages) formed a List Processing Task Force the network CODASYL database model was published. 1980s interest in CODASYL gradually faded due to growing interest in relational databases. Year Event 1970 Dr. E. F. Codd develops the relational database model IBM develops the predecessor to SQL Relational Software, Inc. (later renamed Oracle) releases the first relational DBMS, Oracle IBM releases their first RDBMS, SQL/DS (SQL/Data System) IBM released DB2 (Database 2) Microsoft releases SQL Server ANSI publishes first SQL standards (ANSI/ISO SQL-89, or SQL1) ANSI revises standards (ANSI/ISO SQL-92, or SQL2) ANSI publishes SQL3 (ANSI/ISO SQL:1999) ANSI publishes SQL4 (ANSI/ISO SQL:2003). Dr. Philip Cannata See humorous video here 2

3 Hierarchical DBMS Example (A Contrived Hierarchical DBMS using HTML) Queries Show all employees: $('emp') Show employee with id=7876: $( #7876 ) Show all departments: $( dept') Show department of employee with id=7876: $('#7876').parent().andSelf() Show department of employees: $('emp').parent().andself() Show all employees with their department name: var r=$('emp').each(function( index, element ){console.log( $ (this).attr('empno'), $ (this).attr('ename'), $(this).attr('job'), $ (this).attr('mgr'), $ (this).attr('hiredate'), $(this).attr('sal'), $(this).attr('comm'), $ (this).parent().attr('dname') )}) Show all employees with their Dr. Philip Cannata manager s name: 3

4 Hierarchical DBMS Example (A Contrived Hierarchical DBMS using HTML) Queries Show all employees with their manager s name: var a = $ ("emp").each(function(){ try{ console.log( " EMPLOYEE: " + $ (this).attr('ename') + " MANAGER " + $("#"+ $ (this).attr("mgr")).attr('ename') ); }catch(e){} }); Dr. Philip Cannata 4

5 Hierarchical DBMS Example (A Second Contrived Hierarchical DBMS using HTML) Queries Show all managers: $( mgr') Show all employees with their manager s name: var r=$('emp').each(function( index, element ){console.log( $ (this).attr('empno'), $ (this).attr('ename'), $(this).attr('job'), $ (this).attr('mgr'), $ (this).attr('hiredate'), $(this).attr('sal'), $(this).attr('comm'), $ (this).parent().parent().attr('ename') )} ) Dr. Philip Cannata 5

6 Dr. E. F. Codd 1. Humans only know how to store data on Disk or in Memory in the form of Mathematical Relations (Tables). 2. So, the Data Model should be Relational (i.e., Tables). 3. Also, DBMSs should have an ad hoc query language. A mathematical Relation is a subset of the cross-product of a set of Domains. e.g., Example 1 Example 2 <! 0! 1! 0! 2! 0! 3!!! 1! 2! 1! 3!!! 2! 3!!! D1: People! Phil! Rita! D2: Ages! 60! 65! D3: Gender! M! F! People! Ages! Gender! Phil! 60! M! Phil! 60! F! Phil! 65! M! Phil! 65! F! Rita! 60! M! Rita! 60! F! Rita! 65! M! Rita! 65! F! Dr. Philip Cannata 6

7 Traditional Data Management in the 1980s, 1990s, and 2000s Hierarchical Network (Graph) Relational Schema (Model) - first 1956 Vern Watts was IMS's chief architect for many years. Watts joined IBM in 1956 and worked at IBM's Silicon Valley development labs until his death on April 4, IBM began the designed of IMS with Rockwell and Caterpillar for the Apollo program, where it was used to inventory the very large bill of materials (BOM) for the Saturn V moon rocket and Apollo space vehicle The first "IMS READY" message appeared on an IBM 2740 terminal in Downey, California, on 14 August CODASYL (Conference / Committee on Data Systems Languages) formed a List Processing Task Force the network CODASYL database model was published. 1980s interest in CODASYL gradually faded due to growing interest in relational databases. Year Event 1970 Dr. E. F. Codd develops the relational database model IBM develops the predecessor to SQL Relational Software, Inc. (later renamed Oracle) releases the first relational DBMS, Oracle IBM releases their first RDBMS, SQL/DS (SQL/Data System) IBM released DB2 (Database 2) Microsoft releases SQL Server ANSI publishes first SQL standards (ANSI/ISO SQL-89, or SQL1) ANSI revises standards (ANSI/ISO SQL-92, or SQL2) ANSI publishes SQL3 (ANSI/ISO SQL:1999) ANSI publishes SQL4 (ANSI/ISO SQL:2003). Dr. Philip Cannata 7

8 There were many other Data Models, but most were not implemented as DBMSs. ER Data Model is still used in Data Modeling Introduced by Peter Chen in "The Entity Relationship Model: Toward a Unified View of Data" for entity relationship modeling. Dr. Philip Cannata 8

9 There were many other Data Models but, most were not implemented as DBMSs. IDEF1X Data Model is still used in Data Modeling Dr. Philip Cannata 9

10 There were many other Data Models, but most were not implemented as DBMSs. IE Data Model is still used in Data Modeling Dr. Philip Cannata 10

11 There were many other Data Models, but most were not implemented as DBMSs. UML Data Model is still used in Data Modeling ac count account_number balance 0..* 0..1 branch branch_name branch_city assets 0..* * customer customer_name customer_street customer_city 0..* 0..* loan 0..* loan_number amount Dr. Philip Cannata 11

12 There were many other Data Models, but most were not implemented as DBMSs. OO DBMSs were implemented but failed Dr. Philip Cannata 12

13 There were many other Data Models, but most were not implemented as DBMSs. LDAP was implemented and still exists today as Directory Servers (e.g., Active Directory) Dr. Philip Cannata 13

14 There were many other Data Models, but most were not implemented as DBMSs. EAV was implemented Dr. Philip Cannata 14

15 EAV Evolved into RDF and other NoSQL DBMSs Dr. Philip Cannata 15

16 Data Management Today NoSQL Schema(Model)-later Big Data Schema(Model)-never Relational (YeSQL) Schema(Model)-first Dr. Philip Cannata 16

17 Data Management Today NoSQL Schema(Model)-later Big Data Schema(Model)-never NoSQL-SQL-Professionals/ html? srchtrk=index:1%0alinktypeid: 2%0Aq:nosql%0Apage:1%0As:relevance %0Asa:true%0Aproducttypeid:2 Relational (YeSQL) Schema(Model)-first I don t endorse all of the statements in this course but it does represent what many people are preaching. Dr. Philip Cannata 17

18 Dr. Philip Cannata 18

19 cs347 Data Management NoSQL Schema(Model)-later Big Data Schema(Model)-never 2 Relational (YeSQL) Schema-first Dr. Philip Cannata The Complexity goes Somewhere! SIM Circa 1980! 19

20 DataStax Graph Databases - minute 10:45-28:00 Dr. Philip Cannata 20

21 Dr. Philip Cannata 21

22 Demo of Neo4j Cypher Graph Database Language I ll be using the Apex 5 REST interface for this demo. Dr. Philip Cannata 22

23 Demo of Neo4j Cypher Graph Database Language The Apex 5 REST interface is similar to POSTMAN. Dr. Philip Cannata 23

24 Demo of Neo4j Cypher Graph Database Language "CREATE (:emp { EMPNO : 7369, ENAME : 'SMITH', JOB : 'CLERK', MGR : 7902, HIREDATE : '17- DEC-80', SAL : 800, COMM : 0, DEPTNO : 20})" Notice: I didn t create an emp table before this insert. Dr. Philip Cannata 24

25 Demo of Neo4j Cypher Graph Database Language "CREATE (:dept { DEPTNO : 20, DNAME : 'RESEARCH', LOC : 'DALLAS' })" Dr. Philip Cannata 25

26 Demo of Neo4j Cypher Graph Database Language "MATCH (a:emp),(b:dept) WHERE a.deptno = 20 AND b.deptno = 20 CREATE (a)<-[:employees]-(b)" Creating a Relationship. Dr. Philip Cannata 26

27 Demo of Neo4j Cypher Graph Database Language "MATCH(a:emp) RETURN a" Dr. Philip Cannata 27

28 Demo of Neo4j Cypher Graph Database Language "MATCH(a:emp)<-[:employees]-(b:dept) WHERE b.deptno = 20 RETURN b, a.ename, a.job, a.mgr, a.deptno, a.hiredate" Dr. Philip Cannata 28

29 Demo of Neo4j Cypher Graph Database Language I ll load this behind the scenes and then do a few more examples. Dr. Philip Cannata 29

30 SIM DDL for the emp/dept Database Creating a Relationships. Dr. Philip Cannata 30

31 Demo of Neo4j Cypher Graph Database Language "MATCH(a:emp)-[:dept]->(b:dept) RETURN b.dname, a.ename" Dr. Philip Cannata 31

32 Demo of Neo4j Cypher Graph Database Language "MATCH(a:emp)<-[:employees]-(b:dept) WHERE b.deptno = 20 RETURN b, a.ename, a.job, a.mgr, a.deptno, a.hiredate" Dr. Philip Cannata 32

33 cs347 Data Management NoSQL Schema(Model)-later Big Data Schema(Model)-never 2 4 Relational (YeSQL) Schema-first The Complexity goes Somewhere! 1 3 Dr. Philip Cannata SIM Circa 1980! 33

34 All Data Models are Ignoring the First Data Model! from Dr. Philip Cannata 34

35 Except for SIM, which was implemented in the 1980s and still exists at UT with Me Based on the 1981 paper Database Description with SDM: A Semantic Database Model by Michael Hammer and. Dennis McLeod, see class calendar for a pdf version of the paper. Doug Tolbert Dr. Philip Cannata 35

36 Dr. Philip Cannata 36

37 SIM DDL for the emp/dept Database Creating a Relationships. Dr. Philip Cannata 37

38 Revealing a Secret Dr. Philip Cannata 38

39 {'query': u'"match(a:emp) RETURN a"'} ReL statement is: MATCH(a:emp) RETURN a SIM is: FROM EMP RETRIEVE *; Revealing a Secret ReL is a research system that I ve built with the help of my students. ReL is a fork of jython, which is a python interpreter written in Java that incorporates various Data Models. In this case, ReL converts Neo4j/Cypher into SIM and then converts SIM to SPARQL. ReL is also being run as my RESTful Server. SPARQL is: SELECT DISTINCT empno, deptno, comm, job, mgr, ename, hiredate, sal from table( sem_match('select * where { GRAPH <emp_schema> {?indiv rdf:type :emp }?indiv :empno?empno.?indiv :deptno?deptno.?indiv :comm?comm.?indiv :job?job.?indiv :mgr?mgr.?indiv :ename?ename.?indiv :hiredate?hiredate.?indiv :sal?sal. }', SEM_MODELS('F2015_CS370_SIM'), null, SEM_ALIASES( SEM_ALIAS('', '#')), null) ) (('EMPNO', 'DEPTNO', 'COMM', 'JOB', 'MGR', 'ENAME', 'HIREDATE', 'SAL'), (7900, 30, 0, 'CLERK', 7698, 'JAMES', '03-DEC-81', 950), (7698, 30, 0, 'MANAGER', 7839, 'BLAKE', '01-MAY-81', 2850))... Dr. Philip Cannata 39

40 Revealing a Secret When I loaded the full Cypher, this was built under the covers by ReL. Dr. Philip Cannata 40

41 Revealing a Secret "MATCH(a:emp)<-[:employees]-(b:dept) WHERE b.deptno = 20 RETURN b, a.ename, a.job, a.mgr, a.deptno, a.hiredate" Dr. Philip Cannata 41

42 Revealing a Secret {'query': u'"match(a:emp)<-[:employees]-(b:dept) WHERE b.deptno = 20 RETURN b, a.ename, a.job, a.mgr, a.deptno, a.hiredate"'} ReL statement is: MATCH(a:emp)<-[:employees]-(b:dept) WHERE b.deptno = 20 RETURN b, a.ename, a.job, a.mgr, a.deptno, a.hiredate SIM is: FROM DEPT RETRIEVE *, ENAME OF employees, JOB OF employees, MGR OF employees, DEPTNO OF employees, HIREDATE OF employees WHERE DEPTNO = 20; SPARQL is: SELECT DISTINCT deptno, dname, loc, x0_1, x1_1, x2_1, x3_1, x4_1 from table( sem_match('select * where { GRAPH <dept_schema> {?indiv rdf:type :dept }?indiv :deptno?deptno.?indiv :dname?dname.?indiv :loc?loc.?indiv :deptno :20. OPTIONAL {?indiv :employees?x0_0.?x0_0 :ename?x0_1.?indiv :employees?x0_0.?x0_0 :job?x1_1.?indiv :employees?x0_0.?x0_0 :mgr?x2_1.?indiv :employees?x0_0.?x0_0 :deptno?x3_1.?indiv :employees?x0_0.?x0_0 :hiredate?x4_1. } }', SEM_MODELS('F2015_CS370_SIM'), null, SEM_ALIASES( SEM_ALIAS('', '#')), null) ) (('DEPTNO', 'DNAME', 'LOC', 'X0_1', 'X1_1', 'X2_1', 'X3_1', 'X4_1'), (20, 'RESEARCH', 'DALLAS', 'ADAMS', 'CLERK', 7788, 20, '12-JAN-83'), (20, 'RESEARCH', 'DALLAS', 'SCOTT', 'ANALYST', 7566, 20, '09-DEC-82'))... Dr. Philip Cannata 42

43 How is the Neo4j/Cypher to SIM translation done in ReL? MATCH (e:emp)-[:deptno]->(d:dept)-[:orgnum]->(o:org) WHERE e.empno = 111 RETURN e.empno, e.ename, e.sal, d.dname, o.oname In CS345, you learn to parse this kind of statement and build an Abstract Syntax Tree (AST): You use a set of formal grammars to do this. You also learn how to travers the AST and output something like SIM (I teach how to do this using a Visitor pattern). This work was done by Carson Ball as a CS370 project this past Summer. Carson has had cs345, cs375, and cs347. I would hire him! Dr. Philip Cannata 43

44 Revealing another Secret Dr. Philip Cannata 44

45 Others are Starting to Discover this Secret DataStax Graph Databases - minute 29:00 to the end Dr. Philip Cannata 45

46 But there s More that Others are Not Discovering Yet I ll do this behind the scenes. What would expect from this SPARQL query? Dr. Philip Cannata 46

47 class/subclass Inferencing is just the beginning For more information on Inferencing, see Chapter 9 of the Learning SPARQL book (see the Books and Papers Tab on the class website). Dr. Philip Cannata 47

48 cs347 Data Management NoSQL Schema(Model)-later Big Data Schema(Model)-never Relational (YeSQL) Schema-first The Complexity goes Somewhere! 1 3 Dr. Philip Cannata SIM Circa 1980! 48

49 Useful Free Books! See the Books and Papers Tab on the class website. Dr. Philip Cannata 49