DBMS SQL JDBC 1. Integrated data

Transcription

1 Integrated data DBMS SQL JDBC 1 Often an enterprize or problem domain consists of data that can be organized on multiple attributes (multiple organizations). This classroom has people with different roles {student, professor}, associations {major, departments}, courses {sections and classes} In a single program collections can be used to "manage" data objects (references to data objects). For example, the CompSci department could use: a TreeMap to hold professors data with their names "Barnes" as keys. a HashMap to hold employment data using an employee id as a key an ArrayList to hold professors that serve on a committee. The three collections hold references to the same data objects

2 Data w/ multiple organizations DBMS SQL JDBC 2 Barnes Computer Science renzo@csun.edu Barkataki Computer Science shan@csun.edu Noga Computer Science noga@csun.edu All dynamic objects are "reference" variables their "value" is a pointer / reference to their memory location.

3 Retrievals Vs Updates DBMS SQL JDBC 3 Retrieval operations on integrated is easy. Access by key retrieval with the TreeMap or HashMap or search with the Arraylist accesses data object and "getter" methods can be invoked. Retrievals do not affect the organizations. What happens with insertion or deletions operations that update the organization? All collections must be kept in sync. Insert / delete into one requires associated insert / delete into all. The program must "know" and maintain the "semantics" of the associations among the data records. The relationships exist and are defined by the collections. There is no independence between data and management between storage and application. Soln: Database Management Systems.

4 Databases DBMS SQL JDBC 4 Database = collection of data organized for access, updated and managed. Information is the interpretation of data Database: collection of tables (relations), files, or datasets table is a collection of fields (columns of data items) attributes = table columns domain of values tuples = table rows attribute values of an entity primary key = field(s) that uniquely selects table row foreign key = fields in one table that are primary keys in another table. no duplicates of tuples or attributes order of rows and columns is not important cells = data items, single valued (int, float, string, boolean )

5 CSUN example DBMS SQL JDBC 5

6 Relation && Relationships RelationName (attribute 0, attribute 1, attribute n ) key = attribute DBMS SQL JDBC 6 profs ( pid, name, office) section ( snum, profid, studentid, class) students( sid, name, major) deptfacutly( dept, profid) Relationships exist between tables w/ shared values section ( snum, profid, studentid, class) profs ( pid, name, office) memberof teaches deptfacutly( dept, profid) attends students( sid, name, major)

7 Java JDBC SQLite3 Use posted SQLite Java Installation page notes to download, configure, and test your sqlite3 Java JDBC installation. SQLite (sqlite3) is a zeroinstallation, serverless, SQL compliant open source database. It is widely used in common applications. java application JDBC SQLite DB DBMS SQL JDBC 7 sqlite-jdbc connector These notes cover SQL (mysql examples) and sqlite3. The sqlite3 shell program can be used to test sqlite3 queries. We will use an "elections" database of USA Presidential elections since A Windows "dos command prompt" window (console session) for sqlite3 is on the next page

8 sqlite3 session DBMS SQL JDBC 8 C: sqlite3 // run sqlite SQLite version :27:36 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite>.open elections.db // creates empty elections.db sqlite>.mode columns // set output mode to columns sqlite>.mode tabs // set input separator to tabs sqlite>.read createtables.txt // enable tables from file sqlite>.schema CREATE TABLE President( // use blanks if not.mode tabs Name text not null, Party text not null, Year int not null, primary key (Name, Year)); //... shows VicePresident and Vote tables also... // the relations, tables have been defined, next load the data sqlite>.import President.txt President sqlite>.import VicePresident.txt VicePresident sqlite>.import Vote.txt Vote sqlite>.tables President VicePresident Vote

9 DBMS SQL JDBC 9 // count: how many entries are there for President? sqlite> select count(*) from President; 48 // select: who ran for President in 2000? sqlite> select Name, Year, Party from President where Year = 2000; Albert Gore 2000 Democratic Ralph Nader 2000 Green Harry Browne 2000 Libertarian Patrick Bucannan 2000 Reform George W Bush 2000 Republican

10 SQL DBMS SQL JDBC 10 SQL ("ess-que-el") Structured Query Language. SQL is used to communicate with a relational database. SQL statements are used to perform tasks :update or retrieve Relational database management systems that use SQL: Oracle, Sybase, Microsoft SQL Server, Access, MySql, Ingres Use SQL and have proprietary extensions SQL commands: DML Data Manipulation Language: SELECT, INSERT, UPDATE, DELETE, and other commands to load, read, data. DDL Data Definition Language: CREATE TABLE, ALTER TABLE, and others can change the database's design. DCL Data Control Language: GRANT, REVOKE, set privilege.

11 elections database mysql> SHOW DATABASES; Database information_schema elections mysql rows in set (0.00 sec) DBMS SQL JDBC 11 sqlite and mysql are case insensitive SQL commands shown uppercase sqlite commands shown lowercase sqlite shell has its own commands (lowercase only) ".<cmd>" sqlite3>.open elections.db mysql> USE elections; Database changed The elections mysql (and sqlite elections.db) database has 3 tables President(Name, Year, Party ) VicePresident ( Name, Year, Party ) Vote ( Party, Year, Percent )

12 DBMS SQL JDBC 12 mysql> SHOW TABLES; Tables_in_elections President VicePresident Vote rows in set (0.00 sec) sqlite>.table President VicePresident Vote sqlite>.schema President CREATE TABLE President( Name text not null, Party text not null, Year int not null, primary key (Name, Year)); mysql> DESCRIBE President; Field Type Null Key Default Extra Name varchar(30) NO PRI Party varchar(20) YES NULL Year int(4) NO PRI rows in set (0.00 sec)

13 DBMS SQL JDBC 13 The schema of tables below is shown with tabs and line feeds, as I created the elections.db from a text files (createtables.txt, President.txt, VicePresident.txt and Vote.txt). I had.mode tabs set so the data values are tab separated. sqlite>.schema CREATE TABLE President( Name text not null, Party text not null, Year int not null, primary key (Name, Year)); CREATE TABLE VicePresident( Name text not null, Party text not null, Year int not null, primary key (Name, Year)); CREATE TABLE Vote( Party text not null, Percent int not null, Year int not null, primary key (Party, Year));

14 SELECT SELECT column1 [,"column2",...] FROM tablename [WHERE condition]; DBMS SQL JDBC 14 [] = optional mysql> SELECT * FROM President; Name Party Year John Kennedy Democratic 1960 Richard Nixon Republican 1960 Lyndon Johnson Democratic Barak Obama Democratic 2008 John McCain Republican 2008 Ralph Nader Independent 2008 Bob Barr Libertarian rows in set (0.00 sec)

15 DBMS SQL JDBC 15 Note when using SQL commands (not sqlite shell commands) there is no "." before the command and the command terminates with a ";" sqlite> select * from President; Name Party Year Barack H. Obama Democratic 2012 Jill Stein Green 2012 Gary Johnson Libertarian 2012 Willard Mitt Romney Republican Lyndon Johnson Democratic 1964 Barry Goldwater Republican 1964 John Kennedy Democratic 1960 Richard Nixon Republican 1960

16 ORDER BY DBMS SQL JDBC 16 sqlite> select Name, Year from President order by Year; Name Year John Kennedy 1960 Richard Nixon 1960 Lyndon Johnson 1964 Barry Goldwater Barack H. Obama 2012 Jill Stein 2012 Gary Johnson 2012 Willard Mitt Romney 2012 Various sort orders can be used depending on column's data type.

17 WHERE DBMS SQL JDBC 17 mysql> SELECT Name, Year, Party FROM President WHERE Year = '2000'; Name Year Party Albert Gore 2000 Democratic Ralph Nader 2000 Green Harry Browne 2000 Libertarian Patrick Bucannan 2000 Reform George W Bush 2000 Republican rows in set (0.00 sec) mysql> SELECT COUNT (Year) FROM President; COUNT(Year) row in set (0.00 sec)

18 DBMS SQL JDBC 18 sqlite> select Name, Year, Party from President...> where Year = 2000; Name Year Party Albert Gore 2000 Democratic Ralph Nader 2000 Green Harry Browne 2000 Libertarian Patrick Bucannan 2000 Reform George W Bush 2000 Republican sqlite> select count (*) from President; count (*) The elections database for the mysql examples had data errors. The count for the sqlite election.db is "more" correct.

19 SQL Joins DBMS SQL JDBC 19 JOIN: Return rows when there is at least one match in both tables An INNER JOIN is the same as a JOIN. OUTER JOIN: return rows even when there are no matches through the JOIN critieria on the second table LEFT JOIN: Return all rows from the left table, even if there are no matches in the right table. A type of "outer join". RIGHT JOIN: Return all rows from the right table, even if there are no matches in the left table A type of "outer join" FULL JOIN: Return rows when there is a match in one of the tables A type of "outer join" CROSS JOIN: Cartesian product of the sets of rows from the joined tables. There is no "on condition". SQLite has three joins: cross, inner (default join), and left outer joins can be tricky, test/try them out, learn more about them...

20 Cross join DBMS SQL JDBC 20 Cross joins can return large tables the product of table attributes sqlite> select count (*) from President cross join Vote; count (*) President has 48 rows, Vote has 48 rows (2304 = 48 * 48) Does join, inner join, and left outer join give the same result for the above query with the election.db? Try it.

21 Join DBMS SQL JDBC 21 Query results can create "new tables" that are joins of existing tables. WHERE clause specifies the values to join tables on. mysql> SELECT President.Name, VicePresident.Name FROM President, VicePresident WHERE President.Year = VicePresident.Year AND President.Party = VicePresident.Party; Name Name John Kennedy Lyndon Johnson Richard Nixon Henry Lodge Lyndon Johnson Hubert Humphrey Barry Goldwater William Miller George Wallace Curtis LeMay

22 AS column alias DBMS SQL JDBC 22 PresidentName and VicePresidentName might be better attribute labels, wrt to queries, or use an alias sqlite> select President.Name as 'President',...> VicePresident.Name as 'Vice President'...> from President, VicePresident...> where President.Year = VicePresident.Year...> and President.Party = VicePresident.Party; President Vice President Barack H. Obama Joseph R. Biden Jill Stein Cheri Honkala Gary Johnson James P. Gray Willard Mitt Romney Paul Ryan... John Kennedy Lyndon Johnson Richard Nixon Henry Lodge

23 DISTINCT DBMS SQL JDBC 23 DISTINCT removes duplicate rows from results mysql> SELECT DISTINCT President.Party FROM President JOIN Vote; Party Democratic Republican American Libertarian Independent Green Reform rows in set (0.00 sec) sqlite> select distinct President.Party...> from President join Vote; Party Democratic Green Libertarian Republican Independent Reform American

24 sqlite> select distinct Name, Vote.Year...> from President, Vote...> where Name = 'Ralph Nader'; DBMS SQL JDBC 24 Name Year Ralph Nader 2012 Ralph Nader 2008 Ralph Nader 2000 Ralph Nader 1996 Ralph Nader 1992 Ralph Nader 1988 Ralph Nader 1984 Ralph Nader 1980 Ralph Nader 1976 Ralph Nader 1972 Ralph Nader 1968 Ralph Nader 1964 Ralph Nader 1960 implicit join creates too many entries for 'Ralph Nader'

25 ON DBMS SQL JDBC 25 ON clause determines columns to search for matches between tables. sqlite> select Name, President.Party, Vote.Percent,...> Vote.Year...> from President join Vote...> on President.Party = Vote.Party...> and President.Year = Vote.Year...> where Name = 'Ralph Nader'; Name Party Percent Year Ralph Nader Green Ralph Nader Green Ralph Nader Independent

26 INSERT DBMS SQL JDBC 26 Insert values into tables. When a new row is to be inserted the columns are not needed with values in column order. INSERT INTO tablename (column i,... column n VALUES (value i... value n ); sqlite> insert into President (Name, Party, Year)...> values ('Renzo Ghia', 'Yes please', 1999); sqlite> insert into President...> values ('Renzo Ghia', 'Halloween', 2010); sqlite> select * from President...> where Name = 'Renzo Ghia'; Name Party Year Renzo Ghia Yes please 1999 Renzo Ghia Halloween 2010

27 UPDATE DBMS SQL JDBC 27 When inserting values into tables that are related to other tables (foreign keys) all tables must have appropriate values inserted. UPDATE is used to alter (modify) tables UPDATE TableName SET columni = valuei,... WHERE...; sqlite> update President...> set Party = 'Halloween'...> where Year = 1999; sqlite> select * from President...> where Name = 'Renzo Ghia'; Name Party Year Renzo Ghia Halloween 1999 Renzo Ghia Halloween 2010

28 DELETE DBMS SQL JDBC 28 If not WHERE clause is used with UPDATE then all rows in table are updated. Delete a record from a table. Delete used w/o a WHERE clause deletes all rows from the table (no undo). DELETE FROM tablename WHERE...; mysql> DELETE FROM President WHERE Name = 'Renzo Ghia'; Query OK, 2 rows affected (0.00 sec) mysql> SELECT * FROM President WHERE Name = 'Renzo Ghia'; Empty set (0.00 sec) Deleting related records from a table. DELETE ColumnName i FROM TableName i WHERE condition i ;

29 ALTER, DROP DBMS SQL JDBC 29 Use ALTER to change a table's design. ALTER TABLE tablename CHANGE columnnameold columnname New columntype columnoptions; ALTER TABLE tablename ADD newcolumnname columntype columnoptions; ALTER TABLE tablename ADD newcolumnname columntype columnoptions; other add options: primary key, foreign keys, index, ALTER TABLE tablename DROP columnname; sqlite's alter can't add composite keys To delete a table DROP TABLE tablename;

30 CREATE DBMS SQL JDBC 30 There are GUI based mysql administration tools to use for large databases: phpmyadmin, MySqlWorkbench (updates MySqlAdministrator) CREATE DATABASE databasename; CREATE TABLE [IF NOT EXISTS] tablename ( columnname i, columntype i, columnoptions i ); mysql> CREATE TABLE Vote ( Party VARCHAR(20), int(2), Year int(4), PRIMARY KEY (Party, Year) ); mysql> CREATE TABLE President ( Name VARCHAR(30), Party VARCHAR(20), Year int(4), PRIMARY KEY (Name, Year) ); Percent mysql> CREATE TABLE VicePresident ( Name VARCHAR(30), Party VARCHAR(20), Year int(4), PRIMARY KEY (Name, Year) );

31 load data from file DBMS SQL JDBC 31 CREATE TABLE tablename SELECT * FROM columnname WHERE condition; Fill table with data from local file mysql> LOAD DATA LOCAL INFILE 'Vote.txt' INTO TABLE Vote; Query OK, 50 rows affected (0.00 sec) Records: 50 Deleted: 0 Skipped: 0 Warnings: 0 mysql> LOAD DATA LOCAL INFILE 'President.txt' INTO TABLE President;... mysql> LOAD DATA LOCAL INFILE 'VicePresident.txt' INTO TABLE VicePresident;... ========================================================== sqlite see page 8 for creating tables and loading them from files with sqlite shell. Files posted on class page.

32 SQL Backup / restore DBMS SQL JDBC 32 Create can be used to make a backup of tables CREATE TABLE backuptablename SELECT * FROM TableName; Restore a table from a backup DELETE FROM TableName; INSERT INTO TableName SELECT * FROM BackupTableName; Make a backup copy of entire database done outside mysql at command prompt. $ mysldump u loginname p databasename > backupfile Enter password: xxxx Restore a database. $ mysql u loginname p dbname < backupfile Enter password: xxxx

33 DBMS SQL JDBC 33 sqlite shell dump sqlite shell dump command can be used to create a text file of sqlite commands to create (re-create) the database. sqlite>.output election.bak // output sent to filename sqlite>.dump // selected, representative, lines from election.bak PRAGMA foreign_keys=off; BEGIN TRANSACTION; CREATE TABLE President( Name text not null, Party text not null, Year int not null, primary key (Name, Year)); INSERT INTO "President" VALUES('Barack H. Obama','Democratic',2012);... INSERT INTO "President" VALUES('Richard Nixon','Republican',1960); CREATE TABLE VicePresident( Name text not null, Party text not null, Year int not null, primary key (Name, Year)); INSERT INTO "VicePresident" VALUES('Joseph R. Biden','Democratic',2012);... COMMIT;.quit // too close and save output file

34 sqlite shell backup and restore DBMS SQL JDBC 34 sqlite shell backup command can be used to create a "binary" file backup of named or current "main" sqlite>.backup election.bak2 sqlite>.quit // restart sqlite3 sqlite>.open election.bak2 sqlite>.tables President VicePresident Vote sqlite>.schema CREATE TABLE President( Name text not null, Party text not null, Year int not null, primary key (Name, Year));... ===================================================================== Restore database from text file dump sqlite>.read election.bak sqlite>.tables President VicePresident Vote

35 Java + sqlite DBMS SQL JDBC 35 // Code fragments from posted MakeDB.java example import java.sql.*; public class MakeDB { Connection connect = null; Statement stmt = null; ResultSet rs = null; String sql = null; // connection to database // a SQL statement // values returned from SQL query // string for the sql statement public int insertdataobjects(int n) { // create values for id, age, name and gender here try { sql = String.format("insert into DataObject (ID, Name, Age, Gender) values (%d, '%s', %d, '%s')", id, name, age, gender); stmt.executeupdate(sql); } catch ( Exception e ) { System.err.println( e.getclass().getname() + " insert into table " + i + " " + e.getmessage() ); System.exit(0); } //... return i; }

36 create db and table DBMS SQL JDBC 36 public makedb() { try { Class.forName("org.sqlite.JDBC"); connect = DriverManager.getConnection("jdbc:sqlite:dataObject.db"); // create statement stmt = connect.createstatement(); // create table sql = "drop table if exists DataObject"; stmt.executeupdate(sql); sql = "create table if not exists DataObject " + "(ID int primary key not null," + " Name varchar(10) not null," + " Age int not null, " + " Gender varchar(10))"; stmt.executeupdate(sql); System.out.println("defined table DataObject"); } catch ( Exception e ) { System.err.println( e.getclass().getname() + ". attempt to create table: " + e.getmessage() ); System.exit(0); }

37 select query with result set DBMS SQL JDBC 37 try { // Query table rs = stmt.executequery( "select * from DataObject where Name = 'ginger';"); int id, age; String name, gender; while ( rs.next() ) { id = rs.getint("id"); name = rs.getstring("name"); age = rs.getint("age"); gender = rs.getstring("gender"); System.out.printf("%d \t %s \t %d \t %s \n", id, name, age, gender); } rs.close(); stmt.close(); } catch ( Exception e ) { System.err.println( e.getclass().getname() + ". attempt to query DataObject: " + e.getmessage() ); System.exit(0); }

38 DBMS Normalization DBMS SQL JDBC 38 Entity specification (aka relation) declares an entity s attributes Normalization process designs an optimum set of entity specifications for all types of modifications. create many simple entity specifications from many associations minimizes update anomalies and data redundancy single-valued dependencies: 1:1 primary key is the determinant A s value determines B s value (where B is all attributes) multi-valued dependencies: 1:N and M:N A determines many B and B determines single A A s value determines finite set of B values A determines many B and B determines many A A s value determines finite set of B values B s value determines finite set of A values

39 prof office example DBMS SQL JDBC 39 Prof ( pid, office, name, dept) Barnes CS Smith CS Gumb CS Wong EE Katz EE Schwartz EE Schwartz CS Member( office, dept) 414 CS 413 CS 404 CS 304 EE 402 EE pid name, office, dept office dept name pid office dept

40 single-valued dependencies (SVD) DBMS SQL JDBC 40 If attributes of an entity are determined by the value of the determinant the semantics (associations that exist among attribute values) can be represented in the data not in the application program (or database). functional partial transitive Boyce-Codd non-key attributes of entity are determined by the entire set of the primary key (not a subset). entity non-key attributes is dependent on a subset of the primary key. exist within non-key attributes of an entity when a functional dependency exists among themselves. exists when a non-key attribute determines part of a composite primary key set. Infrequent, not discussed.

41 prof office example improved DBMS SQL JDBC 41 Prof ( pid, office, name,) Barnes Smith Gumb Wong Katz Schwartz Schwartz Member( office, dept) pid pid name, office, office dept name 414 CS 413 CS 404 CS 304 EE 402 EE office dept

42 un-normalized example DBMS SQL JDBC 42 Entities: E1 ( A1, A2, A3, B, C ) E2 ( A1, A3, C ) E3 ( F, G, H, I ) E4 ( H, I ) F G H A1 A3 A2 B C I Normalization seeks to make all single-valued dependencies functional dependencies.

43 Normal Forms DBMS SQL JDBC 43 First Normal Form An entity specification is in first normal form if all its attributes are atomic values (not structures, or repeating groups). All "flat files" (common files, files with nonvariant records) are 1NF. All the entities on the previous slide are 1NF. Second Normal Form An entity specification is 2NF if it is 1NF and there are no partial dependencies. A partial dependency requires a composite key. There exists a partial dependency in E1, C is functionally dependent only on A1, A3 A2 is not required to determine C. E1.1 ( A1, A2, A3, B ) // remove C from E1. E2 ( A1, A3, C )

44 3NF DBMS SQL JDBC 44 Third Normal Form An entity specification is 3NF if it is 2NF and there are no transitive dependencies. E3 is not 3NF H is not a key (in E3) and it determines I E3.1 ( F, G, H ) // remove I from E3 Normalized example Entities: E1.1 ( A1, A2, A3, B ) E2 ( A1, A3, C ) E3.1 ( F, G, H ) E4 ( H, I ) F G H I A1 A3 A2 B C

45 multivalued dependencies DBMS SQL JDBC 45 A multivalued dependency (MVD) exists when key can determine more than one value (from a finite set of values) for another attribute. Texts ( bid, course, section) 20 cs cs cs cs cs cs cs cs cs cs cs A course could have multiple sections. Each section could require a different textbook. The same text could be used for multiple courses. bid course course bid course section course section bid

46 4NF DBMS SQL JDBC 46 Fourth Normal Form An entity specification is 4NF if it is BCNF and there is at most 1 MVD within the entity. violations of 4NF have high data redundancy with possible update anomalies Texts entity has 1 : N between course and section 1 : N between course and bid (book id) M : N between course and bid ClassTexts (bid, course) ClassSections ( course, section ) bid course course section section course 20 cs cs cs cs cs cs440 bid section course cs cs cs cs cs cs cs

47 5NF, DKNF DBMS SQL JDBC 47 Fifth Normal Form An entity specification is 5NF if it is 4NF and dependencies cannot be decomposed into simpler entity specifications w/o loss of information. Domain Key Normal Form Is the goal of normalization. All dependencies in the data are represented by keys determining nonkeys. Practically, SVDs are designed to remove violations of 3NF MVDs are designed to remove violations of 4NF. Complex entity specifications (stuff all attributes into 1 relation or 1 file) are prone to be poorly designed because dependencies can't be expressed as a function of primary keys. (BCNF and 5NF violations are infrequent and usually non-intuitive entity specifications.)

48 exercise DBMS SQL JDBC 48 Determine the normal form for each entity specification and normalize the set of entitites. R1 ( B, E, A, C, H ) R2 ( S, U, R, F ) R3 ( N, O, W ) ( B, E) A, C, H C H (S, U) R, F U F N O, W W O

49 references SQLite tutorial DBMS SQL JDBC 49 SQLite org, command syntax diagrams MySQL on-line reference manual On-line SQL guides, tutorials MongoDB, an open-source no SQL document database