This Session. Databases. Data Base Management System. Current DBMS Systems. Gives Different Views on Data. Data Base Management System Today

Transcription

1 This Session Databases I303, Session 11 Fall 2003 Thomas Haigh Introduction to Database Concepts Databases today Key points in text The four pillars of DB education Origins of Database Data base concept File management DBMS concept Advantages and limitation of DBMS in practice Data Warehouse concept (if time ) i303, Session 11, Thomas Haigh 1 i303, Session 11, Thomas Haigh 2 Data Base Management System Today Single most important kind of corporate IT Foundation of almost every Advanced web site Administrative application Increasing use in science Experimental results, clinical trials, etc. Data acquired from equipment Gene sequences, etc. Data Base Management System Standard piece of system software Oracle, SQL Server, DB2, Access are most widely used Supports multiple data bases Creation, modification of data structures (i.e. tables) Via Data Definition Language (DDL) Retrieval, insertion, deletion, updating of data (i.e. rows) Via Data Manipulation Language (DML) i303, Session 11, Thomas Haigh 3 i303, Session 11, Thomas Haigh 4 Current DBMS Systems Are relational Follow relational model Use SQL (Structured Query Language) to define and manipulate data Support multiple simultaneous users Can be ad-hoc individuals Can be batch jobs or programs Have many advanced capabilities E.g. code triggered when data changes Transactions to protect against data loss i303, Session 11, Thomas Haigh 5 Gives Different Views on Data Different users have different permissions View, change, delete On various parts of the data base Views are used to present Data joined, grouped or filtered in particular ways Can include results of calculations or functions This allows Avoidance of duplicated data Store it once; present in many different ways Called normalization i303, Session 11, Thomas Haigh 6

2 Key Concepts from Textbook Don t Worry about Data Base Data Base Management System Logical vs. Physical design Schemas, etc Types of Database General stuff on data HTML and Web stuff ( ) But search engine vs. database matters Models stuff ( ) Data Access Methods ( ) ERD (141-44) Very important, but needs more space i303, Session 11, Thomas Haigh 7 i303, Session 11, Thomas Haigh 8 4 Pillars of Database Education DBMS Concept Relational Model SQL Entity Relationship Modeling I 308 covers all of these items Advanced DB course is good idea 1: Database Management System DBMS as software layer between data, users Different interfaces, languages for Programs & programmers Ad-hoc managerial reporting Data definition and maintenance i303, Session 11, Thomas Haigh 9 i303, Session 11, Thomas Haigh 10 2: The Relational Model 3: SQL E. F. Codd, 1970 Simple, elegant, mathematically grounded Abstracts data from underlying representations Relations specified by query, not by DDL Looking back on it, I don't think the problem we thought we were solving was where we had the most impact. What we thought we were doing was making it possible for non-programmers to interact with databases. Don Chamberlin System R SQL Language group, i303, Session 11, Thomas Haigh 11 i303, Session 11, Thomas Haigh 12

3 4: Entity Relationship Modeling Formulated by Chen, 1976 Links database entities to real-world functions and processes Easy to convert to relational design i303, Session 11, Thomas Haigh 13 Origin of DB Concept Data base concept of military system origin Probable source is SDC circa 1960 a RAND corporation spin-off doing USAF systems SDC made deliberate attempts to promote systems approach for corporate management Crucial Characteristics Constantly updated Accessed interactively ( real-time ) Shared between users/systems, gives different views to each i303, Session 11, Thomas Haigh 14 Information Retrieval DB in Corporate World New concept circa 1950 New technologies & techniques for searching data Tied to cold war information explosion Increasingly associated with computer & electronics Contemporaneous with Information Theory (late 1940s) Information Science (coined 1959?) Information Technology (1958) Discussion of information in generalized way is new, particularly to business i303, Session 11, Thomas Haigh 15 Data base concept crosses over to corporate use in early 1960s Total Systems Concept (aka Management Information System) Integrated command system for management real time, on-line Data base gains scattered usage as crucial, shared bucket of facts for this system Discussion of DB predates discussion of DBMS by about seven years. i303, Session 11, Thomas Haigh 16 Data Base and MIS MIS relies on a body of data, a veritable bucket of facts, [as] the source into which information seeking ladles of various sizes and shapes are thrust in different locations. (Milt Stone, 1959) Variations in 1961/1962: data hub, data bank, pool of information Data Base spreads in mid-1960s The Corporate War Room the warroom atmosphere is growing up fast (The Corporate Command Post, 1968) one by one, the same applications that are pioneered and proven in military use ultimately find their way into business (1968) Information shared i303, Session between 11, Thomas systems Haigh 17 i303, Session 11, Thomas Haigh 18

4 a Utopian Vision Widener, W. Robert. "New Concepts of Running a Business." Business Automation 13, no. 4 (1966): 38-43, 63. a more relaxed, leisurely management environment. The uneasiness will be replaced by a feeling of confidence in the completeness and timeliness of information and in the decisions based on that information. i303, Session 11, Thomas Haigh 19 In contrast, Data Processing Tasks Payroll, accounting, invoicing Taking over jobs from existing punched card machines Slow evolution hardware of hardware, practice Intended to automate clerical work Success means replacing clerks Justified on basis of lower operating costs i303, Session 11, Thomas Haigh 20 A File Based System, 1962 File Management Software As old as corporate computing First documented in GE, mid-1950s Generalized set of subroutines to update, query, maintain sequential files By mid-1960s, becoming more sophisticated Offered as commercial products Working with new random-access devices Mark IV (Informatics) is huge success Also IDS (GE), IMS (IBM) i303, Session 11, Thomas Haigh 21 i303, Session 11, Thomas Haigh 22 State of Play circa 1967 Data base concept is Fashionable Widely promoted as key to MIS Vaporware, revolutionary Real-time, on-line, total system Closely tied to information retrieval File management software is Growth area Data processing tool (batch mode) Practical, batch-oriented, evolutionary i303, Session 11, Thomas Haigh 23 The DBMS & CODASYL New concept Data Base Management System appears circa 1968 CODASYL Data Base Task Group Originally in context of extensions to COBOL Based on consideration of current file management products, directions for future. One system must offer Real Time & Batch operation Capabilities for programmers Ability to query directly i303, Session 11, Thomas Haigh 24

5 DBMS Foundational Concept DBMS as software layer between data, users Different interfaces, languages for Programs & programmers Ad-hoc managerial reporting Data definition maintenance and administration Sets up links between files BUT rigid, standardized format remain ANSI-SPARC Three-level Architecture i303, Session 11, Thomas Haigh 25 i303, Session 11, Thomas Haigh 26 DBMS as a Product Components of a DBMS Term DBMS applied widely to new & existing products CODASYL standard influential but not dominant Guides evolution of packages DBMS key part of software industry TOTAL, IDMS, SYSTEM 2000, IMS (IBM) Even in late 1970s, used mostly in batch mode Real-time very inefficient Big cost in hardware and software New specialists needed to configure i303, Session 11, Thomas Haigh 27 i303, Session 11, Thomas Haigh 28 Hopes for MIS reborn with DB Writings on MIS have waned recently and have largely been replaced by writings on the Data Base (1973) The Data Base Administrator Originally expected to take responsibility for data as a resource much broader than machine readable data (1974) something of a superstar (1975) DBMS technology expected to build DBMS usages in the 1970s Advantages mostly for programmers easier reporting, Program/data independence faster application development, easier maintenance better integration of different applications Integration proves harder than expected Help with conversion to disk and multitasking operating system integrated, company wide DB i303, Session 11, Thomas Haigh 29 i303, Session 11, Thomas Haigh 30

6 Post 1980: DBMS Concept Spreads Shift to relational model Devised in 1970s, spreads in 1980s SQL emerges as standard Costs lower, performance improves But still tool mostly of new programmers Extension to new kinds of hardware Minicomputers Microcomputers Pocket computers! i303, Session 11, Thomas Haigh 31 Impact of Relational Model SQL becomes more of a tool for programmers and interoperability than end users Some separation of joins between tables (relations) from definition of tables Makes joining information in different ways easier, which makes big DBs more flexible Also makes querying harder Relational systems gradually gain ground Efficiency improves; hardware costs drop Become dominant on minicomputers & UNIX Much mainframe data remains in non-relational formats today i303, Session 11, Thomas Haigh 32 Not One Big Database Big central database doesn t work Finish up with dozens/hundreds of little data bases Physically separate All incomplete Different data formats Different concepts of data Dominant model is relational (eg Oracle) Good for updating Flexible Can be slow & complex to extract data for reports i303, Session 11, Thomas Haigh 33 DBMS as Information Technology Compared to 1960s data base ideas New concept of database is narrower More general information retrieval problems are excluded DBMS is not well suited for Irregular records Full text or even keyword searching Ad-hoc linkages between records Context, relevance (in IS terms) Only with search engines of 90s Is much attention given to unstructured data i303, Session 11, Thomas Haigh 34 Implications Despite IR, IT, etc. hard to deal with information in general Routine administrative (dominant in business use) file management, DBMS Scientific and bibliographical (library) specialized on-line system In practice, data bases fragment New challenge is reuniting them! New dreams of integrated systems Data warehouse (reporting) Enterprise Resource Planning (operational) i303, Session 11, Thomas Haigh 35 From Recent DB Textbook Management Hierarchy Top (strategic) Middle (tactical) Lower (operational) External data sources and summarized, tactical databases Summarized, integrated operational databases Individual operational databases Operational databases i303, Session 11, Thomas Haigh 36

7 Data Warehouse Concept Emerges early 1990s One big DB for everything has failed, so Leave transactional systems spread out (physically, organizationally), BUT Make a second, read-only copy of everything in a centralized data warehouse. Update regularly. Lots of work for consultants Copying, cleaning, reformatting data Restructuring data around business areas and for easy querying Providing query tools for managerial users i303, Session 11, Thomas Haigh 37 Data Warehouse Information Flows i303, Session 11, Thomas Haigh 38