Data Warehousing. Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig

Size: px
Start display at page:

Download "Data Warehousing. Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs."

Transcription

1 Data Warehousing & Mining Techniques Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig

2 2. Architecture 2. Architecture 2.1 Basics 2.2 Storage structures 2.3 Tier architectures 2.4 Distributed DW 2.5 Middleware Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 2

3 2.1 Basics Architecture of a DW Data is stored in a predefined database Maintenance of the database is performed as in OLTP by a DBMS Usual functionality of the database is ensured Storage, Update, Delete, Locate Individually Structured Departmentally Structured Organizationally Structured Source Systems Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 3

4 2.1 Basics: Databases & DBMS Databases & DBMS Application SELECT id FROM revenues WHERE val> DBMS Disk 1 Disk N ID VAL Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 4

5 2.1 Basics: Indexing DW is characterized by Large volume of information Mostly used for reading the information and not for updating or deleting operations outside the ETL phase This characteristics suggest indexes as a must have in DW so let s remember indexes Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 5

6 2.1 Basics: Indexing Indexes are additional data structures which help locating records in a DB Creation of indexes is part of the physical tuning task of the DB administrators Indexes can influence the actual location of storage for a record Sequential storage, or via a hash function If the location is determined by the index not all attributes can be directly indexed (primary vs. secondary indexes) Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 6

7 2.1 Basics: Indexing Indexes are useful for speeding up access to the data They are ordered by indexing field (search key) search key is the attribute used to look up records into a file An index file consists of index entries Records of the form search key location Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 7

8 2.1 Primary Index Primary indexes Order data by some unique attribute as indexing field (primary key), store database records in this order An index record contains a pointer to the respective storage place Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 8

9 2.1 Secondary Index Secondary indexes point to locations of records regarding non-ordering attribute Indexing does not affect storage order There can be multiple secondary indexes for the same DB file Secondary indexes are usually dense Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 9

10 2.1 Secondary Index Characteristics of secondary indexes Speeds up retrieval, if secondary index on the searched attribute does not exist, the entire file has to be searched linearly Use more time and space, because they are dense Provide logical ordering Accessing records in this order might not be the most efficient way regarding block accesses In DW, due to the large amount of data, multilevel ordered indexes are used Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 10

11 2.1 Indexes Here s a great idea: Why not index every attribute? Have a physical index on the primary key, and logical indexes on every other attribute This results in good read efficiency, but really terrible write/update efficiency But Data Warehouses only need good read efficiency? Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 11

12 2.1 Indexes Whenever a DB is modified, most of the indexes have to be updated This result in a large amount of overhead on operations like insert, delete or update If the indexes are multi-level every level has to be updated Why should we care? We have a DW not an OLTP system The majority of the operations in a DW are reads But remember ETL? We should use considerable more indexes than in OLTP, but loading data into the DW should not last forever!!! Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 12

13 2.1 Indexes in DW In DW the underlying technology has to support Creation and loading of new indexes Efficient access to the indexes Efficient access can be accomplished in different ways Using bit maps Having multi-leveled indexes Storing all or parts of an index in main memory Compacting the index entries when the order of the data being indexed allows such compaction Creating selective indexes and range indexes Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 13

14 2.1 Indexes in DW Recommended index structures are: B-tree indexes, on high cardinality attribute columns (due to the bushy nature of B-Trees) Bitmap indexes on all medium and low cardinality attributes Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 14

15 2.1 B-Trees Rel. DB 2: Basic structure of a B-Tree node Node contains key values and respective data (block) pointers to the actual data records Additionally, there are node pointers for the left respectively right interval around a key value Key Value Data Pointer Tree Node Node Pointers Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 15

16 2.1 Bitmap indexes Bitmap indexes: Work well with small number of distinct values E.g., gender data Identifier Gender Bitmaps 1 Female Female Male Unspecified Male 0 1 Have a significant space and performance advantage over other structures for this type of data Useful in DW for joining a large fact table to smaller dimension tables F M Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 16

17 2.1 Basic architecture Architecture of a DW Data Sources Staging Area Warehouse Data Marts Users Operational System Purchasing Analysis Operational System Summary Data Metadata Raw Data Sales Reporting Flat files Inventory Mining Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 17

18 2.1 Basic architecture The Data Staging Area Is both a storage and process area (the ETL process) It represents everything that Data Sources Staging Area Warehouse Data Marts happens between the operational source system and the data presentation area The key architectural requirement for data staging area is that it is off-limits to business users and does not provide query and presentation services Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 18

19 2.1 Basic architecture Customers aren t invited to visit the kitchen Similar to a restaurant s kitchen, the data staging area should be accessible only to skilled professionals Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 19

20 2.1 Basic architecture The Data Presentation Area Is where data is organized, stored and made available for queries, report writers, and other analytical processing Data Sources Staging Area Warehouse This area is the Warehouse as far as the business community is concerned Data Marts Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 20

21 2.2 Storage structures Storage structure After extraction from the operational data, in DW information is stored in databases The databases are operated by a DBMS Different database structures can be used for a DW: Relational model (RDB) operated by a RDBMS MultiDimensional model (MDB) operated by a MDBMS Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 21

22 2.2 Storage structures RDB and MDB are complementary and do not have to exclude each other In the staging area some RDBMS can be used, however it must be off-limits to user queries because of performance reasons By default, normalized databases are excluded from the presentation area, which should be strictly multi-dimensionally (MDBMS) Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 22

23 2.2 Relational DB DB in relational model A database is seen as a collection of predicates over a finite set of variables The content of the DB is modeled as a set of relations in which all predicates are satisfied Books Title ISBN (PK) Price Publisher(FK) Category (FK) Publisher Name ID (PK) BookCategory Cat_ID(PK) Description Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 23

24 2.2 Relational DB A relation is defined as a set of tuples that have the same attributes It is usually described as a table Attribute Tuple Relation Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 24

25 2.2 Multidimensional DB A Multidimensional DB (MDB) is optimized for DW and OLAP applications They are created using input from the staging area Their purpose is to answer questions like How many Nokia 5800 have we sold so far this year in Braunschweig? MDBs are RDBS optimized for OLAP queries Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 25

26 2.2 Multidimensional DB MDB are Designed for efficient and convenient storage and retrieval of large volumes of data Stored, viewed and analyzed from different perspectives called dimensions Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 26

27 2.2 Multidimensional DB MDB example An automobile manufacturer wants to increase sale volumes Evaluation requires to view historical sale volume figures from multiple dimensions Sales volume by model, by color, by dealer, over time Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 27

28 2.2 Multidimensional DB A relational structure of the given evaluation would be Model Color Sales volume Mini VAN Blue 324 Mini VAN Black 113 Mini VAN Red 18 Sedan Black 160 Sedan Blue 115 Sedan Red 6 Sports coupe Red 16 Sports coupe Black 16 Sports coupe Blue 12 Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 28

29 2.2 Multidimensional structure * Mini VAN Sedan Coupe Black Blue Red * Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 29

30 2.2 Multidimensional DB The complexity grows quickly with the number of dimensions and the number of positions Example: 3 dimensions with 10 values each and no indexes If we consider viewing information in a RDB it would result in a worst case of 10 3 =1000 records view Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 30

31 2.2 Multidimensional DB Now, if we consider performance For responding to a query when car type = Sedan, color = Blue, and dealer = Berg RDBMS has to search through 1000 records to find the right record MDB has more knowledge about where data lies The maximum of searches in the case of MDB is of 30 positions Average case 18 vs. 501 positions Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 31

32 2.2 Multidimensional DB If the query is more relaxed Total sales across all dealers for all colors when car type = sedan RDBMS still has to go through the 1000 records MDB, however, goes only through a slice of 10x10 Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 32

33 2.2 Multidimensional DB Performance advantages MDBs are an order of magnitude faster than RDBMSs Performance benefits are more for queries that generate cross-tab views of data (the case of DW) Conclusion The performance advantages offered by MDBs facilitates the development of interactive decision support applications like OLAP that can be impractical in a relational environment Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 33

34 2.2 RDB vs. MDB Any database manipulation is possible with both technologies MDBs however offer some advantages in the context of DW: Ease of data presentation Ease of maintenance Performance Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 34

35 2.2 RDB vs. MDB Ease of data presentation Data views are natural output of the MDBs Obtaining the same views in RDB requires a complex query Example with Walmart and Sybase: select sum(sales.quantity_sold) from sales, products, product_categories, manufacturers, stores, cities where manufacturer_name = Colgate and product_category_name = toothpaste and cities.population < and trunc(sales.date_time_of_sale) = trunc(sysdate-1) and sales.product_id = products.product_id and sales.store_id = stores.store_id and products.product_category_id = product_categories.product_category_id and products.manufacturer_id = manufacturers.manufacturer_id and stores.city_id = cities.city_id Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 35

36 2.2 RDB vs. MDB Ease of data presentation Top k queries cannot be expressed well in SQL Find the five cheapest hotels in Frankfurt SELECT * FROM hotels h WHERE h.city = Frankfurt AND 5 > (SELECT count(*) FROM hotels h1 WHERE h1.city = Frankfurt AND h1.price < h.price); Some RDBMS extended the functionality of SQL with STOP AFTER functionality SELECT * FROM hotels WHERE city = Frankfurt Order By price STOP AFTER 5; Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 36

37 2.2 RDB vs. MDB Ease of maintenance No additional overhead to translate user queries into requests for data Data is stored as it is viewed RDBs use indexes and sophisticated joins which require significant maintenance and storage to provide same intuitiveness Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 37

38 2.2 RDB vs. MDB Performance Performance of MDBs can be matched by RDBs through database tuning Not possible to tune the database for all possible ad- hoc queries Aggregate navigators are helping RDBs to catch up with MDBs as far as aggregation queries are concerned Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 38

39 2.2 MDB When MDBs are in-appropriate? If the dataset types are not highly related, using a MDB results in a sparse representation Mini VAN Smith 34 Sedan James Coupe Fox 45 Black Blue Red Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 39

40 2.2 MDB When MDBs are appropriate? In the case of highly interrelated dataset types MDBs are recommended for greatest ease of access and analysis Examples of applications Financial Analysis and Reporting Budgeting Promotion Tracking Quality Assurance and Quality Control Product Profitability Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 40

41 2.3 Tier architectures Popular DW architectures Generic Two-Tier Architecture Independent Data Mart Dependent Data Mart and Operational Data Store Logical Data Mart and Active Warehouse Three-Tier Architecture Other One-Tier Architecture N-Tier Architecture Web-based Architecture Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 41

42 2.3 Layered architectures Generic Two-Tier Architecture Data is not completely current in the DW Periodic extraction Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 42

43 2.3 Layered architectures Data analysis comes in two flavors Depending on the execution place of the analysis Thin Client Analytics are executed on the server Client just displays This architecture fits well for Internet/Intranet DW access Client HTTP, IIOP Server Analysis Data storage Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 43

44 2.3 Layered architectures Fat Client The server just delivers the data Analytics are executed on the client Communication between client and server must be able to sustain large data transfers ODBC, JDBC, NFS Client Server Analysis Data storage Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 44

45 2.3 Layered architectures Independent Data Mart Mini warehouses limited in scope Separate ETL for each independent Data Mart High Data Marts access complexity Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 45

46 2.3 Layered architectures Dependent Data Mart and Operational Data Store Single ETL for the DW Data Marts are loaded from the DW More simple data access than in the previous case Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 46

47 2.3 Layered architectures Logical Data Mart and Active Warehouse The ETL is near real-time Data Marts are not separate databases, but logical views of the DW Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 47

48 2.3 DW vs. Data Marts DW Application independent Centralized, Planned DW Historical, detailed, summarized Lightly denormalized Scope Data Marts Specific DSS application Decentralized by user area Organic, possibly not planned Data Data Marts Some history,detailed, summarized Highly denormalized DW Multiple subjects DW Many internal and external sources DW Flexible Data-oriented Long life Large Single complex structure Subjects Sources Data Marts One central subject Data Marts Few internal and external sources Other characteristics Data Marts Restrictive Project oriented Short life Start small, becomes large Multiple,semi-complex structure, together complex Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 48

49 2.3 Layered architectures Generic Three-Tier Architecture Derived data Data that had been selected, formatted, and aggregated for DSS support Reconciled data Detailed, current data intended to be the single, authoritative source for all decision support Derived data Data Mart Reconciled data DW and ODS Operational data Operational data Data mart metadata DW metadata Operational metadata Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 49

50 2.3 Layered architectures One-Tier Architecture Theoretically possible Might be interesting for mobile applications N-Tier Architecture Higher tier architecture is also possible But the complexity grows with the number of tier-interfaces Web-based Architecture Advantages: Usage of existing software, reduction of costs, platform independence Disadvantages: Security issues: data encryption/user access and identification Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 50

51 2.4 Distributed DW In most cases the economics and technology greatly favor a single centralized DW But in some cases, distributed DW make sense Types of distributed DW Geographically distributed Local DW/global DW Technologically distributed DW Logically one DW, physically more DW Independently evolving distributed DW Uncontrolled growth Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 51

52 2.4 Distributed DW Geographically distributed In the case of corporations spread around the world Information is needed both locally and globally A distributed DW makes sense When much processing occurs at the local level Even though local branches report to the same balance sheet, the local organizations are their own companies Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 52

53 2.4 Distributed DW All IBM Europe Site A Local DW Local operational processing IBM/Teradata USA Local DW HQ Local operational processing Sybase Asia Site B Local DW Local operational processing Global DW Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 53

54 2.4 Distributed DW Technologically distributed DW Placing the DW on the distributed technology of a vendor Advantages The entry cost is cheap large centralized hardware is expensive No theoretical limit to how much data can be placed in the DW we can add new servers to the network Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 54

55 2.4 Distributed DW As the DW starts to expand network data communication starts playing an important role Example: Let s simplify and consider we have 4 nodes holding each data regarding the last 4 years Now let s consider we have a query which needs to access the data from the last 4 years: such a query arises the issue of transporting large amount of data between processors Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 55

56 2.4 Distributed DW Independently evolving distributed DW In practice there are many cases in which independent DW are developed concurrently and uncontrolled in the same organization The first step many corporations make is to build a DW for financial or marketing Once it is successfully set up, other parts of the organization follow independently the process resulting in the coexistence of more independent DW in the same organization This problem will be addressed later Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 56

57 2.5 Middleware Middleware-Systems Provide an inter-connectivity layer between heterogeneous platforms and the applications that come on top Application Application Application APIs Middleware-System Platform-interface Platform -Hardware - Operating System Platform-interface Platform -Hardware - Operating System Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 57

58 2.5 Middleware Middleware in DW? DW usually implies Heterogeneous hardware, databases, operating systems, networks and applications Middleware serves both users and developers It shields both users and developers from differences in services and resources used by applications Without middleware Changes at the lower layers could imply propagating changes by updating the higher layers Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 58

59 2.5 Middleware Roles of Middleware Assist the developer in ETL and populating the DW Assist DW users in accessing the DW It is therefore needed at different points in the life cycle Types: Copy management: data extraction, transformation, Gateways: DB and independent gateways Program to program: RPC, ORBs Message oriented Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 59

60 2.5 Middleware Most common middleware technologies CORBA (Common Object Request Broker Architecture) DCOM (Distributed Component Object Model) J2EE in DW Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 60

61 2.5 Middleware CORBA Mechanism for normalizing method-call semantics between application objects on the same host or on remote host (client) main() Object reference (server) main() Object implementation reference ORB vendor code ORB vendor-tool generated code User-defined application code Generated stub code ORB network Generated skeleton code ORB Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 61

62 2.5 Middleware ORB Is a middleware technology that manages communication and data exchange between objects in object-oriented programming and databases ORB Client app Establish connection Client Service communication Object implementation (service) Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 62

63 2.5 Middleware Client The application program that invokes a method or operation on an object implementation Stub Precompiled interface between the client and the ORB, generated by the ORB tool ORB An interface containing help functions and APIs that can be used by a client or an object implementation BOA (Basic Object Adapter) Refers to the part of the ORB responsible for managing server-side operations Replaced by the POA (Portable Object Adapter) Skeleton The server-side analogue of stubs Implementation Called a service or method in object-oriented terminology, defines the operations supporting an interface definition language(idl) Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 63

64 2.5 How CORBA works 1. the client makes the call through the stub to the ORB 2. the ORB dispatches the call to the BOA, that does the object activation 3. the implementation registers itself, if necessary, and declares itself ready 4. the BOA, now signaled ready, invokes the implementation via the skeleton from IDL 5. a response or exception propagates up to the client caller. Client Stub ORB BOA Skeleton Implementation Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 64

65 2.5 Middleware CORBA in DW Query Service Supports SQL and OQL Object Transaction Service Ensure correct state of transactional objects Distributed commit/rollback Guarantees ACID properties It is able to send copies of multidimensional data Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 65

66 2.5 Middleware DCOM Microsoft's concurrent for CORBA Can access distributed stored data through ADO (ActiveX Data Objects) ADO uses for the actual database access OLE DB (Object Linking and Embeding DB) and ODBC (Open DB Conectivity) There is also a multidimensional ADO ADO MD It contains objects for communication of data cubes Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 66

67 2.5 Middleware J2EE in DW Not fit for storage and analyze of a multidimensional DB JOLAP offers a programming interface for analytical access to the DW A Java community initiative, sustained by SUN and Oracle Lack of effective support OLAP4J Is simply put a multidimensional JDBC Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 67

68 2.5 Middleware So why is middleware important? Heterogeneous Hardware, Data sources, Data targets, Platforms, Operating systems Communication protocols Connectivity Platform and Application independence Support of standard protocols and interfaces Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 68

69 Next lecture Modeling Basics of data modeling Data models in DW Data Warehousing & OLAP Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 69

2.1 Basics: Indexing. 2.1 Primary Index. 2.1 Secondary Index. 2.1 Secondary Index. 2.1 Indexes. 2.1 Indexes 14.04.2009.

2.1 Basics: Indexing. 2.1 Primary Index. 2.1 Secondary Index. 2.1 Secondary Index. 2.1 Indexes. 2.1 Indexes 14.04.2009. 2. Architecture Data Warehousing & Mining Techniques Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 2. Architecture 2.1

More information

Data Warehousing. Jens Teubner, TU Dortmund jens.teubner@cs.tu-dortmund.de. Winter 2015/16. Jens Teubner Data Warehousing Winter 2015/16 1

Data Warehousing. Jens Teubner, TU Dortmund jens.teubner@cs.tu-dortmund.de. Winter 2015/16. Jens Teubner Data Warehousing Winter 2015/16 1 Jens Teubner Data Warehousing Winter 2015/16 1 Data Warehousing Jens Teubner, TU Dortmund jens.teubner@cs.tu-dortmund.de Winter 2015/16 Jens Teubner Data Warehousing Winter 2015/16 13 Part II Overview

More information

Introduction. Introduction: Database management system. Introduction: DBS concepts & architecture. Introduction: DBS versus File system

Introduction. Introduction: Database management system. Introduction: DBS concepts & architecture. Introduction: DBS versus File system Introduction: management system Introduction s vs. files Basic concepts Brief history of databases Architectures & languages System User / Programmer Application program Software to process queries Software

More information

BUILDING BLOCKS OF DATAWAREHOUSE. G.Lakshmi Priya & Razia Sultana.A Assistant Professor/IT

BUILDING BLOCKS OF DATAWAREHOUSE. G.Lakshmi Priya & Razia Sultana.A Assistant Professor/IT BUILDING BLOCKS OF DATAWAREHOUSE G.Lakshmi Priya & Razia Sultana.A Assistant Professor/IT 1 Data Warehouse Subject Oriented Organized around major subjects, such as customer, product, sales. Focusing on

More information

Introduction: Database management system

Introduction: Database management system Introduction Databases vs. files Basic concepts Brief history of databases Architectures & languages Introduction: Database management system User / Programmer Database System Application program Software

More information

B.Sc (Computer Science) Database Management Systems UNIT-V

B.Sc (Computer Science) Database Management Systems UNIT-V 1 B.Sc (Computer Science) Database Management Systems UNIT-V Business Intelligence? Business intelligence is a term used to describe a comprehensive cohesive and integrated set of tools and process used

More information

DATABASE MANAGEMENT SYSTEM

DATABASE MANAGEMENT SYSTEM REVIEW ARTICLE DATABASE MANAGEMENT SYSTEM Sweta Singh Assistant Professor, Faculty of Management Studies, BHU, Varanasi, India E-mail: sweta.v.singh27@gmail.com ABSTRACT Today, more than at any previous

More information

Introduction to Oracle Business Intelligence Standard Edition One. Mike Donohue Senior Manager, Product Management Oracle Business Intelligence

Introduction to Oracle Business Intelligence Standard Edition One. Mike Donohue Senior Manager, Product Management Oracle Business Intelligence Introduction to Oracle Business Intelligence Standard Edition One Mike Donohue Senior Manager, Product Management Oracle Business Intelligence The following is intended to outline our general product direction.

More information

DATA WAREHOUSING AND OLAP TECHNOLOGY

DATA WAREHOUSING AND OLAP TECHNOLOGY DATA WAREHOUSING AND OLAP TECHNOLOGY Manya Sethi MCA Final Year Amity University, Uttar Pradesh Under Guidance of Ms. Shruti Nagpal Abstract DATA WAREHOUSING and Online Analytical Processing (OLAP) are

More information

ICOM 6005 Database Management Systems Design. Dr. Manuel Rodríguez Martínez Electrical and Computer Engineering Department Lecture 2 August 23, 2001

ICOM 6005 Database Management Systems Design. Dr. Manuel Rodríguez Martínez Electrical and Computer Engineering Department Lecture 2 August 23, 2001 ICOM 6005 Database Management Systems Design Dr. Manuel Rodríguez Martínez Electrical and Computer Engineering Department Lecture 2 August 23, 2001 Readings Read Chapter 1 of text book ICOM 6005 Dr. Manuel

More information

When to consider OLAP?

When to consider OLAP? When to consider OLAP? Author: Prakash Kewalramani Organization: Evaltech, Inc. Evaltech Research Group, Data Warehousing Practice. Date: 03/10/08 Email: erg@evaltech.com Abstract: Do you need an OLAP

More information

1 File Processing Systems

1 File Processing Systems COMP 378 Database Systems Notes for Chapter 1 of Database System Concepts Introduction A database management system (DBMS) is a collection of data and an integrated set of programs that access that data.

More information

Chapter 3. Database Environment - Objectives. Multi-user DBMS Architectures. Teleprocessing. File-Server

Chapter 3. Database Environment - Objectives. Multi-user DBMS Architectures. Teleprocessing. File-Server Chapter 3 Database Architectures and the Web Transparencies Database Environment - Objectives The meaning of the client server architecture and the advantages of this type of architecture for a DBMS. The

More information

Data Warehousing: Data Models and OLAP operations. By Kishore Jaladi kishorejaladi@yahoo.com

Data Warehousing: Data Models and OLAP operations. By Kishore Jaladi kishorejaladi@yahoo.com Data Warehousing: Data Models and OLAP operations By Kishore Jaladi kishorejaladi@yahoo.com Topics Covered 1. Understanding the term Data Warehousing 2. Three-tier Decision Support Systems 3. Approaches

More information

Indexing Techniques for Data Warehouses Queries. Abstract

Indexing Techniques for Data Warehouses Queries. Abstract Indexing Techniques for Data Warehouses Queries Sirirut Vanichayobon Le Gruenwald The University of Oklahoma School of Computer Science Norman, OK, 739 sirirut@cs.ou.edu gruenwal@cs.ou.edu Abstract Recently,

More information

GEOG 482/582 : GIS Data Management. Lesson 10: Enterprise GIS Data Management Strategies GEOG 482/582 / My Course / University of Washington

GEOG 482/582 : GIS Data Management. Lesson 10: Enterprise GIS Data Management Strategies GEOG 482/582 / My Course / University of Washington GEOG 482/582 : GIS Data Management Lesson 10: Enterprise GIS Data Management Strategies Overview Learning Objective Questions: 1. What are challenges for multi-user database environments? 2. What is Enterprise

More information

OLAP and OLTP. AMIT KUMAR BINDAL Associate Professor M M U MULLANA

OLAP and OLTP. AMIT KUMAR BINDAL Associate Professor M M U MULLANA OLAP and OLTP AMIT KUMAR BINDAL Associate Professor Databases Databases are developed on the IDEA that DATA is one of the critical materials of the Information Age Information, which is created by data,

More information

Understanding Client/Server Computing

Understanding Client/Server Computing Understanding Client/Server Computing By Will Fastie PC Magazine; "Enterprize Computing;" Back in the eighties when network equipment vendors first started to hawk their wares, the term client/server was

More information

Relational Database Systems 2 1. System Architecture

Relational Database Systems 2 1. System Architecture Relational Database Systems 2 1. System Architecture Wolf-Tilo Balke Philipp Wille Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 1 Organizational Issues

More information

Multi-dimensional index structures Part I: motivation

Multi-dimensional index structures Part I: motivation Multi-dimensional index structures Part I: motivation 144 Motivation: Data Warehouse A definition A data warehouse is a repository of integrated enterprise data. A data warehouse is used specifically for

More information

Enterprise Application Integration

Enterprise Application Integration Enterprise Integration By William Tse MSc Computer Science Enterprise Integration By the end of this lecturer you will learn What is Enterprise Integration (EAI)? Benefits of Enterprise Integration Barrier

More information

Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 29-1

Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 29-1 Slide 29-1 Chapter 29 Overview of Data Warehousing and OLAP Chapter 29 Outline Purpose of Data Warehousing Introduction, Definitions, and Terminology Comparison with Traditional Databases Characteristics

More information

Data Warehouse: Introduction

Data Warehouse: Introduction Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of base and data mining group,

More information

Oracle BI Standard Edition One The Right BI Foundation for the Emerging Enterprise

<Insert Picture Here> Oracle BI Standard Edition One The Right BI Foundation for the Emerging Enterprise Oracle BI Standard Edition One The Right BI Foundation for the Emerging Enterprise Business Intelligence is the #1 Priority the most important technology in 2007 is business intelligence

More information

System types. Distributed systems

System types. Distributed systems System types 1 Personal systems that are designed to run on a personal computer or workstation Distributed systems where the system software runs on a loosely integrated group of cooperating processors

More information

C/S Basic Concepts. The Gartner Model. Gartner Group Model. GM: distributed presentation. GM: distributed logic. GM: remote presentation

C/S Basic Concepts. The Gartner Model. Gartner Group Model. GM: distributed presentation. GM: distributed logic. GM: remote presentation C/S Basic Concepts The Gartner Model Contents: 2-tier Gartner Model Winsberg s Model / Balance Example 3-tier n-tier Became de facto reference model Recognizes 5 possible modes of distribution: distributed

More information

Chapter 14: Databases and Database Management Systems

Chapter 14: Databases and Database Management Systems 15 th Edition Understanding Computers Today and Tomorrow Comprehensive Chapter 14: Databases and Database Management Systems Deborah Morley Charles S. Parker Copyright 2015 Cengage Learning Learning Objectives

More information

Concepts of Database Management Seventh Edition. Chapter 9 Database Management Approaches

Concepts of Database Management Seventh Edition. Chapter 9 Database Management Approaches Concepts of Database Management Seventh Edition Chapter 9 Database Management Approaches Objectives Describe distributed database management systems (DDBMSs) Discuss client/server systems Examine the ways

More information

IAF Business Intelligence Solutions Make the Most of Your Business Intelligence. White Paper November 2002

IAF Business Intelligence Solutions Make the Most of Your Business Intelligence. White Paper November 2002 IAF Business Intelligence Solutions Make the Most of Your Business Intelligence White Paper INTRODUCTION In recent years, the amount of data in companies has increased dramatically as enterprise resource

More information

BUILDING OLAP TOOLS OVER LARGE DATABASES

BUILDING OLAP TOOLS OVER LARGE DATABASES BUILDING OLAP TOOLS OVER LARGE DATABASES Rui Oliveira, Jorge Bernardino ISEC Instituto Superior de Engenharia de Coimbra, Polytechnic Institute of Coimbra Quinta da Nora, Rua Pedro Nunes, P-3030-199 Coimbra,

More information

Tier Architectures. Kathleen Durant CS 3200

Tier Architectures. Kathleen Durant CS 3200 Tier Architectures Kathleen Durant CS 3200 1 Supporting Architectures for DBMS Over the years there have been many different hardware configurations to support database systems Some are outdated others

More information

Cache Database: Introduction to a New Generation Database

Cache Database: Introduction to a New Generation Database Cache Database: Introduction to a New Generation Database Amrita Bhatnagar Department of Computer Science and Engineering, Birla Institute of Technology, A 7, Sector 1, Noida 201301 UP amritapsaxena@gmail.com

More information

ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS

ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS PRODUCT FACTS & FEATURES KEY FEATURES Comprehensive, best-of-breed capabilities 100 percent thin client interface Intelligence across multiple

More information

Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database

Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Managing Big Data with Hadoop & Vertica A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Copyright Vertica Systems, Inc. October 2009 Cloudera and Vertica

More information

OLAP and Data Mining. Data Warehousing and End-User Access Tools. Introducing OLAP. Introducing OLAP

OLAP and Data Mining. Data Warehousing and End-User Access Tools. Introducing OLAP. Introducing OLAP Data Warehousing and End-User Access Tools OLAP and Data Mining Accompanying growth in data warehouses is increasing demands for more powerful access tools providing advanced analytical capabilities. Key

More information

ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS

ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS Oracle Fusion editions of Oracle's Hyperion performance management products are currently available only on Microsoft Windows server platforms. The following is intended to outline our general product

More information

SOFT 437. Software Performance Analysis. Ch 5:Web Applications and Other Distributed Systems

SOFT 437. Software Performance Analysis. Ch 5:Web Applications and Other Distributed Systems SOFT 437 Software Performance Analysis Ch 5:Web Applications and Other Distributed Systems Outline Overview of Web applications, distributed object technologies, and the important considerations for SPE

More information

Client/server is a network architecture that divides functions into client and server

Client/server is a network architecture that divides functions into client and server Page 1 A. Title Client/Server Technology B. Introduction Client/server is a network architecture that divides functions into client and server subsystems, with standard communication methods to facilitate

More information

low-level storage structures e.g. partitions underpinning the warehouse logical table structures

low-level storage structures e.g. partitions underpinning the warehouse logical table structures DATA WAREHOUSE PHYSICAL DESIGN The physical design of a data warehouse specifies the: low-level storage structures e.g. partitions underpinning the warehouse logical table structures low-level structures

More information

Data Warehousing and Data Mining

Data Warehousing and Data Mining Data Warehousing and Data Mining Part I: Data Warehousing Gao Cong gaocong@cs.aau.dk Slides adapted from Man Lung Yiu and Torben Bach Pedersen Course Structure Business intelligence: Extract knowledge

More information

Multimedia Databases. Wolf-Tilo Balke Philipp Wille Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.

Multimedia Databases. Wolf-Tilo Balke Philipp Wille Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs. Multimedia Databases Wolf-Tilo Balke Philipp Wille Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 14 Previous Lecture 13 Indexes for Multimedia Data 13.1

More information

CONCEPTUALIZING BUSINESS INTELLIGENCE ARCHITECTURE MOHAMMAD SHARIAT, Florida A&M University ROSCOE HIGHTOWER, JR., Florida A&M University

CONCEPTUALIZING BUSINESS INTELLIGENCE ARCHITECTURE MOHAMMAD SHARIAT, Florida A&M University ROSCOE HIGHTOWER, JR., Florida A&M University CONCEPTUALIZING BUSINESS INTELLIGENCE ARCHITECTURE MOHAMMAD SHARIAT, Florida A&M University ROSCOE HIGHTOWER, JR., Florida A&M University Given today s business environment, at times a corporate executive

More information

What is a database? COSC 304 Introduction to Database Systems. Database Introduction. Example Problem. Databases in the Real-World

What is a database? COSC 304 Introduction to Database Systems. Database Introduction. Example Problem. Databases in the Real-World COSC 304 Introduction to Systems Introduction Dr. Ramon Lawrence University of British Columbia Okanagan ramon.lawrence@ubc.ca What is a database? A database is a collection of logically related data for

More information

Database System Architecture & System Catalog Instructor: Mourad Benchikh Text Books: Elmasri & Navathe Chap. 17 Silberschatz & Korth Chap.

Database System Architecture & System Catalog Instructor: Mourad Benchikh Text Books: Elmasri & Navathe Chap. 17 Silberschatz & Korth Chap. Database System Architecture & System Catalog Instructor: Mourad Benchikh Text Books: Elmasri & Navathe Chap. 17 Silberschatz & Korth Chap. 1 Oracle9i Documentation First-Semester 1427-1428 Definitions

More information

Data Warehousing and OLAP Technology for Knowledge Discovery

Data Warehousing and OLAP Technology for Knowledge Discovery 542 Data Warehousing and OLAP Technology for Knowledge Discovery Aparajita Suman Abstract Since time immemorial, libraries have been generating services using the knowledge stored in various repositories

More information

PowerDesigner WarehouseArchitect The Model for Data Warehousing Solutions. A Technical Whitepaper from Sybase, Inc.

PowerDesigner WarehouseArchitect The Model for Data Warehousing Solutions. A Technical Whitepaper from Sybase, Inc. PowerDesigner WarehouseArchitect The Model for Data Warehousing Solutions A Technical Whitepaper from Sybase, Inc. Table of Contents Section I: The Need for Data Warehouse Modeling.....................................4

More information

Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization

Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization Turban, Aronson, and Liang Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization

More information

Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence

Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence Appliances and DW Architectures John O Brien President and Executive Architect Zukeran Technologies 1 TDWI 1 Agenda What

More information

Oracle Warehouse Builder 10g

Oracle Warehouse Builder 10g Oracle Warehouse Builder 10g Architectural White paper February 2004 Table of contents INTRODUCTION... 3 OVERVIEW... 4 THE DESIGN COMPONENT... 4 THE RUNTIME COMPONENT... 5 THE DESIGN ARCHITECTURE... 6

More information

LITERATURE SURVEY ON DATA WAREHOUSE AND ITS TECHNIQUES

LITERATURE SURVEY ON DATA WAREHOUSE AND ITS TECHNIQUES LITERATURE SURVEY ON DATA WAREHOUSE AND ITS TECHNIQUES MUHAMMAD KHALEEL (0912125) SZABIST KARACHI CAMPUS Abstract. Data warehouse and online analytical processing (OLAP) both are core component for decision

More information

European Archival Records and Knowledge Preservation Database Archiving in the E-ARK Project

European Archival Records and Knowledge Preservation Database Archiving in the E-ARK Project European Archival Records and Knowledge Preservation Database Archiving in the E-ARK Project Janet Delve, University of Portsmouth Kuldar Aas, National Archives of Estonia Rainer Schmidt, Austrian Institute

More information

In-memory databases and innovations in Business Intelligence

In-memory databases and innovations in Business Intelligence Database Systems Journal vol. VI, no. 1/2015 59 In-memory databases and innovations in Business Intelligence Ruxandra BĂBEANU, Marian CIOBANU University of Economic Studies, Bucharest, Romania babeanu.ruxandra@gmail.com,

More information

www.ijreat.org Published by: PIONEER RESEARCH & DEVELOPMENT GROUP (www.prdg.org) 28

www.ijreat.org Published by: PIONEER RESEARCH & DEVELOPMENT GROUP (www.prdg.org) 28 Data Warehousing - Essential Element To Support Decision- Making Process In Industries Ashima Bhasin 1, Mr Manoj Kumar 2 1 Computer Science Engineering Department, 2 Associate Professor, CSE Abstract SGT

More information

Chapter 3 - Data Replication and Materialized Integration

Chapter 3 - Data Replication and Materialized Integration Prof. Dr.-Ing. Stefan Deßloch AG Heterogene Informationssysteme Geb. 36, Raum 329 Tel. 0631/205 3275 dessloch@informatik.uni-kl.de Chapter 3 - Data Replication and Materialized Integration Motivation Replication:

More information

Course 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization

Course 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization Oman College of Management and Technology Course 803401 DSS Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization CS/MIS Department Information Sharing

More information

What is Data Virtualization? Rick F. van der Lans, R20/Consultancy

What is Data Virtualization? Rick F. van der Lans, R20/Consultancy What is Data Virtualization? by Rick F. van der Lans, R20/Consultancy August 2011 Introduction Data virtualization is receiving more and more attention in the IT industry, especially from those interested

More information

Extending Hyperion BI with the Oracle BI Server

<Insert Picture Here> Extending Hyperion BI with the Oracle BI Server Extending Hyperion BI with the Oracle BI Server Mark Ostroff Sr. BI Solutions Consultant Agenda Hyperion BI versus Hyperion BI with OBI Server Benefits of using Hyperion BI with the

More information

A Framework for Developing the Web-based Data Integration Tool for Web-Oriented Data Warehousing

A Framework for Developing the Web-based Data Integration Tool for Web-Oriented Data Warehousing A Framework for Developing the Web-based Integration Tool for Web-Oriented Warehousing PATRAVADEE VONGSUMEDH School of Science and Technology Bangkok University Rama IV road, Klong-Toey, BKK, 10110, THAILAND

More information

Chapter 5. Warehousing, Data Acquisition, Data. Visualization

Chapter 5. Warehousing, Data Acquisition, Data. Visualization Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization 5-1 Learning Objectives

More information

Data Warehousing Concepts

Data Warehousing Concepts Data Warehousing Concepts JB Software and Consulting Inc 1333 McDermott Drive, Suite 200 Allen, TX 75013. [[[[[ DATA WAREHOUSING What is a Data Warehouse? Decision Support Systems (DSS), provides an analysis

More information

In principle, SAP BW architecture can be divided into three layers:

In principle, SAP BW architecture can be divided into three layers: Unit 1(Day 2): Data Warehousing Against this background, SAP decided to create its own data warehousing Solution that classifies reporting tasks as a self-contained business component. To circumvent the

More information

The difference between. BI and CPM. A white paper prepared by Prophix Software

The difference between. BI and CPM. A white paper prepared by Prophix Software The difference between BI and CPM A white paper prepared by Prophix Software Overview The term Business Intelligence (BI) is often ambiguous. In popular contexts such as mainstream media, it can simply

More information

BUSINESS ANALYTICS AND DATA VISUALIZATION. ITM-761 Business Intelligence ดร. สล ล บ ญพราหมณ

BUSINESS ANALYTICS AND DATA VISUALIZATION. ITM-761 Business Intelligence ดร. สล ล บ ญพราหมณ 1 BUSINESS ANALYTICS AND DATA VISUALIZATION ITM-761 Business Intelligence ดร. สล ล บ ญพราหมณ 2 การท าความด น น ยากและเห นผลช า แต ก จ าเป นต องท า เพราะหาไม ความช วซ งท าได ง ายจะเข ามาแทนท และจะพอกพ นข

More information

3-Tier Architecture. 3-Tier Architecture. Prepared By. Channu Kambalyal. Page 1 of 19

3-Tier Architecture. 3-Tier Architecture. Prepared By. Channu Kambalyal. Page 1 of 19 3-Tier Architecture Prepared By Channu Kambalyal Page 1 of 19 Table of Contents 1.0 Traditional Host Systems... 3 2.0 Distributed Systems... 4 3.0 Client/Server Model... 5 4.0 Distributed Client/Server

More information

ORACLE DATA INTEGRATOR ENTERPRISE EDITION

ORACLE DATA INTEGRATOR ENTERPRISE EDITION ORACLE DATA INTEGRATOR ENTERPRISE EDITION ORACLE DATA INTEGRATOR ENTERPRISE EDITION KEY FEATURES Out-of-box integration with databases, ERPs, CRMs, B2B systems, flat files, XML data, LDAP, JDBC, ODBC Knowledge

More information

Big Data Analytics with IBM Cognos BI Dynamic Query IBM Redbooks Solution Guide

Big Data Analytics with IBM Cognos BI Dynamic Query IBM Redbooks Solution Guide Big Data Analytics with IBM Cognos BI Dynamic Query IBM Redbooks Solution Guide IBM Cognos Business Intelligence (BI) helps you make better and smarter business decisions faster. Advanced visualization

More information

Global Data Integration with Autonomous Mobile Agents. White Paper

Global Data Integration with Autonomous Mobile Agents. White Paper Global Data Integration with Autonomous Mobile Agents White Paper June 2002 Contents Executive Summary... 1 The Business Problem... 2 The Global IDs Solution... 5 Global IDs Technology... 8 Company Overview...

More information

OBIEE 11g Data Modeling Best Practices

OBIEE 11g Data Modeling Best Practices OBIEE 11g Data Modeling Best Practices Mark Rittman, Director, Rittman Mead Oracle Open World 2010, San Francisco, September 2010 Introductions Mark Rittman, Co-Founder of Rittman Mead Oracle ACE Director,

More information

Data Warehousing. Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.

Data Warehousing. Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs. Data Warehousing & Data Mining Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 13. Decision Support Systems 13. Decision

More information

Week 3 lecture slides

Week 3 lecture slides Week 3 lecture slides Topics Data Warehouses Online Analytical Processing Introduction to Data Cubes Textbook reference: Chapter 3 Data Warehouses A data warehouse is a collection of data specifically

More information

TRENDS IN THE DEVELOPMENT OF BUSINESS INTELLIGENCE SYSTEMS

TRENDS IN THE DEVELOPMENT OF BUSINESS INTELLIGENCE SYSTEMS 9 8 TRENDS IN THE DEVELOPMENT OF BUSINESS INTELLIGENCE SYSTEMS Assist. Prof. Latinka Todoranova Econ Lit C 810 Information technology is a highly dynamic field of research. As part of it, business intelligence

More information

BENEFITS OF AUTOMATING DATA WAREHOUSING

BENEFITS OF AUTOMATING DATA WAREHOUSING BENEFITS OF AUTOMATING DATA WAREHOUSING Introduction...2 The Process...2 The Problem...2 The Solution...2 Benefits...2 Background...3 Automating the Data Warehouse with UC4 Workload Automation Suite...3

More information

Introductory Concepts

Introductory Concepts Introductory Concepts 5DV119 Introduction to Database Management Umeå University Department of Computing Science Stephen J. Hegner hegner@cs.umu.se http://www.cs.umu.se/~hegner Introductory Concepts 20150117

More information

FROM RELATIONAL TO OBJECT DATABASE MANAGEMENT SYSTEMS

FROM RELATIONAL TO OBJECT DATABASE MANAGEMENT SYSTEMS FROM RELATIONAL TO OBJECT DATABASE MANAGEMENT SYSTEMS V. CHRISTOPHIDES Department of Computer Science & Engineering University of California, San Diego ICS - FORTH, Heraklion, Crete 1 I) INTRODUCTION 2

More information

Lection 3-4 WAREHOUSING

Lection 3-4 WAREHOUSING Lection 3-4 DATA WAREHOUSING Learning Objectives Understand d the basic definitions iti and concepts of data warehouses Understand data warehousing architectures Describe the processes used in developing

More information

Lost in Space? Methodology for a Guided Drill-Through Analysis Out of the Wormhole

Lost in Space? Methodology for a Guided Drill-Through Analysis Out of the Wormhole Paper BB-01 Lost in Space? Methodology for a Guided Drill-Through Analysis Out of the Wormhole ABSTRACT Stephen Overton, Overton Technologies, LLC, Raleigh, NC Business information can be consumed many

More information

Real-time Data Replication

Real-time Data Replication Real-time Data Replication from Oracle to other databases using DataCurrents WHITEPAPER Contents Data Replication Concepts... 2 Real time Data Replication... 3 Heterogeneous Data Replication... 4 Different

More information

Databases in Organizations

Databases in Organizations The following is an excerpt from a draft chapter of a new enterprise architecture text book that is currently under development entitled Enterprise Architecture: Principles and Practice by Brian Cameron

More information

ORACLE BUSINESS INTELLIGENCE, ORACLE DATABASE, AND EXADATA INTEGRATION

ORACLE BUSINESS INTELLIGENCE, ORACLE DATABASE, AND EXADATA INTEGRATION ORACLE BUSINESS INTELLIGENCE, ORACLE DATABASE, AND EXADATA INTEGRATION EXECUTIVE SUMMARY Oracle business intelligence solutions are complete, open, and integrated. Key components of Oracle business intelligence

More information

Chapter 6 Basics of Data Integration. Fundamentals of Business Analytics RN Prasad and Seema Acharya

Chapter 6 Basics of Data Integration. Fundamentals of Business Analytics RN Prasad and Seema Acharya Chapter 6 Basics of Data Integration Fundamentals of Business Analytics Learning Objectives and Learning Outcomes Learning Objectives 1. Concepts of data integration 2. Needs and advantages of using data

More information

CHAPTER 5: BUSINESS ANALYTICS

CHAPTER 5: BUSINESS ANALYTICS Chapter 5: Business Analytics CHAPTER 5: BUSINESS ANALYTICS Objectives The objectives are: Describe Business Analytics. Explain the terminology associated with Business Analytics. Describe the data warehouse

More information

MDM and Data Warehousing Complement Each Other

MDM and Data Warehousing Complement Each Other Master Management MDM and Warehousing Complement Each Other Greater business value from both 2011 IBM Corporation Executive Summary Master Management (MDM) and Warehousing (DW) complement each other There

More information

DISTRIBUTED AND PARALLELL DATABASE

DISTRIBUTED AND PARALLELL DATABASE DISTRIBUTED AND PARALLELL DATABASE SYSTEMS Tore Risch Uppsala Database Laboratory Department of Information Technology Uppsala University Sweden http://user.it.uu.se/~torer PAGE 1 What is a Distributed

More information

Data Warehouses & OLAP

Data Warehouses & OLAP Riadh Ben Messaoud 1. The Big Picture 2. Data Warehouse Philosophy 3. Data Warehouse Concepts 4. Warehousing Applications 5. Warehouse Schema Design 6. Business Intelligence Reporting 7. On-Line Analytical

More information

ETL-EXTRACT, TRANSFORM & LOAD TESTING

ETL-EXTRACT, TRANSFORM & LOAD TESTING ETL-EXTRACT, TRANSFORM & LOAD TESTING Rajesh Popli Manager (Quality), Nagarro Software Pvt. Ltd., Gurgaon, INDIA rajesh.popli@nagarro.com ABSTRACT Data is most important part in any organization. Data

More information

Lecture Data Warehouse Systems

Lecture Data Warehouse Systems Lecture Data Warehouse Systems Eva Zangerle SS 2013 PART A: Architecture Chapter 1: Motivation and Definitions Motivation Goal: to build an operational general view on a company to support decisions in

More information

Foundations of Business Intelligence: Databases and Information Management

Foundations of Business Intelligence: Databases and Information Management Chapter 5 Foundations of Business Intelligence: Databases and Information Management 5.1 Copyright 2011 Pearson Education, Inc. Student Learning Objectives How does a relational database organize data,

More information

Technology in Action. Alan Evans Kendall Martin Mary Anne Poatsy. Eleventh Edition. Copyright 2015 Pearson Education, Inc.

Technology in Action. Alan Evans Kendall Martin Mary Anne Poatsy. Eleventh Edition. Copyright 2015 Pearson Education, Inc. Copyright 2015 Pearson Education, Inc. Technology in Action Alan Evans Kendall Martin Mary Anne Poatsy Eleventh Edition Copyright 2015 Pearson Education, Inc. Technology in Action Chapter 9 Behind the

More information

Extraction Transformation Loading ETL Get data out of sources and load into the DW

Extraction Transformation Loading ETL Get data out of sources and load into the DW Lection 5 ETL Definition Extraction Transformation Loading ETL Get data out of sources and load into the DW Data is extracted from OLTP database, transformed to match the DW schema and loaded into the

More information

Building Cubes and Analyzing Data using Oracle OLAP 11g

Building Cubes and Analyzing Data using Oracle OLAP 11g Building Cubes and Analyzing Data using Oracle OLAP 11g Collaborate '08 Session 219 Chris Claterbos claterbos@vlamis.com Vlamis Software Solutions, Inc. 816-729-1034 http://www.vlamis.com Copyright 2007,

More information

Module 17. Client-Server Software Development. Version 2 CSE IIT, Kharagpur

Module 17. Client-Server Software Development. Version 2 CSE IIT, Kharagpur Module 17 Client-Server Software Development Lesson 42 CORBA and COM/DCOM Specific Instructional Objectives At the end of this lesson the student would be able to: Explain what Common Object Request Broker

More information

1. INTRODUCTION TO RDBMS

1. INTRODUCTION TO RDBMS Oracle For Beginners Page: 1 1. INTRODUCTION TO RDBMS What is DBMS? Data Models Relational database management system (RDBMS) Relational Algebra Structured query language (SQL) What Is DBMS? Data is one

More information

CS2032 Data warehousing and Data Mining Unit II Page 1

CS2032 Data warehousing and Data Mining Unit II Page 1 UNIT II BUSINESS ANALYSIS Reporting Query tools and Applications The data warehouse is accessed using an end-user query and reporting tool from Business Objects. Business Objects provides several tools

More information

Bussiness Intelligence and Data Warehouse. Tomas Bartos CIS 764, Kansas State University

Bussiness Intelligence and Data Warehouse. Tomas Bartos CIS 764, Kansas State University Bussiness Intelligence and Data Warehouse Schedule Bussiness Intelligence (BI) BI tools Oracle vs. Microsoft Data warehouse History Tools Oracle vs. Others Discussion Business Intelligence (BI) Products

More information

JDBC Drivers, Bridges and SequeLink. Edward M. Peters Vice President & General Manager DataDirect

JDBC Drivers, Bridges and SequeLink. Edward M. Peters Vice President & General Manager DataDirect JDBC Drivers, Bridges and SequeLink Edward M. Peters Vice President & General Manager DataDirect DataDirect s Business Provide world class data connectivity from any application, to any data source, from

More information

Breadboard BI. Unlocking ERP Data Using Open Source Tools By Christopher Lavigne

Breadboard BI. Unlocking ERP Data Using Open Source Tools By Christopher Lavigne Breadboard BI Unlocking ERP Data Using Open Source Tools By Christopher Lavigne Introduction Organizations have made enormous investments in ERP applications like JD Edwards, PeopleSoft and SAP. These

More information

Oracle9i Data Warehouse Review. Robert F. Edwards Dulcian, Inc.

Oracle9i Data Warehouse Review. Robert F. Edwards Dulcian, Inc. Oracle9i Data Warehouse Review Robert F. Edwards Dulcian, Inc. Agenda Oracle9i Server OLAP Server Analytical SQL Data Mining ETL Warehouse Builder 3i Oracle 9i Server Overview 9i Server = Data Warehouse

More information

A Technical Review on On-Line Analytical Processing (OLAP)

A Technical Review on On-Line Analytical Processing (OLAP) A Technical Review on On-Line Analytical Processing (OLAP) K. Jayapriya 1., E. Girija 2,III-M.C.A., R.Uma. 3,M.C.A.,M.Phil., Department of computer applications, Assit.Prof,Dept of M.C.A, Dhanalakshmi

More information