Introduction to SAS ETL Studio

Transcription

1 Introduction to SAS ETL Studio Krzysztof Dembczyński Institute of Computing Science Laboratory of Intelligent Decision Support Systems Politechnika Poznańska (Poznań University of Technology) Software Development Technologies Master studies, third semester Academic year 2008/09 (winter course)

2 1 Introduction to SAS ETL Studio 2 Case Study 3 SAS ETL Studio Metadata 4 SAS ETL Studio Process Flow Diagram 5 SAS ETL Studio Run the Job

3 1 Introduction to SAS ETL Studio 2 Case Study 3 SAS ETL Studio Metadata 4 SAS ETL Studio Process Flow Diagram 5 SAS ETL Studio Run the Job

4 SAS ETL Studio is an application that enables to create and manage ETL process (extraction, transformation and load of data).

5 We will learn: How to define data source tables, How to define data target tables, How to create and run ETL jobs.

6 We will use: SAS Management Console (for accessing metadata), SAS ETL Studio (for creating and managing ETL process), SAS Metadata Server (for storing and managing metadata), SAS Workspace Server (for running SAS programs).

7 Metadata Repository Metadata object is a set of attributes that describe a resource: tables, users, servers, etc. Definitions of data sources and targets, and all transformations performed on these data are also metadata. Metadata are stored in repositories (SAS Main is a name of a repository of type Foundation).

8 Metadata Repository

9 SAS ETL Studio

10 SAS ETL Studio

11 1 Introduction to SAS ETL Studio 2 Case Study 3 SAS ETL Studio Metadata 4 SAS ETL Studio Process Flow Diagram 5 SAS ETL Studio Run the Job

12 Case Study The fictional Orion Sports & Outdoors is an international retail company that sells sports and outdoor products. The headquarters is based in the United States, and retail stores are situated in a number of other countries. Products are sold through physical retail stores, as well as through mail-order catalogs and the Internet. Customers who sign up as members of the Orion Star Club organization can receive favorable special offers; therefore, most customers enroll in the Orion Star Club. The sales data in the scenario only include the purchases by Orion Star Club members in the years 1998 through 2002.

13 Case Study Some queries: What products are sold to which customers? What products are sold when? What is the average delivery time for orders placed through the catalog and Internet sales channels?

14 Data source tables: CUSTOMER contains information about the customers who have placed orders with the company. The table includes information such as the name, address, and birth date of each customer as well as other demographic data. ORDER_ITEM contains detailed information about catalog orders, Internet orders, and retail sales. The data includes product ID numbers, the quantity of each item purchased, and the price of each item. ORDERS contains information about catalog orders, Internet orders, and retail sales. The data includes the date, sales person ID, and customer ID for each order.

15 Target data tables: ORDER_FACT contains selected attributes from CUSTOMER, ORDER_ITEM, ORDERS tables, and additional fields What products are sold when? (Order_Year, Order_Month) CATALOG_INTERNET include all of the columns in ORDER_FACT as well as an additional column Delivery_Time.

16 Metadata Profile Machine: localhost Username: sasadm Password: sas Repository: Foundation

17 Data Tables Location Create directories for source data:... /Sources Create directories for target data:... /Targets Grant SAS Server Users all permissions to these folders.

18 1 Introduction to SAS ETL Studio 2 Case Study 3 SAS ETL Studio Metadata 4 SAS ETL Studio Process Flow Diagram 5 SAS ETL Studio Run the Job

19 Inventory tab: Foundation Library New Library SAS Base Engine Library Name: Source Tables and Target Tables Libref: Sources and Targets Engine: BASE Path: [...]/Sources and [...]/Targets Server: SASMain

20 Define metadata for source tables (CUSTOMER, ORDERS, ORDER_ITEM): Source Designer Sources SAS server: SASMain SAS Library: Source Tables Select All Tables Group Location: Foundation View the source table metadata and data: CUSTOMER Properties, CUSTOMER View Data.

21 Define metadata for target table ORDER_FACT: Target Designer Targets Target Table Name: Order_Fact Available Columns: Table to Select From CUSTOMER ORDER_ITEM ORDERS Columns to Select Customer_ID Street_ID Order_ID Product_ID Quantity Total_Retail_Price CostPrice_Per_Unit Employee_ID Order_Type Order_Date Delivery_Date

22 Specify new columns that will be added to ORDER_FACT: Columns New Name Description Length Type Is Nullable Order_Year Year Order was Placed 4 Numeric Yes Order_Month Month Order was Placed 4 Numeric Yes Specify a library and review the final information: Library: Target Tables

23 Specify metadata for the CATALOG_INTERNET target table: Target Designer Targets Target Table Name: Catalog_Intenet Available Columns: ORDER_FACT (all) New column added to the target table: Columns New Name Description Length Type Is Nullable Deliver_Time DeliveryTime-OrderTime 4 Numeric Yes Review the final information: Library: Target Tables

24 1 Introduction to SAS ETL Studio 2 Case Study 3 SAS ETL Studio Metadata 4 SAS ETL Studio Process Flow Diagram 5 SAS ETL Studio Run the Job

25 A job is a metadata object that specifies processes that create output. Process flow diagram:

26 Create a process flow diagram for the ORDER_FACT target table: Process Designer Name: Load Orion Star Tables Select Tables: Order_Fact

27 Create a process flow diagram for the ORDER_FACT target table: Process Library tab Data Transforms SQL Join Invertory tab drag&drop CUSTOMER, ORDERS, ORDER_ITEM (from Source Tables Library)

28 CATALOG_INTERNET target table is constructed in similar way as ORDER_FACT table, but contains filtered data: Process Library tab Data Transforms SQL Splitter

29 CATALOG_INTERNET target table is constructed in similar way as ORDER_FACT table, but contains filtered data: Inventory tab drag&drop Catalog_Internet (from Target Tables Library) Note: Loader for Catalog_Internet table is added automatically File Save

30 SQL Join: SQL Join Properties Tables tab right click Advanced Joins Mappings tab

31 SQL Join: Import metadata describing Order_Year, Order_Month, and Delivery_Time: Import Columns Ungrouped Available Columns Catalog_Internet select: Order_Year, Order_Month, Delivery_Time Specify expressions defining these columns.

32 SQL Join: Specify that the values for Order_Year are extracted from the values of ORDER_Data by using the YEAR function: double-click on Expresions Category: Data and Time Functions: Year (Date) Insert Data Elements:Orders Order_Date Insert OK

33 SQL Join: Specify that the values for ORDER_FACT are extracted from the values of Order_Date by using the MONTH function. double-click on Expresions Category: Data and Time Functions: MONTH(Date) Insert Data Elements: Orders Order_Date Insert OK

34 SQL Join: specify that the values for Delivery_Time are calculated by using the values for Delivery_Date and Order_Date. double-click on Expresions Data Sources: Orders Deliver_Date- Order_Date Insert OK

35 SQL Join: Select rows where value of the derived column, Order_Year, is greater than or equal to 2001: Where tab Available Columns Result Columns Order_Year 2001 OK File->Save

36 SQL Splitter: Specify that the target table Catalog_Internet will contain only observations where the value of OrderType equals Catalog or Internet: SAS Splitter Properties Row Selection tab Target Table: Catalog_Internet Row Selection Type: Row Selection Conditions Selection Condition: OrderType = Catalog OrderType = Internet

37 SQL Splitter: Propagate the new columns SAS Splitter Properties Mappings tab Right click: Quick Propagate Select Target Table: All File Save

38 SAS Loader: Specify loader properties for Order_Fact and Catalog_Internet: SAS Loader Properties Tab: Load Technique Load Technique: Refresh Before Loading: Drop Table Update the column mappings for Order_Fact and Catalog_Internet: SAS Loader Properties Tab: Mapping Right click: Quick Map File Save

39 1 Introduction to SAS ETL Studio 2 Case Study 3 SAS ETL Studio Metadata 4 SAS ETL Studio Process Flow Diagram 5 SAS ETL Studio Run the Job

40 Run the Job: Right click Submit

41 Target data tables: Target table right click View Data

42 Bibliography Getting Started with SAS ETL Studio, 2006