How to Load Data from Flat Files into BW PSA Applies to: SAP BI/BW 7.0 and above. Summary The document is aimed at all those who are at beginner level in BW. It details out step by step processes that are involved in loading data from a flat file into BW. Author: Umesh Pednekar Company: Howdens Joinery Co. Created on: 27 Oct 2011 Author Bio Umesh is working as a SAP BI/ BW Analyst in Howdens Joinery co and has technofunctional experience of over 7 years in designing, developing and implementing/upgrading SAP BW-BI solutions across various Industries in India and in the UK. 2010 SAP AG 1
Table of Contents Overview... 3 Assumptions... 3 Business Scenario... 3 Step by Step Solution... 3 Identifying the file that needs to be loaded into BW... 4 Creating Data source structure for the File... 4 Creating Info package to Load the file till the PSA... 13 Execute the Info package... 14 Validate the Results... 15 Follow these DO s and DONT s... 17 Related Content... 18 Disclaimer and Liability Notice... 19 2010 SAP AG 2
Overview Purpose of this document is to demonstrate a step by step solution that will help understand the mechanism involved in loading data from the flat file, in CSV format, into BW till the PSA. Often, loading data from flat files becomes an activity of paramount importance more importantly when the BW professionals are faced with the task of testing that solutions without sufficient availability of test data or the task that calls for making historical data available on quality, testing or pre-production servers. This document gives a simplistic approach that can help understand the solution and implement it quickly with positive results. Assumptions The document assumes that the target audience is aware of the processes that involve Creating BW Data targets / Info Providers Creating flat (CSV) files The document provides the mechanism that will help load data from flat files till the PSA. Further, it is expected that the audience know creating data targets and mapping the PSA with the data targets using transformations and moving the data using the Data Transfer Process (DTP). Business Scenario Classic business scenario which I had to encounter in one of my implementations project it was APO project and the requirement called for extraction of historical sales data from BW into APO. We wanted to test the solution with near live data and this meant making the historical sales data available from BW production to APO test server. (When we implemented the solution on the Production server we gave away the use of the flat files as we could extract data directly from one of the Sales Info providers but alas, this wasn t the case of Test server as BW Test server did not have the required historical sales data). The historical data from the Sales Info provider was extracted into flat files (you can refer HOW TO RSCRM_BAPI related documentation for this ). The next step was to load the flat files into the Test server first the PSA and then into the Data target. Step by Step Solution We now take a look at the steps involved in the process of loading data from a flat file into the PSA. These steps are covered in detail. 1. Identify the file that needs to be loaded into BW. 2. Make sure that the file structure coincides with the structure of the Data Target. 3. Execute activities for loading the file a. Create data source structure for the file b. Create Info package to load the file into PSA. c. Execute the Info package. d. Validate the Results 2010 SAP AG 3
Identifying the file that needs to be loaded into BW Let s assume that we need to load the following file into BW. The file structure has 3 Key figures in column K, L and M. Rest of the fields are to be treated as characteristics. Creating Data source structure for the File Our first step is to create a data source structure based on the file format. Go to the DW workbench using RSA1. In the Modeling section Go to the tab Source system The system displays folders all the possible source systems available this would include SAP, BI, other external systems and Flat File I am assuming that the source system creation is already done. You need to create the source system ( FLATFILE in this case) under this folder ( FILE in this case). 2010 SAP AG 4
Select the Flat file source system Right click the mouse and select Display Datasource Tree. All the application components are displayed for the source system. For the sake of simplicity, we will not create any additional app component but create the data source in the Unassigned nodes application component. At the desired application component, right click the mouse and select the option Create DataSource. 2010 SAP AG 5
You are now asked to enter the Datasource details viz. Technical name, Source system and the Type of the Datasource. Data source created in this case is called ZFLAT_DS and is identified as a Transaction data data source. Select Transfer to go to the various tabs that would help take the process ahead. On the General Info tab the details of the data source covering it s description and other critical parameters. PSA for All Segments in CHAR Format If this indicator is set, the PSA is not generated with a typed structure, but with character-type fields of type CHAR. The use of conversion exits (where available) only takes place after the data from the PSA has been read. Use this option if the conversion causes problems during loading, for example, because no suitable conversion routines currently exist if the source cannot guarantee that the data can be loaded to the correct data type. In this case, the data can nevertheless be loaded into the PSA and be corrected there. Data Is Language Dependent This indicator is set if the DataSource delivers language-dependent data. A DataSource is language dependent if it has a selection field that is defined as a language field. Data Is Time Dependent This flag is set if the DataSource delivers time-dependent data.. A DataSource is time-dependent if it has a selection field that defines either the lower or upper limit of the validity of the data. 2010 SAP AG 6
Delivery of Duplicate Data Recs : This indicator gives information on how the DataSource behaves within a request with regard to duplicate records: ' ' The status of the DataSource is unknown. '0' The DataSource does not deliver any duplicate records within a request, with reference to its key. '1' The DataSource can deliver duplicate records within a request, with reference to its key. However, no duplicate records are delivered in a data package. This indicator is particularly important for delta-capable attribute tables and text tables. For the settings '0' and '1' you also have to define a key for the DataSource. This can be either in the DDIC using the maintenance for the corresponding field property of the extract structure fields, or (alternatively or additionally) in the metadata of the Datasource. A field in the DataSource also has the additional attribute ' DataSource Key Field', which transfersor corrects the DDIC property where necessary The example is this document covers the details as follows: The Extraction tab helps enter the details of the Flat file. 2010 SAP AG 7
In the Delta Process option we are selecting Full Load. The file considered is a Text-type file which will be loaded from the local workstation (You can load files situated on the application servers too). No. of lines to be ignored When you load a flat file, its first XX rows are ignored. This means you can hide up to 99 top headers / remark rows. A blank entry (entering 0) means that every row of the file is taken into account. This is the default setting. If the values in your file (CSV file, for example) are separated using a particular character, you enter this separator in the Data Separator field. If the separator that you specified under Data Separator is a part of the value, the file indicates this by enclosing the value in particular start and end characters. Enter these start and end characters under Escape Sign Characters.. User Default or Fixed Entry If you select Default setting, the settings are taken from the user master data of the user used to execute the conversion exit. If you select Direct entry, you can make entries that are always used independent of the current user in the following fields. 2010 SAP AG 8
On the Proposal tab, you need to indicate the file that is under consideration and the system will transfer the fields details. Select limited number of records to create the proposal No. of Data Records selected in this case is 10. Click on Load Example Data. The system might ask you to provide access to the file select the required authorization/access option. When you click OK, system displays the records and creates a proposal which lists the fields from the file. 2010 SAP AG 9
Go to the next tab i.e. Fields. The system prompts if you wish to copy the list of fields from the proposals to the Field Attributes area. Select Yes. In the Fields tab, you now see the list of fields copied from the proposal. In the InfoObject column, enter the corresponding Infoobject technical name for each of the field. Make sure that the contents available in the flat file correspond to the field type/width else you will encounter errors either during loading or the data will not be aligned in appropriate manner. 2010 SAP AG 10
While you try to move to the last tab Preview, the system asks for confirmation that will copy the Infoobject properties onto the fields in the proposal. You will have to select Copy to give the confirmation. Remember that the prompt will appear for every InfoObject/field available in the Fields tab. You will have to select Copy for every prompt. In the Preview tab, select the number of records that you wish to display and validate the specifications. Then click on Read Preview Data. Records selected are 10. System asks for data source activation. Select Activate. 2010 SAP AG 11
Continue at the Log that displays information. Data Source is activated. A preview for the first 10 (or less) is displayed. Data source creation is successful. Go back using back keys to the Datasources tab in the Modeling section. You now see the newly created data source in the FLATFILE source system. 2010 SAP AG 12
Creating Info package to Load the file till the PSA The next process involves loading the data from the flat file into the BW PSA. We create an InfoPackage for this purpose. Select the data source, right click mouse and click on Create InfoPackage On the Create InfoPackage screen, select the tab under the Datasource and enter description for the InfoPackage. Save the InfoPackage. We now need to provide details related to the InfoPackage through various tab. Leave the Data Selection tab. 2010 SAP AG 13
In the Extraction tab, you will find details of the file that you had earlier entered prior to the proposal creation in the Datasource. Make changes, if required. You may have positioned the flat file in a different location. Also check for the number of headers row to be ignored in the file as well as the data separator. In BI 7.0. the load is done only till the PSA (unlike in BW 3.5 wherein you can carry the extraction till the Data Target/Info Provider). We are doing a Full Update as indicated in the Update tab. Execute the Info package In the Schedule tab, select the radio button next to Start Data Load Immediately. You can set the load in the background by providing details regarding the date and time of the load through Start Later in Background option. 2010 SAP AG 14
Click on Start to schedule the load. Data Extraction process is initiated. Validate the Results You can view the output in the PSA, by accessing the Call Monitor. The Call Monitor screen is displayed for the load under consideration. Select the PSA maintenance icon as shown below. 2010 SAP AG 15
From the available requests, select any of the requests for the PSA, with desired number of records and Continue. PSA requests contents are displayed. We have successfully extracted data from a flat file into the PSA. The Data source and the InfoPackage are listed in the source system. The extracted data can now be taken further within BW using transformations and DTPs (in BI 7.0) to desired Data Targets and Info Providers. 2010 SAP AG 16
Follow these DO s and DONT s 1) Flat file fields should correspond with the Info Objects whose properties you will be copying during the proposal in terms of field type and width. 2) During flat file loads, inclusion of special characters which invariably results in a failure while loading the data in the Data targets/info Providers goes unnoticed during load till the PSA. Care should be taken to eliminate presence of special characters. 3) While loading Units/Currencies for various key figures, ensure that the contents included in the Flat file are defined in system. Best way to validate is to check the SID table for 0UNIT, 0CURRENCY or the relevant unit InfoObject used. 2010 SAP AG 17
Related Content SCN Article : Upload Flat File in SAP BI/BW http://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/603efe5a-b9d4-2e10-67a5-f16e9aa6aab8 SCN Article : Extraction of Hierarchy into Flat File from R/3 and Loading in BW System http://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/90859ec4-9c90-2c10-5cbd-933ce3251b13 Thread: http://forums.sdn.sap.com/thread.jspa?threadid=1574128 2010 SAP AG 18
Disclaimer and Liability Notice This document may discuss sample coding or other information that does not include SAP official interfaces and therefore is not supported by SAP. Changes made based on this information are not supported and can be overwritten during an upgrade. SAP will not be held liable for any damages caused by using or misusing the information, code or methods suggested in this document, and anyone using these methods does so at his/her own risk. SAP offers no guarantees and assumes no responsibility or liability of any type with respect to the content of this technical article or code sample, including any liability resulting from incompatibility between the content within this document and the materials and services offered by SAP. You agree that you will not hold, or seek to hold, SAP responsible or liable with respect to the content of this document. 2010 SAP AG 19