Using SAS Enterprise Business Intelligence to Automate a Manual Process: A Case Study Erik S. Larsen, Independent Consultant, Charleston, SC

Transcription

1 Using SAS Enterprise Business Intelligence to Automate a Manual Process: A Case Study Erik S. Larsen, Independent Consultant, Charleston, SC Abstract: Often times while on a client site as a SAS consultant, you are faced with a project which involves many manual steps and can include both legacy code and Excel spreadsheets. Running this code (or possibly segments of code, depending on what the results are) along with some cutting and pasting rows and columns here and there which ultimately results in a report that gets sent out to a client. The client (or clients) like this report and keep asking for more of them until these reports become a task in a business analyst s daily routine. This routine grows as the volume of data or number of reports increase until it consumes the entire day. This paper will illustrate a real-world example of how a report which consisted of legacy SAS programs and Excel spreadsheet manipulation that grew so large that a more efficient way of providing these reports to clients needed to be developed. The SAS Enterprise Business Intelligence platform provided many of the tools necessary to automate this process so that an end user with no programming and basic Excel knowledge could create these reports with minimal effort. The Backstory: A large multinational financial institution which provides lines of credit to numerous companies around the world needed an easy way to measure the utilization of these lines for a given period, typically a year prior to the current date. The company needed a tool that allowed it to be nimble to allow decisions to be made when macroeconomic events occurred around the world, such as the Cyprus banking crisis in early 2013 or European Union problems that started in 2011 and continue today. A tool that would allow the company to terminate a line of credit quickly when financial conditions in a country or region deteriorated in order to protect itself from potential losses. Several business analysts (or BAs) created some code to connect to a multitude of Teradata and Netezza tables to join them together to get both the amount of the line along with the utilization over a certain time period which was edited each time the reports were run. They also received information about utilization daily in s which contained Excel files from each of the company s regional offices around the world. The BAs created a SAS dataset out of the daily feeds after uploading it and checking it against the internal data to see if it matched criteria for inclusion. The resulting dataset would then be filtered to an appropriate date range (typically a year) and compared to several database tables to determine if these lines of credit had United States and non-u.s. allocations (or subsidiary companies) using these loan products. If the query results came up with records for one or both of these queries, a separate program was then submitted for each entity (U.S. and non-u.s.) where allocations were present to come up with the level of utilization for each day in the time period. These results were then summarized for reporting purposes. Both the detail and summary results of these programs were then downloaded into Excel separately, and manually cut and pasted into separate tabs. A pivot table was created to create a report and graph of the utilization on a different tab which contained a filter for each of the company s subsidiaries. The process of loading the daily tables along with creating a report took between 4-6 hours to create. This would be longer if there were multiple requests for reports for that given day as the data used by these reports were rather large. 1

2 Figure 1: Data Flow Diagram of the old process The Solution: There were many moving parts in the above described process which involved a large amount of human intervention. Not only were there a large number of tasks involved, there was always a chance that human errors could be introduced at different steps of the process. A solution to these issues would require removing the BA from the process itself and using the SAS Enterprise Business Intelligence (EBI) package in order to automate the processes. The first step of the process was to automate the daily feeds from around the world. These were being ed daily from subsidiaries around the globe which had to be downloaded out of Microsoft Outlook and copied to the UNIX server. A PC-SAS program would be remote-submitted to read the CSV files and check to see if the data matched account numbers for a given time period by joining to several Teradata tables. Validated data were then appended to a Master file SAS dataset which would be used later in the process. Some help from the suppliers of the data and Data Integration Studio (or DI Studio) were used to automate this process. Instead of ing the data, the global data centers FTPed the data directly to the UNIX server each day. The data landed in the same directory with the same name so that the system would know when all of the data had arrived to start the process. Several DI Studio jobs were created for 2

3 each data center s feed and an LSF data flow was created to look for the data and to control the execution of the jobs. Once all of the data arrived, LSF would kick off all of the DI Studio jobs concurrently with each job appending to the Master file. LSF is included with the SAS EBI platform as a plug-in to SAS Management Console (SMC). Often a SAS Administrator needs to grant end users access to LSF. Figure 2 shows a screen shot of one of the DI Studio jobs. Figure 2: An example Data Integration Studio job The second step in the process would be to take the legacy SAS code that was already written and add in some logic to account for the decision making process which determined which other programs to run if the U.S. entities and non-u.s. entities had allocations to report on for a certain time period. As described above, the time period was entered in a PC SAS program manually along with the subsidiary number and code was executed to determine if allocations existed in either U.S. or non-u.s. A SAS stored process would take all of the code with the added logic and allow the end user to use prompts to enter in the time period and the subsidiary number. Using Enterprise Guide (or EG), the stored process was created along with prompts for the GCI number (or subsidiary number) and the begin and end dates of the period. The prompts, as shown below in Figure 3 generate SAS macro variables that are used in the code. A third prompt, Instrument also selects which currency is used. Figure 3: Prompts created in Enterprise Guide 3

4 Now that all of the code and logic are combined into a nice package, the client still needed a nice way to display the report from the data that was generated from the prompts and the code. In the previous version, SAS data was downloaded into Excel and a pivot table with a filter was created to select each of the accounts in the subsidiary company. Each selection created a line chart showing the line of credit and the utilization of the credit during the time period and a report which showed the raw data. The client wanted to take it a step further and show all of the chart-report combinations of each account in the subsidiary company and not have a filter. This was a challenge as every subsidiary had anywhere from one to up to 40 accounts so the tables and graphs needed to line up next to each other (as opposed to just running a report and plot with a by variable which would list all of the reports and follow with all of the graphs as shown in Figure 4). More logic was added to the SAS stored process to parse through the data Figure 4: Original Output Report by account number so that a graph and report for each account would be generated in order for each and every account in the subsidiary. Figure 5 shows the macro code used to generate the calls to the procedures that would consecutively output each subsidiary s report and graph in order. PROC SQL NOPRINT; SELECT DISTINCT ClientLimit INTO :CL SEPARATED BY "," FROM MaxLimit ORDER BY MaxLimit DESC; QUIT; DATA _NULL_; Num_CL = COUNT("&CL", ',') + 1; CALL SYMPUT("Num_CL", TRIM(LEFT(PUT(Num_CL, BEST.)))); CLString = "&CL"; DO I = 1 TO Num_CL; IF ((INDEX(CLString, ',')) NE 0) THEN DO; LEN = SUBSTR(CLString, 1, INDEX(CLString, ',') - 1); CALL SYMPUT("CL" TRIM(LEFT(PUT(I, BEST.))), TRIM(LEFT(Len))); CLString = SUBSTR(CLString, INDEX(CLString, ',') + 1); END; ELSE DO; CALL SYMPUT("CL" TRIM(LEFT(PUT(I, BEST.))), TRIM(LEFT(CLString))); END; END; RUN; Figure 5: Macro Code to Generate Report 4

5 The final step in this project was to allow for an end-user who may not know SAS or programming in general to be able to access this report without taking any of the BA s time. A SAS stored process has the advantage that sassrv is the ID that will access the databases (as opposed to the user id of the person who wants to generate the report) and thus alleviates each user needing to have access to the databases. Often in large organizations, forms need to be filled out for access and several supervisors, including the DBA need to approve read access before the end user can query the databases. If the SAS stored process server is set up correctly, submitting a stored process alleviates this problem. The SAS Add-in for Microsoft Office product allows end users to open up Excel and, among other things, submit SAS stored process. This allows the end user to use a tool that they are comfortable with, Excel to surface the reports desired by running the SAS stored process, filling out the prompts in the stored process and viewing the report. Figure 6 shows how to submit the SAS stored process in Excel and Figure 7 shows the prompts inside Figure 6: Using the Add-in to Microsoft Office to Submit a Stored Process Excel. End users do not need to have access to the databases used in the stored process. Figure 7: SAS Stored Process Prompts Surfaced in Excel 5

6 Brick Walls Found: When first developing the solution to this project, SAS Web Report Studio (WRS) was used to create the report portion of this project. Web Report Studio had the advantage of accessing any database package through the SAS/ACCESS engines (or ODBC) and with SPDS, which is SAS version of a credentialed DBMS. The SAS Stored Process was set up to create a SPDS data file which would be used by WRS to create the reports with side-by-side listings and graphs. The WRS reports were attractive but had one limitation the download to Excel. The File Export in WRS generated a.zip file which contained an Excel workbook and several.png images for the graphs. When the zip file was extracted, the Excel workbook appeared similar to the previous report but had one serious limitation. When the file was saved, the graphs were saved as links to the.png files which were included in the zip file (as opposed to being embedded in the Excel workbook. The problem was that if the Excel file was moved to a different location (or ed to someone, per se), the links to the graphs would break and show a red x in the Excel workbook. Future Report Enhancements: Development is underway to improve the report once the newest version of SAS 9.4 and the Add-in for Microsoft Excel are released and installed. The new SAS version allows for greater flexibility with SAS PROC TEMPLATE and also allows for output into different worksheets in Excel, which was originally desired in this report. SAS version 9.3 s version of the Add-in ignores any PROC TEMPLATE code. Conclusion: SAS provides many different tools to allow an analyst to solve problems and there are many ways to arrive at the correct answer. This case study showed how a Base SAS application which would output to Excel by a Business Analyst in approximately 4 hours would be reduced to having an end user create the report in minutes by using the SAS Add-in for Microsoft Office and automating the data feeds using SAS Data Integration Studio. Thus freeing up the BA to work on other things and eliminating the key dependency. More enhancements are in the works as new versions of SAS are rolled out to the organization and the demand for information from upper management increases. Acknowledgements: SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are trademarks of their respective companies. Author Contact Information: Erik S. Larsen Independent Consultant Post Office Box 505 Charleston, South Carolina [email protected] A SAS Alliance Consulting Partner, Affiliate Level 6