Importing Excel Files Into SAS Using DDE Curtis A. Smith, Defense Contract Audit Agency, La Mirada, CA

Transcription

1 Importing Excel Files Into SAS Using DDE Curtis A. Smith, Defense Contract Audit Agency, La Mirada, CA ABSTRACT With the popularity of Excel files, the SAS user could use an easy way to get Excel files into SAS. There are many good methods to do so, one of them being Dynamic Data Exchange (DDE). A SAS user can use DDE within SAS code to make importing an Excel file routine and easily repeatable. The author will show how he uses DDE within SAS to routinely import into SAS lookup tables downloaded from an ERP system as Excel files. The basic DDE statements needed to accomplish this are surprising simple. INTRODUCTION What in the world is DDE and why use it? Good question... DDE is Dynamic Data Exchange and is a method of accessing data from one MS-Windows application by another. As a SAS user, you can use DDE within a DATA step to import data into SAS and export data from SAS. Using DDE involves SAS code and statements that other MS-Windows applications understand. But, why bother with DDE when other methods are available? What about the IMPORT and EXPORT procedures? Or, Open Database Connectivity (ODBC)? How about SAS Enterprise Guide? These alternatives are useful under the right conditions. For example, the PROC IMPORT and PROC EXPORT are simple to use, but are limited in the way you can define your data and these procedures require you license SAS/ACCESS For PC Formats. Bummer. ODBC is also simple to use, but importing data into SAS using ODBC requires you license SAS/ACCESS For ODBC. Bummer again. And, the new Excel LIBNAME engine? Oh, that also requires you license SAS/ACCESS For PC Formats. Bummer times three. SAS Enterprise Guide is very simple, and does not require you license anything other than BASE SAS. Hmm. Oh yes, SAS Enterprise Guide doesn t offer quite the flexibility as DDE, because when using DDE you have all the features of the SAS DATA step. Also, using DDE in a SAS DATA step makes it easy to adapt the import process for repeatable uses. A SAS DATA step using DDE only requires BASE SAS running under MS-Windows. In my applications, I routinely download tables from an ERP system that likes to dump the data into an Excel file. Each time I download a fresh table of data into the Excel file, I want to re-run my SAS import process to create a fresh SAS table. I download several similar tables from the ERP system and like to use macro variables within my SAS code to import any one of the several Excel files downloaded from the ERP system. So, a DATA step using DDE works very well for my situation. Throughout the remainder of this paper I will describe only the process of importing Excel worksheets into a SAS data set, as the purpose of this paper is to provide a working introduction to the use of DDE to get Excel data into SAS. To that end, I will use a straightforward example. DDE BASICS Dynamic Data Exchange is a method to dynamically exchange data between MS-Windows applications. DDE uses a client-server relationship to enable client applications to request data from a server application. In the context of SAS and DDE, SAS is always the client, regardless of which application is receiving the data. SAS request data from server applications, sends data to server applications, or sends commands to server applications. DDE has many uses: the one that we will explore is acquiring data from another MS-Windows application, namely Excel. A search of the Microsoft web page for macrofun will reward you with the MS-W indows HLP file macrofun.hlp that will provide all of the wonderful Excel macro commands that you can use within the DDE statements. Although these macros are older technology, for use with DDE they get the job done. COMMUNICATION IS KEY To use DDE, both SAS (the client) and the application owning the data you wish to import (the server) must be running, and the data that is to be read must be open. Communication between two MS-Windows applications is accomplished using what is known as the DDE Triplet. The triplet requires three parameters, in the following form: 1

2 'application-name topic! item' where: application-name is the executable filename of the server application. For example, the application-name for Microsoft Excel is excel. topic is the subject of conversation (between SAS and the DDE server application). This is typically the full drive, path, and filename of the spreadsheet with which you want to share data. As Watts points out, only enough information for identifying a specific worksheet is actually required. For example, If a single workbook is opened on the desktop,"excel sheet1!..." will be enough to identify it. When specifying the entire Excel workbook and worksheet, you enclose the workbook name in brackets [ ]. item is the range of conversation specified between the client and server applications. When reading data from Excel, this will be a range of cells. Notice that the triplet is quoted, the application-name and topic are separated by a vertical bar, and the topic and item are separated by an exclamation mark!. There is an easy to use trick to determine the exact string for the DDE Triplet. Use Excel to copy the desired range of cells to the MS-Windows clipboard. Then, while in SAS for MS-Windows, use the pull-down menu option Solutions\Accessories\DDE triplet. You will see something like the following: You can then select the text, copy, and paste it into your SAS code. In my example the DDE Triplet looks like this: Excel C:\WUSS\[Names Workbook.xls]Names Worksheet!R1C1:R11C6 Microsoft defined the DDE topic SYSTEM to enable commands to be sent to the server application. When sending commands to a server application, we need only specify the application-name and the special DDE topic SYSTEM. Therefore, there is no need for an item parameter when only sending commands to Excel. Thus, the DDE triplet for sending commands to Excel contains only two parameters, with the application-name being set to EXCEL and the topic being set to SYSTEM. The functionality of this special two level DDE Triplet is different from the normal DDE Triplet: whereas the normal DDE Triplet opens a link to a specific cell-range in an Excel workbook/worksheet, the two-level triplet will allow SAS to send system-level commands to the server application (which in our discussion is Excel). IMPORTANT SYSTEM OPTIONS NOXSYNC The NOXSYNC instructs SAS to continue processing as soon as the operating system command is issued. NOXW AIT The NOXWAIT instructs SAS to automatically close the spawned prompt window after the specified command 2

3 has completed. Without this option, the command prompt window just won t close. IMPORTANT FILENAME/INFILE OPTIONS NOTAB The NOTAB option instructs SAS to ignore tab characters between variables. SAS expects to see a tab character placed between each variable that is sent across the DDE link when importing into SAS. Similarly, SAS places a tab character between variables when SAS sends data across the DDE link when exporting from SAS. When the NOTAB option is used, SAS accepts character delimiters other than tabs between variables and the NOTAB option tells SAS not to convert tabs sent from the Excel worksheet into blanks. Therefore, the tab character can be used as the delimiter between data values. DLM The DLM= option specifies the delimiter character, and '09'x is the hexadecimal representation of the tab character. MISSOVER The MISSOVER option prevents SAS from going to a new input line if it does not find values in the current line for all the INPUT statement variables, i.e. one or more cells are blank. W hen the MISSOVER option is used, if SAS encounters a blank cell then SAS sets the input value to missing, and continues to read until the expected end of the input line. DSD The DSD option specifies that two consecutive delimiters represent a missing value. The default delimiter is a comma. USING DDE TO READ EXCEL DATA Okay, we are ready to put some code together to read Excel data into a SAS DATA step. I m excited. The first step is to launch (invoke) Excel. INVOKE MICROSOFT EXCEL You can invoke Excel using the SAS X command. options noxwait noxsync; x '"c:\program files\microsoft office\office11\excel.exe"'; The path for Excel will differ depending on the version you are using and whether or not you installed Excel using the default destination. Searching the hard drive for excel.exe will pinpoint the proper location for the path. The entire x command must be enclosed in single quotes. If the path contains blanks, then the entire path must be enclosed in double quotes. You use the NOXWAIT and NOXSYNC options to ensure that the x command executes independently from the SAS session and the x command window closes. Obviously, if you enter the path or executable incorrectly, Excel will fail to launch. SAS must wait until Excel has completed loading itself before it can begin to exchange data. You cause SAS to wait for Excel to load by using a DATA _NULL_ step with the SLEEP function. How long does SAS need to sleep? That will depend on your computer - trial and error will help to determine the time value to use in the SLEEP function. A value larger than necessary will simply mean SAS will pause longer than necessary, which is not critical. 3

4 x=sleep(5); If you try to proceed with your DDE application before Excel finishes loading, you will get an error message in the SAS log that looks like the following: ERROR: Physical file does not exist, excel system. NOTE: The SAS System stopped processing this step because of errors. Once Excel is running we can define and then open our Excel workbook and worksheet. DEFINE AND OPEN A MICROSOFT EXCEL FILE You are now ready to tell SAS to open an existing Excel workbook. You start by creating a special fileref that will allow the DATA step to communicate with Excel via DDE. This is that two-level DDE Triplet. filename ddecmd dde 'excel system'; Here you define ddecmd as a fileref that signifies DDE Command (of course, the fileref can be any literal you want) and DDE is the SAS device-type keyword that tells SAS you want to use DDE. Next, you use a DATA _NULL_ step to pass to Excel via DDE the Excel macro command to open the desired Excel workbook. You link to the previous FILENAME statement using a FILE statement that references the previously defined fileref, ddecmd. Then, you issue a PUT statement to pass to the two-level DDE Triplet a string containing the Excel macro command FILE-OPEN and the literal drive\path\filename of the desired Excel workbook. file ddecmd; put '[FILE-OPEN("c:\wuss\names workbook.xls")]'; Note that the entire PUT string is in single quotes and the entire FILE-OPEN macro statement is enclosed in brackets [ ]. This will also work using the Excel macro OPEN, as seen below. file ddecmd; put '[OPEN("c:\wuss\names workbook.xls")]'; Running this DATA _NULL_ step will cause Excel to open the desired workbook. The results in the SAS log will look something like the following: NOTE: The file DDECMD is: DDE Session, SESSION=excel system,recfm=v,lrecl=256 NOTE: 1 record was written to the file DDECMD. The minimum record length was 41. The maximum record length was 41. If you enter the path or filename incorrectly, Excel will fail to open your workbook. You will get an error message in the SAS log that looks like the following: NOTE: The file DDECMD is: DDE Session, SESSION=excel system,recfm=v,lrecl=256 ERROR: DDE session not ready. 4

5 FATAL: Unrecoverable I/O error detected in the execution of the data step program. Aborted during the EXECUTION phase. NOTE: 0 records were written to the file DDECMD. NOTE: The SAS System stopped processing this step because of errors. Another unfortunate mistake you can make is to try to open your Excel workbook without first launching Excel. If you try that, you will see an error message in the SAS log that looks something like the following: ERROR: Physical file does not exist, excel system. NOTE: The SAS System stopped processing this step because of errors. If you try opening your Excel workbook with Excel successfully launched but without first establishing the DDE link with the FILENAME statement as shown above, then the SAS log won t reflect an error, but the Excel workbook won t open. Paul Choate suggests an alternate method for launching Excel and opening the desired Excel workbook: simply use %sysexec to launch the desired Excel workbook. %sysexec "c:\wuss\names workbook.xls"; As long as the xls extension is registered in MS-Windows to Excel, then MS-Windows will launch Excel and open the desired Excel workbook. You are now ready to define a SAS fileref to the range of cells within the Excel workbook and worksheet you wish to read data from. READ THE MICROSOFT EXCEL FILE INTO SAS Let s see what your Excel workbook/worksheet looks like. 5

6 Now that you have established an open DDE link to your open Excel workbook, you are ready to tell SAS just where within that workbook you want to read data. You do this with a FILENAME statement that uses the DDE Triplet. filename xlin DDE "excel c:\wuss\[names workbook.xls]names worksheet!r2c1:r65536c6"; Here you define xlin as a fileref that simply indicates Excel In (of course, the fileref can be any literal you want) and DDE is the SAS device-type keyword that tells SAS you want to use DDE. This statement includes the full DDE Triplet that defines Excel as the application-name, the c:\wuss\names workbook.xls, names worksheet workbook/worksheet as the topic, and the range of cells r2c1:r65536c6 as the item. Notice that of the full drive\path\filename, only the entire Excel workbook name is placed inside brackets [ ]. Notice the separation of the three parameters using the vertical bar and the exclamation mark!. Notice also that the item starts on row 2 to skip the column headings and ends with row (Excel s maximum number of rows - how did they decide on that number?). I have found that SAS won t create blank rows in the resulting SAS file when I define more rows in the item that contain data. So, I always set the range to the maximum. But, I set the columns to just those that contain data. W hy? Because when I download fresh data the structure will be the same, so I know the number of columns that will have data. But, the number of rows will vary each time I re-download the table. Running this FILENAME statement does not cause SAS to write anything interesting to the SAS log. Next, you can use the data you just defined from your Excel workbook in a DATA step. Simply use you favorite DATA step statements and options to create the perfect SAS data set from the Excel data. The only trick is to use the fileref you created in the above FILENAME statement to reference the Excel worksheet via DDE. There are some important options that may be necessary, as we discussed earlier. Notice on the INFILE statement that you may need to define the DLM=, NOTAB, MISSOVER, and DSD options to get SAS to read your Excel workbook the way you want. Other than that, the rest of the DATA step is composed of your favorite statements and options (the advance of using DDE). data wuss.employee(label='employee ROSTER' index=(emp_id)); infile xlin dlm='09'x notab missover dsd; informat FIRST $20. LAST $20. DEPT $4. EMP_ID $6. TITLE $30. RATE 8.2; input FIRST LAST DEPT EMP_ID TITLE RATE; format FIRST $20. LAST $20. DEPT $4. EMP_ID $6. TITLE $30. RATE 8.2; label FIRST = 'EMPLOYEE FIRST NAME' LAST = 'EMPLOYEE LAST NAME' DEPT = 'DEPARTMENT' EMP_ID = 'EMPLOYEE ID' TITLE = 'EMPLOYEE TITLE' RATE = 'PAY RATE'; After running the DATA step, SAS will write to the SAS log something like the following: NOTE: The infile XLIN is: DDE Session, SESSION=excel c:\wuss\[names workbook.xls]names worksheet!r2c1:r65536c6, RECFM=V,LRECL=256 NOTE: 794 records were read from the infile XLIN. The minimum record length was 23. The maximum record length was 58. NOTE: The data set WUSS.EMPLOYEE has 794 observations and 6 variables. If you mistype the drive\path\filename\worksheet name in the FILENAME statement, or if you don t open the workbook in Excel first, SAS will let you know with the following error in the SAS log: 6

7 ERROR: Physical file does not exist, excel c:\wuss\[names workbook1.xls]names worksheet!r2c1:r65536c6. NOTE: The SAS System stopped processing this step because of errors. WARNING: The data set WUSS.EMPLOYEE may be incomplete. When this step was stopped there were 0 observations and 6 variables. WARNING: Data set WUSS.EMPLOYEE was not replaced because this step was stopped. CLOSE THE EXCEL FILE Lastly, you may want to close the Excel workbook using the same method you used to open it, but with the FILE-CLOSE Excel macro. The DATA _NULL_ step below has an added bonus: the QUIT Excel macro will close Excel. file xlin; put '[FILE-CLOSE("c:\wuss\names workbook.xls")]'; put [QUIT()] ; THE COMPLETE PROGRAM Here s the complete code, suitable to copying and pasting to your favorite SAS session. /* set options and invoke Excel using DDE */ options noxwait noxsync; x '"c:\program files\microsoft office\office11\excel.exe"'; x=sleep(5); /* open Excel workbook */ filename ddecmd dde 'excel system'; file ddecmd; put '[FILE-OPEN("c:\wuss\names workbook.xls")]'; /* specify desired Excel worksheet cell range */ filename xlin DDE "excel c:\wuss\[names workbook.xls]names worksheet!r2c1:r65536c6"; /* read Excel files using DDE into SAS data set*/ data wuss.employee(label='employee ROSTER' index=(emp_id)); infile xlin dlm='09'x notab missover dsd; informat FIRST $20. LAST $20. DEPT $4. EMP_ID $6. TITLE $30. RATE 8.2; input FIRST LAST DEPT EMP_ID TITLE RATE; format FIRST $20. LAST $20. DEPT $4. EMP_ID $6. TITLE $30. RATE 8.2; label FIRST = 'EMPLOYEE FIRST NAME' LAST = 'EMPLOYEE LAST NAME' DEPT = 'DEPARTMENT' EMP_ID = 'EMPLOYEE ID' TITLE = 'EMPLOYEE TITLE' RATE = 'PAY RATE'; /* close Excel workbook and close Excel */ 7

8 file xlin; put '[FILE-CLOSE("c:\wuss\names workbook.xls")]'; put [QUIT()] ; CONCLUSION Don t let anyone tell you DDE is dead. It continues to do its job reading and writing data between MS-Windows applications. It is especially useful because we can use all the power of the SAS DATA step with DDE, and DDE requires we only have BASE SAS. REFERENCES SAS Institute Inc. SAS Companion For Windows. Cary, NC: SAS. SAS Institute Inc. SAS Institute Technical Support document TS325. Christopher A. Roper, Intelligently Launching Microsoft Excel from SAS, Using SCL Functions Ported to Base SAS. Proceedings of the Twenty-Fifth Annual SAS Users Group International Conference. RHA (Minisystems) Ltd. Dynamic Data Exchange (DDE) and NetDDE FAQ. UCLA Academic Technology Services SAS Library, Reading Data into SAS. Koen Vyverman Using Dynamic Data Exchange to Export Your SAS Data to MS Excel Against All ODS, Part I. Proceedings of the Twenty-Seventh Annual SAS Users Group International Conference. Perry Watts Using Single-Purpose SAS Macros to Format EXCEL Spreadsheets with DDE. Proceedings of the Thirtieth Annual SAS Users Group International Conference. SAS-L Posting, Paul Choate ACKNOWLEDGMENTS I appreciate all the fine technical support and advice from the sources listed above as well as the many fine posts on SAS-L. CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the author at: Curtis A,. Smith P.O. Box Fountain Valley, CA Fax: [email protected] SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are trademarks of their respective companies. 8