ABSTRACT INTRODUCTION

Transcription

1 Automating Concatenation of PDF/RTF Reports Using ODS DOCUMENT Shirish Nalavade, eclinical Solutions, Mansfield, MA Shubha Manjunath, Independent Consultant, New London, CT ABSTRACT As part of clinical trial reporting, a large number of PDF/RTF reports are created and at the completion of a major milestone in a study, it is required to combine all reports in a single document for easy delivery and review. Output Delivery System (ODS) in V9.2 provides new and enhanced capabilities to programmers for reporting and displaying clinical trials results. One such enhancement to ODS is the introduction of PROC DOCUMENT, which provides greater flexibility to modify/update reports. Apart from selectively displaying listing/graphs in the report, we can utilize ODS DOCUMENT capability to concatenate files and replay it in different destinations as a single document. Currently, SAS does not provide a capability to append RTF/PDF files, appending file is feasible only by using third party software outside SAS. We will explore a scenario that requires appending number of PDF files and provides details on a utility program that automates the concatenation process by using the output saved as an ODS DOCUMENT itemstore. This presentation will demonstrate how to dynamically handle output objects that are captured/saved from PROC DOCUMENT. Also illustrated in this paper is how to capture output objects from a reporting procedure, create document folders, list all permanent documents in a library, list and store information of a document in a dataset and copy and replay all objects to PDF, or any other destination (LISTING, HTML, RTF etc). INTRODUCTION With ODS DOCUMENT in production in SAS9 and with new enhancements in V9.2, it is now possible to accomplish the task of concatenating files with relative ease. From our earlier implementation of DOCUMENT procedure, we were aware of modifying bookmark nodes, updating titles, replaying selective reports and so we decided to explore more on other capabilities of this procedure. As a result, this utility was created to automate concatenation process within SAS. This utility provides user the functionality to combine any number of objects and any types of objects (reports, graphs, and tables) stored in different document folders and replay them in a single PDF report. Before implementation of our PROC DOCUMENT solution, concatenation of PDF reports was done by using Adobe s Acrobat Professional product. This was a two step process, one to create PDF reports using SAS and then use Adobe s product outside SAS to combine all individual files and create one document. Similarly, to concatenate RTF files using SAS, there was no easy way to combine reports other than having knowledge of RTF native code and manipulate it in SAS. This process is very well explained in reference 2. But these two processes either required manual interruption or required to learn a new coding language which was not very desirable. Our scenario had lots of reporting programs that created PDF reports of tables, listings, and graphs. For a typical study, the number of reports can go well beyond a hundred. For the purposes of illustration and to explain the core functionality of this utility macro, we will use simplified examples of SAS programs that create listing, graph and summary reports with and without BYGROUPS. These examples will illustrate how the DOCUMENT objects are defined and stored when used with different procedures (PROC 1

2 REPORT, PROC GPLOT) and using different options like BYGROUPS. We will also illustrate how to dynamically handle these objects to replay them to a single PDF or any other ODS destination. ODS DOCUMENT SYNTAX The following section provides the core structure of the utility. We will review the syntax of ODS DOCUMENT which is essential to understand the concepts presented in this paper. For a complete understanding of this procedure, readers are encouraged to read reference 2, which introduces DOCUMENT procedure in greater detail. First, let us look at how a document object is created for each reporting program; we will add documents from three procedures to illustrate how the document information is stored when created with different procedures. Document stores are created by enclosing the procedure within ODS DOCUMENT statements as shown below ODS DOCUMENT NAME =rptout.doc1(write); title1 "Listing of Site By Country"; title2 "Provides Sales By Quarter"; PROC REPORT data = totals NOWD; column dept site quarter sales; RUN; footnote j=r "Created on &sysdate9."; ODS DOCUMENT CLOSE; Opens the DOCUMENT destination and creates the document store doc1. For our application, as we wanted to utilize these objects for concatenation, we wrote the objects to a permanent location given by libref rptout. The NAME= option assigns the name doc1 to the ODS DOCUMENT store that contains the information from reporting procedure program with write access. With write access, if ODS DOCUMENT already exists in SAS session then all its contents are overwritten. This is essential because we only need the latest output every time the reporting macro is executed. If no option is specified then the default is update and every time the report is executed, then new outputs are appended to the existing object in the document store. This is not desirable because during replaying, we will end up displaying all the iteration of this report. We only need the most updated entry in the document. 2 Corresponding statement to close a current document or to stop storing output. Similarly, we will add documents for other procedures, one created by proc report using BY statement and the other from proc gchart that creates a basic graph. ODS DOCUMENT NAME =rptout.doc2(write); title1 "Listing of Site By Country"; title2 "Provides Sales By Quarter"; PROC REPORT data = totals NOWD; by site; column dept site quarter sales; RUN; footnote j=r "Created on &sysdate9."; ODS DOCUMENT CLOSE; ODS DOCUMENT NAME =rptout.doc3(write); goptions reset=all border; title "Total Sales"; footnote j=r "Created on &sysdate9."; 2

3 PROC GCHART data=totals; format sales dollar8.; block site / sumvar=sales; RUN; QUIT; ODS DOCUMENT CLOSE; When using interactive SAS environment, these documents can be explored by issuing odsdocument or odsdoc command in the command line (Fig1 below displays all documents created). In later sections, we will see how to dynamically handle the documents properties, (specifically the document path), to combine all itemstores from individual documents to a single document and replay them to the output destination. Fig1: Documents created in permanent location. MACRO CALL Once all reporting procedures are executed and corresponding PDF reports and DOCUMENT objects created and stored in a permanent library, the next step is to call the utility macro to combine and replay all DOCUMENTS. As our reporting scenario created individual document store from each reporting program, we create a new document in work directory that contains all itemstores from individual DOCUMENT and replay them to a single PDF or any ODS destination requested. The call to the utility macro looks like below 3

4 %AppendFiles(indir =, /*location of the directory to read individual documents */ outdir=, /*Output directory where the concatenated document should be saved */ filename=, /* filename of the final document */ filetype= /*the output file type required (permissible values are PDF RTF HTML LISTING delimited by space).*/ ); The processing sequence of the utility macro is 1. Create a temporary dataset containing a list of all document names located in the permanent directory referred by indir parameter. 2. Loop through each document to get the property information of itemstores in each document and output this to another dataset. 3. Finally, copy all objects to a temporary document to replay to a PDF destination or any destination requested through filetype parameter. Sounds like a complex process but actually it simpler when we look at the actual program, so let s jump into it. This process is depicted in figure2 below Individual documents created from reporting programs Create a list of documents to concatenate and loop through each of them to extract path information of each object Copy path information of reporting objects from all documents to a temporary new document Replay all objects from new document to the requested output destination(s) Document 1 (a) Report #1(b) New Document Document 2 Report #1 Report #2.... Document n Document 1 Document 2 Document n Document 1/Report #1 Document 2/Report #1 Document 2/Report #2.. Document n/graph #1.. Graph #1 Fig2: Utility Processing First step is to get a list of documents created during the reporting process. Using PROC DOCUMENT, we can get a list of all documents that were created and stored in a permanent location during the reporting process by using following code 4

5 libname rptout "&indir"; ods output documents = ods_obj_library (keep=name); PROC DOCUMENT; doc library=rptout quit; ods output close; 4 3 Creates a libname reference from the indir input parameter. 4 Creates a temporary dataset ods_obj_library containing the document names. The documents option creates a list of all documents stored in the location referenced by rptout. DOC statement lists the ODS DOCUMENTs in SAS library rptout in alphabetical order. 5 Figure3 below shows the content of temporary dataset ods_obj_library. Figure3: List of documents Next, from the temporary dataset ods_obj_library created above, we get a count of all documents and create a macro variable of each document name as shown below data _null_; set ods_obj_library end=last; call symput(compress("name" _n_),trim(left(name))); if last then call symput( 'lib_count', trim(left(put(_n_,best.))) ); run; This is required to loop through each document and create temporary dataset to store path information of all itemstores, which is explained below ODS DOCUMENT NAME = all_doc(write); ODS DOCUMENT CLOSE; %if &lib_count ne 0 %then %do; /* count not zero*/ %local _j; %do _j = 1 %to &lib_count; /* document loop */ 8 ods output properties = ods_obj_properties; PROC DOCUMENT NAME = &&name&_j; list /levels = all; quit; ods output close; 9 ods listing; 6 Creates a new document in work folder to temporarily hold all itemstore objects to be replayed 7 It is important to note that, the newly created document must be closed. If not closed then below message is displayed in the log when trying to add itemstores later 5

6 ERROR: Cannot open document Work.all_doc. An application already has it open. Did you forget to do "ODS DOCUMENT CLOSE;"?. 8 Checks for zero counts, if no documents found at the rptout location then exit the utility else loop through each document. 9 Creates a temporary dataset with document properties containing path information of each itemstore in the document. Figure4 below shows listing of document properties created during each iteration of the loop. This information is also stored in ods_obj_properties dataset created above. At each iteration of the loop, the contents of this dataset is updated with property information of the document in current loop. For Ex: if it s the first loop then ods_obj_properties dataset will contain properties of Rptout.Doc1 document store and so on Figure4: Document properties Once we get the path information of all items in the document store, the next step is to iterate through each of the entries and select the ones to be replayed in the final document. Specifically, we are interested in the objects with Type other than Dir. In our example, these entries are highlighted above. To extract only these entries we need to loop through each document store to get a count and path of entries and copy those to a new document to be replayed later in a single document. This is explained below 6

7 data _null_; set ods_obj_properties end=last; if type ne "Dir" then do; count+1; call symput('path' trim(left(count)),path); end; if last then call symput('total',count); run; PROC DOCUMENT name=&&name&_j(read); doc all_doc; %do i=1 %to &total; copy &&path&i to \work.all_doc\; %end; quit; 14 %end; /* document loop */ 10 Creates a macro parameter of the path for each entry corresponding to a Table/Graph in the document. 11 Gets the total count of objects in the current document. This count may vary depending on the options used to create the report. In our example, we created one report without by group and one with by group. For the table created without by group there is only one entry for table but there will be a corresponding entry in the document for each by group for the table created using by groups. (Refer Figure 3). 12 Specifies the name of current document in the loop and its access mode (Read). 13 Opens the newly created document (in step 6 above) in update mode (default). Copies all objects from individual document to the new document in work directory. 14 Once we loop through all individual documents and copy all objects to a single document, the next step is to replay these objects to the requested output destination. The code to replay all objects is below %if %length(&filetype) > 0 %then %do; %local _i _file; %let _i=1; %do %while(%scan(&list,&_i,%str( )) ne %str()); %let _file = %scan(&list,&_i,%str( )); %if %upcase(&_file) eq PDF %then ods pdf file= &outdir\&filename..pdf ;; %if %upcase(&_file) eq PDF %then ods rtf file= &outdir\&filename..rtf ;; %if %upcase(&_file) eq HTML %then ods html file= &outdir\&filename..htm ;; %end; %let _i = %eval(&_i+1); %end; PROC DOCUMENT name=work.all_doc; replay / dest=( &filetype ); quit; ods _all_ close; ods listing; %end; /* count not zero*/ 7

8 15 Through the filetype macro parameter, there can be multiple output destinations requested. Based on this list, loop through the list and open appropriate ODS destination. The final document will be stored in the libref passed through outdir macro parameter. 16 Replays all objects from all_doc document to destinations requested through filetype parameter. Figure5 below shows screenshot of concatenated reports in RTF. Figure5: Concatenated output. Above example illustrated how we can dynamically extract object s path information and concatenate them using list, copy, and replay statement in PROC DOCUMENT. Apart from these statements, PROC DOCUMENT offers numerous other options to enhance the output. We can insert notes before or after the table, update bookmark nodes and also update the title and footnotes. Using these options user can have complete control to update and replay the objects. We will not discuss these options but readers are encouraged to refer ODS user guide for more information on enhancements. Below are a few assumptions to remember before running this utility. 1. All individual reporting programs are executed and document objects are created and stored in a permanent library. 2. There is at least one document output object created by executing the reporting macro. 3. The ordering of reports in the final document depends on the order in which the original documents are read. 8

9 CONCLUSION Using PROC DOCUMENT procedure it is now possible to streamline the process of concatenating PDF reports which otherwise was done using third party software outside SAS. The enhancement options in PROC DOCUMENT provide a complete control on how a report should be structured and displayed. Also, as the objects are stored in a permanent library we have the flexibility to pick and choose what reports to replay in the concatenated document without re-running the reporting programs. REFERENCES 1. SAS(R) 9.2 Output Delivery System: User's Guide 2. Have It Your Way: Rearrange and Replay Your Output with ODS DOCUMENT by Cynthia L. Zender, SAS Institute Inc., Cary, NC 3. Utilizing SAS to Automate the Concatenation of Multiple Proc Report RTF Tables by Andrew Newcomer DISCLAIMER SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the authors at: Shirish Nalavade eclinical Solutions, A Division of Eliassen Group 603 West St Mansfield, MA Work Phone: snalavade@eclinicalsol.com or at shirish.nalavade@gmail.com Shubha Manjunath Independent Consultant, New London, CT skashyap301@gmail.com Code presented in this paper is simplified version of the production code to illustrate the core functionality of the utility. Sample code is available from the author upon request. 9