PS004 SAS , using Data Sets and Macros: A HUGE timesaver! Donna E. Levy, Dana-Farber Cancer Institute

Transcription

1 PS004 SAS , using Data Sets and Macros: A HUGE timesaver! Donna E. Levy, Dana-Farber Cancer Institute Abstract: SAS V8+ has the capability to send by using the DATA step, procedures or SCL. But what if ongoing s must be generated at unscheduled/scheduled times? This SAS program has been developed to 'automatically' send a unique, customized and interactive to the appropriate recipient(s). The program uses SAS , MERGING datasets, DO loops, SUBSTR and SCAN functions as well as MACROS. Introduction: Each day, we spend countless hours reading, sending and responding to s. Though many s that are sent cannot be automated due to their unique content, there are many instances when an automatic can be generated. For example, when similar text is sent to multiple individuals --- this is a perfect opportunity to save some time and effort by using a program to do the work for you. This program does just that by using many features of SAS V8 including: SAS , MERGING data sets, DO loops, MACROS, MACRO VARIABLES and most importantly SUBSTRing searches and SCANS. Background: Our group maintains a website that includes general public information as well as specific details pertaining to data capture, protocol amendments and other administrative details. The webmasters in general are not responsible for maintaining content pertaining to any of the protocols. They simply receive the request to update or modify the material on the website, identify which documents that need to be changed/updated/replaced, and then update the document with the new material that has been submitted. A quality assurance (QA) check was implemented as there were several protocols active simultaneously each with their supporting material being maintained on the website. The QA was put in place to verify that the correct document was sent by the sender and that the document was correctly replaced on the website. Also the affiliate members (everyone in the group and/or data managers) would also need to be informed that there was updated material on the website as well as where this material was located on the site. Since a log of changes to the site was already being maintained including who requested the change and who completed the change, little additional work was needed except for writing the SAS code. The Content of the Example 1 below is the general that is sent to the person who made the request to change/update the document on the website. The type of document, determines the content of the as well as additional recipients of notification of the update. In example 1 and 2 below the update is an SAE (Serious Adverse Event). In this case, the general is sent to the requestor and a similar is also sent to all the affiliates in the group according to the rules of the program (example 2). For certain documents like updated address listings and contact information, this document is also saved on the administration site. The type of document (done manually) will determine whether the note pertaining to the administration site is included in the to the requestor or not

2 Example 1: to the Requestor: To: CC: Subject: DFCI/ALL Website Protocol SAE 28 Ginger, We have completed your website request pertaining to Protocol SAE 28. Please go to: to check the accuracy of your document. You will need your DFCI/ALL username and password to access this document. Please respond to Donna ( within 1 working day of receiving this . Note: This document has been added to the administration site: for any future updates and modifications. Example 2: to the Group: To= dfciconsortium@partners.org CC= all_webmaster@jimmy.harvard.edu Subject=DFCI/ALL Protocol SAE 28 Dear DFCI/ALL Consortium Member, To view Protocol SAE 28 Go to: You will need your username and password to access these files. If you have any questions or problems accessing the file, please all_webmaster@jimmy.harvard.edu Thanks. There are 2 Excel files that are necessary to use in the program. In one Excel document ( list.xls) that is used in the program (blue, bold) contains s of all the people that could send updates for the website [file 1 Listing]. This auto program is used by the three webmasters at our institution and we all do not use the same software. As a result, this file is kept as a shared file rather than within our program. The other Excel document (red, italic) contains the name of the document that was changed/updated, the directory that the document was located, who requested the changes and who performed the changes on the website [file 2 - Log]. See below for content examples of the first [file 1] and the second Excel file [file 2]

3 File 1: addresses of all individuals who could submit changes for website updates Name Kathryn McBride Kirsten Tunney Montse Rue Donna Levy Ginger Dalton Jennifer Cronin File 2: Listing of Content Changes Performed on the website Req # Request Date Request From Request To Specific Request 13 01/21/03 Ginger Dalton Kathryn McBride 14 01/21/03 Ginger Dalton Kathryn McBride 15 01/21/03 Kirsten Tunney Kirsten Tunney Amendment 16 approved for posting 01/22/03 Amendment 16 protocol approved for posting 01/23/03 DMC recommendations Nov /22/ /21/03 Ginger Dalton Kathryn McBride Post SAE /22/03 Date Completed filepath /consortium/00001/protocol /amendments/amend00001 _16.pdf /consortium/00001/protocol /protocol00001.pdf /consortium/dmcopen/dmcr ecommend2002.pdf /consortium/sae/sae00001/ sae00001_28.pdf Added to administration page No No No No Key Components of the Program: list.xls (file 1): list of anyone who contributes information to the website logofwebsite.xls (file 2): contains who the request came from, who fulfilled the request, the file that was changed, the path to the file and whether an editable version sits in the administration folder IF obs IN (&linenum) THEN DO; linenum is the only line in the program that changes from run to run. As a result of this line of code, the data set only includes the line of information that will be sent in the . CALL SYMPUT: creates macro variables out of data contained in the above data sets. The SYMPUT routine assigns a value produced in the DATA step to a macro variable. If the macro variable does not exist, the symput routine creates it. SYMPUT (name, value) creates a global variable name= the name of the macro variable value= contains the value to be assigned Must use the symput routine as part of the call statement How many s will be sent and what will be contained in the ? The following lines of code determine how many s will be sent: IF added_to_administration_page= <Yes, No> IF SUBSTR of file location= <substring> to_group= <Yes, No> to_dm= <Yes, No> - 3 -

4 In the program attached, there is always a minimum of 1 sent in total. If the file is maintained on the administration page, there will be an additional 3 lines of code in the with the path to the editable document on the administration page. The SUBSTR function searches the file name and path to determine what type of document the file is ie. QOL, SAEs, Amendments etc. If the file pertains to QOL, then an goes to the data managers (to_dm= Yes ) but not to the entire group. If the file is an SAE or an amendment, then an goes to the entire group (to_group= Yes ). Going Through the Code: Line 10 [%LET linenum=16;] This is the only line of code that needs to be changed each time you run the code. This global variable indicates that line 16 of the log (file 2) will be run through the program. *** need to add where used in the program *** Line 20 [%LET to_group='no'] and line 30 [%LET to_dm='no';] Line 20 and line 30 are initializing the dichotomous variables to_group and to_dm to 'No. Currently an will not be sent to the entire group or the data managers. Depending upon the particular change that has been requested, will determine whether or not these variables will be reassigned to 'Yes' and thus an will be sent to the group and/or data managers. Line [DATA WORK. ListFrom;] and Line [DATA WORK. ListTo;] This is file 1 ( Listing) being used as a dataset and is used twice. We will be merging these 2 files with file 2 (Log) to establish who the request came from (variables: request_from, request_from_ ) as well as who will be performing the request (variables: request_to, request_to_ ). Line 60 [MERGE LogOfWebsite ListFrom;] Merges the log (file 2) with the listing (file 1). These lines of code will allow us to determine who the request for the update came from as well as determining their address. Line 70 [DATA LogOfWebsite;] Merges the log with the list to (so now the addresses 'to' have been added to the dataset (the webmaster's ). This step would not be needed if there was only 1 webmaster. Line [CALL SYMPUT('request_number', request_number);] Is creating macro variables (? global variables) out of the merged data set that we will use in our . CALL SYMPUT(<macro variable name>, <original variable value>); In this case, I named the macro variable the same name as the original variable. This is not necessary. I only did this for simplicity of identifying variables and values. Line 90 [ _to_name=scan(request_from,-2, ' ');] The SCAN function searches through the specified variable in search of the specified string. For example: _to_name=scan(request_from, -2, ' '); If _to_name='donna Levy', in this case, we are scanning through this variable starting from the end (the negative of the -2 indicates that. If you wanted to start from the beginning you would have used a 2 or +2) and searching for the second space (' ') which is located in front of the 'D' in 'Donna' technically. I look at it like this: start and the end of the string then jump over 2 'words', then take the word to your left then assign the variable that value

5 Line 100 [IF added_to_administration_page IN ('Yes','yes') THEN DO;] Is a variable contained in the log (added_to_administration_page). If this field is yes, then 3 more lines of text will be added to the to let the requestor know that the file has been added to the administration page. Line 110 [CALL SYMPUT('adminyesno1',adminyesno1);] Once again we use CALL SYMPUT to assign a value to a macro value. Line [amendsae1='protocol' ' ' SUBSTR(SCAN(filepath,-1,'/'),4,5) ' ' UPCASE(SUBSTR(SCAN(filepath,-1,'/'),1,3)) ' Number ' SUBSTR(SCAN(filepath,-1,'/'),10,2); amendsae2="&livesite./sae/saesummarylist/saesummary.xls"; If the file is an SAE (determined by the SUBSTR function and scanning the filepath variable as we described previously), then the title is created for the to identify what has been updated on the website as well as the link to the file on the website which will also be included in the . The is simply adjoining 2 text components. For example do 123 is simply do123. Line 130 [to_group='yes';] In this example, we have determined that the file is an SAE, then the to_group is assigned to be a 'Yes' thus an will be sent to the entire group. Line 140 [protocol=substr(scan(filepath,-1,'/'),6,5);] If the file is an amendment or an SAE, we need to determine what protocol has been affected. This code will search the string to determine what protocol has been amended. Line 150 [to_group='yes';] Once again, if the file is an amendment, an will be sent to the group (to_group assigned to equal 'yes'). Line 160/170 [IF SUBSTR(SCAN(filepath,-1,'/'),1,3)='qol' THEN DO; to_dm='yes';] If the file pertains to quality of life (qol), then this only goes to the data managers (to_dm assigned to be equal to 'yes'). Once again, to determine what type of file has been submitted, the string is searched for key components, in this case qol. Line 180/190/200 [% GROUP; % DM; % REQUESTOR;] Lines and 200 calls the macros (group, data manager and requestor). When we go to the group macro, if to_group is Yes then an will be sent to the group. Otherwise an will not be sent to the group (to_group= No ). Similar events occur with the data manager macro ( dm). As previously stated, an always goes to the person who made the request for the website file update. Tying it all Together: Although this program has been designed for a very specific example, the idea can be easily expanded and applied to any individual s or group s needs. The text of the is easily edited and the program is expandable if another subgroup forms within the consortium. By using features of SAS, specifically , SUBSTR, SCAN and MACROS, a tedious job of multiple s with the same or similar content, can be completed by simply --- the push of a button

6 The Code: OPTIONS NOCENTER LS=256 SGEN MERROR; **** **** * Enter the line number that you would like to send an out for here; %LET linenum=16; *line 10; ***** ***** * ; * Where the LogofWebsite.xls is located; %LET logpath=w:all_web; * Where the list.xls as well as datamanager.xls is located; %LET path=w:all_web\auto ; * ; * The URL to the live site; %LET livesite= * ; * The URL to the development site; %LET devsite= * ; * THE URL to the administration page; %LET adminsite=&livesite./administration; * ; * Protocols that any s may apply to. When a new protocol is added be sure to; * update this line of code; %LET protocols=00001, 95001, 01175, 99137; * ; * to_group= sent to group (yes, no). Initially set at no. Program will; * determine and update this variable if the file that is updated is an SAE or ; * Amendment. If in the future, the decision is made to send more s to ; * the group (other than for an SAE or Amendment), then the decision rules ; * within the code, for this variable will need to be updated ; %LET to_group='no'; * line 20; * to_dm = sent to DMs only (yes, no). Initially set at no. Program will; * determine and update this variable if the file that is updated is an QOL ; * Schedule. If in the future, the decision is made to send more s only ; * to the DMs;(other than for QOL schedule), then the decision rules within the; * code, for this variable will need to be updated ; %LET to_dm='no'; * line 30; - 6 -

7 * ; * where macros are located that need to be included in this program; %LET macropath=f:todo\auto ; %INCLUDE "&macropath\ .mac"; ************************ READING IN DATA ************************************; PROC IMPORT OUT=WORK.LogOfWebsite DATAFILE="&logpath.\LoGofWebsiteRequests.xls" DBMS=EXCEL2000 REPLACE; GETNAMES=YES; PROC IMPORT OUT=WORK. List DATAFILE="& path.\ list.xls" DBMS=EXCEL2000 REPLACE; GETNAMES=YES; PROC IMPORT OUT=WORK.DataManagers DATAFILE="& path.\datamanagers.xls" DBMS=EXCEL2000 REPLACE; GETNAMES=YES; **** * Data set with the and name of the person who submitted the request; * for the website update; DATA WORK. ListFrom; * line 40; SET WORK. List; request_from= name; request_from_ = ; * Data set with the and name of the person who received the request; * for the website update; DATA WORK. ListTo; * line 50; SET WORK. List; request_to= name; request_to_ = ; PROC SORT DATA=WORK.LogOfWebsite; BY request_from; PROC SORT DATA=WORK. ListFrom; BY request_from; DATA LogOfWebsite; MERGE LogOfWebsite ListFrom; *line 60; BY request_from; PROC SORT DATA=WORK.LogOfWebsite; BY request_to; PROC SORT DATA=WORK. ListTo; BY request_to; DATA LogOfWebsite; * line 70; MERGE LogOfWebsite ListTo; BY request_to; - 7 -

8 DATA LogOfWebsite; SET LogOfWebsite; IF request_number=. THEN DELETE; PROC SORT DATA=LogOfWebsite; BY request_number; DATA LogOfWebsite; SET LogOfWebsite; obs=_n_; %MACRO DATASETLINE; * this macro creates macro variables that will be included in the auto s; IF obs IN (&linenum) THEN DO; CALL SYMPUT('request_number', request_number); * line 80; CALL SYMPUT('request_date', request_date); CALL SYMPUT('request_from', CALL SYMPUT('request_to', CALL SYMPUT('specific_request', CALL SYMPUT('date_completed', CALL SYMPUT('other', request_from); request_to); specific_request); date_completed); other); CALL SYMPUT('filepath', filepath); CALL SYMPUT('request_from_ ', request_from_ ); CALL SYMPUT('request_to_ ', request_to_ ); _to_name=scan(request_from,-2, ' '); *line 90; CALL SYMPUT(' _to_name', _to_name); FORMAT adminyesno1 adminyesno2 adminyesno3 $70.; IF added_to_administration_page IN ('Yes','yes') THEN DO; * line 100; adminyesno1= "Note: This document has been added to the administration site:"; adminyesno2="&adminsite."; adminyesno3="for any of your future updates and modifications."; END; ELSE DO; adminyesno1=" "; adminyesno2=" "; adminyesno3=" "; END; CALL SYMPUT('adminyesno1',adminyesno1); * line 110; CALL SYMPUT('adminyesno2',adminyesno2); CALL SYMPUT('adminyesno3',adminyesno3); - 8 -

9 IF SUBSTR(SCAN(filepath,-1,'/'),1,3)='sae' AND SUBSTR(SCAN(filepath,-1,'/'),4,5) IN (&protocols) THEN DO; amendsae1= 'Protocol' ' ' SUBSTR(SCAN(filepath,-1,'/'),4,5) ' ' UPCASE(SUBSTR(SCAN(filepath,-1,'/'),1,3)) ' Number ' SUBSTR(SCAN(filepath,-1,'/'),10,2); amendsae2= "&livesite./sae/saesummarylist/saesummary.xls"; * line 120; CALL SYMPUT('amendsae1',amendsae1); CALL SYMPUT('amendsae2',amendsae2); to_group='yes'; * line 130; CALL SYMPUT('to_group', to_group); END; IF SUBSTR(SCAN(filepath,-1,'/'),1,5)='amend' THEN DO; FORMAT protocol $5.; protocol=substr(scan(filepath,-1,'/'),6,5); * line 140; CALL SYMPUT('protocol', protocol); amendsae1= 'Protocol' ' ' protocol ' Amendment ' SUBSTR(SCAN(filepath,-1,'/'),12,2); amendsae2="&livesite./&protocol./protocol/protocol&protocol..pdf"; CALL SYMPUT('amendsae1',amendsae1); CALL SYMPUT('amendsae2',amendsae2); to_group='yes'; * line 150; CALL SYMPUT('to_group', to_group); END; IF SUBSTR(SCAN(filepath,-1,'/'),1,3)='qol' THEN DO; * line 160; to_dm='yes'; * line 170; CALL SYMPUT ('to_dm', to_dm); END; END; %MEND DATASETLINE; %DATASETLINE; TITLE1'Check of Run'; PROC PRINT DATA=LogofWebsite N NOOBS; VAR to_group to_dm protocol filepath; % GROUP; * line 180; % DM; * line 190; % REQUESTOR; * line 200; - 9 -

10 The Macros: %MACRO DM; * This macro sends a mass to the DM if to_dm='yes'; * the upper limit of the DO loop is the number of data managers; * that are listed in the datamanagers.xls file; %DO i= 1 %TO 14; DATA Fake&i; SET WORK.DataManagers; IF num=&i THEN DO; name=scan(data_manager,-2, ' '); CALL SYMPUT('name', name); CALL SYMPUT('data_manager', data_manager); CALL SYMPUT(' ', ); CALL SYMPUT('site', site); CALL SYMPUT('qol_path', qol_path); END; FILENAME DO 2 ; %IF "&to_dm"="yes" %THEN %DO; DATA Test; FILE DO 2 TO=("& ") CC=("levy@jimmy.harvard.edu" "ktunney@partners.org" "mcbride.kathryn@jimmy.harvard.edu" "all_webmaster@jimmy.harvard.edu") SUBJECT="DFCI/ALL &site QOL Schedule" TYPE="TEXT/HTML"; PUT "&name"; PUT "The new QOL patient schedules have been posted on the DFCI/ALL Consortium website."; PUT "Please go to:"; PUT "&livesite.&qol_path"; PUT "You will need your username and password to access these files."; PUT "If you have any questions or problems accessing the file, please all_webmaster@jimmy.harvard.edu"; PUT "Note: Patients that were enrolled between now and the last report have been included. Please try to"; PUT "collect this information at the earliest convenience"; PUT "Thanks."; %END; %MEND DM;

11 %MACRO GROUP; * This macro sends a mass to the group if to_group='yes'; FILENAME DO 2 ; %IF "&to_group"="yes" %THEN %DO; DATA Test; FILE DO 2 TO= ("dfciconsortium@partners.org") CC= ("levy@jimmy.harvard.edu" "ktunney@partners.org" "mcbride.kathryn@jimmy.harvard.edu" "all_webmaster@jimmy.harvard.edu" ) SUBJECT="DFCI/ALL &amendsae1" TYPE="TEXT/HTML"; PUT "Dear DFCI/ALL Consortium Member,"; PUT "To view &amendsae1."; PUT "Go to: &livesite.&filepath"; PUT "&amendsae2"; PUT "You will need your username and password to access these files."; PUT "If you have any questions or problems accessing the file, please all_webmaster@jimmy.harvard.edu"; PUT "Thanks."; %END; %MEND GROUP;

12 %MACRO REQUESTOR; * The following code sends an to the individual who made the requested; * change to the site; FILENAME do ; DATA _NULL_; FILE DO TO=("&request_from_ ") CC=("levy@jimmy.harvard.edu" "Monisha_Verma@dfci.harvard.edu" "ktunney@partners.org" "mcbride.kathryn@jimmy.harvard.edu" "all_webmaster@jimmy.harvard.edu") SUBJECT="DFCI/ALL Website Changed/Addition Request &specific_request" TYPE="TEXT/HTML"; PUT "& _to_name"; PUT "We have completed your website request pertaining to:"; PUT "&specific_request"; PUT "Please go to:"; PUT "&livesite.&filepath"; PUT "to check the accuracy of your document."; PUT "Please respond to &request_to &request_to_ . within 1 working day of receiving this ."; PUT "Thanks."; PUT "You will need your username and password to access these files."; PUT "&adminyesno1"; PUT "&adminyesno2"; PUT "&adminyesno3"; %MEND REQUESTOR;

13 Program Documentation: **** * AUTO FOR ALL/DFCI WEBSITE ; **** * This program is used when a request is made to update the website. ; * An is sent to the person who requested the change to the website ; * to inform them that the change has been made to the live site and to ; * inform them to check the live site to make sure the correct posting has ; * been done. ; * When the change to the site is an SAE or amendment to the protocol, a ; * group wide is sent at the same time using this program. Nothing ; * different needs to be done by the webmaster, to send the 2nd to the ; * group. ; **** * Necessary files and permissions: ; * * website_ .sas ; * - SAS program to run auto for website ; * only need to know what line to run in the Log file for ; * auto to be sent ; * - file located on the W(SFA8) drive in ALL_WEB folder ; * - need read and write privileges ; * * LogOfWebsiteRequests.xls ; * - contains list of changes that have been made to website ; * - located on the W(SFA8) drive in ALL_WEB folder ; * - need read and write privileges ; * * list.xls ; * - contains addresses of MAIN ALL members that ; * contribute information to be posted on the website ; * - located on the W(SFA8) drive in ALL_WEB folder ; * - need read and write privileges ; * - when new contributing members are added to the group, will ; * need to update this list ; * * datamanagers.xls ; * - contains name, address and site of data managers ; * for QOL auto ; * - located on the W(SFA8) drive in ALL_WEB folder ; * - need read and write privileges ; * - when new datamanagers are added to the group, will ; * need to update this list ; * * Need PC SAS to run website_ .sas ; **** SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies

14 References: Thanks to the Boston SAS Users Group for sharing their many ideas, SAS techniques and expertise. Contact Information: Donna E Levy Dana-Farber Cancer Institute 44 Binney Street, Biostatistical Science Department Boston MA levy@jimmy.harvard.edu