PS004 SAS Email, using Data Sets and Macros: A HUGE timesaver! Donna E. Levy, Dana-Farber Cancer Institute Abstract: SAS V8+ has the capability to send email by using the DATA step, procedures or SCL. But what if ongoing emails must be generated at unscheduled/scheduled times? This SAS program has been developed to 'automatically' send a unique, customized and interactive email to the appropriate recipient(s). The program uses SAS EMAIL, MERGING datasets, DO loops, SUBSTR and SCAN functions as well as MACROS. Introduction: Each day, we spend countless hours reading, sending and responding to emails. Though many emails that are sent cannot be automated due to their unique content, there are many instances when an automatic email can be generated. For example, when similar text is sent to multiple individuals --- this is a perfect opportunity to save some time and effort by using a program to do the work for you. This program does just that by using many features of SAS V8 including: SAS email, MERGING data sets, DO loops, MACROS, MACRO VARIABLES and most importantly SUBSTRing searches and SCANS. Background: Our group maintains a website that includes general public information as well as specific details pertaining to data capture, protocol amendments and other administrative details. The webmasters in general are not responsible for maintaining content pertaining to any of the protocols. They simply receive the request to update or modify the material on the website, identify which documents that need to be changed/updated/replaced, and then update the document with the new material that has been submitted. A quality assurance (QA) check was implemented as there were several protocols active simultaneously each with their supporting material being maintained on the website. The QA was put in place to verify that the correct document was sent by the sender and that the document was correctly replaced on the website. Also the affiliate members (everyone in the group and/or data managers) would also need to be informed that there was updated material on the website as well as where this material was located on the site. Since a log of changes to the site was already being maintained including who requested the change and who completed the change, little additional work was needed except for writing the SAS code. The Content of the Email: Example 1 below is the general email that is sent to the person who made the request to change/update the document on the website. The type of document, determines the content of the email as well as additional recipients of email notification of the update. In example 1 and 2 below the update is an SAE (Serious Adverse Event). In this case, the general email is sent to the requestor and a similar email is also sent to all the affiliates in the group according to the rules of the program (example 2). For certain documents like updated address listings and contact information, this document is also saved on the administration site. The type of document (done manually) will determine whether the note pertaining to the administration site is included in the email to the requestor or not. - 1 -
Example 1: Email to the Requestor: To: ginger@dfci.harvard.edu CC: all_webmaster@jimmy.harvard.edu Subject: DFCI/ALL Website Protocol 00001 SAE 28 Ginger, We have completed your website request pertaining to Protocol 00001 SAE 28. Please go to: http://allconsortium.dfci.harvard.edu/consortium/sae/sae00001/sae00001_28.pdf to check the accuracy of your document. You will need your DFCI/ALL username and password to access this document. Please respond to Donna (email: levy@jimmy.harvard.edu) within 1 working day of receiving this email. Note: This document has been added to the administration site: http://allconsortium.dfci.harvard.edu/administration/sae00001_28.doc for any future updates and modifications. Example 2: Email to the Group: To= dfciconsortium@partners.org CC= all_webmaster@jimmy.harvard.edu Subject=DFCI/ALL Protocol 00001 SAE 28 Dear DFCI/ALL Consortium Member, To view Protocol 00001 SAE 28 Go to: http://allconsortium.dfci.harvard.edu/consortium/sae/sae00001/sae00001_28.pdf You will need your username and password to access these files. If you have any questions or problems accessing the file, please email all_webmaster@jimmy.harvard.edu Thanks. There are 2 Excel files that are necessary to use in the program. In one Excel document (emaillist.xls) that is used in the program (blue, bold) contains emails of all the people that could send updates for the website [file 1 Email Listing]. This autoemail program is used by the three webmasters at our institution and we all do not use the same email software. As a result, this file is kept as a shared file rather than within our email program. The other Excel document (red, italic) contains the name of the document that was changed/updated, the directory that the document was located, who requested the changes and who performed the changes on the website [file 2 - Log]. See below for content examples of the first [file 1] and the second Excel file [file 2]. - 2 -
File 1: Email addresses of all individuals who could submit changes for website updates Name Email Kathryn McBride kdmcbride@partners.org Kirsten Tunney ktunney@partners.org Montse Rue rue@jimmy.harvard.edu Donna Levy levy@jimmy.harvard.edu Ginger Dalton virginia_dalton@dfci.harvard.edu Jennifer Cronin jennifer_cronin@dfci.harvard.edu File 2: Listing of Content Changes Performed on the website Req # Request Date Request From Request To Specific Request 13 01/21/03 Ginger Dalton Kathryn McBride 14 01/21/03 Ginger Dalton Kathryn McBride 15 01/21/03 Kirsten Tunney Kirsten Tunney Amendment 16 approved for posting 01/22/03 Amendment 16 protocol approved for posting 01/23/03 DMC recommendations Nov2002 01/22/03 16 01/21/03 Ginger Dalton Kathryn McBride Post SAE 00001 28 01/22/03 Date Completed filepath /consortium/00001/protocol /amendments/amend00001 _16.pdf /consortium/00001/protocol /protocol00001.pdf /consortium/dmcopen/dmcr ecommend2002.pdf /consortium/sae/sae00001/ sae00001_28.pdf Added to administration page No No No No Key Components of the Program: emaillist.xls (file 1): email list of anyone who contributes information to the website logofwebsite.xls (file 2): contains who the request came from, who fulfilled the request, the file that was changed, the path to the file and whether an editable version sits in the administration folder IF obs IN (&linenum) THEN DO; linenum is the only line in the program that changes from run to run. As a result of this line of code, the data set only includes the line of information that will be sent in the email. CALL SYMPUT: creates macro variables out of data contained in the above data sets. The SYMPUT routine assigns a value produced in the DATA step to a macro variable. If the macro variable does not exist, the symput routine creates it. SYMPUT (name, value) creates a global variable name= the name of the macro variable value= contains the value to be assigned Must use the symput routine as part of the call statement How many emails will be sent and what will be contained in the email? The following lines of code determine how many emails will be sent: IF added_to_administration_page= <Yes, No> IF SUBSTR of file location= <substring> to_group= <Yes, No> to_dm= <Yes, No> - 3 -
In the program attached, there is always a minimum of 1 email sent in total. If the file is maintained on the administration page, there will be an additional 3 lines of code in the email with the path to the editable document on the administration page. The SUBSTR function searches the file name and path to determine what type of document the file is ie. QOL, SAEs, Amendments etc. If the file pertains to QOL, then an email goes to the data managers (to_dm= Yes ) but not to the entire group. If the file is an SAE or an amendment, then an email goes to the entire group (to_group= Yes ). Going Through the Code: Line 10 [%LET linenum=16;] This is the only line of code that needs to be changed each time you run the code. This global variable indicates that line 16 of the log (file 2) will be run through the program. *** need to add where used in the program *** Line 20 [%LET to_group='no'] and line 30 [%LET to_dm='no';] Line 20 and line 30 are initializing the dichotomous variables to_group and to_dm to 'No. Currently an email will not be sent to the entire group or the data managers. Depending upon the particular change that has been requested, will determine whether or not these variables will be reassigned to 'Yes' and thus an email will be sent to the group and/or data managers. Line 40-43 [DATA WORK.EmailListFrom;] and Line 50-53 [DATA WORK.EmailListTo;] This is file 1 (Email Listing) being used as a dataset and is used twice. We will be merging these 2 files with file 2 (Log) to establish who the request came from (variables: request_from, request_from_email) as well as who will be performing the request (variables: request_to, request_to_email). Line 60 [MERGE LogOfWebsite EmailListFrom;] Merges the log (file 2) with the email listing (file 1). These lines of code will allow us to determine who the request for the update came from as well as determining their email address. Line 70 [DATA LogOfWebsite;] Merges the log with the email list to (so now the email addresses 'to' have been added to the dataset (the webmaster's email). This step would not be needed if there was only 1 webmaster. Line 80-89 [CALL SYMPUT('request_number', request_number);] Is creating macro variables (? global variables) out of the merged data set that we will use in our email. CALL SYMPUT(<macro variable name>, <original variable value>); In this case, I named the macro variable the same name as the original variable. This is not necessary. I only did this for simplicity of identifying variables and values. Line 90 [email_to_name=scan(request_from,-2, ' ');] The SCAN function searches through the specified variable in search of the specified string. For example: email_to_name=scan(request_from, -2, ' '); If email_to_name='donna Levy', in this case, we are scanning through this variable starting from the end (the negative of the -2 indicates that. If you wanted to start from the beginning you would have used a 2 or +2) and searching for the second space (' ') which is located in front of the 'D' in 'Donna' technically. I look at it like this: start and the end of the string then jump over 2 'words', then take the word to your left then assign the variable that value. - 4 -
Line 100 [IF added_to_administration_page IN ('Yes','yes') THEN DO;] Is a variable contained in the log (added_to_administration_page). If this field is yes, then 3 more lines of text will be added to the email to let the requestor know that the file has been added to the administration page. Line 110 [CALL SYMPUT('adminyesno1',adminyesno1);] Once again we use CALL SYMPUT to assign a value to a macro value. Line 120-121 [amendsae1='protocol' ' ' SUBSTR(SCAN(filepath,-1,'/'),4,5) ' ' UPCASE(SUBSTR(SCAN(filepath,-1,'/'),1,3)) ' Number ' SUBSTR(SCAN(filepath,-1,'/'),10,2); amendsae2="&livesite./sae/saesummarylist/saesummary.xls"; If the file is an SAE (determined by the SUBSTR function and scanning the filepath variable as we described previously), then the title is created for the email to identify what has been updated on the website as well as the link to the file on the website which will also be included in the email. The is simply adjoining 2 text components. For example do 123 is simply do123. Line 130 [to_group='yes';] In this example, we have determined that the file is an SAE, then the to_group is assigned to be a 'Yes' thus an email will be sent to the entire group. Line 140 [protocol=substr(scan(filepath,-1,'/'),6,5);] If the file is an amendment or an SAE, we need to determine what protocol has been affected. This code will search the string to determine what protocol has been amended. Line 150 [to_group='yes';] Once again, if the file is an amendment, an email will be sent to the group (to_group assigned to equal 'yes'). Line 160/170 [IF SUBSTR(SCAN(filepath,-1,'/'),1,3)='qol' THEN DO; to_dm='yes';] If the file pertains to quality of life (qol), then this email only goes to the data managers (to_dm assigned to be equal to 'yes'). Once again, to determine what type of file has been submitted, the string is searched for key components, in this case qol. Line 180/190/200 [%EMAILGROUP; %EMAILDM; %EMAILREQUESTOR;] Lines 180. 190 and 200 calls the email macros (group, data manager and requestor). When we go to the emailgroup macro, if to_group is Yes then an email will be sent to the group. Otherwise an email will not be sent to the group (to_group= No ). Similar events occur with the data manager macro (emaildm). As previously stated, an email always goes to the person who made the request for the website file update. Tying it all Together: Although this program has been designed for a very specific example, the idea can be easily expanded and applied to any individual s or group s needs. The text of the email is easily edited and the program is expandable if another subgroup forms within the consortium. By using features of SAS, specifically EMAIL, SUBSTR, SCAN and MACROS, a tedious job of multiple emails with the same or similar content, can be completed by simply --- the push of a button. - 5 -
The Code: OPTIONS NOCENTER LS=256 SGEN MERROR; **** **** * Enter the line number that you would like to send an email out for here; %LET linenum=16; *line 10; ***** ***** * ---------------------------------------------------------------------------; * Where the LogofWebsite.xls is located; %LET logpath=w:all_web; * Where the emaillist.xls as well as datamanager.xls is located; %LET emailpath=w:all_web\autoemail; * ---------------------------------------------------------------------------; * The URL to the live site; %LET livesite=http://allconsortium.dfci.harvard.edu; * ---------------------------------------------------------------------------; * The URL to the development site; %LET devsite=http://alldev.dfci.harvard.edu; * ---------------------------------------------------------------------------; * THE URL to the administration page; %LET adminsite=&livesite./administration; * ---------------------------------------------------------------------------; * Protocols that any emails may apply to. When a new protocol is added be sure to; * update this line of code; %LET protocols=00001, 95001, 01175, 99137; * ---------------------------------------------------------------------------; * to_group= email sent to group (yes, no). Initially set at no. Program will; * determine and update this variable if the file that is updated is an SAE or ; * Amendment. If in the future, the decision is made to send more emails to ; * the group (other than for an SAE or Amendment), then the decision rules ; * within the code, for this variable will need to be updated ; %LET to_group='no'; * line 20; * to_dm =email sent to DMs only (yes, no). Initially set at no. Program will; * determine and update this variable if the file that is updated is an QOL ; * Schedule. If in the future, the decision is made to send more emails only ; * to the DMs;(other than for QOL schedule), then the decision rules within the; * code, for this variable will need to be updated ; %LET to_dm='no'; * line 30; - 6 -
* ----------------------------------------------------------------------------; * where macros are located that need to be included in this program; %LET macropath=f:todo\autoemail; %INCLUDE "¯opath\email.mac"; ************************ READING IN DATA ************************************; PROC IMPORT OUT=WORK.LogOfWebsite DATAFILE="&logpath.\LoGofWebsiteRequests.xls" DBMS=EXCEL2000 REPLACE; GETNAMES=YES; PROC IMPORT OUT=WORK.EmailList DATAFILE="&emailpath.\emaillist.xls" DBMS=EXCEL2000 REPLACE; GETNAMES=YES; PROC IMPORT OUT=WORK.DataManagers DATAFILE="&emailpath.\datamanagers.xls" DBMS=EXCEL2000 REPLACE; GETNAMES=YES; **** * Data set with the email and name of the person who submitted the request; * for the website update; DATA WORK.EmailListFrom; * line 40; SET WORK.EmailList; request_from= name; request_from_email=email; * Data set with the email and name of the person who received the request; * for the website update; DATA WORK.EmailListTo; * line 50; SET WORK.EmailList; request_to= name; request_to_email=email; PROC SORT DATA=WORK.LogOfWebsite; BY request_from; PROC SORT DATA=WORK.EmailListFrom; BY request_from; DATA LogOfWebsite; MERGE LogOfWebsite EmailListFrom; *line 60; BY request_from; PROC SORT DATA=WORK.LogOfWebsite; BY request_to; PROC SORT DATA=WORK.EmailListTo; BY request_to; DATA LogOfWebsite; * line 70; MERGE LogOfWebsite EmailListTo; BY request_to; - 7 -
DATA LogOfWebsite; SET LogOfWebsite; IF request_number=. THEN DELETE; PROC SORT DATA=LogOfWebsite; BY request_number; DATA LogOfWebsite; SET LogOfWebsite; obs=_n_; %MACRO DATASETLINE; * this macro creates macro variables that will be included in the auto emails; IF obs IN (&linenum) THEN DO; CALL SYMPUT('request_number', request_number); * line 80; CALL SYMPUT('request_date', request_date); CALL SYMPUT('request_from', CALL SYMPUT('request_to', CALL SYMPUT('specific_request', CALL SYMPUT('date_completed', CALL SYMPUT('other', request_from); request_to); specific_request); date_completed); other); CALL SYMPUT('filepath', filepath); CALL SYMPUT('request_from_email', request_from_email); CALL SYMPUT('request_to_email', request_to_email); email_to_name=scan(request_from,-2, ' '); *line 90; CALL SYMPUT('email_to_name', email_to_name); FORMAT adminyesno1 adminyesno2 adminyesno3 $70.; IF added_to_administration_page IN ('Yes','yes') THEN DO; * line 100; adminyesno1= "Note: This document has been added to the administration site:"; adminyesno2="&adminsite."; adminyesno3="for any of your future updates and modifications."; END; ELSE DO; adminyesno1=" "; adminyesno2=" "; adminyesno3=" "; END; CALL SYMPUT('adminyesno1',adminyesno1); * line 110; CALL SYMPUT('adminyesno2',adminyesno2); CALL SYMPUT('adminyesno3',adminyesno3); - 8 -
IF SUBSTR(SCAN(filepath,-1,'/'),1,3)='sae' AND SUBSTR(SCAN(filepath,-1,'/'),4,5) IN (&protocols) THEN DO; amendsae1= 'Protocol' ' ' SUBSTR(SCAN(filepath,-1,'/'),4,5) ' ' UPCASE(SUBSTR(SCAN(filepath,-1,'/'),1,3)) ' Number ' SUBSTR(SCAN(filepath,-1,'/'),10,2); amendsae2= "&livesite./sae/saesummarylist/saesummary.xls"; * line 120; CALL SYMPUT('amendsae1',amendsae1); CALL SYMPUT('amendsae2',amendsae2); to_group='yes'; * line 130; CALL SYMPUT('to_group', to_group); END; IF SUBSTR(SCAN(filepath,-1,'/'),1,5)='amend' THEN DO; FORMAT protocol $5.; protocol=substr(scan(filepath,-1,'/'),6,5); * line 140; CALL SYMPUT('protocol', protocol); amendsae1= 'Protocol' ' ' protocol ' Amendment ' SUBSTR(SCAN(filepath,-1,'/'),12,2); amendsae2="&livesite./&protocol./protocol/protocol&protocol..pdf"; CALL SYMPUT('amendsae1',amendsae1); CALL SYMPUT('amendsae2',amendsae2); to_group='yes'; * line 150; CALL SYMPUT('to_group', to_group); END; IF SUBSTR(SCAN(filepath,-1,'/'),1,3)='qol' THEN DO; * line 160; to_dm='yes'; * line 170; CALL SYMPUT ('to_dm', to_dm); END; END; %MEND DATASETLINE; %DATASETLINE; TITLE1'Check of Run'; PROC PRINT DATA=LogofWebsite N NOOBS; VAR to_group to_dm protocol filepath; %EMAILGROUP; * line 180; %EMAILDM; * line 190; %EMAILREQUESTOR; * line 200; - 9 -
The Macros: %MACRO EMAILDM; * This macro sends a mass email to the DM if to_dm='yes'; * the upper limit of the DO loop is the number of data managers; * that are listed in the datamanagers.xls file; %DO i= 1 %TO 14; DATA Fake&i; SET WORK.DataManagers; IF num=&i THEN DO; name=scan(data_manager,-2, ' '); CALL SYMPUT('name', name); CALL SYMPUT('data_manager', data_manager); CALL SYMPUT('email', email); CALL SYMPUT('site', site); CALL SYMPUT('qol_path', qol_path); END; FILENAME DOEMAIL2 EMAIL; %IF "&to_dm"="yes" %THEN %DO; DATA Test; FILE DOEMAIL2 TO=("&email") CC=("levy@jimmy.harvard.edu" "ktunney@partners.org" "mcbride.kathryn@jimmy.harvard.edu" "all_webmaster@jimmy.harvard.edu") SUBJECT="DFCI/ALL &site QOL Schedule" TYPE="TEXT/HTML"; PUT "&name"; PUT "The new QOL patient schedules have been posted on the DFCI/ALL Consortium website."; PUT "Please go to:"; PUT "&livesite.&qol_path"; PUT "You will need your username and password to access these files."; PUT "If you have any questions or problems accessing the file, please email all_webmaster@jimmy.harvard.edu"; PUT "Note: Patients that were enrolled between now and the last report have been included. Please try to"; PUT "collect this information at the earliest convenience"; PUT "Thanks."; %END; %MEND EMAILDM; - 10 -
%MACRO EMAILGROUP; * This macro sends a mass email to the group if to_group='yes'; FILENAME DOEMAIL2 EMAIL; %IF "&to_group"="yes" %THEN %DO; DATA Test; FILE DOEMAIL2 TO= ("dfciconsortium@partners.org") CC= ("levy@jimmy.harvard.edu" "ktunney@partners.org" "mcbride.kathryn@jimmy.harvard.edu" "all_webmaster@jimmy.harvard.edu" ) SUBJECT="DFCI/ALL &amendsae1" TYPE="TEXT/HTML"; PUT "Dear DFCI/ALL Consortium Member,"; PUT "To view &amendsae1."; PUT "Go to: &livesite.&filepath"; PUT "&amendsae2"; PUT "You will need your username and password to access these files."; PUT "If you have any questions or problems accessing the file, please email all_webmaster@jimmy.harvard.edu"; PUT "Thanks."; %END; %MEND EMAILGROUP; - 11 -
%MACRO EMAILREQUESTOR; * The following code sends an email to the individual who made the requested; * change to the site; FILENAME doemail EMAIL; DATA _NULL_; FILE DOEMAIL TO=("&request_from_email") CC=("levy@jimmy.harvard.edu" "Monisha_Verma@dfci.harvard.edu" "ktunney@partners.org" "mcbride.kathryn@jimmy.harvard.edu" "all_webmaster@jimmy.harvard.edu") SUBJECT="DFCI/ALL Website Changed/Addition Request &specific_request" TYPE="TEXT/HTML"; PUT "&email_to_name"; PUT "We have completed your website request pertaining to:"; PUT "&specific_request"; PUT "Please go to:"; PUT "&livesite.&filepath"; PUT "to check the accuracy of your document."; PUT "Please respond to &request_to email: &request_to_email. within 1 working day of receiving this email."; PUT "Thanks."; PUT "You will need your username and password to access these files."; PUT "&adminyesno1"; PUT "&adminyesno2"; PUT "&adminyesno3"; %MEND EMAILREQUESTOR; - 12 -
Program Documentation: **** * AUTO EMAIL FOR ALL/DFCI WEBSITE ; **** * This program is used when a request is made to update the website. ; * An email is sent to the person who requested the change to the website ; * to inform them that the change has been made to the live site and to ; * inform them to check the live site to make sure the correct posting has ; * been done. ; * When the change to the site is an SAE or amendment to the protocol, a ; * group wide email is sent at the same time using this program. Nothing ; * different needs to be done by the webmaster, to send the 2nd email to the ; * group. ; **** * Necessary files and permissions: ; * * website_email.sas ; * - SAS program to run autoemail for website ; * only need to know what line to run in the Log file for ; * autoemail to be sent ; * - file located on the W(SFA8) drive in ALL_WEB folder ; * - need read and write privileges ; * * LogOfWebsiteRequests.xls ; * - contains list of changes that have been made to website ; * - located on the W(SFA8) drive in ALL_WEB folder ; * - need read and write privileges ; * * emaillist.xls ; * - contains email addresses of MAIN ALL members that ; * contribute information to be posted on the website ; * - located on the W(SFA8) drive in ALL_WEB folder ; * - need read and write privileges ; * - when new contributing members are added to the group, will ; * need to update this list ; * * datamanagers.xls ; * - contains name, email address and site of data managers ; * for QOL autoemail ; * - located on the W(SFA8) drive in ALL_WEB folder ; * - need read and write privileges ; * - when new datamanagers are added to the group, will ; * need to update this list ; * * Need PC SAS to run website_email.sas ; **** SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. - 13 -
References: Thanks to the Boston SAS Users Group for sharing their many ideas, SAS techniques and expertise. Contact Information: Donna E Levy Dana-Farber Cancer Institute 44 Binney Street, Biostatistical Science Department Boston MA 02215 levy@jimmy.harvard.edu - 14 -