Swap the DATA Step for PROC TRANSPOSE Katie Joseph, U.S. Office of Personnel Management, Washington, DC

Transcription

1 Swap the DATA Step for PROC TRANSPOSE Katie Joseph, U.S. Office of Personnel Management, Washington, DC ABSTRACT Anyone who has spent an unspeakable amount of time using arrays and do loops in a DATA step to convert observations to variables or vice versa has certainly thought, Isn't there an easier way to do this? The answer is YES! SAS offers a convenient way of circumventing these lengthy DATA steps with PROC TRANSPOSE. This paper presents an introduction to PROC TRANSPOSE, the useful SAS procedure that rotates SAS datasets. Through examples, I will compare code that transposes data with the DATA step to much simpler code using PROC TRANSPOSE and discuss ways to name and shape variables in the output dataset using the PREFIX= option, the ID statement, and BY processing. INTRODUCTION SAS programmers are often faced with the task of transposing data sets. You may need to transpose your data set in order to merge it with another data set, to feed into a SAS/STAT procedure, or to simply create a better-looking report. You can write a DATA step to do this, but the useful TRANSPOSE procedure can do the job in a fraction of the code, with knowledge of a few simple options and statements. SAMPLE DATA The following portion of the SAS data set SURVEY in Figure 1 will be used for demonstration. The task is to restructure the data so that there is one observation per question, and the observations of RESPONSE are four separate variables, as in Figure 2. Figure 1 Sample Dataset SURVEY Q Q Q Q Q Q Q Q Q Q Q Q Figure 2 Transposed Data Q Q Q Q Q Q Q Q Q Q

2 TRANSPOSING WITH THE DATA STEP The DATA step can accomplish this transposition with arrays and do loops with the following code: proc sort data=survey; data survey2 (keep=question response1-response4); retain response1-response4; array responses{4} response1-response4; set survey; if first.question then do; i=1; do j=1 to 4; responses{j}=.; end; end; responses{i}=percent; if last.question then output; i + 1; This DATA step, in addition to being lengthy, may produce unwanted results if the data set does not have placeholders for missing observations. In the data set SURVEY, none of the survey respondents chose response 2 for question 12, so no observation exists for question= Q12 and response= 2. The do loop processing assigns the value 21.3 to the variable response2, the value.9 to response3, and the value missing to response4, as shown in Figure 3. This is clearly not correct. In absence of an observation for question Q12 and response 2, the DATA step essentially shifts the values for question 12 and sets response4 to missing. Figure 3 Incorrect Results of Transposing with the DATA step Question 12 in data set SURVEY DATA step processing Question 12 in data set SURVEY2 Q Some manipulation to this DATA step can fix the problem, but an intimate familiarity with do loops, arrays, the first. and last. variables, and the program data vector is essential for correctly transposing data with the DATA step. In addition, a minor change to the data set structure may require significant recoding. For example, adapting this program to a data set with 5 rows of data for each by group would require updating multiple statements in the program. Fortunately, PROC TRANSPOSE provides an easier, less volatile path to data transposition. USING PROC TRANSPOSE PROC TRANSPOSE will correctly align the variables in an output data set with a few simple lines of code, and I will discuss the options and statements below. proc transpose data=survey out=survey3 (drop=_name LABEL_) prefix=response; id response; var percent; 2

3 PROC TRANSPOSE OPTIONS As with all SAS procedures, the DATA= option names the input data set and the OUT= option names the output data set. SAS will use the data set naming convention DATA1, DATA2, etc. in absence of an OUT= option. The PREFIX= option assigns a prefix to the transposed variables in the output data set. In our output data set SURVEY3, the prefix is response so the transposed variables are named response1, response2, response3, and response4. _NAME_ and _LABEL_, variables that SAS automatically generates in PROC TRANSPOSE, contain the names and labels, of the variables that were transposed. These may not be useful if you are only transposing one variable and can be dropped using the DROP= data set option. BY STATEMENT We use the BY statement in PROC TRANSPOSE to create an observation for each value of the BY variable for each transposed variable. The number of observations in the transposed data set is the number of unique values of the BY variable times the number of variables you are transposing. We are only transposing one variable, PERCENT, and there are 73 questions in the data set SURVEY, so our transposed data set, SURVEY3, will have 73 observations. PROC TRANSPOSE supports multiple BY variables. For example, if the sample data consisted of survey results by question for 80 different agencies, I could sort the data by question and agency, then use question and agency in the BY statement in PROC TRANSPOSE to create one observation for each combination of question and agency, resulting in 80 agencies X 73 questions X 1 variable that we are transposing = 5,840 observations. ID STATEMENT The variable you specify in the ID statement will be used to name the transposed variables in the output data set. If you use the PREFIX= option, the ID variable values will be concatenated to the prefix you specify to name the new variables. In the SAMPLE data, the ID variable is response (whose values are 1, 2, 3, and 4 ) and our prefix is response so our variable names in SURVEY3 are response1, response2, response3, and response4. If you use the ID statement without the PREFIX= option, the values of the ID variable will simply name the new variables. This is particularly useful if the values of the ID variable are descriptive (i.e. age, race, weight, etc.), as opposed to numbers in our example. However, if the first character of the ID variable is a number, or if it is a numeric variable, SAS will prefix the variable names with an underscore, since SAS variable names cannot begin with a number. If you do not use the PREFIX= option or the ID statement, the transposed variables will follow the naming convention COL1, COL2, etc. If you specify a prefix in the PREFIX= option and you do not use the ID statement, SAS will using the naming convention <prefix>1, <prefix>2, <prefix>3, etc. Because the ID variable RESPONSE has values 1, 2, 3, and 4, our variables in the transposed data set would have the same name, regardless of whether you use the ID statement or not. However, the ID statement is important because it places values within the correct variables, even when there is no placeholder for missing observations, as in question Q12. Without the ID statement, the output data set will look like the data set SURVEY2 that was created by the DATA step (see Figure 3). Figure 4 shows that the ID statement in PROC TRANSPOSE correctly transposes input data with no placeholders for missing values. Figure 4 Correct Results of Transposing with PROC TRANSPOSE and the ID Statement Question 12 in data set SURVEY PROC TRANSPOSE Question 12 in data set SURVEY3 Q

4 VAR STATEMENT The VAR statement tells SAS which variable(s) you want to transpose. Without a VAR statement, all numeric variables not named in the ID or BY statement will be transposed. If you want to transpose a character variable, you must use the VAR statement. The VAR statement could be omitted in this example because PERCENT is the only numeric variable. CONVERTING VARIABLES INTO OBSERVATIONS WITH PROC TRANSPOSE In addition to converting observations into variables, as in the example above, PROC TRANSPOSE can conversely make variables into observations. Several SAS/STAT procedures require a specific layout of the input data set and a transposition may be necessary if the data is not organized in that manner. For example, for each question in a survey, there are 4 possible responses, A, B, C, or D, and I have done a significance test for each of the 6 possible response comparisons. I have a data set with an observation for each question and a variable for each of the 6 possible tests containing the p-values I computed from the significance tests, as shown below in Figure 5. PROC MULTTEST is the SAS/STAT procedure that adjusts raw p-values from multiple hypothesis tests to avoid false significances that can occur with multiple testing. I want to adjust the p-values in the survey data using PROC MULTTEST, but it requires the input data to have one variable named RAW_P that contains the raw p-values. Thus, the data must be transposed to make the 6 p-value variables into observations, as shown in Figure 6. Figure 5 Original Data Set of Raw P-Values question pvalue_a_b pvalue_a_c pvalue_a_d pvalue_b_c pvalue_b_d pvalue_c_d Q Q Q Q Q Q Q Q Q Q Figure 6 Input Data Set Needed for PROC MULTTEST question test Raw_P Q01 pvalue_a_b Q01 pvalue_a_c Q01 pvalue_a_d Q01 pvalue_b_c Q01 pvalue_b_d Q01 pvalue_c_d Q02 pvalue_a_b Q02 pvalue_a_c Q02 pvalue_a_d Q02 pvalue_b_c Q02 pvalue_b_d Q02 pvalue_c_d : : Q10 pvalue_c_d

5 The following code effortlessly changes the original data into a data set suitable for PROC MULTTEST: proc transpose data=rawpvalues out=data4multtest (rename=(col1=raw_p _NAME_=test)); Recall that if you do not use a VAR statement, all numeric variables not named in the BY or ID statement will be transposed, so a VAR statement is not necessary. The RENAME= data set option renames automatic SAS variable names into variable names that PROC MULTTEST will recognize. CONCLUSION The ability to transpose data sets is a powerful data manipulation skill for SAS programmers. The DATA step is one method of transposing, but the code can be long and produce undesired results in some situations. With the few simple options and statements described in this paper, you can save time and hassle by swapping the DATA step for PROC TRANSPOSE. REFERENCES SAS Institute Inc Base SAS 9.1 Procedures Guide. Cary, NC: SAS Institute Inc. SAS Institute Inc SAS/STAT 9.1 User s Guide. Cary, NC: SAS Institute Inc. CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the author at: Katie Joseph U.S. Office of Personnel Management 1900 E St, NW, Room 7439 Washington, DC Work Phone: (202) Fax: (202) katie.joseph@opm.gov SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are trademarks of their respective companies. 5