HELP! - My MERGE Statement Has More Than One Data Set With Repeats of BY Values!
|
|
- Shanon Ball
- 8 years ago
- Views:
Transcription
1 HELP! - My MERGE Statement Has More Than One Data Set With Repeats of BY Values! Andrew T. Kuligowski, The Nielsen Company ABSTRACT Most users of the SAS system have encountered the following message: NOTE: MERGE statement has more than one data set with repeats of BY values. There are many papers in the Proceedings from past SAS User Group conferences that describe how to programatically force this "many-to-many" merge to occur. These presentations are built upon the assumption that the user wants to merge datasets that have repeats of BY values. However, it is also possible that the coder did not expect this condition and wishes to avoid and eliminate it. This presentation will demonstrate a technique using a series of ad hoc programs that will isolate the observations causing the many-to-many merge, so that we can modify our underlying data assumptions, and hopefully eliminate this condition. MERGE Statement: Repeats of BY Values As previously mentioned, most users of the SAS system have encountered the following message: NOTE: MERGE statement has more than one data set with repeats of BY values. This implies an error in your SAS routine, caused by a misunderstanding of the input data. We want to isolate these cases and modify our assumptions, so that we can correct our MERGE and eliminate this condition. The following will illustrate an example of "repeats of BY values". We are going to merge a dataset containing a list of dog breeds [see Table "1-A"] against a dataset containing the dogs owned by sample households [see Table "1-B"] using the variable BREED, keeping only those records from the Breed file that have a corresponding record in the Household file. As one would expect (knowing, of course, that this example was created to illustrate the problem at hand), the merge results in "more than one data set with repeats of BY values" [illustrated in Table "1-C"]. The problem is to determine which BY values had repeats, which records (on which files) are affected, and what additional information needs to be included to make the "BY values", also referred to as "merge variables", unique on at least one of the files. BREED VARIETY OTHER INFO Bulldog Dalmatian Dachshund Mini Dachshund MiniLonghair Dachshund MiniWirehair Dachshund Std BREED VARIETY OTHER INFO Dachshund Std Longhair Dachshund Std Wirehair Ger Sheprd GoldRetrvr Husky,Sib Lab Retrvr Table "1-A" : Breed File 1
2 HHLD ID BREED VARIETY BIRTHDAY GOAWAYDY Dalmatian 07/31/ Dalmatian 12/23/ Bulldog English 05/19/91 02/20/ Dachshund Std Longhair 09/17/ Dachshund Std Longhair 10/29/ Ger Sheprd 11/24/89 12/07/ Husky,Sib 05/26/ Lab Retrvr 02/28/94 05/06/ GoldRetrvr 03/06/95 Table "1-B" : Dogs per Household File 177 DATA DOGDTL; 178 MERGE DOGOWNED (IN=IN_OWNED) 179 DOGBREED (IN=IN_BREED); 180 BY BREED ; 181 IF IN_OWNED ; 182 RUN; NOTE: MERGE statement has more than one data set with repeats of BY values. NOTE: The data set WORK.DOGDTL has 13 observations and 6 variables. Table "1-C" : SASLOG - MERGE with "repeats of BY values" We can use basic elements of the SAS System to do most of the analysis for us; our most powerful tool will be the MEANS procedure. PROC MEANS is traditionally used to compute descriptive statistics for numeric variables in a SAS Dataset. In this situation, we simply need a list of the unique values of our merge variable (or, in a more complex case, the unique combinations of values for our merge variables), along with a count of the number of occurrences of each in both of our datasets. The NWAY option on the PROC MEANS statement will limit the output dataset to only those observations with the "highest interaction among CLASS variables" -- that is, only those records containing the unique values or combinations of values for the BY variables will be written to the output dataset. NOPRINT is optional; it can be used or excluded based on personal preference. The variable used in the VAR statement can be any numeric variable in the dataset; the only condition is that it must be a numeric variable. Date variables and ID values should be considered, since many / most datasets contain them and they are normally stored as numeric values. (In the rare event your dataset contains exclusively character variables, you will need to add a numeric variable to either the original dataset or a copy of it prior to issuing PROC MEANS.) The OUTPUT statement must specify an OUT= dataset. It must also include the statistics keyword N=, so that a record count -- and only a record count -- for each unique combination of values for the BY variables will be written to each observation of the output dataset. [Table "1-D" illustrates this use of PROC.] The next step is to take the outputs of our PROC MEANS for each affected dataset and merge them together. We cannot encounter the "... repeats of BY values" note, since we now only have one observation per unique combination of BY values! Each observation does have a count of the number of observations that contain the combination of BY values in their original dataset, stored as _FREQ_ by default. The RENAME= option the datasets in the MERGE statement can be used to give the _FREQ_ variable in each dataset a unique name in our output dataset. (Alternatively, a clearner approach would be to change the N= option on the OUTPUT statement in PROC MEANS to N=varname, avoiding the need for the subsequent RENAME.) We will use a subsetting IF, so that we only keep those observations that have multiple occurrences in each input dataset. The output of this step will contain the unique combinations of values that are causing the "...repeats of BY values" note. [The routine is contained in Table "1-E", and the output depicted in Table "1-F".] 2
3 208 PROC MEANS DATA=DOGOWNED NOPRINT NWAY; 209 CLASS BREED ; 210 VAR HHLD_ID ; 211 OUTPUT OUT=SUMOWNED N=CNTOWNED; 212 RUN; NOTE: The data set WORK.SUMOWNED has 7 observations and 4 variables. NOTE: The PROCEDURE MEANS used 0.05 seconds. 213 PROC MEANS DATA=DOGBREED NOPRINT NWAY; 214 CLASS BREED ; 215 VAR OTHRINFO ; 216 OUTPUT OUT=SUMBREED N=; 217 RUN; NOTE: The data set WORK.SUMBREED has 7 observations and 4 variables. NOTE: The PROCEDURE MEANS used 0.05 seconds. Table "1-D" : SASLOG - PROC MEANS example 585 DATA SUMMERGE (KEEP=BREED CNTBREED CNTOWNED); 586 MERGE SUMBREED (RENAME=(_FREQ_=CNTBREED)) 587 SUMOWNED ; 588 BY BREED ; 589 IF CNTBREED > 1 AND CNTOWNED > 1; 590 RUN ; NOTE: The data set WORK.SUMMERGE has 1 observations and 3 variables. Table "1-E" : SASLOG - Merging the PROC MEANS outputs SAS Dataset WORK.SUMMERGE OBS BREED CNTBREED CNTOWNED 1 Dachshund 6 2 Table "1-F" : Results of Merging the PROC MEANS outputs Up to this point, we have not discussed one important factor in this analysis - the human element. The process described in this section is meant to be used as a tool to guide the analyst through the unknown elements in their data - once these areas become known - there is no need to continue this analysis. The listing of unique combinations of values that occur multiple times in each dataset will often be that stopping point for an analyst. The information obtained will allow them to make modifications to their assumptions and corresponding changes to their routines. However, in the event that the oddities are still not clear, we can employ one or more additional MERGE steps, taking the merged outputs from the PROC MEANS, and merging those dataset against each of the original datasets. [Table "1-G" shows how this is done, while Table 1-H displays the results of this analysis.] This final step should provide sufficient clarification for the analyst to determine which factor(s) are missing in their assumptions and adjust their routines accordingly. 3
4 599 DATA CHKOWNED ; 600 MERGE DOGOWNED (IN=IN_BREED ) 601 SUMMERGE (IN=IN_MERGE ); 602 BY BREED ; 603 IF IN_MERGE ; 604 RUN ; NOTE: The data set WORK.CHKOWNED has 2 observations and 7 variables. 605 DATA CHKBREED ; 606 MERGE DOGBREED (IN=IN_BREED ) 607 SUMMERGE (IN=IN_MERGE ); 608 BY BREED ; 609 IF IN_MERGE ; 610 RUN ; NOTE: The data set WORK.CHKBREED has 6 observations and 5 variables. Table "1-G" : SASLOG - Merging the analysis back to the original input SAS Dataset WORK.CHKOWNED OBS HHLD_ID BREED VARIETY BIRTHDAY GOTCHADY CNTBREED CNTOWNED Dachshund Std Longhair Dachshund Std Longhair SAS Dataset WORK.CHKBREED OBS BREED VARIETY OTHRINFO CNTBREED CNTOWNED 1 Dachshund Mini Dachshund MiniLonghair Dachshund MiniWirehair Dachshund Std Dachshund Std Longhair Dachshund Std Wirehair Table "1-H" : Results of merging the analysis back to the original input The end result of this analysis is the discovery that the BY statement in the routine does not contain the proper variables to uniquely identify each record. By adding the extra variable or variables to the original BY statement, the routine works without error and without the offending NOTE. [Table "1-I" shows the corrected MERGE statement.] 682 DATA DOGDTAIL; 683 MERGE DOGOWNED (IN=IN_OWNED) 684 DOGBREED (IN=IN_BREED); 685 BY BREED VARIETY; 686 IF IN_OWNED ; 687 RUN; NOTE: The data set WORK.DOGDTAIL has 9 observations and 6 variables. Table "1-I" : SASLOG - MERGE without "repeats of BY values" 4
5 CONCLUSION This paper addressed the use of ad hoc routines to explore WHY a common SASLOG message occurred, and covered how to correct a routine to prevent the recurrence of that message. It is hoped that the mechanisms discussed in this paper might be used by the readers in their daily jobs. However, this paper is a failure -- at least in part -- if the process stops there. It is hoped, even more strongly, that the concepts of developing and using ad hoc routines to fully understand ones data are the true lessons that the reader retains from this paper. REFERENCES / FOR FURTHER INFORMATION Burlew, Michele M. (2001). Debugging SAS Programs A Handbook of Tools and Techniques. Cary, NC: SAS Institute, Inc. Cody, Ron (2008). Cody s Data Cleaning Techniques Using SAS, Second Edition. Cary, NC: SAS Institute, Inc. Kuligowski, Andrew T. (2003), "The BEST. Message in the SASLOG". Proceedings of the Twenty- Eighth Annual SAS Users Group International Conference. Cary, NC: SAS Institute, Inc. SAS Institute, Inc. (2008), SAS 9.2 Online Documentation. Cary, NC: SAS Institute, Inc. SAS Institute, Inc. (1990), SAS Language: Reference, Version 6, First Edition. Cary, NC: SAS Institute, Inc. SAS Institute, Inc. (2007), SAS OnlineDoc, Cary, NC: SAS Institute, Inc. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. The author can be contacted via at: A_Kuligowski@msn.com (preferred) or Andy.Kuligowski@Nielsen.com ACKNOWLEDGMENTS The author wishes to acknowledge those individuals, whose identities have been lost to the ages, whose data oddities across the years provided the true inspiration behind this paper. 5
Paper 109-25 Merges and Joins Timothy J Harrington, Trilogy Consulting Corporation
Paper 109-25 Merges and Joins Timothy J Harrington, Trilogy Consulting Corporation Abstract This paper discusses methods of joining SAS data sets. The different methods and the reasons for choosing a particular
More informationIntro to Longitudinal Data: A Grad Student How-To Paper Elisa L. Priest 1,2, Ashley W. Collinsworth 1,3 1
Intro to Longitudinal Data: A Grad Student How-To Paper Elisa L. Priest 1,2, Ashley W. Collinsworth 1,3 1 Institute for Health Care Research and Improvement, Baylor Health Care System 2 University of North
More informationNew Tricks for an Old Tool: Using Custom Formats for Data Validation and Program Efficiency
New Tricks for an Old Tool: Using Custom Formats for Data Validation and Program Efficiency S. David Riba, JADE Tech, Inc., Clearwater, FL ABSTRACT PROC FORMAT is one of the old standards among SAS Procedures,
More informationChristianna S. Williams, University of North Carolina at Chapel Hill, Chapel Hill, NC
Christianna S. Williams, University of North Carolina at Chapel Hill, Chapel Hill, NC ABSTRACT Have you used PROC MEANS or PROC SUMMARY and wished there was something intermediate between the NWAY option
More informationThe Essentials of Finding the Distinct, Unique, and Duplicate Values in Your Data
The Essentials of Finding the Distinct, Unique, and Duplicate Values in Your Data Carter Sevick MS, DoD Center for Deployment Health Research, San Diego, CA ABSTRACT Whether by design or by error there
More informationSubsetting Observations from Large SAS Data Sets
Subsetting Observations from Large SAS Data Sets Christopher J. Bost, MDRC, New York, NY ABSTRACT This paper reviews four techniques to subset observations from large SAS data sets: MERGE, PROC SQL, user-defined
More informationStatistics and Analysis. Quality Control: How to Analyze and Verify Financial Data
Abstract Quality Control: How to Analyze and Verify Financial Data Michelle Duan, Wharton Research Data Services, Philadelphia, PA As SAS programmers dealing with massive financial data from a variety
More informationSearch and Replace in SAS Data Sets thru GUI
Search and Replace in SAS Data Sets thru GUI Edmond Cheng, Bureau of Labor Statistics, Washington, DC ABSTRACT In managing data with SAS /BASE software, performing a search and replace is not a straight
More informationOne problem > Multiple solutions; various ways of removing duplicates from dataset using SAS Jaya Dhillon, Louisiana State University
One problem > Multiple solutions; various ways of removing duplicates from dataset using SAS Jaya Dhillon, Louisiana State University ABSTRACT In real world, analysts seldom come across data which is in
More informationEXST SAS Lab Lab #4: Data input and dataset modifications
EXST SAS Lab Lab #4: Data input and dataset modifications Objectives 1. Import an EXCEL dataset. 2. Infile an external dataset (CSV file) 3. Concatenate two datasets into one 4. The PLOT statement will
More informationSalary. Cumulative Frequency
HW01 Answering the Right Question with the Right PROC Carrie Mariner, Afton-Royal Training & Consulting, Richmond, VA ABSTRACT When your boss comes to you and says "I need this report by tomorrow!" do
More informationSQL SUBQUERIES: Usage in Clinical Programming. Pavan Vemuri, PPD, Morrisville, NC
PharmaSUG 2013 Poster # P015 SQL SUBQUERIES: Usage in Clinical Programming Pavan Vemuri, PPD, Morrisville, NC ABSTRACT A feature of PROC SQL which provides flexibility to SAS users is that of a SUBQUERY.
More informationSUGI 29 Coders' Corner
Paper 074-29 Tales from the Help Desk: Solutions for Simple SAS Mistakes Bruce Gilsen, Federal Reserve Board INTRODUCTION In 19 years as a SAS consultant at the Federal Reserve Board, I have seen SAS users
More informationTHE POWER OF PROC FORMAT
THE POWER OF PROC FORMAT Jonas V. Bilenas, Chase Manhattan Bank, New York, NY ABSTRACT The FORMAT procedure in SAS is a very powerful and productive tool. Yet many beginning programmers rarely make use
More informationTales from the Help Desk 3: More Solutions for Simple SAS Mistakes Bruce Gilsen, Federal Reserve Board
Tales from the Help Desk 3: More Solutions for Simple SAS Mistakes Bruce Gilsen, Federal Reserve Board INTRODUCTION In 20 years as a SAS consultant at the Federal Reserve Board, I have seen SAS users make
More informationUsing Pharmacovigilance Reporting System to Generate Ad-hoc Reports
Using Pharmacovigilance Reporting System to Generate Ad-hoc Reports Jeff Cai, Amylin Pharmaceuticals, Inc., San Diego, CA Jay Zhou, Amylin Pharmaceuticals, Inc., San Diego, CA ABSTRACT To supplement Oracle
More informationThe SET Statement and Beyond: Uses and Abuses of the SET Statement. S. David Riba, JADE Tech, Inc., Clearwater, FL
The SET Statement and Beyond: Uses and Abuses of the SET Statement S. David Riba, JADE Tech, Inc., Clearwater, FL ABSTRACT The SET statement is one of the most frequently used statements in the SAS System.
More informationCounting the Ways to Count in SAS. Imelda C. Go, South Carolina Department of Education, Columbia, SC
Paper CC 14 Counting the Ways to Count in SAS Imelda C. Go, South Carolina Department of Education, Columbia, SC ABSTRACT This paper first takes the reader through a progression of ways to count in SAS.
More informationLabels, Labels, and More Labels Stephanie R. Thompson, Rochester Institute of Technology, Rochester, NY
Paper FF-007 Labels, Labels, and More Labels Stephanie R. Thompson, Rochester Institute of Technology, Rochester, NY ABSTRACT SAS datasets include labels as optional variable attributes in the descriptor
More informationTable Lookups: From IF-THEN to Key-Indexing
Paper 158-26 Table Lookups: From IF-THEN to Key-Indexing Arthur L. Carpenter, California Occidental Consultants ABSTRACT One of the more commonly needed operations within SAS programming is to determine
More informationAlternatives to Merging SAS Data Sets But Be Careful
lternatives to Merging SS Data Sets ut e Careful Michael J. Wieczkowski, IMS HELTH, Plymouth Meeting, P bstract The MERGE statement in the SS programming language is a very useful tool in combining or
More informationPaper 2917. Creating Variables: Traps and Pitfalls Olena Galligan, Clinops LLC, San Francisco, CA
Paper 2917 Creating Variables: Traps and Pitfalls Olena Galligan, Clinops LLC, San Francisco, CA ABSTRACT Creation of variables is one of the most common SAS programming tasks. However, sometimes it produces
More informationA Method for Cleaning Clinical Trial Analysis Data Sets
A Method for Cleaning Clinical Trial Analysis Data Sets Carol R. Vaughn, Bridgewater Crossings, NJ ABSTRACT This paper presents a method for using SAS software to search SAS programs in selected directories
More informationConstructing a Table of Survey Data with Percent and Confidence Intervals in every Direction
Constructing a Table of Survey Data with Percent and Confidence Intervals in every Direction David Izrael, Abt Associates Sarah W. Ball, Abt Associates Sara M.A. Donahue, Abt Associates ABSTRACT We examined
More information4 Other useful features on the course web page. 5 Accessing SAS
1 Using SAS outside of ITCs Statistical Methods and Computing, 22S:30/105 Instructor: Cowles Lab 1 Jan 31, 2014 You can access SAS from off campus by using the ITC Virtual Desktop Go to https://virtualdesktopuiowaedu
More informationEffective Use of SQL in SAS Programming
INTRODUCTION Effective Use of SQL in SAS Programming Yi Zhao Merck & Co. Inc., Upper Gwynedd, Pennsylvania Structured Query Language (SQL) is a data manipulation tool of which many SAS programmers are
More informationFoundations & Fundamentals. A PROC SQL Primer. Matt Taylor, Carolina Analytical Consulting, LLC, Charlotte, NC
A PROC SQL Primer Matt Taylor, Carolina Analytical Consulting, LLC, Charlotte, NC ABSTRACT Most SAS programmers utilize the power of the DATA step to manipulate their datasets. However, unless they pull
More informationReturn of the Codes: SAS, Windows, and Your s Mark Tabladillo, Ph.D., MarkTab Consulting, Atlanta, GA Associate Faculty, University of Phoenix
Paper AP-11 Return of the Codes: SAS, Windows, and Your s Mark Tabladillo, Ph.D., MarkTab Consulting, Atlanta, GA Associate Faculty, University of Phoenix ABSTRACT Robust applications participate in the
More informationThe Science and Art of Market Segmentation Using PROC FASTCLUS Mark E. Thompson, Forefront Economics Inc, Beaverton, Oregon
The Science and Art of Market Segmentation Using PROC FASTCLUS Mark E. Thompson, Forefront Economics Inc, Beaverton, Oregon ABSTRACT Effective business development strategies often begin with market segmentation,
More informationMWSUG 2011 - Paper S111
MWSUG 2011 - Paper S111 Dealing with Duplicates in Your Data Joshua M. Horstman, First Phase Consulting, Inc., Indianapolis IN Roger D. Muller, First Phase Consulting, Inc., Carmel IN Abstract As SAS programmers,
More informationPaper 208-28. KEYWORDS PROC TRANSPOSE, PROC CORR, PROC MEANS, PROC GPLOT, Macro Language, Mean, Standard Deviation, Vertical Reference.
Paper 208-28 Analysis of Method Comparison Studies Using SAS Mohamed Shoukri, King Faisal Specialist Hospital & Research Center, Riyadh, KSA and Department of Epidemiology and Biostatistics, University
More informationPaper 70-27 An Introduction to SAS PROC SQL Timothy J Harrington, Venturi Partners Consulting, Waukegan, Illinois
Paper 70-27 An Introduction to SAS PROC SQL Timothy J Harrington, Venturi Partners Consulting, Waukegan, Illinois Abstract This paper introduces SAS users with at least a basic understanding of SAS data
More informationPreparing your data for analysis using SAS. Landon Sego 24 April 2003 Department of Statistics UW-Madison
Preparing your data for analysis using SAS Landon Sego 24 April 2003 Department of Statistics UW-Madison Assumptions That you have used SAS at least a few times. It doesn t matter whether you run SAS in
More informationSDTM, ADaM and define.xml with OpenCDISC Matt Becker, PharmaNet/i3, Cary, NC
PharmaSUG 2012 - Paper HW07 SDTM, ADaM and define.xml with OpenCDISC Matt Becker, PharmaNet/i3, Cary, NC ABSTRACT Standards are an ongoing focus of the health care and life science industry. Common terms
More informationYou have got SASMAIL!
You have got SASMAIL! Rajbir Chadha, Cognizant Technology Solutions, Wilmington, DE ABSTRACT As SAS software programs become complex, processing times increase. Sitting in front of the computer, waiting
More informationUsing the Magical Keyword "INTO:" in PROC SQL
Using the Magical Keyword "INTO:" in PROC SQL Thiru Satchi Blue Cross and Blue Shield of Massachusetts, Boston, Massachusetts Abstract INTO: host-variable in PROC SQL is a powerful tool. It simplifies
More informationWhat You re Missing About Missing Values
Paper 1440-2014 What You re Missing About Missing Values Christopher J. Bost, MDRC, New York, NY ABSTRACT Do you know everything you need to know about missing values? Do you know how to assign a missing
More informationPROC SUMMARY Options Beyond the Basics Susmita Pattnaik, PPD Inc, Morrisville, NC
Paper BB-12 PROC SUMMARY Options Beyond the Basics Susmita Pattnaik, PPD Inc, Morrisville, NC ABSTRACT PROC SUMMARY is used for summarizing the data across all observations and is familiar to most SAS
More informationImporting Excel File using Microsoft Access in SAS Ajay Gupta, PPD Inc, Morrisville, NC
ABSTRACT PharmaSUG 2012 - Paper CC07 Importing Excel File using Microsoft Access in SAS Ajay Gupta, PPD Inc, Morrisville, NC In Pharmaceuticals/CRO industries, Excel files are widely use for data storage.
More informationSAS Views The Best of Both Worlds
Paper 026-2010 SAS Views The Best of Both Worlds As seasoned SAS programmers, we have written and reviewed many SAS programs in our careers. What I have noticed is that more often than not, people who
More informationSimulate PRELOADFMT Option in PROC FREQ Ajay Gupta, PPD, Morrisville, NC
ABSTRACT PharmaSUG 2015 - Paper QT33 Simulate PRELOADFMT Option in PROC FREQ Ajay Gupta, PPD, Morrisville, NC In Pharmaceuticals/CRO industries, table programing is often started when only partial data
More informationPaper DV-06-2015. KEYWORDS: SAS, R, Statistics, Data visualization, Monte Carlo simulation, Pseudo- random numbers
Paper DV-06-2015 Intuitive Demonstration of Statistics through Data Visualization of Pseudo- Randomly Generated Numbers in R and SAS Jack Sawilowsky, Ph.D., Union Pacific Railroad, Omaha, NE ABSTRACT Statistics
More informationUtilizing Clinical SAS Report Templates Sunil Kumar Gupta Gupta Programming, Thousand Oaks, CA
Utilizing Clinical SAS Report Templates Sunil Kumar Gupta Gupta Programming, Thousand Oaks, CA ABSTRACT SAS programmers often have the responsibility of supporting the reporting needs of the Clinical Affairs
More informationEverything you wanted to know about MERGE but were afraid to ask
TS- 644 Janice Bloom Everything you wanted to know about MERGE but were afraid to ask So you have read the documentation in the SAS Language Reference for MERGE and it still does not make sense? Rest assured
More informationMEASURES OF LOCATION AND SPREAD
Paper TU04 An Overview of Non-parametric Tests in SAS : When, Why, and How Paul A. Pappas and Venita DePuy Durham, North Carolina, USA ABSTRACT Most commonly used statistical procedures are based on the
More informationSAS Comments How Straightforward Are They? Jacksen Lou, Merck & Co.,, Blue Bell, PA 19422
SAS Comments How Straightforward Are They? Jacksen Lou, Merck & Co.,, Blue Bell, PA 19422 ABSTRACT SAS comment statements typically use conventional symbols, *, %*, /* */. Most programmers regard SAS commenting
More informationSUGI 29 Applications Development
Backing up File Systems with Hierarchical Structure Using SAS/CONNECT Fagen Xie, Kaiser Permanent Southern California, California, USA Wansu Chen, Kaiser Permanent Southern California, California, USA
More informationTips, Tricks, and Techniques from the Experts
Tips, Tricks, and Techniques from the Experts Presented by Katie Ronk 2997 Yarmouth Greenway Drive, Madison, WI 53711 Phone: (608) 278-9964 Web: www.sys-seminar.com Systems Seminar Consultants, Inc www.sys-seminar.com
More informationMore Tales from the Help Desk: Solutions for Simple SAS Mistakes Bruce Gilsen, Federal Reserve Board
More Tales from the Help Desk: Solutions for Simple SAS Mistakes Bruce Gilsen, Federal Reserve Board INTRODUCTION In 20 years as a SAS consultant at the Federal Reserve Board, I have seen SAS users make
More informationPaper 232-2012. Getting to the Good Part of Data Analysis: Data Access, Manipulation, and Customization Using JMP
Paper 232-2012 Getting to the Good Part of Data Analysis: Data Access, Manipulation, and Customization Using JMP Audrey Ventura, SAS Institute Inc., Cary, NC ABSTRACT Effective data analysis requires easy
More informationIntroduction to Criteria-based Deduplication of Records, continued SESUG 2012
SESUG 2012 Paper CT-11 An Introduction to Criteria-based Deduplication of Records Elizabeth Heath RTI International, RTP, NC Priya Suresh RTI International, RTP, NC ABSTRACT When survey respondents are
More informationTaming the PROC TRANSPOSE
Taming the PROC TRANSPOSE Matt Taylor, Carolina Analytical Consulting, LLC ABSTRACT The PROC TRANSPOSE is often misunderstood and seldom used. SAS users are unsure of the results it will give and curious
More informationSAS Decision Services Engine XXXXXX Loggers Logger(file:///myMachine/Lev1/Web/Common/LogConfig/SASDecisionService Level sengine-log4j.
SAS Decision Services Engine XXXXXX Loggers Logger(file:///myMachine/Lev1/Web/Common/LogConfig/SASDecisionService Level sengine-log4j.xml) Root Logger com.sas.services.deployment.omi com.sas.rtdm.implementation.eventexecutor.activity_time
More informationPreparing Real World Data in Excel Sheets for Statistical Analysis
Paper DM03 Preparing Real World Data in Excel Sheets for Statistical Analysis Volker Harm, Bayer Schering Pharma AG, Berlin, Germany ABSTRACT This paper collects a set of techniques of importing Excel
More informationSAS Information Delivery Portal: Organize your Organization's Reporting
SAS Information Delivery Portal: Organize your Organization's Reporting Kevin Davidson Texas Institute for Measurement, Evaluation, and Statistics University of Houston, Houston, TX ABSTRACT The SAS Information
More informationPost Processing Macro in Clinical Data Reporting Niraj J. Pandya
Post Processing Macro in Clinical Data Reporting Niraj J. Pandya ABSTRACT Post Processing is the last step of generating listings and analysis reports of clinical data reporting in pharmaceutical industry
More informationTable 1. Demog. id education B002 5 b003 8 b005 5 b007 9 b008 7 b009 8 b010 8
Adding PROC SQL to your SAS toolbox for merging multiple tables without common variable names Susan Wancewicz, Moores Cancer Center, UCSD, La Jolla, CA ABSTRACT The SQL procedure offers an efficient method
More informationTransferring vs. Transporting Between SAS Operating Environments Mimi Lou, Medical College of Georgia, Augusta, GA
CC13 Transferring vs. Transporting Between SAS Operating Environments Mimi Lou, Medical College of Georgia, Augusta, GA ABSTRACT Prior to SAS version 8, permanent SAS data sets cannot be moved directly
More informationEssential Project Management Reports in Clinical Development Nalin Tikoo, BioMarin Pharmaceutical Inc., Novato, CA
Essential Project Management Reports in Clinical Development Nalin Tikoo, BioMarin Pharmaceutical Inc., Novato, CA ABSTRACT Throughout the course of a clinical trial the Statistical Programming group is
More informationReshaping & Combining Tables Unit of analysis Combining. Assignment 4. Assignment 4 continued PHPM 672/677 2/21/2016. Kum 1
Reshaping & Combining Tables Unit of analysis Combining Reshaping set: concatenate tables (stack rows) merge: link tables (attach columns) proc summary: consolidate rows proc transpose: reshape table Hye-Chung
More informationLet SAS Modify Your Excel File Nelson Lee, Genentech, South San Francisco, CA
ABSTRACT PharmaSUG 2015 - Paper QT12 Let SAS Modify Your Excel File Nelson Lee, Genentech, South San Francisco, CA It is common to export SAS data to Excel by creating a new Excel file. However, there
More informationAn Approach to Creating Archives That Minimizes Storage Requirements
Paper SC-008 An Approach to Creating Archives That Minimizes Storage Requirements Ruben Chiflikyan, RTI International, Research Triangle Park, NC Mila Chiflikyan, RTI International, Research Triangle Park,
More informationPaper 23-28. Hot Links: Creating Embedded URLs using ODS Jonathan Squire, C 2 RA (Cambridge Clinical Research Associates), Andover, MA
Paper 23-28 Hot Links: Creating Embedded URLs using ODS Jonathan Squire, C 2 RA (Cambridge Clinical Research Associates), Andover, MA ABSTRACT With SAS/BASE version 8, one can create embedded HTML links
More informationDo It Yourself (DIY) Data: Creating a Searchable Data Set of Available Classrooms using SAS Enterprise BI Server
Paper 8801-2016 Do It Yourself (DIY) Data: Creating a Searchable Data Set of Available Classrooms using SAS Enterprise BI Server Nicole E. Jagusztyn, Hillsborough Community College ABSTRACT At a community
More informationAlternative Methods for Sorting Large Files without leaving a Big Disk Space Footprint
Alternative Methods for Sorting Large Files without leaving a Big Disk Space Footprint Rita Volya, Harvard Medical School, Boston, MA ABSTRACT Working with very large data is not only a question of efficiency
More informationHealth Services Research Utilizing Electronic Health Record Data: A Grad Student How-To Paper
Paper 3485-2015 Health Services Research Utilizing Electronic Health Record Data: A Grad Student How-To Paper Ashley W. Collinsworth, ScD, MPH, Baylor Scott & White Health and Tulane University School
More informationPH 7525 Introduction to Data & Statistical Packages Course Reference #: 18254 Spring 2011
PH 7525 Introduction to Data & Statistical Packages Course Reference #: 18254 Spring 2011 Scott R. Weaver, Ph.D. Division of Epidemiology & Biostatistics Course Basics Faculty Accessibility Class Day/Time:
More informationImelda C. Go, South Carolina Department of Education, Columbia, SC
PO 003 Matching SAS Data Sets with PROC SQL: If at First You Don t Succeed, Match, Match Again ABSTRACT Imelda C. Go, South Carolina Department of Education, Columbia, SC Two data sets are often matched
More informationPharmaSUG 2013 - Paper DG06
PharmaSUG 2013 - Paper DG06 JMP versus JMP Clinical for Interactive Visualization of Clinical Trials Data Doug Robinson, SAS Institute, Cary, NC Jordan Hiller, SAS Institute, Cary, NC ABSTRACT JMP software
More informationKEYWORDS ARRAY statement, DO loop, temporary arrays, MERGE statement, Hash Objects, Big Data, Brute force Techniques, PROC PHREG
Paper BB-07-2014 Using Arrays to Quickly Perform Fuzzy Merge Look-ups: Case Studies in Efficiency Arthur L. Carpenter California Occidental Consultants, Anchorage, AK ABSTRACT Merging two data sets when
More informationAN INTRODUCTION TO MACRO VARIABLES AND MACRO PROGRAMS Mike S. Zdeb, New York State Department of Health
AN INTRODUCTION TO MACRO VARIABLES AND MACRO PROGRAMS Mike S. Zdeb, New York State Department of Health INTRODUCTION There are a number of SAS tools that you may never have to use. Why? The main reason
More informationKEY FEATURES OF SOURCE CONTROL UTILITIES
Source Code Revision Control Systems and Auto-Documenting Headers for SAS Programs on a UNIX or PC Multiuser Environment Terek Peterson, Alliance Consulting Group, Philadelphia, PA Max Cherny, Alliance
More informationUNIX Comes to the Rescue: A Comparison between UNIX SAS and PC SAS
UNIX Comes to the Rescue: A Comparison between UNIX SAS and PC SAS Chii-Dean Lin, San Diego State University, San Diego, CA Ming Ji, San Diego State University, San Diego, CA ABSTRACT Running SAS under
More informationTips for Constructing a Data Warehouse Part 2 Curtis A. Smith, Defense Contract Audit Agency, La Mirada, CA
Tips for Constructing a Data Warehouse Part 2 Curtis A. Smith, Defense Contract Audit Agency, La Mirada, CA ABSTRACT Ah, yes, data warehousing. The subject of much discussion and excitement. Within the
More informationSAS, Excel, and the Intranet
SAS, Excel, and the Intranet Peter N. Prause, The Hartford, Hartford CT Charles Patridge, The Hartford, Hartford CT Introduction: The Hartford s Corporate Profit Model (CPM) is a SAS based multi-platform
More informationInterfacing SAS Software, Excel, and the Intranet without SAS/Intrnet TM Software or SAS Software for the Personal Computer
Interfacing SAS Software, Excel, and the Intranet without SAS/Intrnet TM Software or SAS Software for the Personal Computer Peter N. Prause, The Hartford, Hartford CT Charles Patridge, The Hartford, Hartford
More informationSAS Rule-Based Codebook Generation for Exploratory Data Analysis Ross Bettinger, Senior Analytical Consultant, Seattle, WA
SAS Rule-Based Codebook Generation for Exploratory Data Analysis Ross Bettinger, Senior Analytical Consultant, Seattle, WA ABSTRACT A codebook is a summary of a collection of data that reports significant
More informationProgramming Tricks For Reducing Storage And Work Space Curtis A. Smith, Defense Contract Audit Agency, La Mirada, CA.
Paper 23-27 Programming Tricks For Reducing Storage And Work Space Curtis A. Smith, Defense Contract Audit Agency, La Mirada, CA. ABSTRACT Have you ever had trouble getting a SAS job to complete, although
More informationMethodologies for Converting Microsoft Excel Spreadsheets to SAS datasets
Methodologies for Converting Microsoft Excel Spreadsheets to SAS datasets Karin LaPann ViroPharma Incorporated ABSTRACT Much functionality has been added to the SAS to Excel procedures in SAS version 9.
More informationUsing SAS to Examine Health-Promoting Life Style Activities of Upper Division Nursing Students at USC
SESUG 2015 Paper PO-46 Using SAS to Examine Health-Promoting Life Style Activities of Upper Division Nursing Students at USC Abbas S. Tavakoli, DrPH, MPH, ME, Mary Boyd, Phd, RN, ABSTRACT Health promotion
More informationData-driven Validation Rules: Custom Data Validation Without Custom Programming Don Hopkins, Ursa Logic Corporation, Durham, NC
Data-driven Validation Rules: Custom Data Validation Without Custom Programming Don Hopkins, Ursa Logic Corporation, Durham, NC ABSTRACT One of the most expensive and time-consuming aspects of data management
More informationUsing DATA Step MERGE and PROC SQL JOIN to Combine SAS Datasets Dalia C. Kahane, Westat, Rockville, MD
Using DATA Step MERGE and PROC SQL JOIN to Combine SAS Datasets Dalia C. Kahane, Westat, Rockville, MD ABSTRACT This paper demonstrates important features of combining datasets in SAS. The facility to
More informationUsing SAS Views and SQL Views Lynn Palmer, State of California, Richmond, CA
Using SAS Views and SQL Views Lynn Palmer, State of Califnia, Richmond, CA ABSTRACT Views are a way of simplifying access to your ganization s database while maintaining security. With new and easier ways
More informationInnovative Techniques and Tools to Detect Data Quality Problems
Paper DM05 Innovative Techniques and Tools to Detect Data Quality Problems Hong Qi and Allan Glaser Merck & Co., Inc., Upper Gwynnedd, PA ABSTRACT High quality data are essential for accurate and meaningful
More informationDemand for Analysis-Ready Data Sets. An Introduction to Banking and Credit Card Analytics
Demand for Analysis-Ready Data Sets. An Introduction to Banking and Credit Card Analytics Copyright 2003 by Bikila bi Gwet Research papers and training manuals often use data that are too clean to reflect
More informationUSING SAS WITH ORACLE PRODUCTS FOR DATABASE MANAGEMENT AND REPORTING
USING SAS WITH ORACLE PRODUCTS FOR DATABASE MANAGEMENT AND REPORTING Henry W. Buffum, R. O. W. ScIences, Inc. Darryl J. Keith, U.S. Environmental Protection Agency Abstract: Data for a large environmental
More informationProject Request and Tracking Using SAS/IntrNet Software Steven Beakley, LabOne, Inc., Lenexa, Kansas
Paper 197 Project Request and Tracking Using SAS/IntrNet Software Steven Beakley, LabOne, Inc., Lenexa, Kansas ABSTRACT The following paper describes a project request and tracking system that has been
More informationAutomate Data Integration Processes for Pharmaceutical Data Warehouse
Paper AD01 Automate Data Integration Processes for Pharmaceutical Data Warehouse Sandy Lei, Johnson & Johnson Pharmaceutical Research and Development, L.L.C, Titusville, NJ Kwang-Shi Shu, Johnson & Johnson
More informationA Day in the Life of Data Part 2
ABSTRACT Paper 117-2013 A Day in the Life of Data Part 2 Harry Droogendyk, Stratia Consulting Inc. As a new SAS programmer, you may be overwhelmed with the variety of tricks and techniques that you see
More informationCustomer Profiling for Marketing Strategies in a Healthcare Environment MaryAnne DePesquo, Phoenix, Arizona
Paper 1285-2014 Customer Profiling for Marketing Strategies in a Healthcare Environment MaryAnne DePesquo, Phoenix, Arizona ABSTRACT In this new era of healthcare reform, health insurance companies have
More informationData Mining: An Overview of Methods and Technologies for Increasing Profits in Direct Marketing. C. Olivia Rud, VP, Fleet Bank
Data Mining: An Overview of Methods and Technologies for Increasing Profits in Direct Marketing C. Olivia Rud, VP, Fleet Bank ABSTRACT Data Mining is a new term for the common practice of searching through
More informationIntelligent Query and Reporting against DB2. Jens Dahl Mikkelsen SAS Institute A/S
Intelligent Query and Reporting against DB2 Jens Dahl Mikkelsen SAS Institute A/S DB2 Reporting Pains Difficult and slow to get information on available tables and columns table and column contents/definitions
More informationAbbas S. Tavakoli, DrPH, MPH, ME 1 ; Nikki R. Wooten, PhD, LISW-CP 2,3, Jordan Brittingham, MSPH 4
1 Paper 1680-2016 Using GENMOD to Analyze Correlated Data on Military System Beneficiaries Receiving Inpatient Behavioral Care in South Carolina Care Systems Abbas S. Tavakoli, DrPH, MPH, ME 1 ; Nikki
More informationStrategic Procurement: The SAS Solution for Supplier Relationship Management Fritz Lehman, SAS Institute Inc., Cary, NC
Paper 146-27 Strategic Procurement: The SAS Solution for Supplier Relationship Management Fritz Lehman, SAS Institute Inc., Cary, NC ABSTRACT The SAS Solution for supplier relationship management (SRM)
More informationStatistics & Analysis
NESUG How to Increase Sales of Orthopedic Equipment in United States: Factor and Cluster Analysis using SAS and R George Obsekov American College of Radiology Research Center Philadelphia, PA INTRODUCTION
More informationExploit SAS Enterprise BI Server to Manage Your Batch Scheduling Needs
Exploit SAS Enterprise BI Server to Manage Your Batch Scheduling Needs Troy B. Wolfe, Qualex Consulting Services, Inc., Miami, Florida ABSTRACT As someone responsible for maintaining over 40 nightly batch
More informationHOW TO USE SAS SOFTWARE EFFECTIVELY ON LARGE FILES. Juliana M. Ma, University of North Carolina
INTRODUCTION HOW TO USE SAS SOFTWARE EFFECTIVELY ON LARGE FILES The intent of this tutorial was to present basic principles worth remembering when working with SAS software and large files. The paper varies
More informationTop Ten Reasons to Use PROC SQL
Paper 042-29 Top Ten Reasons to Use PROC SQL Weiming Hu, Center for Health Research Kaiser Permanente, Portland, Oregon, USA ABSTRACT Among SAS users, it seems there are two groups of people, those who
More informationTHE TERMS AND CONCEPTS
Calculating Medication Compliance, Adherence, and Persistence in Administrative Pharmacy Claims Databases R. Scott Leslie, MedImpact Healthcare Systems, Inc., San Diego, CA ABSTRACT Compliance, adherence,
More informationPaper PO 015. Figure 1. PoweReward concept
Paper PO 05 Constructing Baseline of Customer s Hourly Electric Usage in SAS Yuqing Xiao, Bob Bolen, Diane Cunningham, Jiaying Xu, Atlanta, GA ABSTRACT PowerRewards is a pilot program offered by the Georgia
More information