Get in Control! Configuration Management for SAS Projects John Quarantillo, Westat, Rockville, MD

Transcription

1 AD004 Get in Control! Configuration Management for SAS Projects John Quarantillo, Westat, Rockville, MD Abstract SAS applications development can benefit greatly from the use of Configuration Management/Source Control software. Not only can multiple developers stay on track, but also they can work independently on different modules without stepping on each other s code or inadvertently working on the same module simultaneously. The development area can be configured in such a way that each developer has his own independent work area that has a separate copy of all finalized programs, and interim copies of programs assigned to that developer. This paper will illustrate techniques used to develop SAS programs for a production reporting system running against an Oracle database. Development teams of any size can implement these concepts that structure and organize the development, testing and deployment of SAS applications. Although this example shows SAS development using Microsoft Visual SourceSafe, the same techniques can be implemented with almost any configuration management software. Introduction SAS developers are not immune to the challenges faced by developers using such programming languages as Visual Basic, C++, etc. All developers need to address the issues of change management, source code control, and configuration management. Proper use of version control software such as Visual SourceSafe can easily meet the challenges and address the issues. In addition to using version control software, it is equally important to design the development environment to take full advantage of the benefits of using version control software. ******************** Evolution of Development Environment Design Initially, programmers would often develop systems in an ad hoc manner. There was little or no separation between development and production. Later, the need for program accuracy and data integrity brought about the need to create a separate development environment. Larger development efforts brought about the need for separate testing staff that worked independently from the developers. Finally, with the addition of version control, and multi-developer teams, a separate development environment was added for each developer. Creating Multi-Environment Programs The key to creating SAS programs that work in more than one environment is to remove all references to locations of data and other programs and place them in a driver program. The remainder of the programs is then defined as SAS Macros, which are called by the driver program. (Sample SAS code can be found at the end of the paper.) Version Control/Source Control Software There are many popular version control or source control software packages in use today. Basically, these programs allow developers to control and track changes to source code. This project was already using Microsoft Visual SourceSafe to manage the development of Active Server Page (ASP) code for a web-based interface, as well as middleware components developed using Microsoft Visual Basic. Visual SourceSafe integrates nicely with those development environments. Although Visual 1

2 SourceSafe does not integrate directly into the SAS development environment, it can still be used to manage SAS programs. Basic Model Our development life cycle has three separate phases. Development, Testing, and Production. Figure 1 illustrates the logical layout of our basic environment. Each workspace contains all the necessary programs and directories necessary to run the system. This includes a driver program, a macro library, a directory in which work data sets can be dumped for troubleshooting, and an output directory. A separate work area is created in the workspace for each developer (see Figure 3). After programs are developed, they move to the testing phase. Programs that pass testing, move into production, programs that fail testing, go back to development. Because the programs are developed using the driver approach, they can be freely moved from one step of the development life cycle (and its corresponding environment) to another without having to modify the source code. Setting Up the Workspace The development workspace is set up on a network server volume. This is mainly done to take advantage of corporate infrastructure that provides backup of network volumes. The network volume also comes into play for the testing area. For illustrative purposes, we will outline a team of 4 developers (Al, Bob, Chris, and Don.) Figure 2 shows the logical design of the multi-user environment from the perspective of the development life cycle. In addition to creating a work area for each user, there is also a work area for testing. This area is not directly accessed by any of the developers. Source programs that are ready for testing are automatically placed here during the development process (this will be further explained in the Using Shadow Folders section below). Figure 4 shows the top-level layout for the development and testing workspace. ******************** 2

3 Note that in Figure 4 there is no representation of the production environment. The production environment is often on a different network server, and frequently on the opposite side of a firewall from a development/testing environment. Programs are usually migrated to the production environment manually, or as part of a separate install script. Add the Programs to Visual Source- Safe Projects The SAS programs are then added to Visual SourceSafe projects which roughly correspond to directories or folders. This project structure should be as close as possible to the structure outlined in the previous discussion of workspaces. A good rule of thumb to follow is something does not exist unless it has been checked in to Visual SourceSafe. cannot be directly accessed without using the Visual SourceSafe interface. The Visual SourceSafe Administrator interface allows you to create what it calls shadow folders. Each Visual SourceSafe project can be associated with a physical directory on a network volume. Every time a developer checks in a program, the copy of that program is updated in the shadow folder. The main project folder should be set up with a shadow folder directed to the Testing work area. Driver programs should not be put in shadow folders. Development The development process is fairly simple, once a user/developer has been set up correctly. Each time a developer begins to work on the project they will need to do the following: Update their work area by choosing Get Latest Version on the Visual SourceSafe project folder. (see screenshot) Setting up Users in Visual Source- Safe Each developer is given a user account in Visual SourceSafe. Users with administrative rights can control what rights other users have, and can control their access to source programs on a project" level. After a developer s user account has been established, they need to open the Visual SourceSafe program, and set working folders for each directory set up in the workspace discussion above. Using Shadow folders When a program is initially added to a Visual SourceSafe project, or subsequently checked in to Visual SourceSafe, that program code is stored in the Visual SourceSafe database. It 3 Choose the program(s) that they need to modify, right click and Check Out those programs. Open SAS, or some other program editor, and make the modifications.

4 Unit test the modifications by submitting their user-specific driver program. Because the developer has a current copy of all working programs (see the first step) the newly modified program is tested against the current programs in the testing environment. If the program modifications pass unit testing, check the programs back in to Visual SourceSafe, making sure to indicate what significant changes were made in the Comments section of the Check In dialog box. Another good rule of thumb is to never check anything in without making a comment here. These comments become incredibly useful when reviewing a program s history in Visual SourceSafe. Testing After a developer has successfully modified a program, and unit tested it, the program is ready for full testing. This is usually communicated to the testing unit either through a change management tracking database, or possibly something as simple as ing the testing team. Because all programs currently checked into Visual SourceSafe are updated in the Testing shadow folder, the testers can run the current system by simply submitting their user-specific driver program. Production After a number of program changes have been made, and successfully passed testing, they are ready to be migrated to the production environment. This can be done manually, or by using an automated installation builder that will draw the current programs from the testing shadow folder, and install them on the production environment. At this point, it is important for the Project Administrator to Label the project by right clicking the project, and choosing Label from the menu, and labeling the project with a release number, or some other milestone (see screenshot below). Visual SourceSafe will place an entry in the History of every file in that project. These labels are necessary to facilitate identifying the specific contents of a release. If a developer needs to create a patch of a production program independent of the current development process, they can obtain a copy of the exact version of every program at the time of the release using these labels. Conclusions As outlined above, with a bit of front-end environment design and planning, and a small learning curve to comfortably navigate the Visual SourceSafe interface, a SAS project can benefit greatly. The hidden beauty of managing projects with source code control or version control comes to light when there is a problem with the system. A developer can compare different versions of a program and see exactly what changes were made and when. If developers followed the never check in a program without adding a comment mantra, troubleshooting program changes should be greatly simplified. In addition, a developer can be given an entire working copy of a system at any Label point. (see Production above). Projects can also be branched to allow for multiple development paths, and later merged, using a built in differences interface. ******************** 4

5 References Ray, C. (1998) Establishing Production and Development Environments for Base SAS Software Development Proceedings of the Twenty-Third Annual SAS Users Group International Conference, Paper 26. Acknowledgments SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Microsoft and Visual SourceSafe are registered trademarks of Microsoft, of Redmond, WA. Many thanks to Craig Ray, for laying the foundation for this structured approach. Contact John Quarantillo Westat, Inc Research Blvd. Rockville, MD Sample Code for a Simple Driver Program: /*******************************************************************/ /* John Q's simplified driver program */ /* */ /* simple_driver.sas 6/16/2003 */ /* */ /* This program creates and sets the global macro variables */ /* necessary to use a Multi-Environment development approach. */ /* */ /*******************************************************************/ /* Macro that loops and calls the report for each region*/ %macro callrpts; %local j; /*Local loop counter */ %do j=1 %to &numregions; /* 1 -> &numregions run each report for all regions */ %let Region = &j; /* this should evaluate to actual region number */ %let Report = SimpleReport; %SimpleReport; /* Run the report */ %end; /* of %do j=1 %to &numregions; */ %mend callrpts; %global Environ; /*var that indicates the Environment */ %global Developer; /*used to identify correct developer (and their environment)*/ %global ProgRoot; /*main directory where the macros are kept */ %global OutputRoot1; /*main directory where the output should go */ %global OutputRoot2; /*directory where the output for archival goes */ %global SaveWork; /*Save SAS work library? (1 = Yes) */ %global DB; /*database name */ %global LoginID; /*database loginid */ %global PW; /*database password */ %global UserName; /*database UserName */ %global Report; /*Name of Report (and program) */ %global Region; /*Region Number */ %global NumRegions; /*Number of Regions */ (continued) 5

6 Sample Code for a Simple Driver Program (continued): /* vvvv Use these to Run in Development environment vvvv */ %let Developer = Bob; *%let Developer = Testing; /*for Testing*/ %let Environ = DEV; %let ProgRoot = \\rk24\vol2404\kccs\ad004\&developer; %let OutputRoot1 = &ProgRoot\Output; %let OutputRoot2 = \\rk24\vol2404\kccs\ad004\&developer\reportsarchive; %let savework = 1; /* ^^^^ Use these to Run in Development environment ^^^^ */ /* vvvv Use these lines to Run in PRODUCTION environment vvvv */ *%let Environ = PRD; *%let ProgRoot = d:\kccs; *%let OutputRoot1 = \\FOSTS1\KCCSRPT; *%let OutputRoot2 = \\rk24\vol2404\kccs\reportsarchive; /* ^^^^ Uncomment these to Run in PRODUCTION environment ^^^^ */ %let LoginID = kccs_user; /* Same across all environments */ %let pw = kccs_pass; /* Same across all environments */ %let UserName = 0; /* Same across all environments */ %let DB = KCCS1&Environ; /* Varies based on value of Environ */ %let NumRegions = 2; /* Project Specific */ libname ora ORACLE user=&loginid orapw=&pw path=&db; libname workstuf "&ProgRoot\Work Dump"; options ls=100 ps=58 missing=' ' nofmterr mprint /*mlogic*/ notes mautosource sasautos=("&progroot\macros"); %callrpts 6

7 Sample Code for a Simple Report Program (Macro): %macro SimpleReport; /**************************************************************** /* Name: SimpleReport.sas /* Date: 6/16/2003 /* Desc: Simple Report Program /****************************************************************/ ods pdf(1) file="&outputroot1\region&region&report..pdf" notoc; ods pdf(2) file="&outputroot2\region&region&report..pdf" notoc; proc sql; connect to oracle (user=&loginid orapw=&pw path="@&db"); /* EXTRACT REPORT DATA */ create table ReportData as select * from connection to oracle ( SELECT * FROM vroleregion a WHERE ) as sasfile; /* FILTER FOR REGION */ %if &Region ^= %THEN a.region=&region; quit; disconnect from oracle; /*for testing and debugging*/ %if &savework = 1 %then %do; proc datasets library=work nolist; copy out=workstuf; run; quit; %end; /* INSERT CODE HERE TO CREATE REPORT OUTPUT (PROC REPORT/PRINT/TABULATE) */ Title1 "Report &Report for Region &Region"; proc print data=reportdata; run; ods pdf(1) close; ods pdf(2) close; %mend SimpleReport; 7