U.S. Energy Information Administration: The Move to SAS 9.2

Transcription

1 U.S. Energy Information Administration: The Move to SAS 9.2 Michael Bretz, Chenega Government Consulting, Washington, DC ABSTRACT The U.S. Energy Information Administration (EIA) is currently using the following SAS solutions SAS Foundation SAS Enterprise BI Server SAS Forecast Server This paper discusses the migration strategy implemented at EIA and the move to the SAS 9.2 platform. An overview of the hardware and software architecture will be discussed and our experiences migrating from SAS to SAS 9.2 in the following areas: Installation, configuration and maintenance upgrades of SAS 9.2 software An overview of the security model changes from SAS to SAS 9.2 Migration of SAS artifacts from SAS to SAS 9.2 INTRODUCTION SAS Enterprise BI Server and SAS Forecast Server were purchased in 2009 in support of migrating SAS programs and applications from the mainframe. Applications and programs written using SAS Foundation software were migrated to these new software solutions and provided the opportunity to use new enhancements and features within the SAS Intelligence Platform. EIA also has a PC SAS user base which provided an option for users to migrate programs and applications to a server based solution with these new solutions. The software was licensed for a UNIX server running Linux. These new products provided options for publishing including web distribution capabilities for EIA survey products and information for internal users as well as provide opportunities to explore the use of business analytics like high-performance forecasting. As applications were being converted to the SAS server environment, EIA wanted to move forward with the upgrade to SAS 9.2 software for the Linux UNIX server. A migration plan and strategy was necessary to ensure a successful upgrade of SAS files and applications to the new release while also ensuring user customizations were honored along with validation of data and business processes. MIGRATION STRATEGY There are two primary approaches available when upgrading to SAS 9.2. Promote install SAS 9.2 and upgrade portions of the current SAS content using the export and import SAS Package wizards Migrate install SAS 9.2 and upgrade all of the SAS content using the SAS Migration Utility with the SAS Deployment Wizard Promotion was chosen as the migration strategy at EIA for the following reasons: The SAS topology and operating system changed for the SAS 9.2 environment Existing SAS applications could continue production processing cycles while the new SAS 9.2 environment was configured SAS 9.2 enhancements and features could be tested and the security model implementation could be improved Migrated applications and business processes could be run to verify and validate all output and results in a separate environment ARCHITECTURE The hardware and software architecture is setup for internal EIA SAS user access. The following information provides a summary of those architecture configurations for the SAS environments. 1

2 HARDWARE ARCHITECTURE The SAS environment was configured as two-tier architecture. The middle/web tier is a Windows server configured with Tomcat while the SAS Metadata and SAS Server Compute tiers are installed and configured on 32-bit Linux Server. The EIA SAS 9.2 environment combines the SAS Metadata Server, SAS Compute and Web tiers on a 64-bit Linux server. The middle/web tier utilizes the JBoss application server and the SAS Content Server. SOFTWARE ARCHITECTURE The SAS Enterprise BI Server and SAS Forecast Server software were originally installed and configured with SAS 9.2 (TS2M0). The SAS Hot Fix update process has been replaced with a maintenance strategy that allows for updates to the software through the SAS Deployment Wizard. The SAS Deployment Wizard allows for the silent update of all components within the software architecture. The EIA SAS administrators have applied two maintenance releases to the SAS 9.2 environment using this process (TS2M2 and TS2M3). The SAS 9.2 environment is now at SAS 9.2 (TS2M3). This maintenance process is a significant improvement that allows for minimal manual configuration of the software during the upgrade process. Deployment summaries are generated and available for review as well as detailed log information. These logs capture and collect detailed install and configuration operations for review should a step within the process fail. The SAS maintenance update documentation provides reference information to outline the SAS components in scope for the maintenance update. Product release numbers for all SAS products can be determined by generating a deployment registry report. For SAS 9.2 Phase 2 (TS2M0) and later, the installation of maintenance updates are logged in the SAS Deployment Registry. A reporting utility named View Registry processes the deployment registry and generates a report in HTML format. A version of this utility is provided with the software and installed in the SAS installation directory under the SASHOME/deploymntreg directory starting with the SAS 9.2 TS2M2 software release. The client tier provides users with desktop access to data through a set of role-based interfaces. For many information consumers, reporting and analysis can be performed using a Web browser. For more advanced design and analysis tasks, SAS Enterprise Guide software can be used along with many of the SAS Enterprise BI Server clients. EIA changed the deployment strategy for the licensed SAS 9.2 clients. The clients are not installed on each user s desktop. Instead all licensed, SAS clients are installed on a Windows 2008 Terminal server farm. Centralization of the SAS clients allows for easier maintenance and updates to the client software as well as load balancing user s access to these clients as our user base grows. SAS9 SECURITY The SAS 9.2 environment allowed the SAS Administrators to improve the SAS Metadata Server security model as compared to the SAS environment. The key improvements made to the SAS9 Security model are: Flattening of the SAS Folders hierarchy to Store and Organize BI and Forecast Server Content Increased Usage of Access Control Templates (ACTs) Use of Roles within User Manager of SAS Management Console Our security business requirements were to organize content and access within an Office > Division > Group hierarchy that mimics the EIA organizational structure. Multiple offices exist within the EIA organization. Authorization is the process of determining a user s grant/deny permissions for SAS resources. The SAS Intelligence Platform includes an authorization mechanism that consists of access controls that you define and store in the metadata repository. These metadata-based controls supplement protections and controls at the host system level. You can use the metadata authorization layer to manage access to any metadata object like reports, data definitions, information maps, stored processes and SAS Forecast Server projects etc. In the EIA environment, the user s login id is mapped to a metadata identity and assigned to one or more metadata groups. These metadata groups are then used to define a set of Access Control Templates to assign the effective permissions to SAS resources. SAS FOLDERS The metadata authorization layer is used to manage access to content such as reports, data, information maps, and stored processes by utilizing custom folders created with permissions set so that each folder offers appropri- 2

3 ate access. SAS 9.2 Management Console now has a Folders view displaying the folders that are used to store SAS metadata objects. The EIA SAS environment had the SAS Folders organized in a hierarchical order based on the EIA Organization Chart. The SAS 9.2 SAS Folder structure was flattened by business unit. Figure 1 illustrates the organization of the SAS Folders in each environment. The images on the left (Item 1 and 2) are the SAS Folder organizations as setup within SAS Management Console. The image on the right (Item 3) is a single user s view of the SAS Folders accessible to the user when referenced within SAS Enterprise Guide. For example, SAS Folder and Library access is controlled by their authorization permissions. In this example, the user is within the Office of Oil and Gas (OOG), Petroleum Marketing (PM) Group within the Petroleum Division (PD). As shown in item 3, the user is granted access to the SAS Folders within their group (OOG_PD_PM) allowing BI content to be accessible to other users within the same group. In addition, SAS folders were created to facilitate sharing of BI content across the Division (OOG_PD) or within the Office (OOG). These office and division folders are setup with read-only access. A user also has the option to save content in a personal (not shared) SAS folder called My Folder. The Shared Data folder allows for content to be shared and accessed by all users. We define global resources like SAS/GRAPH maps as an example within this area for user s to reference when developing content. Figure 1: SAS Folder Organization and Access ACCESS CONTROL TEMPLATES (ACTS) The SAS environment used a limited number of ACTS against the metadata content. ACTs were used to assign grant/deny permissions to the SAS Libraries and SAS BI Report content within the business unit. SAS Libraries where defined not only to access SAS data sets but also Oracle DBMS data sources. EIA BI Content includes SAS Stored Processes, Information Maps and Web Report Studio Reports. The SAS Folder structure implemented within the SAS environment was hierarchical mimicking EIA s organizational chart as noted in Figure 1. Access Control Entries were applied at the highest level of the SAS Folders allowing objects to receive the necessary grant/deny permissions via inheritance. The SAS 9.2 Environment used a combination of ACTs broken out by the Office/Division business units. An ACT was applied to the appropriate SAS folder still allowing for inheritance to grant/deny permission to the BI content based on parent object permissions. The ACT allows for the changing of permissions within the template vs. adjusting Access Control Entries (ACE) permissions assigned on the individual SAS folders. This method allows for 3

4 easier maintenance of permissions and reduces manual errors applying ACEs incorrectly. Figure 2 illustrates the approach used to setup Read and ReadMetadata via an ACT as shown at the Office level. Figure 2: ACT Properties at the Office Level Figure 3 depicts the permissions applied to the SAS Folder for the group. Since this is the lowest level business unit, users will be assigned ReadMetadata and WriteMemberMetadata within the folder as shown. Note the Read, Write, Create and Delete permissions. In this case we are allowing these rights to allow users within this unit read, add or update content within the SAS folder. Figure 3: Group SAS Folder Access ROLE SUPPORT The User Manager plug-in of SAS Management Console provides support for roles. A role manages the availability of application features such as menu items, and any user who is a member of a role has all of that role s capabilities. In our SAS 9.2 environment, a role was created to restrict user access to SAS Administrators functions within SAS Management Console. The role allows one more of the plug-ins in SAS Management Console to not be accessible by a general user. The following figure depicts what an end user and a SAS Administrator would see logging into the SAS Management Console while assigned a role that disables plug-in access. 4

5 Figure 4: SAS Management Console Role Based Views Role support allows for additional security and access to SAS Management Console. In the future, we can deploy this client and assign roles to advanced users allowing access to any of the appropriate plug-ins. As an example we can allow access to the SAS Library Manager Plug-in. This will allow our advanced users the ability to create and manage the SAS Library definitions accessing DBMS data within their business unit. SAS TO SAS 9.2 MIGRATION Promotion was the preferred migration strategy for migrating SAS content to SAS 9.2. Promotion capabilities in SAS 9.2 are embedded within the SAS folder structure allowing for import of SAS content. The Export SAS Package and Import SAS Package wizards support partial promotion of content. The following are highlights of the content tested for migration from SAS to SAS 9.2 in our environment. SAS LIBRARIES Exporting SAS libraries and tables from the SAS environment requires these objects to be in a folder. To move these definitions to folders, the Collect Ungrouped Library Metadata option within BI Manager must be executed as shown. Figure 5: BI Manager - Library Metadata The collected SAS library folder(s) can then be exported to a SAS package for import to the SAS 9.2 environment. The SAS package can be imported by going to the Folders tab in SAS Management Console and selecting import with the following selections: 5

6 The Include access controls option was not selected since we have a new set of ACTs defined. In addition, the ACTS currently applied allow for these newly imported objects to accept permissions by inheritance of the defined template. The SAS library and tables are selected (Item 2) The target SAS application server is reset from SASMain to SASApp (Item 3) The target location path is assigned. Note the physical folder needs to exist and the physical tables will need to be migrated to the target server file system (Item 4) The following figure illustrates those options set during the SAS package import. Figure 6: SAS Package Import SAS FORECAST SERVER ENVIRONMENTS AND PROJECTS SAS Forecast Studio organizes content into a project. Each project is saved on the server where the processing occurs. In previous releases, the ability to group projects together into workspaces was not available. The new release supports environments to address this issue. An environment defines basic attributes including the logical name, host workspace server and content path on the server. An environment defines a container for projects, a workspace for users to execute code and a partition of a workspace server (Item 1). Environments can be used to allocate multiple project locations for each workspace server, allowing for more flexibility to apply security requirements. In our SAS 9.2 Server environment, we establish SAS Forecast Server Environments for each business unit. We can assign metadata permissions to control access to each environment. Users who invoke the SAS Forecast Studio client will be assigned their environment (Item 2). A SAS Forecast Administrator may have access to more than one Environment as shown (Item 3). In our environment the SAS Administrator assumes the SAS Forecast Administrator role and creates the SAS Forecast Environments for each business unit. 6

7 Figure 7: SAS Forecast Server Environment By default, the project can only be opened by the author of the project. An option exists to allow sharing of the project during the initial creation for users within the business unit based on the defined security. Details on the setup of SAS 9 security are outlined in the SAS Forecast Server Administrator s Guide as there are additional protections and controls required to allow proper creation of projects and content. Forecast Studio projects can be migrated from a SAS environment to a SAS 9.2 environment. Below outlines the migration steps to migrate a project. Use the %fsexport to export the project to an archive file Copy the archive to the new environment Register the project in the new Environment via the SAS Forecast Server Plug-in in SAS Management Console %fsexport( projectname=exampleproject, ArchiveName=Arch_ExampleProject, description=exportexample, user=<user-id>, password=<pw>, workspace=sasmain, cport=yes ); SAS Forecast Server Macros provide an interface to the SAS Forecast Server Java API. The macros allow you to perform administrative functions on a single project or a group of projects. The %FSEXPORT macro exports a single SAS Forecast Server project to an archive file. The archive file can then be imported within the new environment. The following is example code to export a SAS Forecast Studio 2.1 project to an archive file. 7

8 In the code syntax example, the project and archive name are provided along with a description. User credentials allow for access to the project for export. Note that the workspace=sasmain option was added to assign which SAS application server should be used to export the project. The cport=yes parameter exports the project to allow for our migration from a Linux 32-bit system to a 64-bit system. The archive file is placed in the default archive folder location as defined within the Forecast Studio plug-in in the SAS Environment. The archive file is copied to the new SAS 9.2 Server environment to be imported to the new environment. Using the SAS Forecast Server plug-in in SAS 9.2 SAS Management Console, we select Unarchive Project and select the archive file as shown. Figure 8: Project Archive The project is un-archived and imported for use within the new SAS Forecast Studio 3.1 client as shown in Figure 9. Note the version is in a 2.1 format. Figure 9: Register Archive The project will be updated from the 2.1 to 3.1 format when the project is opened as noted in the next figure. Figure 10: Update Project Version 8

9 Once the project is executed and saved, the project is updated to the new version. The next figure shows the newly imported and executed project is converted to the new format. Figure 11: Environment Assignment Additional Forecast Server Administrative macros exist to export more than one archive as well as developed reports. Refer to the SAS Forecast Server migration documentation for additional information. SAS ENTERPRISE GUIDE PROJECTS SAS Enterprise Guide 4.1 projects will be migrated for use within the SAS 9.2 environment. The migration consists of a two part process: migrating the metadata (i.e., SAS Libraries and tables) and migrating the SAS Enterprise Guide project(s). All of the SAS metadata must be migrated before migrating any SAS Enterprise Guide projects. The original project must be converted to SAS Enterprise Guide 4.2 and any references to metadata objects that have been migrated or updated within the new SAS 9.2 environment must be changed. A SAS Enterprise Guide 4.1 project opened in SAS Enterprise Guide 4.2 is automatically converted to the correct format. Note that once opened, you must manually change the references for each changed metadata object in the project. This manual approach was not a viable option for EIA due to the large number of SAS Enterprise Guide projects created by our users. A utility included with SAS Enterprise Guide 4.2 enables groups of projects to be migrated to the correct format while also making any necessary global changes to the metadata references. The Migration Wizard exists in the SAS Enterprise Guide installation directory at C:\Program Files\SAS\EnterpriseGuide\4.2 directory. A key metadata change was the update of the SAS application server logical name from SASMain to SASApp. In some cases, we also modified SAS Library definitions which can also be remapped through the wizard. The SAS administrators obtained a sample group of SAS Enterprise Guide projects and tested the migration of these projects into the new environment. This testing also allowed the option to build a mapping reference file for later use. In addition, the wizard provides the option to create backups of all SAS Enterprise Guide 4.1 projects. Detailed information on the wizard and options can be found in the SAS Enterprise Guide 4.2 administration documentation. SAS STORED PROCESSES A SAS Stored Process is a SAS report/program that is stored centrally on the server. A client application can then execute the report and receive and process the results. Stored processes are like other SAS programs except they have an additional features that enable customization of the program s execution and also allow for invoking an application interface to supply parameters and pass these parameters to the program when invoked. SAS Stored Processes can be exported to a SAS package for import to the SAS 9.2 environment. Option selections during stored process import are similar to the process reviewed migrating a SAS library. The EIA SAS administrators conducted migration tests of production stored processes. All stored process metadata, programs and parameter attributes migrated successfully to our SAS 9.2 environment. Note that dependent metadata (i.e., SAS library and tables) referenced by these programs as well as the physical data sources must be migrated prior to the import and testing a migrated stored process. Some of our applications use the SAS Stored Process Web Application to call stored processes from another system. Starting with SAS 9.2, the default mechanism for a user to login to the SAS Stored Process Web Application is to use the SAS Logon Manager. This is the standard application used by the SAS 9.2 Web products. In our scenario, we will use the _USERNAME and _PASSWORD options as input parameters in the URL. Note that by default the SAS 9.2 Stored Process Web Application does not accept encoded passwords. The AllowEncoded- Password initialization parameter must be set to true to allow encoded passwords. 9

10 The SAS 9.2 BI Web Services provide enhanced web services. The Deploy As Web Service wizard enables a set of stored processes in SAS Management Console to be deployed as a generated Web service. This enhancement allows us to explore new integration options not available in our existing SAS environment, such as integrating with non-sas applications. ON-GOING SAS 9.2 ENVIRONMENT SUPPORT The EIA SAS Administrator manages all metadata content and administrates the hardware and software portions of the environment. Once the SAS 9.2 environment became operational, the following activities provide a summary of on-going support work performed by the SAS Administrators: Creating and scheduling scripts that monitor and clean up the work space on the server Monitoring SAS sessions for orphaned processes and canceling these process to free up resources Administering security for all users who access reporting content As a best practice, the environment should have an established, formal and regularly scheduled backup process. The following SAS components should be included as part of the backup process: SAS metadata repositories and related files The SAS Content Server SAS Table Server Metadata Server backup and restore utilities are provided as part of the installation and can be executed manually. Operating system commands to backup the metadata repositories can be used and are provided with the installation and configuration of the environment. Note that to ensure metadata integrity of all backups, the SAS Content Server and SAS Table Server content must be part of the backup process and have a defined dependency to the metadata repository backup. The SAS Content Server contains web content that is associated with metadata objects including content for the SAS Information Delivery Portal, report definition files and other supporting files for reports. Metadata cannot be used without the associated content that is stored on this server. At the time of our installation, there were no utilities or system commands provided as part of the installation for the SAS Content Server or the SAS Table Server. The SAS administrators needed to build a set of UNIX scripts to backup and restore all of this metadata content as outlined in the SAS documentation for these components. Time should be allocated for the development and testing of these scripts to verify backup and restore processes of metadata work as expected. The SAS 9.2 Intelligence Platform: System Administration Guide outlines the best practices for backup of your environment. CONCLUSION The success of implementing the SAS 9.2 environment within EIA can be attributed to the following factors: SAS9 SECURITY PLANNING AND TESTING Our SAS environment provided a back drop of lessons learned while implementing and maintaining protections and controls within the environment. Improving the security strategy using Access Control Templates along with the SAS Folder redesign provides better management and administration of content while meeting our security business requirements. The newly implemented design was thoroughly tested using a set of test cases and scenarios to ensure all SAS clients honored the correct protections and controls for role based access. The new roles and capabilities within the User Manager plug-in of SAS Management Console provided easier management and additional security and allowing the SAS Administrators the option in the future to hand off key tasks to advanced users once the environment is established and the end user audience matures using the new environment. 10

11 EARLY ADOPTION OF THE SAS ADMINISTRATOR ROLE EIA was quick to assign resources to the SAS Administrator role required to support the SAS and SAS 9.2 environments. The individuals in this role shadowed all aspects of the installation and configuration process. New features now exist for the SAS Administrator that were not available for use in SAS Improvements to SAS Management Console allow for server management and monitoring features for the SAS Administrator in the Server Manager Plug-in. Role based access to capabilities within the SAS Management Console provide the option to deploy this component to advanced users to manage their own metadata content. The BI Manager Plug-in import and export wizards have been replaced with the Export and Import SAS Package Wizards. These wizards have been enhanced to allow for migration of additional metadata objects over the prior release to assist with the promotion process. Enhancements to SAS Forecast Studio and SAS Forecast Server allow for projects to be managed more effectively while providing additional features through a plug-in to SAS Management Console. SAS TO SAS 9.2 MIGRATION PLANNING New product features and enhancements not only exist for the SAS Administrator but also to the end user. As part of the migration planning, the SAS Administrators provided knowledge transfer sessions to the end users of the SAS 9.2 products to educate the users of design changes and new features in all licensed products. In addition, Standard Operating Procedures (SOPS) were developed as part of user documentation to supplement SAS documentation. This user documentation provides information on the new SAS 9.2 Server environment and how to get setup and connected. Additional SOPS will be created based on user feedback as users adopt the new SAS 9.2 server environment. Successful migration practices include testing migration of SAS artifacts (applications, programs etc.) over to the new environment. The SAS Administrators worked with key program units within the organization to identify SAS applications and programs to migrate to the SAS 9.2 environment. Activities included prototyping the migration of existing applications to the new environment and running parallel testing to ensure data validation of content and verifying results were accurate and processes perform to the level of the current business processes. REFERENCES SAS Institute Inc., Migration, (June, 2010). SAS Institute Inc SAS 9.2 Intelligence Platform: Security Administration Guide. Cary, NC: SAS Institute Inc. SAS Institute Inc SAS Forecast Server 3.1: Administrator s Guide. Cary, NC: SAS Institute Inc. ACKNOWLEDGMENTS SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the author at: Michael Bretz U.S. Energy Information Administration 1000 Independence Ave SW Washington, DC Work Phone: Michael.Bretz@eia.gov 11