Quest for a Business Rules Management Environment (BRME) in the Internal Revenue Service

Business Rules and Requirements Management Internal Revenue Service Business Rules and Requirements Management Office (BRRM) Quest for a Business Rules Management Environment (BRME) in the Internal Revenue Service Debbie Ware Debbie.S.Ware@irs.gov 0 0

in the Internal Revenue Service The intent of this presentation is to share the requirements and approach used by the Internal Revenue Service to select tools to support its current business rules management environment and not to endorse any particular products 1 1

in the Internal Revenue Service Background The Internal Revenue Service has over 500 systems listed in its As-Built Architecture Traditional methodologies used by the IRS to develop these systems have been varied and their effectiveness and efficiency A high rate of errors occur in systems development because the methods used to document the problem and propose solutions do not communicate consistent and unambiguous specifications As experienced in private industry as well as in government This leads to systems that do not meet business expectations, and that overrun forecasts of schedule and cost It costs far more to fix requirements errors downstream, after programming the solution -- e.g., ~ 5X cost compared to getting the requirements right the first time These are well-researched and documented facts, documented by leading institutions such as Harvard, MIT, SEI, etc., for example... 2 2

CIO Magazine 11-15-2005 Fixing the Requirements Mess Analysts report that as many as 71 percent of software projects that fail do so because of poor requirements management, making it the single biggest reason for project failure bigger than bad technology, missed deadlines or change management fiascoes. [ ] It's up to the CIO to establish an overall requirements process that works and to support it with the political skills necessary to get buy-in from both the business and development sides None of this is easy. Business users often: Don't know exactly what they want Can't prioritize what they do want Request things IT simply can't deliver (because of complexity or cost) Or can't describe their desires in terms that translate accurately into code On the IT side, analysts, architects and coders regularly: try too hard to please don't set realistic expectations for projects don't use every means possible to guarantee that what they're building is what the user really needs sometimes they even fail to make sure that they're talking to all the right stakeholders In short, the traditional practice of requirements is broken 3 3

in the Internal Revenue Service.. Historical Overview In 1999, the IRS initiated the Business Systems Modernization Effort with the intent to replace many of the major IRS business infrastructure systems The Customer Account Data Engine (CADE)* is one of the immensely complex IRS modernization efforts designed to replace its existing master file system. Once completed, CADE will post and settle all taxpayer account activity The original design for CADE included a business rules engine In 2003, SEI was asked to review the CADE project and perform a health check on the proposed use of a business rules engine SEI reported skepticism around predicted business rules coding efficiencies through use of an engine and IRS ability to harvest rules quickly enough (over 20,000 pages of tax code) They also reported that the harvesting, adjudicating and managing of business rules needed to be done regardless of how the rules are to be technically implemented. * (The CADE Project has made presentations at previous Business Rules Forums) 4 4

in the Internal Revenue Service.. Methodology Development With that as the backdrop, the IRS with its Prime contractor spent the next 2 years establishing a business rules and requirements management methodology, prototyped the methodology*, and established a business rules and requirements management office During this same period, the IRS s Prime contractor published The Business Rules Enterprise Management (BREM) Business Rule Tool Acquisition Guidance Report containing the initial set of rules management requirements The IRS used the findings in this report as the basis for a firm fixed price solicitation for an integrated, comprehensive business rules management solution *(The Submission and Settlement Harvesting Project (SSHP) presented at the Business Rules Forum in 2005, was one of the IRS s first implementations of the new methodology) 5 5

in the Internal Revenue Service.. Requirements The IRS had over 400 requirements in the following general areas: Administration API/Interface Architecture Business Process Modeling Change Control Diagramming & Mapping Enterprise Capabilities Environment, Platform, Capacity Fact Modeling/Vocabulary Management Metadata Extensibility Metadata Integration Reporting Rule Analysis Rule Structuring Rule Simulation Security Workflow Management 6 6

in the Internal Revenue Service.. Requirements Technical Administration The ability to perform initial tool installation, day to day administration jobs like backup, restore of projects and managing of the BRME environment Install/Uninstall the Tool Creation of the enterprise repository Seed project work areas Backup/Restore repository Administrative reporting 7 7

in the Internal Revenue Service.. Requirements Technical API Interface The ability to provide access to the data maintained within the tool suite through import and export facilities Provide an API Use standard formats such as XML, SMI, CSV Interface with other standard tools such as MS Office or Rational Suite Use Specific programming languages such as Java, C++, or MS Visual Basic 8 8

in the Internal Revenue Service.. Requirements Technical Architecture & Environment/Platform/Capacity The ability for the product to fit into IRS s standard platforms with minimal extension, configuration, and customization The ability to be scalable and deliver specific performance 9 9

in the Internal Revenue Service.. Requirements Technical/Functional Security The ability to ensure that only properly entitled users and systems have the ability to access and modify the repository s objects Define roles and responsibilities Assign roles to users Designate groups that have access to sensitive rules Check sensitive rules accessibility Check for security enforcement Audit log of security administrator activities 10 10

in the Internal Revenue Service.. Requirements Technical/Functional Meta data Extensibility & Metadata Integration The ability to extend the tool suite s meta model to support new object types, object properties and relationships between object types & The ability to integrate metadata from multiple external sources, ensuring that all change control, versioning and security policies are enforced. The ability to specify the mapping between meta model of the repository and external tools 11 11

in the Internal Revenue Service.. Requirements Functional Enterprise Capabilities The ability to support enterprise-level requirements including Maintain an enterprise repository Establish projects with see-thru capability to the enterprise repository Promote approved project-level objects into the enterprise repository Change Control The ability to assess the impact of a potential business rule change Keep track and report on changes at the most granular level Version individual objects, as well as groups of objects Compare and report on the differences between those versions 12 12

in the Internal Revenue Service.. Requirements Functional Business Process Modeling, Fact Modeling, Diagramming, and Mapping The ability to diagram business processes, document them, and relate them to the business rules that control those processes. The ability to develop a fact model (vocabulary) of terms and business concepts and the relationships between those business concepts using business terminology. The ability to represent that vocabulary through fact model diagrams Development and modification of the overall business process Simulation of the throughput of business events through the process Association of rules, to the business processes, to terms, to the technical objects that support them across the application portfolio Diagramming and Standards support Versioning of business processes 13 13

in the Internal Revenue Service.. Requirements Functional Reporting The ability to perform ad-hoc queries or develop and run pre-defined reports, outputting the results in various formats (e.g. to the tool s browser, printer, web publishing). The ability to create project deliverables relating to business rules Workflow The ability to enforce a governance process. The ability to notify the appropriate stakeholders when significant business rule management events occur (e.g. business concept or term is added or changed, deliverable is ready for review) 14 14

in the Internal Revenue Service.. Requirements Functional Rule Authoring & Structuring The ability to develop business rule statements that use a vocabulary and enforce standard grammar Expressed in natural business language Re-use of terms Enforce grammar Intelligent prompts to build the structure Rule Simulation & Analysis The ability to assess the impact of a potential business rule change Identify inconsistent rules Generate dependency diagrams Develop test scenarios Simulate rule sets Compare results Generate flow of decisions Generate scenarios and identify the rules that fired and the resulting output values 15 15

in the Internal Revenue Service Administration Security Administration Business Rule Repository Metadata Integration API/Interface Metadata Extensibility Business to Technical Mapping Adhoc Query Documentation Generation Web Publishing Reporting Business Process Modeling Authoring, Modeling and Analysis Business Rule Structuring Diagramming Simulation Mining Business Business Rule Rule Business Engine Engine Rule Engine Database Management System Governance Enforcement Change Control Impact Analysis Audit Life Cycle Workflow Application Application Application Code Code Code Technical Environment 16 16

in the Internal Revenue Service.. Solicitation Our original solicitation process took 1 year and was unsuccessful There were no tools on the market meeting all of the IRS requirements The type of solicitation put too much risk on the vendor to demonstrate a high cost completed solution Vendors did not respond with a complete integrated solution The IRS had to establish a new approach to find tools to support it s new business rules management organization and began a market survey to find the best of breed tools across this broad spectrum of capabilities The market survey was conducted using a two phase approach 17 17

in the Internal Revenue Service.. Survey and Demonstration Phase 1 The IRS began a time-boxed survey of the industry to identify candidate products. Candidate vendors participated in an intensive one day session where they presented their products. A review of products already in use in the IRS environment was done as well. Candidates were included in the list of products considered. Through this survey process we learned that the market was divided into major capability segments and in order to narrow our focus we needed to specifically define the immediate problem to be solved and focus on products meeting those needs 18 18

in the Internal Revenue Service.. Approach Phase 1 Defining the problem Technical Solution v Business Solution Enterprise Solution v Project Solution Based on the mission of the Business Rules and Requirements Management Office, and the maturity of their immediate needs and capabilities, the IRS chose to search for tools that would best meet Business needs of rule authoring, simulation and analysis Project needs for rule management, traceability, meta data integration and extensibility The products surveyed were objectively scored against each IRS requirement category. The categories were also weighted to highlight products that addressed a specific problem. This narrowed the focus for phase 2 evaluation to those products scoring the highest for meeting IRS s specific needs 19 19

in the Internal Revenue Service.. Scoring Criteria Phase 1 Scoring Criteria Score Value Score Explanation 4 Out Of The Box The product provides facilities that support the architecture category 3 Extensions/Configurations Required The product's extensibility or configuration facilities must be used to meet the architecture category. 2 3rd Party Tool Required The product does not support the architecture category, but a 3 rd party product is available that the product may integrate with to support the requirements. 1 Custom Code Must Be Developed Application code might be written using the product's API to support the architecture category. 0 Can't Support The product can not support the architecture category. 20 20

in the Internal Revenue Service.. Weighting Phase 1 Weighting Criteria Solution Weight Explanation Best of Breed: Enterprise 2 Category is required to support the enterprise perspective. 1 Category is required to support project level business rule object management. 0 Not applicable to repository management. Best of Breed: Business Rule Authoring 1 Category is required to support business rule authoring, analysis and simulation. 0 Not applicable to business rule authoring, analysis and simulation. Single Product 1 All categories are required. 21 21

Overall Ratings Product 1 Product 2 Product 3 Product 4 Administration Enterprise Capabilities API/Interface Architecture Business Process Modeling Change Control Diagramming Facility Environment, Platform, Capacity Mapping Metadata (Extensibility) Metadata Integration Fact Modeling Rule Analysis Reporting Rule Structuring Security Rule Simulation Training Usability Workflow Management Enterprise Perspective Weight Product 1 Product 2 Product 3 Product 4 Single Product Solution Administration 1 Enterprise Capabilities 2 API/Interface 1 Administration 1 0 0 0 0 Architecture 1 Enterprise Capabilities 1 0 Rule 0 Authoring 0 0 Business Process Modeling 0 API/Interface 1 0 0 0 0 Change Control 1 Architecture 1 0 0 0 0 Diagramming Facility 0 Business Process Modeling 1 0 0 0 0 Administration 0 0 0 0 Environment, Platform, 0 Change Control 1 0 0 0 0 Enterprise Capabilities 0 0 0 0 Capacity Diagramming Facility 1 API/Interface 0 0 0 0 1 0 0 0 Mapping 1 Environment, Platform, 0 Architecture 0 0 0 0 0 0 0 0 Metadata (Extensibility) 2 Capacity Business Process Modeling 0 0 0 0 Metadata Integration 2 Mapping 1 Change 0 Control 0 0 0 1 0 0 0 Fact Modeling 0 Metadata (Extensibility) 1 Diagramming 0 0 Facility 0 0 0 0 0 0 Metadata Integration 2 Environment, 0 0 Platform, 0 0 0 0 0 0 Rule Analysis 0 Fact Modeling 1 Capacity 0 0 0 0 Reporting 1 Rule Analysis 1 Mapping 0 0 0 0 0 0 0 0 Rule Structuring 0 Reporting 1 Metadata 0 0(Extensibility) 0 0 1 0 0 0 Security 1 0 0 0 0 Metadata Integration 0 0 0 0 Rule Structuring 1 0 0 0 0 Rule Simulation 0 Fact Modeling 1 0 0 0 Security 1 0 0 0 0 Training 0 Rule Analysis 1 0 0 0 Rule Simulation 1 0 0 0 0 Usability 0 Reporting 0 0 0 0 Training 0 0 0 0 0 Workflow Management 1 Rule Structuring 1 0 0 0 Usability 0 Security 0 0 0 0 1 0 0 0 0 0 0 0 Workflow Management 1 Rule 0 Simulation 0 0 0 1 0 0 0 Training 0 0 0 0 0 0 0 0 Usability 0 0 0 0 Workflow Management 0 0 0 0 0 0 0 Weight Product 1 Product 2 Product 3 Product 4 Weight Product 1 Product 2 Product 3 22 22

in the Internal Revenue Service.. Phase 1 Findings The business rules management market remains immature and evolving. There were no tools on the market that met all of the IRS requirements. Certain products evolved from the scoring and weighting as candidates for a "best of breed" solution to be researched further in phase 2 In a "best of breed solution multiple products are considered, each supporting a subset of the architecture categories. Together, the products represent a suite that provides wider (but not complete) coverage of the requirements. Certain products evolved from the scoring and weighting as candidates for a single product solution to be understood further in phase 2 these products represent a single products that provide a wide (but not complete) coverage of the requirements 23 23

in the Internal Revenue Service.. Phase 2 Major Architecture Components & Capabilities for Phase 2 Category Usage Functionality Repository Modeling Business Rule Authoring Central metadata management facility that serves as the data hub for the modeling and authoring environment. All metadata is integrated and made available through the repository. Used to develop the diagrams that support the business process, fact and business rule models. Much of the metadata captured about business concepts, terms, processes and business rules is created and maintained through these diagrams. Use in validating a project's rules. The authoring environment is used to develop the business rule statements using a formal grammar, conducting the consistency analysis of those business rules, identifying the dependencies between rules and simulating rulesets. Metadata Extensibility Metadata Integration and API/Interfaces Administration Security Change Management Reporting Workflow Management Business Process Fact Modeling Diagramming Extensibility of Diagram Types Business Rule Structuring Analysis Simulation 24 24

in the Internal Revenue Service.. Approach Focus of the Phase 2 analysis was on the following architecture categories: Security Change Management and Versioning Metadata Integration Business Rule Authoring from a Business Perspective, including Term and Fact Modeling Support Business Rule Analysis and Simulation Copies of the products were acquired for hands on testing using a fictitious case study Scoring and Weighting were applied a second time Score Value Score 3 Met Majority Of Requirements 2 Met Minimum Requirements 1 Insufficient For Our Needs 0 Can't Support Phase 2 Scoring Scheme 25 25

in the Internal Revenue Service.. Scoring & Weighting Phase 2 Repository and Modelling Product 1 Product 2 Product 3 Product 4 Rule Authoring Product 1 Product 2 Product 3 Product 4 Weight Total Score 0 0 0 0 0 0 0 0 Administration 1 0 Enterprise Capabilities 1 0 API/Interface 2 2 Architecture 1 1 Business Process 2 0 Modeling Change Control 2 1 Diagramming Facility 2 0 Environment, Platform, 1 0 Capacity Mapping 2 0 Metadata (Extensibility) 2 2 Weight Metadata Integration 2 0 Fact Modeling 2 2 Rule Analysis 1 2 Reporting 2 1 Rule Structuring 0 1 Security 2 1 Rule Simulation 0 2 Usability 1 2 Workflow Management 1 0 26 26

in the Internal Revenue Service The IRS selected and acquired two tools to meet the majority of their current requirements We are utilizing these non-integrated tools in a project environment to support rule authoring and rule management We are establishing a prototype technical environment to test the integration and flow of data between each tool and with other tools in our development environment We are refining the workflow process and roles for our business rule and requirements management environment 27 27