White Paper

Transcription

1 What s your story? White Paper

2 Agile Requirements Epics and Themes help get you Started The Task List The Story Basic Story Structure One More Chapter to the Story Use the Story Structure to Define Tasks in WhereScape The Example The Task List Operational/Functional Stories Tracking Agility in WhereScape RED Start Delivering Introducing WhereScape RED

3 What s your story? Turn User Stories into Working Data Warehouses with WhereScape RED Agile Requirements Feature-led or story-led development is a proven Agile method for efficient data warehouse delivery. With this method, developers rely less on an exhaustive specification of requirements, and more on close collaboration with actual users to get a flexible, intuitive understanding of how to create business value through the data warehouse project. WhereScape RED is ideally suited for story-led development. Among the most valuable features are: Automation: Coding of the routine components of the data warehouse is automated by WhereScape RED, so developers can spend more time working with users and refining the final product. If less time is spent on error-handling code, documentation, and the operational framework, then more time can be spent working with users to refine business logic and presentation. Speed: DW development with WhereScape is so quick that the database design and transformation logic can be refined through rapid iterations and user input rather than detailed specifications. Tables can be redefined and reloaded in minutes so that a variety of approaches can be tried out quickly. Quality: Agile methods emphasize quality and supportability throughout the development process. WhereScape RED helps deliver quality product by automating documentation, source data tracking, coding and naming standards, and many other important features as the data warehouse is developed. The core of agile development is the user story. Rather than calling out each specific data element and report options for a data warehouse, the user story describes a user solving a real-world business problem with the perfect software ready at hand. The user story then serves as a guide as developers plan the project and develop the system. The project team breaks down the user stories to determine the development tasks that will guide the project. This document is not intended to be an exhaustive guide to gathering and working with user stories, but rather to give practical examples of how to use user stories with WhereScape RED. More comprehensive information can be found in User Stories Applied: For Agile Software Development by Mike Cohn and Agile Business Intelligence by Ken Collier. This document concentrates on the anlayze or dimensional layer of the data warehouse as this is very connected to user requirements. This dimensional layer should be supported by a normalized layer where appropriate. Epics and Themes help get you Started Here is a quick review of the basic Agile requirements vocabulary: Story: A brief description of one scenario of how one user would use the BI system. These will be discussed in more detail in the next section. Theme: A group of related stories

4 Epic: A broad statement of goals or needs to be solved by the BI system. While Stories drive the development tasks, the Epics and Themes that emerge during user sessions can help developers get started. Epics often point toward the data set that will be needed to support a broad category of stories. Themes often describe a fact table or aggregate that will be able to support a group of stories. The first step is to group the stories, epics and themes, so we can see the commonalities: Epic (Source System) I want to get a handle on sales activity. A quick analysis of these statements indicates that required data will come from the order management source system, that a bookings fact table and a ranked aggregate table will need to be created, and that, on the reporting side, users expect an ed top 10 report, a dashboard with associated data-linked items, and (at least) a pie chart for visualization. The Task List Theme (fact table) Who are the top ten customers by booked orders? Story (report or other visualizatoon) I want an with the top 10 My desktop dashboard should list the top ten. A pie chart would show us where to focus within the top 10. Not much more analysis is required to lay out some initial tasks for data warehouse development. Developers can get started on these while the project team continues to gather and refine user stories. Epic tasks (source system): Set up a sales activity project in WhereScape RED. Determine the source systems for sales data. Create a connection to the sales data system (or systems). Build load tables for order-related data. Theme tasks (fact table): Create orders fact table. Create status dimension. Confirm that booked orders can be identified with order status. Create customer rank aggregate by order value. With very little time or effort, there are 8 tasks on the board that will be valuable as more user stories are uncovered, and others refined. A WhereScape developer with a basic level of experience won t need much more direction than this to get started. Developers new to the product may require an additional level of more detailed tasks. The Story In Agile data warehousing and business intelligence projects, the user stories are limited to a familiar scope. DW/BI user stories usually talk about existing data and how it will be displayed or analyzed, so properly structured stories can be easily broken down into tasks for the WhereScape RED developers. Basic Story Structure The standard Agile story has the structure: As a [role] I want [something] so that [business benefit]

5 For a BI project, we can take advantage of the standard basic BI architecture to enhance the story structure to give just a little more value to the team: As a [role] I want [data] as [visualization] so that [business benefit]. This version of the story helps developers focus on important questions in BI development: which data does the user want to see and how would they like to see it? For example: As Sales Manager, I want to see daily bookings in an in order to keep an eye on sales performance. The amended format helps us break down the required development tasks. Obviously, the visualization points to the report format that the user has in mind (at the moment, anyway). This acts as a starting point for the front-end development team. The data component of our story, which in the example is daily bookings, provides a direction for the data warehouse team. Clearly, at least one fact table with bookings, and a dimensional constraint that will deliver the current, or previous, day s bookings, will be needed. One More Chapter to the Story A great deal of value can be added to the story by the addition of a single sentence to the format: As a [role] I want [data] as [visualization] so that [business benefit]. I define [data] as [description]. This extra sentence in the user story gives the development team a head start in finding the correct transactions to put into the fact table and determining what, if any, business rules need to be applied. In our example: As Sales Manager, I want to see daily bookings in an in order to keep an eye on sales performance. I define daily bookings as all orders from both regions that have completed or pending status. This extra sentence in the user story gives the development team a head start in finding the correct transactions to put into the fact table and determining what, if any, business rules need to be applied. This refinement makes it clear that there is more to this story than previously understood. The two regions may imply two different source systems. Productive, and while it will assist inexperienced people and teams they will not be as productive as more skilled people. Use the Story Structure to Define Tasks in WhereScape Given our standard story format: As [role] I want to see [data] as [visualization] in order to achieve [business benefit]. I define [data] as [description]. A brief analysis defines several useful tasks that we can quickly deliver. [role] describes who the user is who will ultimately sign off on this story. I [data] generally speaking, every story requires at least one fact table. This part of the story provides a quick description of the fact table needed for this story. [visualization] describes the report or other method for visual display of the data. This serves as high-level guidance to the front end developer. [business benefit] this is the core of the acceptance criteria. [description] this description provides an additional head start in tracking down the data for this story. The description indicates which source system the user expects the data to come from, which transactions will be put into fact tables, and - 3 -

6 may also have a starting point for business rules that are required. The Example As Sales Manager, I want to see daily bookings in an in order to keep an eye on sales performance. I define daily bookings as all orders from both regions that have completed or pending status. [role] Sales Manager. [data] Daily Bookings. [visualization] list report delivered via . [business benefit] keeping an eye on sales performance. How will the Sales Manager verify that this really is helping? [description] Orders from both regions with completed or pending status. The Task List An initial list of tasks can be developed from this breakdown for team members, including the end user, the front end developer and the WhereScape RED developer. Develop with a validation strategy for this story (user). Investigate and set up source connections for both sales regions (RED). Create load tables for all order data (RED). Create status dimension for orders (RED). Check the current day attribute in the RED date dimension to make sure it will work with this story (RED). Create the orders fact table (RED). Create nightly load jobs for orders (RED). Develop a detail report showing daily bookings (Front end). Develop an delivery job for delivering the daily bookings report (Front End). Operational/Functional Stories Some user stories gathered will involve the operational or functional aspects of the data warehouse. While it s important that the data warehouse development effort is focused as much as possible on the end value to the business, user stories can be a useful way to gather up operational requirements as well. For example: As the data warehouse administrator, I want the data warehouse to load all tables every night in the correct order so that data quality and integrity is maintained. As the operations manager, I need to be notified if any load jobs have failed and why so that fix any issues quickly and maintain good system availability. As the operations manager, I need to be able to restart the load process quickly and easily and not spend hours figuring out what to do to recover from a failure. I need this to maintain good system availability and data quality. As the data steward, I need my validation scripts to run after each fact table loads so that I can assure the users that the data is correct. It is critical to collect these stories, in addition to more conventional user stories. WhereScape RED automates data warehouse operations, so most of these requirements are already met. The only tasks required will be to confirm that WhereScape RED has been correctly configured and that any required parameters have been set

7 For the story: As the data warehouse administrator, I want the data warehouse to load all tables every night in the correct order so that data quality and integrity is maintained. The tasks are: Create a scheduler load job with all objects grouped by type. Confirm that load, dimension, stage, fact, aggregates, and cubes are loaded in the correct order. Run a test load to confirm. For the story: As the operations manager, I need to be notified if any load jobs have failed and why so that fix any issues quickly and maintain good system availability. The tasks are: 1. Ask the operations manager which addresses should be notified for nightly job success and failure. 2. Follow WhereScape instructions for writing the notification command line for success and failure. 3. Paste the failure command lines into all jobs. 4. Paste the success command lines into the first and last jobs. 5. Run a test with all jobs successful. 6. Run a test with a failed job. Tracking Agility in WhereScape RED WhereScape RED s flexible and comprehensive metadata structure is useful for tracking progress on the story queue. The properties panel below shows how we can record the Epics, Themes, and Stories that are supported by our FACT_ORDERS fact table. A simple metadata report can then show which user story elements have been covered by the development effort so far. Start Delivering The task lists from the previous sections are sufficient for an experienced development team using WhereScape RED and any of the standard front-end tools to complete the first development cycle and get a first version of the story in front of the user within a few days. Some tasks that might be needed in non- WhereScape projects are not required when using WhereScape RED. The ETL code generated by WhereScape RED automatically includes: User documentation Technical documentation Logic for both initial and incremental loads Error handling Integration into the operational framework If these tasks, important to any production data warehouse, do make their way onto the project list, they will be quickly dispatched by the WhereScape RED development team

8 Introducing WhereScape RED Everyone understands the competitive advantage a business intelligence environment can bring the problem is they take too much time to build before they deliver. That s where WhereScape RED comes in. It provides a complete methodology as well as a development and management framework all while leveraging industry standard database technology. WhereScape RED enables the building of fully functional data warehouses in days or weeks, saving weeks or months of consulting time and money. This means you can get user feedback faster, and improve the quality of the warehouse implementation, as well as save time and money. For more information on WhereScape RED please visit - 6 -