Decision Modeling for Dashboard Projects

Transcription

1 Decision Modeling for Dashboard Projects How to Build a Decision Requirements Model that Drives Successful Dashboard Projects Gagan Saxena VP Consulting Decision modeling provides a formal framework to design dashboards that link explicit decision making to the knowledge required for them. CONTENTS An Introduction to Dashboards Why Decision Modeling Building Decision Requirements Models Dashboard Methodology More Information Dashboards are decision support systems but their design does not usually consider decisions. Most designs are driven by user interface and information visualization requirements and by the underlying data models. A dashboard design based on decision modeling provides a logical structure for going beyond information presentation to include predictions and recommendations so users can progress from what happened? to what can happen?, and on to what action can be taken? Decision modeling is an ideal technique for capturing business requirements for a dashboard, and then for driving the design, implementation and maintenance processes. Decisions are captured and maintained in context of goals, metrics, business processes, trigger events, systems and stakeholders. This ensures a comprehensive, future-proof dashboard design - and as a side-effect enables formal Performance Management, Governance, Compliance, Escalations and Training. Decision models enable dashboard implementation by tying data requirements and knowledge requirements to presentation elements. These sources are defined in context of stakeholders and systems so that governance and control requirements are simultaneously obtained. The graphical nature of decision models allows visual partitioning to drive phase-wise implementation, as well as a robust and logical workbreakdown-structure for formal project management 2015 Decision Management Solutions 1

2 An Introduction to Dashboards The missing Decision in Decision Support Systems Business Intelligence dashboards and reports are known as 'Decision Support Systems' since they support business decisions made by business users. The irony is that the requirements and design of these systems does not consider decisions explicitly. Implementation is heavily focused on user interface requirements, and therefore on individual or a team's implicit decision models. No wonder most projects and maintenance teams are constantly chasing the next set of requirements from the next set of users in a never-ending game of whack-a-mole. Dashboards are key elements for human decision making - and their implementation needs to be based on decision models - not just on information models and process models. Most repeatable business decisions can be standardized and need not depend on individual preferences. In fact, one of the key goals of most organizations is to be able to make consistent decisions aligned with strategic metrics - for governance, risk-management and compliance with regulations. Dashboard Maturity Dashboards can range in sophistication from Informational to Predictive to Prescriptive. Figure 1: From Information to Recommendations All decision-making progresses through the same steps: gathering relevant information, analyzing causes, predicting possible future outcomes and then selecting 2015 Decision Management Solutions 2

3 the most appropriate action from a defined set of choices. The perfect dashboard would be one that can support decision making through these steps. Presenting information is relatively easy since the data can be piped in from core systems without too much additional work. Some basic extrapolation techniques can also be applied to get a sense of trends. A very large proportion of dashboards belong in this Informational category. Figure 2: Dashboard Maturity Stages Advanced analytic models are now increasingly being generated with the help of abundant data, cheap computing power and friendly workbenches. These models uncover a number of patterns that can be beneficially applied to regular repeatable decision-making. While most analytical models today are being treated as one-off discoveries that need to be acted upon with a one-off response, these models are also being embedded within some advanced dashboards as a means of running regular business operations. These Predictive Dashboards add a substantial level of knowledge to decision-making. For example, not only are the Forecast Sales figures presented for all markets and products, but they are qualified with a percentage score indicating probabilities of achieving those figures. Finally, for most repeatable decisions there are a defined set of action choices possible. The core of decision making revolves around picking the best choice given all that is known at that time. Apart from the raw information and predictive models, decisions need to be made within the constraints of business rules. These are policies, procedures, government regulations, best practices and even all the learning from previous actions. Ideally all these diverse requirements and constraints 2015 Decision Management Solutions 3

4 should be applied to rank possible actions in a preference order, and even filter out the actions flagged as illegal by one or more of the rules. The Prescriptive Dashboard thus presents a ranked order of actions to the decision maker - based on all knowledge available at that time, derived analytically or stated explicitly. Dashboards for Decisions vs. Data Visualization for Data Mining Dashboards are popular decision support aids because they are designed to be visually appealing. Decision makers need not pore through figures on a spreadsheet but can use color-coded instrumentation on the dashboard to quickly focus on relevant pieces of information - highlighted in red or the pointer on a gauge falling below safe levels, for example. Good graphic design, visual display techniques and user interface design are all important for designing a great dashboard - but all of that is not the same as Data Visualization. Figure 3: Data Visualization is for Pattern Discovery Data Visualization is a knowledge 'discovery' tool that presents detailed data in a variety of visualizations for users to discover underlying patterns in the data. The science and art of 'Data Visualization' is critical and has its place in knowledge discovery (data-mining), very similar to analytical models being built out of the same data using mathematics. While people are great at discovering patterns in visual representations, machines need to apply math for finding the patterns. In the end both approaches result in insights and knowledge that can be applied to making better decisions. Data Visualization is thus not the same as a Dashboard Decision Management Solutions 4

5 In fact true dashboards are quite often rendered useless by placing all possible data charts on them. This cluttered dashboard is a result of not enough time spent in identifying the decisions that need to be supported by a finite set of information. Instead the so-called dashboards are jam-packed with a variety of colorful charts and dials for the decision maker to go find whatever they need to make their decisions. This might generate the occasional flash of insight but is completely useless in formalizing decisions and getting better at them as an organization. The Dashboard has to be tightly designed for focused decision support, while Data Visualization tools needs to be flexible and leave the user in charge of undirected discovery and exploration. They are different tools meant for different stages of the decision making process. Dashboard Design is not the same as Information Design Classic dashboard design considers a formal Information Architecture to organize and structure information sources and then uses principles of User Interface Design to ensure usability. Both of these are important techniques and required for effective design, but they come into play only after Decision Requirements have been established using decision models. Since Dashboards are intended to support decisions, it would seem that a catalog of decisions needs to be created first so that appropriate dashboards can be designed. Decision modeling establishes the goals and requirements for the dashboard (Figure 4). These are set up in business language and allow business users to define requirements. Figure 4: Business Requirements and Design for Dashboards The Information Architecture organizes all information sources used to power the dashboard. A Knowledge Architecture brings knowledge sources like analytic models and business rules to the dashboard. And finally, best practices from User Interface Design are used to design the presentation of the Dashboard Decision Management Solutions 5

6 Decisions Require Knowledge and Technology The decision making process evaluates a variety of formal and informal knowledge sources in trying to pick the 'best' action. The knowledge sources can be a very specific set of concrete data, or at the other extreme could just be based on human expertise. An ideal dashboard should be able to harness the wide variety of knowledge as effectively as possible in order to support decision making. Figure 5: Knowledge Sources and Technology Capturing various forms of data has been easier than trying to leverage knowledge derived from Advanced Analytics or from Automated Business Rules. These last two have traditionally required specialized technologies, toolsets and skills - and were therefore not as accessible directly to decision makers for repeatable decisions. Some of these challenges have been overcome and now these knowledge sources need to be considered holistically in the design of Dashboards. Advanced Analytics uses mathematics to find hidden patterns and insights in all accessible data. This useful knowledge placed on the dashboard in context with other factors will make the resulting decision that much more effective. If the long term predicted profitability of a channel is high with a high confidence score, a business may decide to invest despite low current sales volumes. Automated Business Rules provide a mechanism to inject business policies, regulatory guidelines, best practices and organizational learning into the Dashboard. These rules are the safety-net or the guide-rails that ensure that various threshold values on the Dashboard are set according to all known rules today - and allow rapid changes to these settings as the environment changes and as the organization learns better ways to manage Decision Management Solutions 6

7 Figure 6: Limitations of not using all Knowledge Sources The challenge has shifted from accessibility to consumption of these advanced technologies. Advanced technologies are accessible today to capture knowledge and even to keep it refreshed as new data and new rules show up. Since these specialized technologies have their own specific notation and implementation, a common format needs to be used that brings the output of various technologies into a common language that can be consumed by a dashboard. The common format and basic vocabulary is provided by decision models that organize information and knowledge according to decisions that need to be made. Decision models are the common, actionable format for representing knowledge residing in diverse technologies for Analytical Models, Automated Business Rules and Optimization Algorithms Decision Management Solutions 7

8 Why Decision Modeling Decision Models connect Knowledge to Decisions Decisions models (Figure 7) are graphical representations showing how a decision is composed of other sub-decisions along with the information and knowledge required to make an effective decision. The notation is based on the Decision Model Notation (DMN) Standard from Object Management Group (OMG). Information Sources are generally data sources and Knowledge Sources could be well-defined Analytical Models, Business Rules and Algorithms or placeholders for human expertise and similar informal knowledge sources. Each element on the Decision Requirements Model can be further described with other relevant characteristics that elaborate on the context (Figure 8). The goal is to articulate all components of decision making without getting distracted with technology or implementation details. These high level requirements will be used later to scope out technology choices, implementation phases and operating specifications related to Governance and Compliance. Figure 7: A Decision Requirements Model Focusing dashboard design on information requirements rather than decision requirements makes the entire process very unsatisfactory. The dashboard becomes cluttered because there is no logical filter available to define relevancy and all possible information is squeezed on to the so-called dashboard. The users have not thought about their decisions and therefore have no formal way to define information needed for a particular decision. So the default choice is, "Give me all the information and I will likely use it all for decisions I need to make (at some later point in time.)" 2015 Decision Management Solutions 8

9 Figure 8: Context for a Decision The Context for Decisions Part of the reason Decisions are difficult to describe formally is that they depend on a wide variety of other contextual factors like Goals, Metrics, Business Processes and Trigger Events. Numerous Stakeholders and Systems are involved either directly or indirectly. In an ideal situation the decision model will simultaneously keep track of all the contextual elements allowing organizations to consider decision making from multiple perspectives. Not only do Decision Contexts make decision-making more transparent, they establish the clear connection between decision-making and other organization elements allowing process rationalization, goals clarification and metrics refinement, for example. Performance Management The explicit definition of a decision pins down the exact criteria that would qualify a decision as a good decision versus a bad decision. Defining a decision precisely requires that a question and a set of allowed answers are laid out. These answers could be simple yes/no or a pick from a predefined list or within a range of values. This level of clarity can then easily be extended into a set of success criteria for that decision to track effectiveness. This effectiveness criterion is a crucial input to the design of a Decision Dashboard and is used to design visual and automated cues for guided decision making Decision Management Solutions 9

10 Governance and Compliance As part of Decision Discovery all decisions stakeholders are identified. Decision stakeholders for a decision are the organization units that make that decision, define decision criteria or are affected by that decision. Identifying these stakeholders ensures that governance and compliance requirements are addressed according to organizational guidelines. An inordinate investment is needed in building so-called personalization options into the dashboard because organizational decision-making has not been described anywhere. Ideally, common elements of standardized decision-making should form the core, stable components of the dashboard with flexible presentation and visualization capabilities sprinkled in. Common elements of the dashboard will drive standardized decision making in the organization - a desirable goal in most organizations. Escalations and Decision Authorizations A side effect of establishing the decision-stakeholder relationship is that new team members have a very convenient mechanism for escalating decisions. Who needs to make what decision and when is all codified in the decision model, and this mechanism has been agreed upon as well as visible across the organization. This facility is very useful not only for new team members but also serves as a refresher for all team members. Training and Help-Wizards Decision models describe how to make decisions - both in terms of the input knowledge needed and in terms of the output choices for the decision maker. Ideally the decision models are extended with organizational context like goals, metrics, processes, events, stakeholders and systems. Users of dashboards based on decision models can readily access the rich set of knowledge built into the decision model via a click-through Help/Information button. Well built decision models are a self-documenting feature that power the Help/Information button on dashboards, allowing very contextual and visual support Decision Management Solutions 10

11 Building Decision Requirements Models Decision modeling has four steps that are performed iteratively: 1. Identify Decisions. Identify the decisions that are the focus of the project. 2. Describe Decisions. Describe the decisions and document how improving these decisions will impact the business objectives and metrics of the business. 3. Specify Decision Requirements. Move beyond simple descriptions of decisions to begin to specify detailed decision requirements. Specify the information and knowledge required to make the decisions and combine into a Decision Requirements Diagram. 4. Decompose and Refine the Model. Refine the requirements for these decisions using the precise yet easy to understand graphical notation of Decision Requirements Diagrams. Identify additional decisions that need to be described and specified. This process repeats until the Figure 9: Iterative Decision Modeling Cycle decisions are completely specified and everyone has a clear sense of how the decisions will be made. At this point you can generate a requirements document, packaging up the decision-making requirements you have identified. This can act as the specification for business rules implementation work or for the development of predictive analytics. Alternatively you can extend the model with decision logic, such as decision tables, to create an executable specification of your decision-making. This process is described in more detail in Decision modeling with DMN, a free white paper available from Decision Management Solutions available at Decision Management Solutions 11

12 Dashboard Methodology Scoping and Requirements Defining a clear scope for a Dashboard Project is a challenge because the functionality cannot be described adequately in the standard systems development vocabulary of processes (and use cases). There are no processes associated with a dashboard except for some workflow or collaboration in some cases. In some cases, an attempt is made to scope a Dashboard in terms of processes or use cases resulting in dissatisfaction all around. A more appropriate scoping is done in terms of information. Since the dashboard is supposed to display information for a particular user group, the scope for Dashboard projects ends up being defined in terms of either all the databases and data warehouses that need to be 'displayed', or in terms of the user groups (departments) that will be using that particular dashboard. These approaches end up with a loosely defined scope but all the details are completely up to the project team and the dashboard owners. A good working relationship will deliver a somewhat useful dashboard, but in other cases there will be unending disputes over scope with nothing getting delivered. Decisions are the building blocks for dashboards. decision models are graphical networks that show the relationship among various decisions, sub-decisions, information sources and knowledge sources. Each decision is also described in terms of the stakeholders, functions, events, metrics, etc. This results in a big-picture view that can be used to define Project Scope by partitioning decision models into logical groupings based on functional and technical dependencies. Figure 10: Drawing Automation or Implementation Boundaries The logical partitions of the decision model result in a set of candidate implementations. These implementation sets can then be sequenced for an 2015 Decision Management Solutions 12

13 implementation roadmap based on relative business priorities and available resources. Using decision models as a scoping tool, Dashboard Projects can be tightly defined and managed through a phase-wise implementation road-map. Project Management Using decision modeling to establish the scope of a Dashboard Project not only results in a concrete definition of all elements but the process of decision modeling involves all stakeholders to achieve consensus. This discussion among stakeholders revolves around formalizing how decisions are made in the organization in business terms with hardly any technology being mentioned at that stage. While this discussion is a discovery process and needs to be managed, business owners rapidly converge to a common representation of the decision model. Figure 11: Project Management Considerations An agreed upon project scope, functional and technical - along with a clear definition of stakeholders - is a wonderful beginning for the project manager. Based on this, project manager can build out estimates, work-breakdown-structures, resourceplans and schedules. Quality, Risk and Communications Management Plans are also defined alongside the overall Work-Breakdown-Structure (WBS) Decision Management Solutions 13

14 The decision model based scope allows project management to be based on concrete elements (Figure 15) instead of relying solely on goodwill among the project team. Overall Architecture The technical elements of a Dashboard Project generally involve three main areas - presentation layer, data layer and a decision management technologies layer. The user interface for dashboards is the presentation layer based on different web, mobile or other specialized technologies. The data layer integrates transactional data, aggregated data and external data into the dashboard, usually passing through numerous cleansing and transformation stages. Finally, the decision management technologies layer includes advanced analytics and business rules capabilities generally deployed as easily consumed web services. Figure 12: Overall Generic Architecture The presentation layer is intended to give users as much flexibility as possible. The focus is on presenting information and knowledge relevant to explicit decision making defined in decision models. Interesting but irrelevant data should be excluded to avoid clutter and distraction. The data layer includes all business intelligence infrastructure and direct access to transactional and external data sources. Substantial effort is requires here to ensure data quality through a series of harmonization and transformation steps. It is important to have this effort be directed by project requirements from decision models that specify the type, quality and frequency needed for decision making. This 2015 Decision Management Solutions 14

15 saves costly efforts in trying to make all the data as complete and clean as possible when only a subset is required for the Dashboard. The decision management technologies layer brings knowledge sources to the dashboard as specified in decision model. Advanced Analytical models and Business Rules are the two main categories of knowledge applied to dashboards, although Optimization Algorithms can also be deployed. Analytical Models and Business Rules are generally created and maintained through dedicated work benches according to decision requirements. The output is generated as a deployable web service that can be plugged into the presentation layer as needed. The workbenches make it easy to keep the knowledge sources updated and a variety of real-time update mechanisms are available to keep the dashboard current. Design and Development Dashboards need to be designed with user interface design and visualization best practices but the core architecture has to be driven by the Decisions that need to be supported by the dashboard. The layout and screen flow is based on logical partitioning of decision models (Figure 13). The wireframes are then fine-tuned on the basis of Decision Characteristics captured during the decision modeling exercise. These characteristics include roles, processes, goals, trigger-events and update frequency. All these design dimensions have a direct bearing on the usefulness of the dashboard and ensure that the design is decision-centric. Figure 13: Layout Structure Driven by Decision Models 2015 Decision Management Solutions 15

16 When dashboard design is driven by decision requirements, a finer level of contextual requirements related to time and place can also be accommodated. Certain decisions can be tagged as mobile-only for example. This would allow specialized display design without the need to come up with a generic design to be adapted to desktop as well as mobile interfaces. Similarly, certain decisions are required only at certain times and can therefore be designed into the interface such that these decisions show up only when needed. Defining decisions explicitly allows a design that is optimized for the media, channel and time when the decision needs to be made. Maintenance Dashboards are decision support tools and need to evolve as decisions get improved. In an ideal situation organizations learn from the decisions it makes and applies this knowledge to subsequent decisions in a continuous improvement process. Dashboards should be able to step up to support these more sophisticated decisions. This closed loop feedback is easier to manage when decision models have been used to build the dashboard, showing the direct relationship between decisions and the information and knowledge sources that drive them. As the dashboard gets used for decisions the user may find that a decision can be better supported by an additional piece of knowledge, or that a particular display on the dashboard is not being used for a decision. Decision models are an easy means to verify if additions or deletions to a dashboard will enhance decision making. Dashboards display relevant information and knowledge to support decision making. Maintaining the dashboard involves activities related to adding and removing information and knowledge. Since the same source may be used in multiple decisions, a decision models is a very useful map to analyze the impact of any changes being considered for the information sources (databases) or the knowledge sources (analytical models and business rules). Decision-centric dashboards can be upgraded rapidly based on new analytic knowledge and new business rules. Thus, decision models help contain the cost and risk of changes by allowing a robust and transparent impact analysis Decision Management Solutions 16

17 More Information DecisionsFirst Modeler DecisionsFirst Modeler is a social, collaborative platform for Decision Requirements Models available from Decision Management Solutions. DecisionsFirst Modeler is available as a free Basic Version (SaaS) and a paid Enterprise Edition (SaaS or onpremise). For more information, or to sign up for the free Basic Version, go to Figure 14: DecisionsFirst Modeler Decision Modeling Decision Management Solutions provides consulting and training in Decision modeling as well as the use of DecisionsFirst Modeler. Contact Us If you have any questions about Decision Management Solutions or would like to discuss engaging us we would love to hear from you. works best but feel free to use any of the methods below. info@decisionmanagementsolutions.com Phone : Web : Decision Management Solutions 17