Reference Code: TA001707DBS Publication Date: July 2009 Author: Michael Thompson TECHNOLOGY AUDIT WhereScape RED v6 WhereScape BUTLER GROUP VIEW ABSTRACT WhereScape RED is an Integrated Development Environment (IDE) that supports the entire data warehouse lifecycle. It consists of an integrated workbench that provides a drag-and-drop interface to building a data warehouse, where Wizards are utilised to take the developer through a process for standard functions such as creating a dimension; an integrated data warehouse scheduler with full dependency management; and a metadata repository. Although it could be seen as purely a development tool, WhereScape RED s additional benefit is really seen when change requirements need to be implemented, as the management aspect of the product allows these to be implemented at a much faster rate than with standard development tools/methods. Although WhereScape RED utilises metadata within its own repository for all aspects of working with the target data warehouse, there is no proprietary lock-in to the product itself, and once the target data warehouse has been built then WhereScape RED could be removed and the target data warehouse would be a stand-alone implementation. The target data warehouse can be any of the three leading relational databases plus Teradata. There is no support for non-relational models and no support for Open Source databases. KEY FINDINGS After development the target data warehouse can be stand-alone. Creation of indexes is automatically taken care of for best performance. Speeds up the development process and handles change management. Aimed at the SME market and departmental-level enterprises. Completely intuitive design and development interface. The metadata repository provides impact analysis and documentation. No support for Open Source database or non-relational models. Designed for data warehouse professionals. LOOK AHEAD The recent agreement signed with IBM means that WhereScape will be looking to integrate its offering with a wider range of IBM products. Butler Group. This Technology Audit is a licensed product and is not to be photocopied Page 1
FUNCTIONALITY Product Analysis WhereScape RED is an Integrated Development Environment (IDE) that supports the entire data warehouse lifecycle. The product was born from the fact that data warehouses take too long to build, and once they are built, take too long to change: a factor that has bedevilled both the data warehouse and the Business Intelligence (BI) vendors for many years. WhereScape believes that experienced data warehouse professionals are not well serviced with IDEs that are powerful and extensible enough for them to build realworld, complex data warehouses, and Butler Group would fully support this view. As an IDE enables rapid development it brings into play agile and iterative techniques that, where applied intelligently, can also lead to better solutions. WhereScape RED has three primary elements: IDE An integrated workbench that provides a drag-and-drop interface to building a data warehouse. Wizards are utilised to take the developer through a process for standard functions such as creating a dimension. Options can be selected to tailor the output. One example of this which highlights the power of the solution is whether a dimension is slowly changing or not, and, if it is, what type of slowly changing dimension it will be. Custom options can be added where appropriate, such as joins or lookup criteria. At the end of the development process standard database objects can be created and code or scripts generated. Impact Analysis and Automated Documentation Metadata is automatically created as the IDE is utilised. This is used for impact analysis and for on-demand documentation. Scheduler WhereScape RED includes an integrated data warehouse scheduler with full dependency management. All generated code has calls to the scheduler. To a large extent the viability of a solution such as WhereScape RED depends upon the ease of use set against the functionality of the developed application (in this case a data warehouse). Having an intuitive development environment is of little benefit if the end result is of limited value. Therefore, one has to measure the value to be gained by implementing this solution. It would be too extreme to state that WhereScape RED could be used effectively by developers with little knowledge of data warehouse design, but it is a close-run thing. The GUI is so intuitive and the underlying engine, which handles many of the more mundane but important aspects of data warehouse design, is so powerful that anyone with a reasonable understanding of databases should be able to use the product to good effect very quickly. It is not just at design time that WhereScape RED scores well: its functionality also makes change management much easier and, more importantly, a much more rapid proposition. This really has been the underlying problem with data warehouse implementations in the past. Organisational requirements change at a faster rate than can be matched by the data warehouse/data marts that are used to reflect that change. Overall, WhereScape RED is a solution that helps developers by taking away much of the minutiae of the development effort allowing them to concentrate on building a data warehouse that addresses the reporting, BI, and analytical requirements of the organisation. Butler Group. This Technology Audit is a licensed product and is not to be photocopied Page 2
Product Operation WhereScape RED automates the creation of database objects to implement a data warehouse and data warehouse processing: the Extract, Transform, and Load (ETL) function. WhereScape RED consists primarily of three major components: WhereScape RED Clients an IDE that manages the data warehouse development, management, and operation. The WhereScape RED client manages metadata in the WhereScape RED metadata repository, as well as objects in the target data warehouse database platform. The WhereScape RED clients consist of a WhereScape RED Desktop, a Setup Administrator, and Scheduler Management. Target Data Warehouse the WhereScape RED managed data warehouse contains: The data warehouse data structures (tables, views, indexes). A staging environment for the data warehouse (typically non-persistent tables, views, indexes). Data warehouse processing implemented as auto-generated or custom command scripts and stored procedures. WhereScape RED metadata repository (tables, views, stored procedures). Scheduler a utility that executes scheduled workflow jobs and tasks, and handles alerting that has been defined in the WhereScape RED metadata repository using WhereScape RED. Optionally WhereScape RED can manage Microsoft On-Line Analytical Processing (OLAP) cubes and file exports from the data warehouse. Figure 1: WhereScape RED High-Level Architecture Source: WhereScape D A T A M O N I T O R Butler Group. This Technology Audit is a licensed product and is not to be photocopied Page 3
WhereScape RED implements the storage, querying and processing within a target relational database environment. WhereScape RED detects the database platform being used as the data warehouse and will implement specific features of that platform to improve scalability and performance of the resulting solution. The question of scalability directly relates to scalability of the database environment used as the target data warehouse. WhereScape RED supports the following target databases: Microsoft SQL Server. Oracle. DB2 for Linux, UNIX, and Windows. Teradata. WhereScape RED has traditionally been used to build dimensional data warehouses or data marts, and many data warehouses managed by WhereScape RED scale to more than a TB and several to more than 10TB. The Teradata version supports normalised data warehouses, and this functionality has recently been added for the other supported databases. With the WhereScape RED Desktop connections are defined to data sources and to the WhereScape RED Repository. The associated data sources can be any data that is accessible natively by the data warehouse database, flat files, XML or via ODBC and can reside on any server accessible to the WhereScape RED Desktop (for development) and the Target Data Warehouse (for data loading). The WhereScape RED Desktop is installed on a developer s PC, the Setup Administrator resides on an administrator s PC and, if required, Scheduler Management clients can be installed on an operator s PC. All the WhereScape RED clients can reside on the same PC. The WhereScape RED Repository resides in the Target Data Warehouse database, and consists of a series of open database tables containing metadata. There is one WhereScape RED Repository installed for each environment development, test, production, etc. Users should note that there is nothing specifically built in to WhereScape RED to support Disaster Recovery (DR). The entire data warehouse and processing built and managed by WhereScape RED utilises native database functionality, so the native DR capabilities of the target database platform or the storage system can easily be used to manage DR scenarios. Apart from the technology aspects of WhereScape RED, the company also brings to the engagement a detailed methodology (Pragmatic Data Warehousing Methodology). Although this has been created based upon many years of data warehouse design knowledge it is not prescriptive in any way and implementing organisations can use their own design techniques and development standards should they so wish. Although the methodology could be considered as an add-on, there is one aspect of it that Butler Group sees as being highly relevant to any good data warehouse development, and that is the focus on data warehouse implementation as a process not a project. Butler Group. This Technology Audit is a licensed product and is not to be photocopied Page 4
Product Emphasis Designed for data warehouse development professionals, WhereScape RED has been designed from the need to speed up the development process. Initially used by WhereScape professionals themselves in data warehouse engagements, the product has now become a full-featured IDE which is aimed at the SME market and the departmental-level enterprise. DEPLOYMENT WhereScape RED is a developer tool that is licensed per data warehouse and per developer. WhereScape RED can be used by in-house staff or external consultants to deliver data warehouse projects more rapidly than by native development. The prerequisite skills for getting the most out of WhereScape RED are data warehouse modelling skills and experience, and database skills. WhereScape RED itself can be installed in 30 minutes. A typical data warehouse project built using WhereScape RED would take between one and three months, using between one and five developers. The actual time is naturally dependent upon the scale of the project, but will in almost every use-case scenario be faster than traditional data warehouse development. WhereScape RED is ideally suited to developing data warehouses using a modular approach as a dimensional or bus architecture data warehouse is by its nature modular. WhereScape RED provides many functions for supporting modular delivery including incremental deployment, metadata projects, and modular documentation. Data warehousing by its nature is an ongoing process any organisation whose business does not evolve will run into problems and the data warehouse must be able to provide relevant reporting and analysis to business users. Ongoing, the data warehouse needs an operational team to manage day-to-day operation of automated tasks, and a project team that enhances existing data warehouse areas or delivers new ones. These teams can be in-house or external staff depending on organisations preferences. As WhereScape RED is intended to be used by data warehouse developers, there is no basic data warehouse development course available, but WhereScape provides a two-day introduction to WhereScape RED (delivered in a workshop environment). Additionally, a number of advanced courses are available and these can be tailored to individual customer requirements. Ongoing support is provided via e-mail, phone, and the WhereScape Forum. The client components of WhereScape RED are installed on a Windows workstation (Windows 2000 upwards), and the product can be deployed on: Windows Servers. UNIX. Linux. There are no dependencies on third-party software with the exception of the target data warehouse database platform. Butler Group. This Technology Audit is a licensed product and is not to be photocopied Page 5
Existing data warehouses can be imported into WhereScape RED to provide a management environment of that data warehouse. Existing database structures can be read into WhereScape RED and managed in the metadata through an ODBC connection. PRODUCT STRATEGY WhereScape competes with three classes of tools lower-end tools that look to simplify data warehousing, traditional ETL tools, and data warehouse automated development toolsets. Simple tools There are a plethora of tools that look to dumb down or simplify the data warehouse build process. These commonly provide black box solutions for creating cubes or data marts and are often sold as part of a software stack. Traditional ETL Traditional ETL tools focus on data movement and are infrastructure rather than project oriented. Data Warehouse Automated Development Toolsets Although there are similarities between this area of the market and WhereScape RED, the latter is differentiated in a couple of ways. Firstly, unlike some other solutions of this type, WhereScape RED is not model based and this ensures that there is no dependency between the development tool and the data warehouse. After development it is possible to remove WhereScape RED from the implementation and the data warehouse would still exist as an independent entity. Secondly, WhereScape RED fully supports standard normalised databases for data warehouses and does not introduce a new concept or create a tie-in to any proprietary data model. As WhereScape RED builds and manages data warehouses there is no specific vertical market focus; this is a horizontal offering and provides equivalent benefits to any organisation that is experiencing the time-based disconnect between business requirements and data warehouse functionality. ROI is achieved through improvements in developer productivity, reduced licensing costs (over existing or alternative solutions) and lower operational costs. WhereScape RED provides an enterprise solution for Small and Medium-sized organisations and a departmental solution for large organisations, and WhereScape focuses in these areas rather than attempting to play at the top end of the market. WhereScape RED is sold directly, via a reseller network and is being embedded by other software companies into their business intelligence offerings with the Powered by WhereScape branding, and the key customer in this space is CDC Software whose offerings include Pivotal and Ross. WhereScape has also recently signed a strategic Independent Software Vendor (ISV) deal with IBM. The other key technology partnerships are Microsoft, Oracle, and Teradata. Following the IBM agreement there are plans to further integrate with other IBM products and offerings. WhereScape RED licensing is perpetual and based on a per data warehouse and a per developer model. An annual maintenance fee of 20% of purchase price provides access to e-mail support, support forums, fixes, and new versions. Butler Group. This Technology Audit is a licensed product and is not to be photocopied Page 6
COMPANY PROFILE WhereScape is a privately owned company headquartered in Portland, Oregon in the USA, with offices in the UK and New Zealand. It currently has a user base of over 300 customers. WhereScape s background is in data warehouse development. WhereScape Software Limited was incorporated in 2001. WhereScape RED version 1 was released a year later to build Oracle data warehouses. WhereScape RED was ported to SQL Server then to Teradata and DB2. Some key clients and usage include: Vodafone a data mart implementation that lets staff access key financial and KPI reports from their desktops. ehealth an online health insurance provider moved from a manual reporting infrastructure to a fully documented automatic environment in three months. Clearwater Seafoods a prototype self-service BI reporting infrastructure was built in 30 days and runs independently of the IT department. ASB Bank a leading New Zealand bank implemented a reporting environment with WhereScape RED. SUMMARY Building data warehouses/data marts can be a complex and time-consuming undertaking, and the use of these can be limited by the change requirements of an organisation, with speed of organisational change not being matched by the technical change implementation. With WhereScape RED, organisations are not only able to build data warehouses faster, but they can manage the change requirement with less involvement of IT, thus helping to speed up the process. Although WhereScape RED is designed for data warehouse design specialists, Butler Group found on reviewing the solution especially the highly intuitive GUI that most people with an understanding of database principles (as the solution uses standard relational model databases as the target data warehouse) would be equally well served by the product. Much of the inherent complexity of data warehouse design is effectively hidden from the developer, who can concentrate on providing the schemas required. One aspect of this that we found especially useful was the way in which relevant indexes were created without a requirement to understand the performance impact that creating the indexes would have, as the required indexes were created automatically. Butler Group. This Technology Audit is a licensed product and is not to be photocopied Page 7
Table 1: Contact Details WhereScape USA Inc. 2100 NW 133 rd Place Suite 76 Portland OR 97229 USA Tel: +1 (503) 466 3979 Fax: +1 (503) 466 3978 www.wherescape.com WhereScape Albany House 14 Shute End Wokingham Berks, RG40 1BJ UK Tel: +44 (0)118 9144509 Fax: +44 (0)118 9144508 Source: WhereScape D A T A M O N I T O R Headquarters Shirethorn House, 37/43 Prospect Street, Kingston upon Hull, HU2 8PX, UK Tel: +44 (0)1482 586149 Fax: +44 (0)1482 323577 For Butler Gr Butler Direct Pty Ltd. Level 46, Citigroup Building, 2 Park Street, Sydney, NSW, 2000, Australia Tel: + 61 (02) 8705 6960 Fax: + 61 (02) 8705 6961 Butler Group 245 Fifth Avenue, 4th Floor, New York, NY 10016, USA Tel: +1 212 652 5302 Fax: +1 212 202 4684 more information on Butler Group s Subscription Services please contact one of the local offices above. Important Notice This report contains data and information upto-date and correct to the best of our knowledge at the time of preparation. The data and information comes from a variety of sources outside our direct control, therefore Butler Direct Limited cannot give any guarantees relating to the content of this report. Ultimate responsibility for all interpretations of, and use of, data, information and commentary in this report remains with you. Butler Direct Limited will not be liable for any interpretations or decisions made by you. oup. This Technology Audit is a licensed product and is not to be photocopied Page 8