Evaluation Checklist Data Warehouse Automation

Transcription

1 Evaluation Checklist Data Warehouse Automation March 2016

2 General Principles Requirement Question Ajilius Response Primary Deliverable Is the primary deliverable of the project a data warehouse, or is end user visualisation and analytics more important? At Ajilius we believe that a project with a data warehouse as its primary deliverable is a failure waiting to happen. Business value comes from analysis, not from data waiting for a use case. Time to deliver Can the project stakeholders wait for a data warehouse to be completed before getting results, or do they need reports and data now? Ajilius conforms to the dimensional modelling methodology, which is renowned for providing data for analysis faster than any other approach. You can test this criterion by asking the vendor of each product to show how you query their warehouse in the visualisation tool of your choice. Frequency of change Do the stakeholders have time to wait while new data requirements are analysed, modelled and work their way through the EDW backlog, or do they need results now? Dimensional modelling enables new star schemas and cubes to be built quickly to solve analytics problems at the speed of Business, not IT. A good test of this criterion is to ask the vendor to demonstrate, live and without notice, the integration of a new data source into their target warehouse. Page 2 / 16

3 Dimensional Modelling Requirement Questions Ajilius Responses Load Tables Is a special class of staging table defined for the loading of source data into the DBMS for further processing? Yes. We refer to this class as a Load Table. Staging Tables Are staging tables supported for the transformation of data, ready to be loaded into dimension and fact tables? Yes. Persistent Staging Tables Can the data in staging tables be persisted across load schedules? Yes, at the choice of the data warehouse developer Master Staging Tables Can persistent master tables be maintained within the data warehouse staging environment? Yes. Ajilius supports the creation of persistent staging tables with insert, update and delete functionality. Types of Staging Table Is there any built-in intelligence associated with specific types of staging table? Yes. Ajilius supports five types of staging table. Transactional tables are sets of data with one row for each source row, and may contain joins across tables. Union tables represent the union of two or more predecessor tables. Aggregate tables contain fewer rows than the predecessor table, and may contain typical aggregate Page 3 / 16

4 values such as sum, count and average. Deduplicate tables remove duplicate rows by selected column values. Unpivot tables convert data from multiple columns to multiple rows. Dimensions What types of dimension table have built-in intelligence supported by the data warehouse automation product? Ajilius generates data structures and code to support Type 0 (date and time) dimensions, Type 1 (overwrite), Type 2 (add row) and Type 3 (add column) dimensions; as well as combinations of these. Fact Tables What types of fact table have built-in intelligence supported by the data warehouse automation product? Ajilius generates data structures and code to support transactional fact tables, periodic snapshots and accumulating snapshots. Data Screens Does the data warehouse automation product have built-in intelligence to support data screens? Yes. Refer to the section on Data Quality. Page 4 / 16

5 Integration Requirement Questions Ajilius Responses Data Warehouse Platform List the target data warehouse platforms supported by the product. Ajilius supports the following target data warehouse platforms in Version 2.1: PostgreSQL EnterpriseDB Greenplum Exasol Amazon Redshift Snowflake Elastic Data Warehouse Microsoft Azure SQL Data Warehouse Data Sources List the sources from which data may be loaded by the product. Ajilius supports the following data sources in V2.1: PostgreSQL EnterpriseDB Microsoft SQL Server ODBC Oracle Sybase MySQL MariaDB AWS Aurora Hadoop SQLite Microsoft Excel Delimited text files XML JSON Additional data sources may be supported through an ODBC driver if available for Page 5 / 16

6 the platform. We will validate new data sources on request. Additional Data Sources Does the product support an interface such as ODBC or JDBC for integration with data sources for which native drivers are not provided? Yes. Ajilius supports a generic ODBC interface, through DSN connections, to additional data sources. Credentials How are database credentials secured in the metadata database? Ajilius uses a combination of SHA256 hashing and AES-256 encryption to ensure database credentials for data sources and the data warehouse are secure from prying eyes. Job Scheduler Does the data warehouse automation product integrate with enterprise job schedulers, such as <insert name here>? Yes. Ajilius generates scripts and batch files appropriate for the target platform. These can be invoked by any known Job Scheduler. We will validate job scheduler integration on request. ETL Tool Can the data warehouse automation product integrate with ETL tools such as <insert name here>? Yes. The scripts generated by Ajilius can typically be integrated with job execution components which are common to most ETL tools. We will validate ETL tool integration on request. Page 6 / 16

7 Existing Data Warehouse Can the data warehouse automation product be integrated into an existing data warehouse? Yes, assuming that the data warehouse platform is one of those supported by Ajilius. Our Virtual Load capability makes it easy to re-use data from an existing warehouse, without having to re-load and re-process it before use. Stored Procedure Dependency Does the data warehouse automation product have a dependency on stored procedures in the target platform? No. Ajilius is designed to be a cross-platform data warehouse product, and some of our supported platforms do not contain a stored procedure language. Beware of products that only generate stored procedures, as you may be locked into a target platform. Data Warehouse Migration Can the data warehouse automation product migrate a data warehouse from one platform to another? Yes, this is a key feature of Ajilius. A data warehouse may be migrated from one platform to another say, SQL Server to PostgreSQL with a simple three-click migration process. If a vendor claims to be able to migrate between data warehouse platforms, make sure that you have this capability demonstrated during the evaluation process. It may be a very complex, manual process in some products. Page 7 / 16

8 Data Quality Requirement Questions Ajilius Responses In-built Data Quality Does the product have in-built support for data quality screens? Yes. Validation Capability Is data validated by type? Yes, validation is provided for all Ajilius data types. Is data validated by range? Yes, both discreet and continuous ranges may be specified for all data types. Is regex validation supported? Yes, regex validation is fully supported through the Python regular expression library. Is lookup validation supported? Not yet. Lookup validation against dimension tables will be delivered in Version 2.3. No 3 rd. party dependencies Does data quality depend on the integration of any third party software? No. All data quality operations are fully integrated into Ajilius. No additional cost Does data quality incur any additional licensing fees? No. Data quality has been implemented as a standard feature of the Ajilius data warehouse automation platform. Page 8 / 16

9 Usability Requirement Questions Ajilius Responses User Languages Which languages are supported for developers of the data warehouse? Ajilius currently supports English, Malay, Indonesian, and Vietnamese. Adding new languages takes 2-3 days of professional translation. We will undertake this work on request, feel free to request it before a demonstration. User Interface What technology delivers the user interface of the application? Like most modern computer systems, Ajilius is delivered through a web browser. We fully support current versions of Internet Explorer, Firefox, Chrome and Safari. We will test other browsers and/or versions on request. Desktop products lock you into legacy architectures that are costly to deliver and support. Device Support What devices can be used to operate all features of the application? Thanks to our responsive application design within the browser, every end-user feature of Ajilius is supported on desktop, tablet and smartphone devices. If a vendor claims that their application can be delivered on a tablet (or smaller) device, make sure that you test all dialogs for fit on the screen, and that the application UI can be scaled to make it usable at Page 9 / 16

10 high resolutions on small devices. Touch Friendly Does the application be driven entirely by touch on tablet and smartphone devices? Ajilius is fully touch compliant. All operations can be conducted on a tablet or smartphone, without needing an external keyboard connection. If a vendor claims touch compliance, make sure that their application is not just a full screen display squashed into a small tablet screen, and that it has plenty of room around controls for finger operation. Screen Scaling Can the user interface scale to support a variety of screen sizes and resolutions? Can the user interface scale to be more readable by vision impaired users? Ajilius supports browser 'zoom' controls, which typically enable the user to make fonts larger and more readable. If a vendor claims to be able to zoom their display, make sure that all user interface elements scale well, including icons and images, and all text fields. Screen Customisation Can the colour, font and size of user interface elements be customised by the user? Ajilius uses the Bootstrap model for browser interfaces, and all user interface elements can be customised through the Bootstrap CSS elements. To evaluate this criterion, ask the vendor to customise specific colour or control elements while you watch. Page 10 / 16

11 Infrastructure Requirement Questions Ajilius Responses Application Dependencies What additional software products must be licensed in order to run the data warehouse automation software? Ajilius has no third-party software dependencies. Make sure that each vendor accounts for the versions of operating systems that may need to be purchased, application framework dependencies (ie,.net 3.5), database management systems that may be required to host their metadata, and additional drivers that may be necessary to connect to your data sources. Server Operating Systems On which server operating systems is the data warehouse automation product supported? Ajilius is fully supported on all versions of Windows, Linux and OSX that are in mainstream support from their respective providers. Beware of products that do not offer a server-based installation, these are typically priced on an expensive peruser basis. Client Operating Systems From which client operating systems may the data warehouse automation product be accessed? We fully support Ajilius on all versions of Windows, Linux and OSX that are in mainstream support from their respective providers. Other operating systems will be validated on request. Page 11 / 16

12 There is no dependency between the client operating system and the server operating system. An OSX client, for example, may access a Windows server. Web Browsers Which web browsers may be used to access the data warehouse automation product? Ajilius is fully supported on current versions of Chrome, Firefox, Internet Explorer and Safari on desktop devices. Ajilius is fully supported on Chrome, Dolphin and Safari browsers on mobile devices. Other browsers and platforms will be validated on request. Metadata Repository Which DBMS is used to store the metadata managed by the data warehouse automation product? Ajilius uses the SQLite3 DBMS to hold all application control data and metadata. This DBMS requires no licence fee. Beware of the potential cost of products that require their metadata to be stored in commercial DBMS platforms. Metadata Location Is the metadata stored independently from the data warehouse? Ajilius stores its metadata in a separate repository for each data warehouse, and that repository is held separately from the warehouse. Products which store their metadata in the warehouse make it difficult to migrate across data warehouse platforms, enforcing a lock-in to your current architecture. Page 12 / 16

13 Constraints Requirement Questions Ajilius Responses Maximum number of tables What is the maximum number of tables that may be supported in a data warehouse? Ajilius supports the maximum number of tables that are supported for the target DBMS. Maximum number of columns What is the maximum number of columns that may be supported in a table? Ajilius supports the maximum number of columns that are supported for the target DBMS. Maximum identifier length What is the maximum length of identifiers such as table names and column names? Ajilius sets a maximum identifier length of 63, to support identical migration across supported data warehouse platforms. Reserved word detection Does the data warehouse automation product support the detection of reserved words in identifier names? How are reserved words resolved? Ajilius holds an internal dictionary of all reserved words for all supported data warehouse platforms. When defining a new object, names are checked against this list. During manual maintenance, such as creating a new table name, the reserved word will trigger an error and must be corrected by the user. During automatic updates, such as importing metadata from a data source, the reserved word will be flagged with the prefix 'reserved_', Page 13 / 16

14 and can later be modified by the user. Duplicate identifier detection Does the data warehouse automation product support the detection of duplicate table and column names? How are duplicate names resolved? Ajilius prevents the creation of objects with duplicate names. Attempting to manually create a table or columns with the same name as one already defined in metadata will trigger an error, and must be corrected by the user. During automatic updates, such as merging the metadata of two or more tables during a join, duplicate column names will have an incrementing suffix appended to keep them unique, which may later be updated by the user. Page 14 / 16

15 Costs Requirement Questions Ajilius Responses Public Price List Does the vendor publish their price list? Ajilius' pricing is directly linked from the front page of our web site. We ve got nothing to hide. Cost Per User What is the licence cost per user? What are the break-points at which licence costs per user are reduced? Ajilius is not licensed per user, it is only licensed per site. Cost Per Site What is the licence cost for a site licence? What is the definition of a site? A site licence costs USD 10,000 per site per year. The definition of a site is an organisation sharing a common domain. Cost Per Warehouse What is the licence cost per data warehouse implementation? Ajilius users can create as many data warehouses as they like, with no additional licence obligation. Cost Per DBMS What is the licence cost per source DBMS? What is the licence cost per target DBMS? Ajilius imposes no licence cost per data source, regardless of whether the data source is running on a free or commercial platform. Ajilius imposes no licence cost per target DBMS, other than the applicable site licence fee. Page 15 / 16

16 Further reducing the DBMS cost of data warehousing, Ajilius supports a number of low-cost and open-source target data warehouse platforms. Additional Unit Costs Are there any additional cost constraints that may require additional licensing? Ajilius has no additional licensing costs. Annual Maintenance Is there an annual maintenance fee? Is the annual maintenance fee mandatory? Ajilius does not have any annual maintenance charge, but see below for our annual renewal policy. Annual Renewal Must annual licences be renewed? No. If you choose not to renew your licence, you must discontinue the use of Ajilius, but may continue to use the scripts which have previously been generated. Page 16 / 16