Data Sheet Denodo Platform 6.0 With the advent of big data and the proliferation of multiple information channels, organizations must store, discover, access and share massive volumes of traditional and new data sources. Data virtualization is a modern approach to data integration that transcends the limitations of traditional techniques. Denodo is the leader in data virtualization, providing agile, high performance data integration and data abstraction across the broadest range of enterprise, cloud, big data and unstructured sources as real-time data services at half the cost of traditional approaches.
What s New in Denodo Platform 6.0? Denodo Platform 6.0 delivers breakthrough performance for big data, logical data warehouses and operational scenarios with the Dynamic Query Optimizer. The Denodo Platform now accelerates data virtualization adoption with cloud support, which enables business users to leverage self-service data discovery and exploration. Dynamic Query Optimizer This feature dynamically determines the lowest-cost query execution plan based on statistics. It is the only data virtualization performance optimization engine in the industry that takes into account all the special characteristics of big data sources (e.g., the number of processing units or partitions), it can handle incremental queries and it enables connectivity to the broadest array of big data sources, including Redshift, Impala and Spark. Denodo Platform for AWS The Denodo Platform 6.0 aims to accelerate the adoption of data virtualization as-a-service with the introduction of Denodo Platform for AWS, another first to the industry which is ready-to-use and available on AWS Marketplace. Denodo Platform for AWS offers elastic infrastructure, flexible options such as rentby-the-hour and a wide range of capacity choices. Self-Service Data Discovery and Exploration The Denodo Platform 6.0 is built to expedite the availability, access and use of data for business users with the self-service data discovery and exploration functionality that comes with google-like search capabilities for data and metadata. It works with all types of data sources, including big data sources, and it offers easy-to-use interfaces for browsing data and metadata as well as viewing data lineage. Users can also create ad-hoc queries using the new Advanced Query Wizard and save them as views in the Denodo Platform Server, and they can now view the graphical representation of entities and relationships. Big Data Connectivity The Denodo Platform offers big data and cloud databases connectivity including Redshift, Apache Spark, HP Vertica and Impala. Unified Security and Governance The Denodo Platform 6.0 offers the following enhanced features to enforce security and governance policies. Kerberos southbound support for databases and Web Services Kerberos pass-through support An API for accessing view dependencies information and data lineage information Resource Management The Denodo Platform 6.0 offers the Resource Manager, which is a new UI based tool for enforcing workload management policies in environments with multiple consuming applications that have varying workloads and usage patterns. DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 2
The Denodo Platform for Data Virtualization Delivers the Following Fundamental Capabilities Universal Connectivity Data Access: The Denodo Platform allows you to connect to any data source including Web, semi-structured and unstructured, in any format or location. The Denodo platform offers high-performance optimized adapters for all the main sources as well as extension points for custom wrappers to unique data sources. Universal Connectivity Data Delivery: The Denodo Platform allows you to publish any combined information as reusable data services in multiple formats (SQL through JDBC and ODBC, SOAP WS, REST WS, XML, JSON, XHTML, etc.) and supported hybrid delivery modes (real-time, cache, batch, etc.) to consuming applications. Optimized Performance: The Denodo Platform is built with features that improve performance, such as the Dynamic Query Optimizer. By always enabling the lowest-cost query execution plan, this feature significantly reduces the amount of data transferred from the data sources through the network and execution time and provides greater benefits in scenarios such as the logical data warehouse scenario. The Optimizer enables the lowest-cost execution plan using multiple optimization features including the following ones that are new to Denodo Platform 6.0: Full aggregation pushdown Partial aggregation pushdown Partition pruning On-the-fly data movement Intelligent Caching: The Denodo Platform offers an advanced caching system that allows both a full-view materialization and query-by-query caching. The cache is configurable on a per-view basis, enabling hybrid query executions that are part real-time and part cached, so the cache can be applied when and where it is needed. The cache system supports manual, scheduled and triggered updates with full or incremental loads, for the flexibility to adapt to any use case. Data Integration: The Denodo Platform flexibly automates web integration processes that are modeled using a library of pre-built templates and components for workflow, navigation and extraction as well as the structuring of Web, semi-structured and unstructured data. Unified Virtual Data Layer: This capability provides an integrated modeling and execution environment that enables users to build powerful transformations and relationships. Because the Unified Virtual Data Layer uses common metadata and semantics, it helps to normalize, transform, improve quality and relate data across heterogeneous source types. The extended relational data model allows disparate data types to be represented natively in the virtual layer, minimizing effort. Advanced Security: The Denodo Platform offers policy-driven row-based and column-based security management features that provide users with controlled access to information. Some of the salient features include role-based security access, schema wide and data-specific permissions, support for security protocols, including single-sign-on using Kerberos in both northbound and southbound connections, and pass-through authentication. Unified Data and Metadata Governance: This feature offers a single, enterprise-wide entry point for data and metadata management, security, audit, logging and monitoring, enabled through built-in tools and instrumentation, which can also integrate with external data management tools. DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 3
Self-Service Data Discovery and Exploration: The Denodo Platform is a feature-rich, single product offering with user-friendly, self-service features that make business-related data accessible and extensively available to business and technology users alike. The Information Self-Service Tool enables business users to easily search and discover data and metadata using a Web interface. It provides browser-based search functionality that enables users to find critical data as well the lineage of the data. Agile Development: The Denodo Platform offers rapid ways to deliver pervasive, self-service data services using graphical, wizard driven UI and discovery tools. The platform hides the complexity to application developers and business users, decouples consuming applications and data sources and enables the easy creation, extension and use of data services. Hybrid Integration Mode: The Denodo Platform supports complex, hybrid integration processes, such as those that integrate ETL within a broader data virtualization approach, to enable the delivery of reliable, high performance virtual data services through the balanced orchestration of real-time, cached, scheduled batch or hybrid execution modes. Denodo Platform Architecture BUSINESS SOLUTIONS Access Informationas-a-Service Analytical DATA CONSUMERS Operational Enterprise Applications, Reporting, BI, Portals, ESB, Mobile, Web, Users Multiple Protocols, Formats Query, Search, Browse Request/Reply, Event Driven Secure Delivery Agile Development DATA VIRTUALIZATION Resource Management DENODO PLATFORM Right Information at the Right Time Performance Cache Hybrid Integration CONNECT Normalized Views of Disparate Data COMBINE Transform, Improve Quality, Integrate PUBLISH Real Time (Right Time) Data Services Governance & Metadata Self-service Information Security SQL, MDX Web Services Big Data APIs Web Automation and Indexing DISPARATE DATA Any Source, any Format More Structured DISPARATE DATA SOURCES Less Structured DOC W Databases & Warehouses, Cloud/Saas Applications, Big Data, NoSQL, Web, XML, Excel,PDF, Word... DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 4
Connect Adapters Point-and-click adapters to enterprise, semi-structured and unstructured data in any format or location. High-performance optimized adapters for all the main sources. Bi-directional read/write. Databases Relational databases such as Oracle, HP Vertica, DB2, Sybase, MS SQL Server, MySQL, PostgreSQL, Informix, MSAccess and Apache Derby, including graphical introspection of tables, views and stored procedures. In-memory relational databases such as SAP HANA and Oracle TimesTen. Parallel databases and appliances such as Teradata, Netezza, Oracle Exadata, Sybase IQ, ParAccel and Greenplum. Big Data/NoSQL Hadoop (Denodo is certified with multiple vendors such as Cloudera, Hortonworks, etc., included Kerberos-secured clusters, and multiple technologies such as Hive, Impala, sequence/maps/key-value/avro files, HDFS, Map/Reduce, HBase), Mongo DB, CouchDB, Neo4J and MarkLogic. Web Automation Sophisticated tools to expose Web, semi-structured and unstructured data as virtual relational data/service. Multi-Dimensional Sources Multidimensional OLAP engines such as SAP BW, MS SQL Server Analysis Services, Mondrian and Essbase. Web Services Flat and Binary Files SOAP / REST Web Services and data feeds, including support for XML, RSS, ATOM, JSON and CSV formats. CSV, pipe-delimited, MS Excel (xls & xlsx), MS Access, XML and JSON. Regular expression-parsed flat files. All files can be locally accessible or in remote filesystems, through FTP/ SFTP/FTPS, and in clear, zipped and/or encrypted format. Active Directory Connect and introspect LDAP and Active Directory services as source data (apart from security access). Cloud, SaaS, Web Sources with Simplified OAuth Security Salesforce, Google, Amazon, LinkedIn, Facebook, Twitter via APIs with simplified OAuth integration (1.0, 1.0a and 2.0); Any website, form, WebApp via browser automation. JMS Queues Consume data from JMS queues by subscribing to the queue. JMS implementations include MQSeries, SonicMQ and ActiveMQ. DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 5
Semantic Repositories Semantic repositories in Triple Stores / RDF accessed through SPARQL endpoints. Mainframe Mainframe/legacy connectivity through third-party adapters. SDK for Custom Extension Unlimited extensibility through a custom connector API that allows users to create connections to other information systems not accessible out-of-the-box. Custom data sources (wrappers), stored procedures, custom functions (both for VDP and ITPilot) and custom policies. Combine Canonical Business Views Extended Relational Model Transform, clean, combine disparate information into virtual canonical business views. Natively represents and seamlessly combines relational, hierarchical, NoSQL and semantic representations of data into abstracted relational model; takes into account source capabilities and constraints for query optimization. Web Integration Automates extraction and integration of less-structured data from Web sites, forms, applications, PDF and MS Word. Automatic Web Navigation: Transparently handles dynamic sites with AJAX, JavaScript, authentication, secure servers, sessions, cookies, popup windows or sequences. Data Extraction: Example-based heuristic extraction of semi-structured content into defined schema from dynamic Web 2.0 content, PDF forms, documents. Search/Index: Built-in tools expose unstructured data as inverted indexes. High Performance: Parallel Execution using MS IE or Denodo browser pools; smart browsers load only optimal navigation sequences to minimize memory usage. Content Integration CMS, file systems, Sharepoint, email servers, knowledge bases, indexes and ontologies. Metadata Driven Integration Graphical wizard-driven UI and tools to introspect siloed data, create unified logical data views, transform, cleanse, group, aggregate, modify output and define data workflow. DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 6
Semantic Integration Transform, relate and merge unstructured data with structured data using text mining, taxonomy filters and semantic tools for textual similarity. Bidirectional Integration Supports read, write and transactions (2-phase commit and XA transactions). Transformation, Quality, Matching Large library of built-in functions for SQL, XSLT, XQuery, XPath, Java and semantic transformations with the ability to plug in external tools or custom functions in every step of the query execution lifecycle. Built-in data workflow tool for complex transformation or data quality processes. Support for arbitrary numeric precision for scenarios that require maximum accuracy. Data Modeling and Model Bridge Data modeling to shape output, flatten or create new hierarchical data structures to match target schema. Support for using elements from other databases through the Model Bridge tool, which enables users to import data models created with modeling tools like ER/Studio, ERwin or PowerDesigner. Contract-first/top-down and bottom-up modeling for greater flexibility, parallel work and change governance. Publish Web Services Fully RESTful interface exposes enterprise data assets as unique URIs accessible via search, browse or query drill-down using standard Web protocols, interfaces and HTTP verbs (GET, POST, PUT, DELETE) Outputs in XHTML, XML or JSON for human and/or machine consumption. Publish SOAP web services that conform to contract-first schema. Supports XSLT tools, WS-security, ESB/JMS access and SOA catalog. Publish services in the Open Data (OData) protocol format. Publish data via JMS (with JSON support), including MQSeries, SonicMQ and Active MQ. Database Views Portal Widgets Delivers virtualized data as SQL views, data services or portlets/widgets to suit every need. Bespoke, optimized SQL views of unified virtual data layer accessed via ODBC, JDBC and ADO.NET. Publish SharePoint WebParts, Java portlets, AJAX widgets to use in major portal/ mashup servers. DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 7
OData Protocol Supports the ability to publish data in the OData standard format so client applications can consume it easily using standard protocols. Semantic Formats Answers SPARQL queries returning RDF (via D2R mappings). Performance Dynamic Query Optimizer Makes automatic optimization decisions on the execution plan that is optimal for each query. Enables the lowest-cost execution plan possible, reduces the amount of data transferred from the data sources through the network and execution time, for a dramatic improvement in performance. Maximizes delegation of operations to the data source. Database Views Full Aggregation Pushdown Supports high-performance sub-queries, access to data sources using native high-performance protocols as well as automatic query rewriting for restructuring SQL sentences into optimal form. Automatically transforms query execution plans to maximize delegation of aggregation to the source level. Partial Aggregation Pushdown Aggregation operations that cannot normally delegated are optimized by splitting them into two steps; then the optimizer is able to push down the most resource intensive step to the source while executing the less taxing step in the virtualization server. Partition Pruning Picks the optimal execution plan where only partitions containing the data relevant to the query are used. On-The-Fly-Data Movement In order for delegation to happen at the source level, a small data set from a data source is temporarily moved to another data source with a larger data set. The decision for data movement is made by the optimizer on its own without the user having to manually specify it. Tunable Optimization Multiple features to manually tune the optimization engine; the user can analyze the behavior of the optimizer using the advanced visual tools for inspecting query execution traces and decide where and when to override the automated decisions. DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 8
Scheduling and Alerts A built-in scheduler, with support for external ETL processes, such as pre-fetching data to cache or materialized source. It also balances source latencies. Intelligent, Multi-Mode Caching Full Materializes the source data into a cached copy with full delegation pushdown and can be incrementally updated. Partial Incremental or Total Refresh Stores the results of the queries on a query-by-query basis to improve the performance of the most frequent queries without having to materialize the entire data set. The cache can be loaded incrementally so only the new rows in the source need to be copied into cache. Event-Based or Scheduled Cache maintenance processes can be executed on a scheduled basis, configured to run when specific events are triggered by the sources, or executed manually. Multi-Mode Caching The cache type can be configured at the view level. Fully cached, partially cached and non-cached views can be combined when necessary. Security Role-Based Access User authentication using external LDAP, Active Directory and/or built-in user directory. Secure and differentiated role-based access control to data services and sources, enterprise tools for management and monitoring dashboards. Single-Sign-On Single sign-on using Kerberos northbound and southbound connections. DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 9
Row/Column- Level Fine- Grained Authorization Support for fine-grained security at virtual data view, column or row level and for masking sensitive data. Integrates with external systems for security, access control and SLA policy management. Pass-Through Supports pass-through authentication to leverage security infrastructure in the data sources. Data Encryption Support for importing and exporting encrypted data; communication between modules can be encrypted and authenticated using SSL also. Firewall Support All components can be distributed in different network segments Resource Management Resource Manager A UI based tool to enforce workload management policies in environments with multiple consuming applications that have varying workloads and usage Dashboard Denodo Dashboard with multiple tools for monitoring of servers, clusters and HA configurations. Server Monitoring Denodo Monitor to view real-time queries and historic audit/logs. Support JMX, SNMP and WS-Management standards to integrate with leading external monitoring tools including HP Openview, IBM Tivoli, Microsoft RM and Nagios. Custom Policies for Workload Management Workload management using custom policies to restrict or constrain service levels and workload based on external factors (source load, network, time). DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 10
Unified Data and Metadata Governance Consistent Data Delivery Delivery of consistent, meaningful data services to users. Metadata Repository Metadata repository with multiple visualization (tree view, linked data, attribute origin, source impact, catalog search, etc.); includes metadata API, model export and introspection from external systems. Metadata Management Discover, introspect and transform source metadata. Refresh or propagate source metadata when it changes. Flatten or create new hierarchical data structures. Controlled Data Service Delivery Data lineage, change impact, dependency analysis, metadata migration tools, version control and granular policy-based tiered security deliver a controlled data virtualization and enterprise data services capability. Self-Service Information Discovery and Search Universal Search Google-like search capabilities for data and metadata. Works on all types of data sources, including big data sources. Data Lineage Intuitive, easy-to-use interfaces for browsing data and metadata and viewing data lineage. Advance Query Wizard Create ad-hoc queries using the Advanced Query Wizard and save them as views in the data virtualization server. Graphical Representation for Relationships and Associations of Data Entities Establishes relationships between virtual data entities using referential constraints (primary keys, foreign keys, varying levels of multiplicity, or conditionals). Associations can be introspected from external metadata/model tools for governance, browsed/traversed by users manually, or programmatically by applications using navigational queries. Data Exploration Sandbox environment for exploring the data before publishing. DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 11
Agile Development User-Friendly Development Environment Easy-to-use, enterprise-class tools. Graphical, wizard-driven UI for all functions and modules. Documented scripting for advanced users. Platform Extensibility Platform extensions enabled via the Eclipse IDE plug-in for developing, testing, debugging and deploying custom functions, connectors and stored procedures. Integration with Version Control Graphical Support for Git, Subversion and TFS Version Control Systems. Support for checkout, commit and update of virtual entities with automatic dependency control directly from within Denodo. Graphical Support for Lifecycle Process Management Graphical support for lifecycle process management (development, staging, production) and/or geographically dispersed environments. VDP Admin Tool UI Enables users to combine elements from several databases, which helps them to manage several open tasks at the same time. Provides docking support, VQL highlighting and auto-complete features. Hybrid Integration Mode Denodo Scheduler Enables complex, hybrid integration processes, integrating ETL within a broader data virtualization approach. Materializes unified data views by exporting to databases, warehouses, flat files, Excel, XML, etc. using the built-in Denodo Scheduler or external ETL tools. Balanced Orchestration Deliver reliable, high performance virtual data services through the balanced orchestration of real-time, cached, scheduled batch or hybrid execution modes. Task Persistence and Support Supports persistent tasks through the continuation of queries after restart and transparent retries in case of failures, intermittent Web access or human intervention. Task Orchestration Task dependencies allow linked tasks to start only when others have finished. DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 12
Denodo Platform Differentiators Broad Connectivity: The Denodo Platform provides universal connectivity and unified access to any type of data source structured or unstructured including big data, IoT and cloud. Operational and Analytical Use Cases: The Denodo Platform can be used for a broad spectrum of industry-specific or horizontal use cases covering both operational and analytical functions. Self-service discovery and search: The Denodo Platform is the only data virtualization software that allows business users to search for data in the data virtualization layer. Superior Performance: The Denodo Platform delivers superior performance through pushdown query optimization, advanced multi-mode caching and workload management. Denodo is the only data virtualization software that can automatically apply sophisticated optimization techniques which are crucial in Big Data scenarios, such as query aggregation, pushdown optimization and on the fly data movement. Benefits Faster Time to Market: A library of high quality data services speeds development on new projects. Unified Ease-of-Use: A platform with unified access, to graphically model all data sources and publish and manage data services easily and intuitively. Scalable Performance: Advanced automatic query optimization with options for manual override control, load balancing and modular clustered scalability. Data Services Reuse: Data is leveraged efficiently across transactional, analytical and informational applications to deliver the full benefits of data virtualization. Denodo Platform Requirements Operating Systems Microsoft Windows (32-bit and 64-bit platforms) Windows XP, Windows Vista, Windows 7, Windows 8, Windows 10 Windows Server 2003, Windows Server 2008, Windows Server 2012 Linux (32-bit and 64-bit platforms) Red Hat Enterprise Linux (RHEL) Oracle Enterprise Linux (OEL) Ubuntu Linux CentOS Linux UNIX (64-bit platforms) Sun Solaris Directory Services LDAP v3 Microsoft Active Directory 2003, 2008 Browser Internet Explorer 10.x, 11.x. Note: The Internet Explorer is only a requirement for using the ITPilot web browsing capabilities. The self-service interface also requires a browser and will work with Firefox and Chrome in addition to IE. DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 13
Deployment Patterns Denodo Platform 6.0 can be deployed in the data center or private cloud (either in physical or virtual servers) or in the public cloud (e.g. Amazon EC2). Strong Web Services support and available automation scripts make Denodo very cloud-friendly. Multiple configurations supported: Basic single server configuration Basic server with proxy configuration for firewalls High availability clusters with load balancer in either active-passive (hot standby) or active-active configurations Clusters with shared cache or distributed local cache Geographically distributed server environments Multiple Denodo instances in peer-to-peer or multi-layered environments Configurations can also be embedded in other product architectures (for example reporting and dashboards, single-view applications, information services delivery platforms, or BPM/Workflow) Visit www.denodo.com Email info@denodo.com twitter.com/denodo NA & APAC (+1) 877 556 2531 EMEA (+44) (0) 20 7869 8053 DACH (+49) (0) 89 203 006 441 Iberia & Latin America (+34) 912 77 58 55 DS-DenodoPlatform6.0-01 Copyright 2016, Denodo Technologies 14