A Fully Standards-Based Approach to Logging High-Throughput Distributed Real-Time Data. Leveraging the DDS and SQL Standards
|
|
- Noah Little
- 8 years ago
- Views:
Transcription
1 A Fully Standards-Based Approach to Logging High-Throughput Distributed Real-Time Data Leveraging the and SQL Standards Mark A. Hamilton And Edwin de Jong Real-Time Innovations 3975 Freedom Circle Santa Clara, CA (408) Abstract The ability to capture run-time activity for future analysis and playback is vital to being able to debug, integrate, test, and support complex distributed systems. Unfortunately, implementing a logging capability is often very difficult and expensive because data must be captured from disparate sources, likely consists of many different native data structures, and often has a higher aggregate throughput (burst or steady-state) than can be supported by traditional commercial databases or even hard disks in some cases. As a result, distributed application developers traditionally have had to implement their own custom logging capabilities and tools to go with them. By integrating widely-used standards and high-performance database technology, a fully standards-based and off-the-shelf approach to logging high-throughput distributed real-time data is now possible. This solution uses the Object Management Group (OMG) Data Distribution Service () standard to distribute real-time data to one or more instances of a Structured Query Language (SQL) database that provides persistent storage. When used with a memory-optimized SQL database, optionally as a cache to a traditional disk-centric database, throughput rates can be supported and sustained that are an order of magnitude higher than most legacy databases support. A significant benefit of this standards-based logging approach is that applications and tools already using the or SQL standards can take advantage of this solution with little or no change. An off-the-shelf implementation of this logging solution is now provided by RTI. RTI Real-Time Connect enables the development of systems that can: efficiently bridge real-time traffic to/from SQL and other enterprise applications, route high-performance traffic between multiple domains, and capture/log message flows persistently. In conjunction with its logging capability, the RTI Real-Time Connect can be leveraged to support the design of a message replay system as well. 1
2 In this paper, we will focus on applying the capabilities provided by RTI Real-Time Connect in the design of a generic message logging subsystem. In addition, we ll also address performance issues related to logging with respect to sustainable rates at which real-time messages or data produced by RTI s solution, RTI Data Distribution Service, can be captured in Oracle TimesTen In-Memory Database and Oracle 10g. We ll also touch on how TimesTen Cache Connect to Oracle can be employed to handle burst message rates during logging activity. Design of a Generic Message Logging Subsystem Being able to persistently log specified messaging activity from various systems and subsystems within a complex distributed system provides a key role in being able to monitor, maintain, and debug system behavior. Once captured, these message logs can be further utilized to support post processing, as well as message replay for further system testing and maintenance within a controlled environment. The standard is particularly well-suited to distributing real-time data for logging as well as for general distributed application development and system integration. The standard has been widely adopted for real-time systems, including in industrial automation, aerospace, and defense applications. specifies an API designed for enabling real-time data distribution. It uses a publish-subscribe communication model and supports both messaging and data-object centric data models. offers several enhanced capabilities with respect to content-based filtering and transformation, per dataflow connectivity monitoring, redundancy, replication, delivery effort and ordering, as well as spontaneous discovery. Furthermore, offers new capabilities with respect to data-object lifecycle management, best-effort and predictable delivery, delivery ordering, resource management, and status notifications. System Requirements Our design objective is a generic Message Logging subsystem. Specifically, we will design a facility that supports the ability to: - log messages from one or more domains, - log messages persistently, - introduce minimal change and negligible performance impact on existing enabled systems, - provide high-performance logging on low-cost off-the-shelf hardware, - provide support for runtime logging configuration, - discover message formats dynamically at runtime, - provide individual message stream logging configuration/control, - provide bounded and unlimited message logging duration. Figure 1 illustrates the fundamental components of a generalized message logging subsystem. The subsystem must interface with applications as well as one or more database management systems. The design will leverage a relational storage model. One of the most compelling reasons for leveraging a relational database is its self describing capability and standards based compliance. In addition, storing information within a relational model opens a vast array of post-processing tools as well. 2
3 Logging Subsystem Message Discovery & Capture Runtime Configuration Engine Relational Database Interface Relational Database Figure 1. Generic Message Logging Subsystem Runtime Configuration The Runtime Configuration Engine controls which messages will be logged to the database. In addition it provides various Quality-of-Service (QoS) settings : - Whether to sub-sample (based on time) the message stream, or simply capture all samples, - the depth of the amount of logging to capture for each message stream, - From which network (i.e. domain) to log a particular message stream from, - The ability to re-configure during runtime. This includes adding a new message to log, indicating which message stream to stop logging, or altering the way in which a current message stream is being logged. Automatic Discovery and Storage Once configured via the Runtime Configuration Engine, the logging subsystem must automatically discover both topics (i.e. unique messages) and their associated message formats (i.e. IDL types). If the message is keyed, then all instances must also be captured. For each message stream that must be logged, as indicated by the Runtime Configuration Engine, the Message Discovery & Capture module simply creates a subscriber for the corresponding topic and data type. The Runtime Configuration Engine controls the actual QoS settings of the subscriber. By using subscribers, the design will introduce minimal performance impact on the existing systems 1. Once the messages have been discovered and are being captured, they will be stored in a relational database. The Relational Database 1 It s important to realize that if every message of a specific topic stream must be logged, then the system that publishes that topic must be configured for Reliable QoS with appropriate History QoS policy. 3
4 Interface module is responsible for inserting each of the message samples into their respective relational tables within the database. Thus if the logging subsystem is configured to capture 50 message types (i.e. topics), then 50 relational database tables (within one database) will be created to persistently store each distinct message stream. Applying RTI Real-Time Connect Next we discuss how RTI Real-Time Connect implements the design of the generic logging subsystem as laid out in the previous section. RTI Real-Time Connect seamlessly integrates applications with enterprise applications by using a relational database as the interface. The database also serves as a buffer giving enterprise applications direct access to real-time data in a way that doesn t negatively impact the performance of real-time applications. Furthermore, RTI Real-Time Connect is designed to integrate existing enabled real-time applications with enterprise based systems with minimal modification to either working application. Figure 2 illustrates the RTI Real-Time Connect architecture. SQL RTI Real-Time Connect Relational Database Figure 2. RTI Real-Time Connect Architecture. RTI Real-Time Connect operationally consists of a configurable number of publishers and subscribers, which act as a bridge between the networks and the relational database. It further enables automatic logging of messages by mapping the Topic to a table in the database and storing an 4
5 instance of a Topic as a row in that table. To be complete, RTI Real-Time Connect can automatically manage the publication of database table changes out to a specified network, but that capability won t be utilized within the logging subsystem design. RTI Real-Time Connect Salient Features In order to better understand how RTI Real-Time Connect can be used as the basis of our logging design, let s briefly list key features that we ll build upon: Automatic discovery of messages being published, Determines message format automatically via typecodes, Automatic storage of received messages within DBMS tables, IDL/SQL Data Mapping, Runtime configuration via meta-tables and QoS, Message History support both bounded and unbounded. Automatic discovery and mapping A key RTI Real-Time Connect feature is that once configured and connected to a network, it will automatically discover the messages (and their type format) being published. Once discovered, message format information is automatically mapped to SQL table schema, and a corresponding database table is created in order to allow the logging of the incoming messages. So, without any user interaction, RTI Real-Time Connect can create, and populate relational tables that will log messages. Each discovered topic will result in a relational database table. Let s explore how RTI Real-Time Connect handles each of these functional steps. Automatic Topic and Type Discovery RTI Real-Time Connect can be configured to listen on one or more domains for message traffic and auto-discover the topic and its data type. RTI Data Distribution Service provides built-in topics that can be used to discover remote entities. By creating a subscriber to the built-in topic DCPSPublication, RTI Real-Time Connect can automatically discover topics. Once the topics have been discovered, their format can be determined by parsing the typecode information. Typecodes provide a standards-based mechanism to represent arbitrarily complex IDL types at run time. They are used to send information about the application defined data format on the wire, thus enabling applications to publish or subscribe to topics of arbitrary types. Since typecode information is automatically propagated during the discovery phase of applications, RTI Real-Time Connect is able to discover and store all of the available typecodes associated with existing Topics. IDL/SQL Data Mapping Once the topic and message format (or type) is known, the relational table can be created in the database (if it doesn t already exist). In order to create the table, the table schema must be decided upon. RTI Real-Time Connect maps the discovered IDL type information to table schema. Fields in IDL types are mapped to columns of a table, and the key fields, if any, form the primary key of the table. Each row in the table represents a different value of the IDL type. Unless RTI Real-Time Connect is configured to keep history data, which will be discussed later, the last value of a instance is stored in the table. So a nonkeyed topic will be stored in a single row whereas for keyed topics, there will be as many rows as there are topic instances. The table below shows how basic IDL types are mapped to SQL data types. Simple IDL structures containing only basic or primitive types directly map to SQL schemas with fields in the structure becoming columns in the table. IDL structures that contain more complex fields, fields that are structures, unions, or sequences and arrays of types other than octet, char or wchar are mapped to SQL tables by flattening the embedded structures so that their fields are all at the top (and only) level. IDL type Char char[n] SQL type CHAR(1) CHAR(n) 5
6 string<n> Boolean Short Int Long long long Float Double Octet octet[n] sequence<octet,n> VARCHAR(n) TINYINT SMALLINT BIGINT REAL DOUBLE BINARY(1)/RAW BINARY(n) VARBINARY(n) RTI Real-Time Connect Runtime configuration Meta-tables and QoS policies Upon startup, RTI Real-Time Connect creates rtidds_subscriptions and rtidds_publications meta-tables in the database. These two relational tables provide the application or user the ability to configure the RTI Real-Time Connect runtime behavior. For the sake of our logging subsystem use case, we ll only focus on the rtidds_subscriptions table. The rtidds_subscriptions table is created empty. The table schema associated with this table is as follows: Table RTI_SUBSCRIPTIONS: *DOMAIN_ID NOT NULL *TOPIC_NAME VARCHAR (200) NOT NULL TABLE_OWNER VARCHAR (128) NOT NULL TABLE_NAME VARCHAR (128) NOT NULL TYPE_NAME VARCHAR (200) TABLE_HISTORY_DEPTH PROCESS_BATCH PROCESS_PERIOD.SEC PROCESS_PERIOD.NSEC COMMIT_TYPE VARCHAR (17) CACHE_MAXIMUM_SIZE CACHE_INITIAL_SIZE IDL_MEMBER_PREFIX_MAX_LENGTH IDL_MEMBER_SUFFIX_MAX_LENGTH SUB.PRESENT.ACCESS_SCOPE VARCHAR (25) SUB.PRESENT.ORDERED_ACCESS TINYINT DR.TIME_FILTER.MIN_SEP.SEC DR.TIME_FILTER.MIN_SEP.NSEC DR.DURABILITY.KIND VARCHAR (30) DR.RELIABILITY.KIND VARCHAR (27) DR.DESTINATION_ORDER.KIND VARCHAR (43) DR.LIVELINESS.LEASE_DUR.SEC DR.LIVELINESS.LEASE_DUR.NSEC DR.DEADLINE.PERIOD.SEC DR.DEADLINE.PERIOD.NSEC DR.HISTORY.KIND VARCHAR (21) DR.HISTORY.DEPTH DR.OWNERSHIP.KIND VARCHAR (23) DR.RES_LIMITS.MAX_SAMPLES DR.RES_LIMITS.MAX_INSTANCES DR.MULTICAST.RECEIVE_ADDRESS VARCHAR (15) 6
7 (primary key columns are indicated with *) Each row in the rtidds_subscriptions table represents a topic in a domain to be logged. The fields of the table expose many of the QoS attributes defined by the standard thus providing the user control over the quality of service when capturing real-time data from the network. More specifically, when a row (corresponding to a specific topic) is inserted into the rtidds_subscriptions meta-table, the RTI Real-Time Connect Daemon creates a DataReader (and Subscriber) according to the provided QoS parameters thus controlling how each unique message is ultimately subscribed to, and logged. As additional records are inserted into the meta table, additional DataReaders are instantiated. By inserting rows, we expand the logging systems ability to capture additional message types. It s not required to specify every column when a row is inserted into the table. In fact, if a row s column is not specified, then a default value will be used. Generally by specifying column-values in the rtidds_subscriptions table the application can configure: Which message or data streams to log; Whether a message stream is captured completely or sub-sampled by time-based filter; How many samples are kept, either all samples (unbounded) or a fixed-length history; Whether samples are committed to the database separately (for better response time) or in batches (for higher throughput); Whether messages are received reliably or best-effort for minimizing the effect on publishers. Once a message stream is being logged, it is possible to reconfigure the logging QoS or to cancel logging the stream by updating or deleting the corresponding row in the rtidds_subscriptions table. For example, to configure RTI Real-Time Connect to log four unique topics (i.e. message streams), the following four SQL statements would be executed on the rtidds_subscriptions table: insert into rtidds_subscriptions values (7, AirCraftPos,'tableowner','AircraftPositionTable','PlanePosType',-1); insert into rtidds_subscriptions (DOMAIN_ID, TOPIC_NAME, TABLE_OWNER, TABLE_NAME, TYPE_NAME, TABLE_HISTORY_DEPTH, DR.TIME_FILTER.MIN_SEP.SEC) values (25, FuseTmp,'tableowner ','FuseTempTable', 'FuseTempType', 20,); insert into rtidds_subscriptions (DOMAIN_ID, TOPIC_NAME, TABLE_OWNER, TABLE_NAME, TYPE_NAME, TABLE_HISTORY_DEPTH, DR.TIME_FILTER.MIN_SEP.NSEC) values (43, Humidity,'tableowner','HumidityTable','HumidType',5, ); insert into rtidds_subscriptions values (43, Pressure,'tableowner ','PressureTable','PressureType',0); As a result, four topics will be logged, each with differing QoS and table history depths. Let s examine each message to be logged: Aircraft Position Message - the topic AirCraftPos of IDL type AircraftPosType will be captured from Domain 7 and stored in a table named AircraftPositionTable. The history column is set to -1 indicating that all samples of the topic will be captured. If the topic has key fields, all instances will be logged in separate rows. Temperature Message - the topic FuseTmp of type FuseTempType will be captured in Domain 25 and stored in a table named FuseTempTable. The history column is set to 20, which indicates that the most recent 20 samples of the topic are kept. In this case the log table acts as a circular buffer. The time based filter QoS policy is set to 1 indicating that only one fuse temperature message per second will be logged. 7
8 Humidity Message - the topic Humidity of type HumidType will be captured from Domain 43 and stored in a table named HumidityTable. The history column is set such that the most recent 5 samples of the topic are kept. The time based filter QoS policy is set such that messages will be logged once per 300ms. Pressure Message - the topic Pressure of type PressureType will be captured in Domain 43 and stored in a table named PressureTable. The history column is set to 0, which indicates that only the most recent sample of the topic is stored. Runtime re-configuration of the Logging Subsystem: Inserting rows into the rtidds_subscriptions meta-table provides the ability to start logging messages in accordance to specified QoS. At some point, it may be desirable to discontinue the capture of a specific message stream. In this case, the row representing that specific message type can simply be deleted from the table. At that point, RTI Real-Time Connect will remove the DataReader, and the message will no longer be logged. The table containing all previously recorded messages continues to be available, for instance for data analysis or simulation/replay. Secondly, if for operational reasons the QoS of the DataReader instantiated by the RTI Real-Time Connect daemon must be altered, one can simply perform an update on the appropriate row within the meta-table, and the existing DataReader will be removed, and a new DataReader (with modified QoS) will be instantiated. This capability can be used, for instance, to dynamically adjust the sample-rate or history depth. For increased flexibility, the rtidds_subscriptions meta-table can be configured and managed either by a dedicated user application or graphical enterprise tools to configure a message logging subsystem with the ability to alter the systems logging functionality during runtime. Performance Considerations: In-memory versus disk-based database Depending on the volume and frequency of messages, RTI Real-Time Connect can be configured to utilize either in-memory, or disk-based relational databases. More specifically, RTI Real Time Connect supports Oracle TimesTen In-Memory Database which, being memory-optimized, delivers exceptionally high performance, enabling it to easily support the high data throughput of real-time applications. Real Time Connect can also be configured to use Oracle Database 10g, for storing large volumes of data. High Performance Logging When high performance logging is required, the TimesTen in-memory database provides the highest transaction rate on commodity hardware because insert and update operations are executed using memoryoptimized data structures. Figure 3 shows the throughput that can be achieved when logging a single message stream into a TimesTen data store. Note that for a single message size of 64 bytes, RTI Real-Time Connect configured with TimesTen can sustain 28,000 messages per second. If RTI Real-Time Connect is configured with Oracle 10g (on commodity hardware), we would achieve a sustainable message capture rate of roughly 8,000 messages per second. 8
9 Message or data rate versus payload 30,000 25,000 20,000 15,000 Oracle Timesten (sustainable rate) Oracle 10g (sustainable rate) TimesTen Cache Connect (burst rate) 10,000 5, ,024 4,096 Payload size (Bytes) Figure 3. RTI Real-Time Connect & Oracle TimesTen Performance (Transactions/Sec). Due to RTI Real-Time Connects scalable design, performance can scale when running on multi-processor or multi-core systems. Figure 4 illustrates RTI Real-Time Connects scalable architecture when logging four message types on a two-processor/dual core system. The results are due to the fact that RTI Real-Time Connect employs a multi-threaded architecture coupled with TimesTen extremely highperformance. Aggregate message or data rate versus payload 90,000 80,000 70,000 60,000 50,000 40,000 30,000 1 topic 2 topics 3 topics 4 topics 20,000 10, ,024 4,096 16,384 Payload size (Bytes) Figure 4. RTI Real-Time Connect & Oracle TimesTen Scalability When Capturing Multiple Messages. 9
10 It s important to note that there are architectural decisions that must be addressed when configuring RTI Real-Time Connect with TimesTen. If your logging system will be running on a 32 bit operating system, you ll be limited to a TimesTen database of roughly 2Gbytes due to the fact that the entire TimesTen database must reside in RAM. In order to support endless logging functionality, the TimesTen tables will need to be persisted to disk, and resized in order to continue to log incoming message samples. This can be achieved by deploying both TimesTen and Oracle 10g with the addition of Cache-Connect to Oracle which allows the seamless flushing of TimesTen tables to Oracle 10g. When TimesTen tables are associated with a Cache-Connect Cache Group, once configured - the Cache Connect daemon will seamlessly flush the designated TimesTen tables to Oracle 10g, and then automatically resize the TimesTen tables in order to allow them to continue to grow with logging. The criteria utilized to determine when to flush the tables, and how the resizing operation occurs is all configurable for each Cache Group managed by the Cache-Connect to Oracle daemon. By leveraging Cache-Connect to Oracle, we can achieve and support endless persistent message logging, with a reasonable amount of system RAM (on a 32 bit operating system). From a performance perspective, it s important to realize that the TimesTen in-memory database serves as a buffer isolating the enterprise infrastructure from potentially overwhelming high-frequency real-time messages. Even though RTI Real-Time Connect with TimesTen can support greater than 80,000 transactions when capturing multiple message streams, this throughput rate is not perpetually sustainable. Oracle TimesTen keeps transactions in an in-memory log buffer before TimesTen Cache Connect propagates these to Oracle 10g. The results for TimesTen Cache Connect shown in Figure 3 were generated using a transaction log buffer size large enough to store all transactions for the duration of a burst. Should the burst length exceed this setting, the data rate would drop to the level as shown by the previous benchmark. By employing TimesTen as an in-memory logging buffer, RTI Real-Time Connect can successfully absorb high message rate spikes without loosing messages. Alternatively, a 64-bit operating system can be deployed with sufficient RAM necessary to support the total volume of messages to be logged. The TimesTen data store can be configured as Persistent, which enables message capture to disk without the use of Oracle 10g. Lastly, as mentioned earlier, if the performance of TimesTen isn t required, then RTI Real-Time Connect can be configured to log directly to Oracle 10g thus allowing virtually unbounded storage of message traffic. In order to increase message throughput when logging to disk, message aggregation or subsampling can be applied if each individual message isn t required to be uniquely logged. Conclusion RTI Real-Time Connect is fully standards-based and is a solution that facilitates the design of a logging capability. From our previously stated design objectives, we have been able to see that RTI Real- Time Connect addresses each of the logging requirements with little or no application specific design or software development. In summary, the RTI Real-Time Connect architecture provides the basis for developing an efficient distributed message auditing subsystem. For more information about RTI Data Distribution Service,, or RTI Real-Time Connect, please visit 10
Unifying the Global Data Space using DDS and SQL
Unifying the Global Data Space using and SQL OMG RT Embedded Systems Workshop 13 July 2006 Gerardo Pardo-Castellote, Ph.D. CTO gerardo.pardo@rti.com www.rti.com Fernando Crespo Sanchez fernando.crespo@rti.com
More informationRepeat Success, Not Mistakes; Use DDS Best Practices to Design Your Complex Distributed Systems
WHITEPAPER Repeat Success, Not Mistakes; Use DDS Best Practices to Design Your Complex Distributed Systems Abstract RTI Connext DDS (Data Distribution Service) is a powerful tool that lets you efficiently
More informationCASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level. -ORACLE TIMESTEN 11gR1
CASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level -ORACLE TIMESTEN 11gR1 CASE STUDY Oracle TimesTen In-Memory Database and Shared Disk HA Implementation
More informationWhat can DDS do for You? Learn how dynamic publish-subscribe messaging can improve the flexibility and scalability of your applications.
What can DDS do for You? Learn how dynamic publish-subscribe messaging can improve the flexibility and scalability of your applications. 2 Contents: Abstract 3 What does DDS do 3 The Strengths of DDS 4
More information<Insert Picture Here> Oracle In-Memory Database Cache Overview
Oracle In-Memory Database Cache Overview Simon Law Product Manager The following is intended to outline our general product direction. It is intended for information purposes only,
More informationRTI Database Integration Service. Release Notes
RTI Database Integration Service Release Notes Version 5.2.0 2015 Real-Time Innovations, Inc. All rights reserved. Printed in U.S.A. First printing. June 2015. Trademarks Real-Time Innovations, RTI, NDDS,
More informationA Data Centric Approach for Modular Assurance. Workshop on Real-time, Embedded and Enterprise-Scale Time-Critical Systems 23 March 2011
A Data Centric Approach for Modular Assurance The Real-Time Middleware Experts Workshop on Real-time, Embedded and Enterprise-Scale Time-Critical Systems 23 March 2011 Gabriela F. Ciocarlie Heidi Schubert
More informationRTI Real-Time Connect. User s Manual
RTI Real-Time Connect User s Manual Version 5.0 2006-2012 Real-Time Innovations, Inc. All rights reserved. Printed in U.S.A. First printing. August 2012. Trademarks Real-Time Innovations, RTI, and Connext
More informationHypertable Architecture Overview
WHITE PAPER - MARCH 2012 Hypertable Architecture Overview Hypertable is an open source, scalable NoSQL database modeled after Bigtable, Google s proprietary scalable database. It is written in C++ for
More informationRTI Real-Time Connect. Release Notes
RTI Real-Time Connect Release Notes Version 4.5f 2012 Real-Time Innovations, Inc. All rights reserved. Printed in U.S.A. First printing. March 2012. Trademarks Real-Time Innovations, RTI, and Connext are
More informationRTI Database Integration Service. User s Manual
RTI Database Integration Service User s Manual Version 5.2.0 2015 Real-Time Innovations, Inc. All rights reserved. Printed in U.S.A. First printing. June 2015. Trademarks Real-Time Innovations, RTI, NDDS,
More informationTips and Tricks for Using Oracle TimesTen In-Memory Database in the Application Tier
Tips and Tricks for Using Oracle TimesTen In-Memory Database in the Application Tier Simon Law TimesTen Product Manager, Oracle Meet The Experts: Andy Yao TimesTen Product Manager, Oracle Gagan Singh Senior
More informationServices. Relational. Databases & JDBC. Today. Relational. Databases SQL JDBC. Next Time. Services. Relational. Databases & JDBC. Today.
& & 1 & 2 Lecture #7 2008 3 Terminology Structure & & Database server software referred to as Database Management Systems (DBMS) Database schemas describe database structure Data ordered in tables, rows
More informationEnabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings
Solution Brief Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Introduction Accelerating time to market, increasing IT agility to enable business strategies, and improving
More informationHow To Create A Table In Sql 2.5.2.2 (Ahem)
Database Systems Unit 5 Database Implementation: SQL Data Definition Language Learning Goals In this unit you will learn how to transfer a logical data model into a physical database, how to extend or
More informationComparing Microsoft SQL Server 2005 Replication and DataXtend Remote Edition for Mobile and Distributed Applications
Comparing Microsoft SQL Server 2005 Replication and DataXtend Remote Edition for Mobile and Distributed Applications White Paper Table of Contents Overview...3 Replication Types Supported...3 Set-up &
More informationSQL Server An Overview
SQL Server An Overview SQL Server Microsoft SQL Server is designed to work effectively in a number of environments: As a two-tier or multi-tier client/server database system As a desktop database system
More informationMemory-Centric Database Acceleration
Memory-Centric Database Acceleration Achieving an Order of Magnitude Increase in Database Performance A FedCentric Technologies White Paper September 2007 Executive Summary Businesses are facing daunting
More informationRaima Database Manager Version 14.0 In-memory Database Engine
+ Raima Database Manager Version 14.0 In-memory Database Engine By Jeffrey R. Parsons, Senior Engineer January 2016 Abstract Raima Database Manager (RDM) v14.0 contains an all new data storage engine optimized
More informationInfiniteGraph: The Distributed Graph Database
A Performance and Distributed Performance Benchmark of InfiniteGraph and a Leading Open Source Graph Database Using Synthetic Data Objectivity, Inc. 640 West California Ave. Suite 240 Sunnyvale, CA 94086
More informationAvailability Digest. www.availabilitydigest.com. Raima s High-Availability Embedded Database December 2011
the Availability Digest Raima s High-Availability Embedded Database December 2011 Embedded processing systems are everywhere. You probably cannot go a day without interacting with dozens of these powerful
More informationResource Utilization of Middleware Components in Embedded Systems
Resource Utilization of Middleware Components in Embedded Systems 3 Introduction System memory, CPU, and network resources are critical to the operation and performance of any software system. These system
More informationTable of Contents. 2015 Cicero, Inc. All rights protected and reserved.
Desktop Analytics Table of Contents Contact Center and Back Office Activity Intelligence... 3 Cicero Discovery Sensors... 3 Business Data Sensor... 5 Business Process Sensor... 5 System Sensor... 6 Session
More informationRTI Database Integration Service. Getting Started Guide
RTI Database Integration Service Getting Started Guide Version 5.2.0 2015 Real-Time Innovations, Inc. All rights reserved. Printed in U.S.A. First printing. June 2015. Trademarks Real-Time Innovations,
More informationCS 464/564 Introduction to Database Management System Instructor: Abdullah Mueen
CS 464/564 Introduction to Database Management System Instructor: Abdullah Mueen LECTURE 14: DATA STORAGE AND REPRESENTATION Data Storage Memory Hierarchy Disks Fields, Records, Blocks Variable-length
More informationFileMaker 13. ODBC and JDBC Guide
FileMaker 13 ODBC and JDBC Guide 2004 2013 FileMaker, Inc. All Rights Reserved. FileMaker, Inc. 5201 Patrick Henry Drive Santa Clara, California 95054 FileMaker and Bento are trademarks of FileMaker, Inc.
More informationOracle BI EE Implementation on Netezza. Prepared by SureShot Strategies, Inc.
Oracle BI EE Implementation on Netezza Prepared by SureShot Strategies, Inc. The goal of this paper is to give an insight to Netezza architecture and implementation experience to strategize Oracle BI EE
More informationOracle SOA Suite: The Evaluation from 10g to 11g
KATTA Durga Reddy TATA Consultancy Services. Oracle SOA Suite: The Evaluation from 10g to 11g Introduction Oracle SOA Suite is an essential middleware layer of Oracle Fusion Middleware. It provides a complete
More informationSAP HANA PLATFORM Top Ten Questions for Choosing In-Memory Databases. Start Here
PLATFORM Top Ten Questions for Choosing In-Memory Databases Start Here PLATFORM Top Ten Questions for Choosing In-Memory Databases. Are my applications accelerated without manual intervention and tuning?.
More informationFileMaker 12. ODBC and JDBC Guide
FileMaker 12 ODBC and JDBC Guide 2004 2012 FileMaker, Inc. All Rights Reserved. FileMaker, Inc. 5201 Patrick Henry Drive Santa Clara, California 95054 FileMaker and Bento are trademarks of FileMaker, Inc.
More informationUsing SQL Server Management Studio
Using SQL Server Management Studio Microsoft SQL Server Management Studio 2005 is a graphical tool for database designer or programmer. With SQL Server Management Studio 2005 you can: Create databases
More informationRTI Monitoring Library Getting Started Guide
RTI Monitoring Library Getting Started Guide Version 5.1.0 2011-2013 Real-Time Innovations, Inc. All rights reserved. Printed in U.S.A. First printing. December 2013. Trademarks Real-Time Innovations,
More informationIkasan ESB Reference Architecture Review
Ikasan ESB Reference Architecture Review EXECUTIVE SUMMARY This paper reviews the Ikasan Enterprise Integration Platform within the construct of a typical ESB Reference Architecture model showing Ikasan
More informationDesktop Activity Intelligence
Desktop Activity Intelligence Table of Contents Cicero Discovery Delivers Activity Intelligence... 1 Cicero Discovery Modules... 1 System Monitor... 2 Session Monitor... 3 Activity Monitor... 3 Business
More informationPhysical Design. Meeting the needs of the users is the gold standard against which we measure our success in creating a database.
Physical Design Physical Database Design (Defined): Process of producing a description of the implementation of the database on secondary storage; it describes the base relations, file organizations, and
More informationDBMS / Business Intelligence, SQL Server
DBMS / Business Intelligence, SQL Server Orsys, with 30 years of experience, is providing high quality, independant State of the Art seminars and hands-on courses corresponding to the needs of IT professionals.
More informationApache Cassandra Query Language (CQL)
REFERENCE GUIDE - P.1 ALTER KEYSPACE ALTER TABLE ALTER TYPE ALTER USER ALTER ( KEYSPACE SCHEMA ) keyspace_name WITH REPLICATION = map ( WITH DURABLE_WRITES = ( true false )) AND ( DURABLE_WRITES = ( true
More informationUsing DDS to Enable The Real-Time Enterprise Service Bus (RT-ESB)
Using DDS to Enable The Real-Time Enterprise Service Bus (RT-ESB) Rajive Joshi, Ph. D. Gerardo Pardo-Castellote, Ph.D. Real-Time Innovations, Inc OMG Real-time and Embedded Systems Workshop Arlington,
More informationOracle Data Integrator 12c: Integration and Administration
Oracle University Contact Us: +33 15 7602 081 Oracle Data Integrator 12c: Integration and Administration Duration: 5 Days What you will learn Oracle Data Integrator is a comprehensive data integration
More informationLinas Virbalas Continuent, Inc.
Linas Virbalas Continuent, Inc. Heterogeneous Replication Replication between different types of DBMS / Introductions / What is Tungsten (the whole stack)? / A Word About MySQL Replication / Tungsten Replicator:
More informationSpeed and Persistence for Real-Time Transactions
Speed and Persistence for Real-Time Transactions by TimesTen and Solid Data Systems July 2002 Table of Contents Abstract 1 Who Needs Speed and Persistence 2 The Reference Architecture 3 Benchmark Results
More informationUsing Object Database db4o as Storage Provider in Voldemort
Using Object Database db4o as Storage Provider in Voldemort by German Viscuso db4objects (a division of Versant Corporation) September 2010 Abstract: In this article I will show you how
More informationOracle Data Integrator 11g: Integration and Administration
Oracle University Contact Us: Local: 1800 103 4775 Intl: +91 80 4108 4709 Oracle Data Integrator 11g: Integration and Administration Duration: 5 Days What you will learn Oracle Data Integrator is a comprehensive
More informationVirtuoso and Database Scalability
Virtuoso and Database Scalability By Orri Erling Table of Contents Abstract Metrics Results Transaction Throughput Initializing 40 warehouses Serial Read Test Conditions Analysis Working Set Effect of
More informationGigaSpaces Real-Time Analytics for Big Data
GigaSpaces Real-Time Analytics for Big Data GigaSpaces makes it easy to build and deploy large-scale real-time analytics systems Rapidly increasing use of large-scale and location-aware social media and
More informationOracle Database 11g Comparison Chart
Key Feature Summary Express 10g Standard One Standard Enterprise Maximum 1 CPU 2 Sockets 4 Sockets No Limit RAM 1GB OS Max OS Max OS Max Database Size 4GB No Limit No Limit No Limit Windows Linux Unix
More informationJitterbit Technical Overview : Microsoft Dynamics CRM
Jitterbit allows you to easily integrate Microsoft Dynamics CRM with any cloud, mobile or on premise application. Jitterbit s intuitive Studio delivers the easiest way of designing and running modern integrations
More informationSQL Server 2008 Performance and Scale
SQL Server 2008 Performance and Scale White Paper Published: February 2008 Updated: July 2008 Summary: Microsoft SQL Server 2008 incorporates the tools and technologies that are necessary to implement
More informationConventional Files versus the Database. Files versus Database. Pros and Cons of Conventional Files. Pros and Cons of Databases. Fields (continued)
Conventional Files versus the Database Files versus Database File a collection of similar records. Files are unrelated to each other except in the code of an application program. Data storage is built
More informationCitusDB Architecture for Real-Time Big Data
CitusDB Architecture for Real-Time Big Data CitusDB Highlights Empowers real-time Big Data using PostgreSQL Scales out PostgreSQL to support up to hundreds of terabytes of data Fast parallel processing
More informationNexenta Performance Scaling for Speed and Cost
Nexenta Performance Scaling for Speed and Cost Key Features Optimize Performance Optimize Performance NexentaStor improves performance for all workloads by adopting commodity components and leveraging
More informationContents. 1010 Huntcliff, Suite 1350, Atlanta, Georgia, 30350, USA http://www.nevatech.com
Sentinet Overview Contents Overview... 3 Architecture... 3 Technology Stack... 4 Features Summary... 6 Repository... 6 Runtime Management... 6 Services Virtualization and Mediation... 9 Communication and
More informationWITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE
WITH A FUSION POWERED SQL SERVER 2014 IN-MEMORY OLTP DATABASE 1 W W W. F U S I ON I O.COM Table of Contents Table of Contents... 2 Executive Summary... 3 Introduction: In-Memory Meets iomemory... 4 What
More informationSpring,2015. Apache Hive BY NATIA MAMAIASHVILI, LASHA AMASHUKELI & ALEKO CHAKHVASHVILI SUPERVAIZOR: PROF. NODAR MOMTSELIDZE
Spring,2015 Apache Hive BY NATIA MAMAIASHVILI, LASHA AMASHUKELI & ALEKO CHAKHVASHVILI SUPERVAIZOR: PROF. NODAR MOMTSELIDZE Contents: Briefly About Big Data Management What is hive? Hive Architecture Working
More informationOracle Enterprise Manager
Oracle Enterprise Manager System Monitoring Plug-in for Oracle TimesTen In-Memory Database Installation Guide Release 11.2.1 E13081-02 June 2009 This document was first written and published in November
More informationSAP Data Services 4.X. An Enterprise Information management Solution
SAP Data Services 4.X An Enterprise Information management Solution Table of Contents I. SAP Data Services 4.X... 3 Highlights Training Objectives Audience Pre Requisites Keys to Success Certification
More informationHigh-Volume Data Warehousing in Centerprise. Product Datasheet
High-Volume Data Warehousing in Centerprise Product Datasheet Table of Contents Overview 3 Data Complexity 3 Data Quality 3 Speed and Scalability 3 Centerprise Data Warehouse Features 4 ETL in a Unified
More informationJitterbit Technical Overview : Salesforce
Jitterbit allows you to easily integrate Salesforce with any cloud, mobile or on premise application. Jitterbit s intuitive Studio delivers the easiest way of designing and running modern integrations
More informationImprove Business Productivity and User Experience with a SanDisk Powered SQL Server 2014 In-Memory OLTP Database
WHITE PAPER Improve Business Productivity and User Experience with a SanDisk Powered SQL Server 2014 In-Memory OLTP Database 951 SanDisk Drive, Milpitas, CA 95035 www.sandisk.com Table of Contents Executive
More informationWhat You Need to Know About Transitioning to SOA
What You Need to Know About Transitioning to SOA written by: David A. Kelly, ebizq Analyst What You Need to Know About Transitioning to SOA Organizations are increasingly turning to service-oriented architectures
More informationAchieving Nanosecond Latency Between Applications with IPC Shared Memory Messaging
Achieving Nanosecond Latency Between Applications with IPC Shared Memory Messaging In some markets and scenarios where competitive advantage is all about speed, speed is measured in micro- and even nano-seconds.
More informationTowards Smart and Intelligent SDN Controller
Towards Smart and Intelligent SDN Controller - Through the Generic, Extensible, and Elastic Time Series Data Repository (TSDR) YuLing Chen, Dell Inc. Rajesh Narayanan, Dell Inc. Sharon Aicler, Cisco Systems
More informationHOUG Konferencia 2015. Oracle TimesTen In-Memory Database and TimesTen Application-Tier Database Cache. A few facts in 10 minutes
HOUG Konferencia 2015 Oracle TimesTen In-Memory Database and TimesTen Application-Tier Database Cache A few facts in 10 minutes Tamas.Kerepes@webvalto.hu What is TimesTen An in-memory relational database
More informationJOURNAL OF OBJECT TECHNOLOGY
JOURNAL OF OBJECT TECHNOLOGY Online at www.jot.fm. Published by ETH Zurich, Chair of Software Engineering JOT, 2008 Vol. 7 No. 7, September-October 2008 Applications At Your Service Mahesh H. Dodani, IBM,
More informationwww.gr8ambitionz.com
Data Base Management Systems (DBMS) Study Material (Objective Type questions with Answers) Shared by Akhil Arora Powered by www. your A to Z competitive exam guide Database Objective type questions Q.1
More informationTIBCO Live Datamart: Push-Based Real-Time Analytics
TIBCO Live Datamart: Push-Based Real-Time Analytics ABSTRACT TIBCO Live Datamart is a new approach to real-time analytics and data warehousing for environments where large volumes of data require a management
More informationSAP HANA SPS 09 - What s New? HANA IM Services: SDI and SDQ
SAP HANA SPS 09 - What s New? HANA IM Services: SDI and SDQ (Delta from SPS 08 to SPS 09) SAP HANA Product Management November, 2014 2014 SAP SE or an SAP affiliate company. All rights reserved. 1 Agenda
More informationFileMaker 11. ODBC and JDBC Guide
FileMaker 11 ODBC and JDBC Guide 2004 2010 FileMaker, Inc. All Rights Reserved. FileMaker, Inc. 5201 Patrick Henry Drive Santa Clara, California 95054 FileMaker is a trademark of FileMaker, Inc. registered
More informationAutomatic Service Migration in WebLogic Server An Oracle White Paper July 2008
Automatic Service Migration in WebLogic Server An Oracle White Paper July 2008 NOTE: The following is intended to outline our general product direction. It is intended for information purposes only, and
More informationAccelerating Complex Event Processing with Memory- Centric DataBase (MCDB)
Accelerating Complex Event Processing with Memory- Centric DataBase (MCDB) A FedCentric Technologies White Paper January 2008 Executive Summary Events happen in real-time; orders are taken, calls are placed,
More informationOracle Total Recall with Oracle Database 11g Release 2
An Oracle White Paper September 2009 Oracle Total Recall with Oracle Database 11g Release 2 Introduction: Total Recall = Total History... 1 Managing Historical Data: Current Approaches... 2 Application
More informationOracle SOA Suite Then and Now:
Oracle SOA Suite Then and Now: The Evolution from 10g to 11g Shane Goss Impac Services Agenda SOA Suite 11g New Features Highlight new features of SOA 11g Some products have added features and functionality
More informationHow To Store Data On An Ocora Nosql Database On A Flash Memory Device On A Microsoft Flash Memory 2 (Iomemory)
WHITE PAPER Oracle NoSQL Database and SanDisk Offer Cost-Effective Extreme Performance for Big Data 951 SanDisk Drive, Milpitas, CA 95035 www.sandisk.com Table of Contents Abstract... 3 What Is Big Data?...
More informationDBMS Questions. 3.) For which two constraints are indexes created when the constraint is added?
DBMS Questions 1.) Which type of file is part of the Oracle database? A.) B.) C.) D.) Control file Password file Parameter files Archived log files 2.) Which statements are use to UNLOCK the user? A.)
More informationAchieving Zero Downtime and Accelerating Performance for WordPress
Application Note Achieving Zero Downtime and Accelerating Performance for WordPress Executive Summary WordPress is the world s most popular open source website content management system (CMS). As usage
More information4D and SQL Server: Powerful Flexibility
4D and SQL Server: Powerful Flexibility OVERVIEW MS SQL Server has become a standard in many parts of corporate America. It can manage large volumes of data and integrates well with other products from
More informationWhite Paper November 2015. Technical Comparison of Perspectium Replicator vs Traditional Enterprise Service Buses
White Paper November 2015 Technical Comparison of Perspectium Replicator vs Traditional Enterprise Service Buses Our Evolutionary Approach to Integration With the proliferation of SaaS adoption, a gap
More informationThe Service Availability Forum Specification for High Availability Middleware
The Availability Forum Specification for High Availability Middleware Timo Jokiaho, Fred Herrmann, Dave Penkler, Manfred Reitenspiess, Louise Moser Availability Forum Timo.Jokiaho@nokia.com, Frederic.Herrmann@sun.com,
More informationCopyright 2014, Oracle and/or its affiliates. All rights reserved.
1 Oracle and Visual Studio 2013: What's New and Best Practices Alex Keh Senior Principal Product Manager, Oracle Program Agenda Introduction to ODAC New Features Schema Compare ODP.NET, Managed Driver
More informationEmpress Embedded Database. for. Medical Systems
Empress Embedded Database for Medical Systems www.empress.com Empress Software Phone: 301-220-1919 1. Introduction From patient primary care information system to medical imaging system to life-critical
More informationIntegrating Content Management Within Enterprise Applications: The Open Standards Option. Copyright Xythos Software, Inc. 2005 All Rights Reserved
Integrating Content Management Within Enterprise Applications: The Open Standards Option Copyright Xythos Software, Inc. 2005 All Rights Reserved Table of Contents Introduction...3 Why Developers Are Choosing
More informationIT2305 Database Systems I (Compulsory)
Database Systems I (Compulsory) INTRODUCTION This is one of the 4 modules designed for Semester 2 of Bachelor of Information Technology Degree program. CREDITS: 04 LEARNING OUTCOMES On completion of this
More informationIn This Lecture. SQL Data Definition SQL SQL. Notes. Non-Procedural Programming. Database Systems Lecture 5 Natasha Alechina
This Lecture Database Systems Lecture 5 Natasha Alechina The language, the relational model, and E/R diagrams CREATE TABLE Columns Primary Keys Foreign Keys For more information Connolly and Begg chapter
More informationWanVelocity. WAN Optimization & Acceleration
WanVelocity D A T A S H E E T WAN Optimization & Acceleration WanVelocity significantly accelerates applications while reducing bandwidth costs using a combination of application acceleration, network
More informationTier Architectures. Kathleen Durant CS 3200
Tier Architectures Kathleen Durant CS 3200 1 Supporting Architectures for DBMS Over the years there have been many different hardware configurations to support database systems Some are outdated others
More informationEMC Backup and Recovery for Microsoft SQL Server 2008 Enabled by EMC Celerra Unified Storage
EMC Backup and Recovery for Microsoft SQL Server 2008 Enabled by EMC Celerra Unified Storage Applied Technology Abstract This white paper describes various backup and recovery solutions available for SQL
More informationWhite Paper. Mimosa NearPoint for Microsoft Exchange Server. Next Generation Email Archiving for Exchange Server 2007. By Bob Spurzem and Martin Tuip
White Paper By Bob Spurzem and Martin Tuip Mimosa Systems, Inc. January 2008 Mimosa NearPoint for Microsoft Exchange Server Next Generation Email Archiving for Exchange Server 2007 CONTENTS Email has become
More informationAn Oracle White Paper March 2014. Best Practices for Real-Time Data Warehousing
An Oracle White Paper March 2014 Best Practices for Real-Time Data Warehousing Executive Overview Today s integration project teams face the daunting challenge that, while data volumes are exponentially
More informationOn- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform
On- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform Page 1 of 16 Table of Contents Table of Contents... 2 Introduction... 3 NoSQL Databases... 3 CumuLogic NoSQL Database Service...
More informationQuestions & Answers from the Webcast: Your Mac Users Deserve Quality Support Too. Sponsors: Atempo. Centrify. Group Logic. LANrev.
Questions & Answers from the Webcast: Your Mac Users Deserve Quality Support Too Sponsors: Atempo Centrify Group Logic LANrev Parallels November 10, 2009 Web Help Desk Contents Compilation of the questions
More informationSCA-based Enterprise Service Bus WebSphere ESB
IBM Software Group SCA-based Enterprise Service Bus WebSphere ESB Soudabeh Javadi, WebSphere Software IBM Canada Ltd sjavadi@ca.ibm.com 2007 IBM Corporation Agenda IBM Software Group WebSphere software
More informationGeodatabase Programming with SQL
DevSummit DC February 11, 2015 Washington, DC Geodatabase Programming with SQL Craig Gillgrass Assumptions Basic knowledge of SQL and relational databases Basic knowledge of the Geodatabase We ll hold
More informationBasic Oracle Database Licensing
By Craig Moir of MyDBA March 2011 Version 2 CONTENTS Introduction Oracle Database Editions Enterprise Edition Features Enterprise Edition Options Management Packs Licensing Types Licensing Metrics Licensing
More informationBenchmarking Cassandra on Violin
Technical White Paper Report Technical Report Benchmarking Cassandra on Violin Accelerating Cassandra Performance and Reducing Read Latency With Violin Memory Flash-based Storage Arrays Version 1.0 Abstract
More informationCisco Prime Optical. Overview
Data Sheet Cisco Prime Optical Overview Service providers face tremendous challenges in managing optical networks. These challenges include accurate capacity planning; maintaining an offline database of
More informationMicrosoft SQL Server performance tuning for Microsoft Dynamics NAV
Microsoft SQL Server performance tuning for Microsoft Dynamics NAV TechNet Evening 11/29/2007 1 Introductions Steven Renders Microsoft Certified Trainer Plataan steven.renders@plataan.be Check Out: www.plataan.be
More informationSAP HANA - Main Memory Technology: A Challenge for Development of Business Applications. Jürgen Primsch, SAP AG July 2011
SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications Jürgen Primsch, SAP AG July 2011 Why In-Memory? Information at the Speed of Thought Imagine access to business data,
More informationMimer SQL Real-Time Edition White Paper
Mimer SQL Real-Time Edition - White Paper 1(5) Mimer SQL Real-Time Edition White Paper - Dag Nyström, Product Manager Mimer SQL Real-Time Edition Mimer SQL Real-Time Edition is a predictable, scalable
More informationThe Advantages of Enterprise Historians vs. Relational Databases
GE Intelligent Platforms The Advantages of Enterprise Historians vs. Relational Databases Comparing Two Approaches for Data Collection and Optimized Process Operations The Advantages of Enterprise Historians
More informationUsing MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam sastry.vedantam@oracle.com
Using MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam sastry.vedantam@oracle.com Agenda The rise of Big Data & Hadoop MySQL in the Big Data Lifecycle MySQL Solutions for Big Data Q&A
More information