TIBCO ActiveSpaces Use Cases How in-memory computing supercharges your infrastructure

Transcription

1 TIBCO Use Cases How in-memory computing supercharges your infrastructure is a great solution for lifting the burden of big data, reducing reliance on costly transactional systems, and building highly scalable, fault-tolerant applications. TIBCO is a distributed in-memory data grid that provides very fast data access and update. While its performance will vary depending on multiple factors, it is not uncommon for to be 100x faster than corresponding database implementations. For this reason and others described here, is part of the TIBCO Fast Data solution for lifting the burden of big data, reducing reliance on costly transactional systems, and building highly scalable, fault-tolerant applications. INTRODUCTION Combining the features and performance of databases, caching systems, and messaging software, TIBCO supports very large, highly volatile data sets and event-driven applications as described in the use cases that follow. In addition, it provides: Distributed, Dynamic Architecture: Scales and handles large data volumes. Dynamically adds or drops nodes without requiring a system restart. Distributed Processing: Processes data on each node, in parallel, in a distributed manner. Handles large processing requirements by distributing the processing over multiple servers. Supports transactions across multiple spaces (tables). Fault Tolerance: Stores backup copies of data on multiple nodes, so in case of node outage(s), processing continues without impact. Data is redistributed as a background task without impacting on-going processing. Persistence Options: Optionally provides no persistence with data stored in a database (shared all) or locally in files on each node (shared nothing). Put and Take: Employs operations similar to messaging. Allows for client applications to register to receive events when certain criteria are met as defined by a SQL-like query.

2 SOLUTION BRIEF 2 Query Support: Supports a subset of SQL-like queries. Configurable Data Purging: Allows time to live (TTL) to be configured for items in the space, automatically removing items when TTL has expired. Data Lock Controls: Provides ability to lock items, with features for lock wait, lock break, and automatic release of the lock. Embedding: Transparently embeds with applications or configures as separate nodes that are accessed by applications. Remote Invocation: Provides the ability to invoke processing on other node(s). YOUR INDUSTRY clients can be written in many technologies including Java,.Net, C++, TIBCO BusinessWorks, and TIBCO BusinessEvents. The examples presented here include specific industry scenarios, but use cases can apply to many industries. CACHING CACHING FOR FAST DATA ACCESS is frequently used as a cache to speed information access. In this example, is not the system of record for the data in the cache. A separate loader process or application is responsible for loading the data into. Because the data is read-only, does not require persistence to save data changes. This makes all ActivesSpaces gets very fast. A side benefit is that the processing load on the back-end data store is reduced because the client applications are not querying the master data set. Loader Data

3 SOLUTION BRIEF 3 DISTRIBUTED, NOT CENTRALIZED Databases are central to many enterprise applications, but also struggle to handle high volumes, rapidly changing values, and large numbers of concurrent client applications. can be used to improve their performance; However, it is rarely a dropin, no-code replacement because the application/ database interchange can be complicated, involving SQL, business logic, and data spread across architectural tiers. Because uses a distributed architecture to achieve its performance and fault tolerance, some database features are not directly supported. For example, the JDBC driver does not support all SQL language syntax but there are ways to provide the same functionality. CACHING + DYNAMIC LOAD A variation on the caching example is to have the data dynamically loaded into when the data is first accessed by a client application. A service is provided that first looks in for the data. If found, the data is directly returned. If the service does not find the data in, it queries a backend data store. If the data is found in the backend data store, it is returned to the client application and stored into. If desired, time to live (TTL) can be configured for the data in so that it is automatically purged. One benefit of this approach is that the service can present a standard interface and the client applications are not required to implement any code. Service Data ACCELERATING SERVICES BROAD PERFORMANCE IMPROVEMENTS There are many ways that can be used to speed services. It can cache information from back-end applications or cache common lookup, index, or transformation information. Any service that retrieves data from an external data store, application, or file system has the potential for performance improvement using. ULTRA-FAST DATA STORE can also be used as a very fast data store. In this case, it allows updates to the data and persists the new data values. It provides two options for persisting data: shared all (database) and shared nothing (distributed file system). The shared nothing approach is illustrated below. With shared nothing, each node maintains a file with only the subset of data that is managed by the node. The system volume and speed is scaled by adding more nodes on different servers.

4 SOLUTION BRIEF 4 ACTIVESPACES AND SORTS queries run in parallel on multiple nodes, and the result is passed back to the client application unsorted. The client application can perform the sort, if desired. ACTIVESPACES AND JOINS Distributed processing prevents from performing joins of multiple tables. So you cannot directly perform an operation like the following, but you could use multiple queries and some processing by the client to arrive at the same result: Select * from Customer, Orders, Inventory where Customer.id = Order.customerId and Order.partnum = Inventory. partnum order by Order.date. One technique for addressing joins is to denormalize data by combining what would be contained in multiple tables in a relational database into a single table, or space, in. There is often sufficient memory to accommodate de-normalized data, and with it is faster to perform one get or set on a large row of data than operations on smaller sets in three separate spaces. Therefore, many implementations focus on flatter data sets; implementations to handle very complex hierarchical data may require additional work. NON-STOP DATA STORE While possible, it can be expensive to achieve 24x7x365 availibility with database technologies because they are frequently deployed on separate, potentially expensive hardware platforms. This hardware can add more failure points and reduce calculated reliability numbers. Further, databases often require planned outages for routine maintenance. With it is often possible to remove the database and the associated failure points from these implementations. persistence is distributed across multiple commodity servers, providing a high availability system at a fraction of the cost of other approaches. FAST DATA STORE WITH SEPARATE MASTER UPDATE A more full-featured use case involves using to manage high volumes of data reads and updates, and to also send and receive data updates from a separate master system of record. This is especially useful when spikes of data access and data updates overload the back office system of record. An event listener gets notified whenever a data value is changed and sends updates through a message queue (for example, TIBCO Enterprise Message Service ) for updating the master system of record. The back office system can also be updated through other channels so any updates are passed through a loader back to. Event Listener Loader Data Base SPIKES IN PROCESSING SPIKES IN PROCESSING: TICKET SALES is frequently used to handle spikes in processing that would overwhelm legacy systems, such as tickets sales made over the web or through multiple channels. When highly sought-after tickets are first offered, a large number of consumers will attempt to purchase immediately. manages the spike in processing volume, providing accurate, real-time inventory and seat reservation data. The spike of data is passed to the back-office system of record at a lower volume that can be more easily handled by these systems.

5 SOLUTION BRIEF 5 IMPLEMENTATION DIFFERENCES It can be possible to use to increase application performance without impacting the application or the application database. Or, development may be required to add to the architecture. Some analysis of the application, data, and process will be required in each situation to determine the best approach for using to speed application processing. SPIKES IN PROCESSING: ONLINE BETTING Certain Internet events can have a large number of users simultaneously logging in horse races, sporting events, election results coverage. If an issue occurs that causes users to be logged off, it will likely cause hundreds of thousands, or even millions, of users to simultaneously try to log back in. This huge spike in processing will overwhelm most conventional systems. In the case of online betting for a horse race, most bets are placed a few minutes before the start of the race. An outage at this time could have a multimillion dollar impact. is used in these types of scenarios to cache user information in a fault-tolerant manner and to handle extremely high spikes in processing. E-COMMERCE SESSION CACHING FOR E-COMMERCE Much e-commerce data is only needed for the current session. is used to cache this information and speed system response. HTTP Web Servers with Inline Cache Load Balancer Requests DYNAMIC INVENTORY Dynamic inventory that is viewed and accessed from a variety of channels provides a great use case for. One nationwide retailer maintains an inventory of returned or damaged items using, which allows local store employees to enter and update their inventory, and local web users to view and purchase items for pick up. With, the retailer can manage the dynamic inventory in real time for hundreds of stores. REAL-TIME OFFERS Formulating and presenting offers customized in real time according to user preferences is an ideal use case for. When a telecom customer is reviewing their usage and remaining minutes, they may be considering moving to a different carrier. In that moment, they are also more likely to accept a special offer designed to keep them. A retail customer is more likely to accept an offer at the point of sale rather than afterwards. Presenting offers in real time requires immediate access to customer information, usage, or spending history, as well as information on prior offers. This information is often contained in overloaded billing or customer relationship management (CRM) systems that cannot respond as quickly as needed. Adding a real-time offer capability to these systems will just further overload them. can provide real-time access to customer preferences, recent spending history, and other relevant information and service a high volume of requests.

6 SOLUTION BRIEF 6 FAULT TOLERANCE AND DISASTER RECOVERY FAULT-TOLERANT ACTIVE-ACTIVE PROCESSING Many applications provide fault tolerance by running in active-standby on multiple servers. The standby server or server application can be either warm (running but not processing) or cold (not running). With it is possible to build fault-tolerant applications that run on multiple servers in an active-active configuration. The data and state information is shared by all servers, and back-up copies of the information can be persisted on multiple servers. provides sophisticated and fast-locking mechanisms that can be used to synchronize processing across multiple servers. Server Server MULTISITE DATA REPLICATION Many organizations would like to run certain applications in multiple data centers in an active-active manner, or at least to have the data synchronized. It is not recommended to run across data centers that are more than a few kilometers apart. The network latencies will impact the overall system performance and can lead to other complications. However, it is possible to have in each data center synchronized. The technique uses a local event listener that can remotely invoke processing in the other data centers. Updates to in one data center are replicated in the other. With this technique, the data centers can operate in an active-active mode, and both have copies of the data in case of disaster recovery. Data Center A Data Center B Event Listener Remote Invocation Event Listener

7 SOLUTION BRIEF 7 DISTRIBUTION OF RAPIDLY CHANGING DATA Database technology is useful for long-term data storage, but not as well suited for managing large volumes of rapidly changing data. There are many situations where only the most recent values in a large, rapidly changing dataset are of interest. Examples include power plant monitoring data, stock market prices, satellite telemetry, and the status and location of packages. The system load becomes more intensive when there are a large number of consuming systems that are interested in subsets of the overall information. can manage large volumes of rapidly changing data. applications register event listeners for specific subsets of the information and are notified in real time when data events of interest occur. OFFLOADING EXPENSIVE RESOURCES Legacy back office applications can be very expensive to operate and scale, especially when mainframes incur a charge on a per transaction basis. When the transaction is a read of non-changing or slowly changing data, can be used as a cache to reduce the load and cost of expensive resources. MESSAGING ROUTING MESSAGES TO BACK OFFICE APPLICATIONS Organizations that receive a common data feed that needs to be parsed and routed to several back-office applications can use as a reference for the routing application. It can hold reference information so the router can quickly determine where to send the data. Bank payments and insurance claims processing are two examples. Data Feed Router

8 SOLUTION BRIEF 8 MULTI-SYSTEM ASYNCHRONOUS REQUEST-REPLY Systems that depend on a request-reply interface to synchronize messages can use to prevent correlation problems. These systems typically are synchronized using a transaction or correlation ID. Problems occur when back office applications cannot process or respond with the correlation ID, and when multiple back-office systems process the same message. A gateway application can be used to send the correlation ID to, which then forwards the data to the back office system for processing. When the back office application responds, the gateway application retrieves the correlation ID and inserts it into the outbound message. Gateway Gateway DUPLICATE MESSAGE DETECTION Anyone implementing a message processing system for critical information will eventually need to define a mechanism for identifying duplicate messages. Sometimes the approach is to pass deduplication downstream to the receiving systems, but it can be much better to implement message deduplication in the enterprise service bus (ESB). can implement duplicate message identification. When a message is received, ActiveMatrix BusinessWorks will query to see if the message has been processed before. The query can use any uniquely identifying information contained in the message. If the message has not been processed, ActiveMatrix BusinessWorks inserts a new row into. Locking strategies can be used to coordinate across multiple ActiveMatrix BusinessWorks engines, processing in an active-active configuration. SAVE MESSAGE INFORMATION Sometimes information must be retained and correlated across multiple messages. A transaction might generate several messages linked by a common transaction ID that should be processed as a group. can retain the information from the message group and correlate them as needed.

9 SOLUTION BRIEF 9 TIBCO BUSINESSWORKS ENHANCEMENTS IN-MEMORY DATA PERSISTENCE FOR BUSINESSWORKS By design, TIBCO ActiveMatrix BusinesWorks, our flagship integration platform, is stateless. It runs a process, and when complete, all of the information is flushed from its memory. This works fine for many use cases where it facilitates integration between systems. However, when it would be better for ActiveMatrix BusinessWorks to retain information across processes, the high-speed, easy-to-implement ActiveMatrix BusinessWorks plugin for fills the bill. BUSINESSWORKS ENGINE SYNCHRONIZATION When processing or state needs to be coordinated across multiple BusinessWorks engines, in-memory data-access provides a very fast solution. Before, the options for implementing this coordination included a database or ActiveMatrix BusinessWorks shared variables stored in a database. This approach was relatively slow and did not meet the performance requirements of many implementations. ActiveMatrix BusinessWorks plugin for is very simple and straightforward to implement. FAST DATA VISUALIZATION FAST DATA STORE FOR TIBCO SPOTFIRE An adapter for TIBCO Spotfire enables rich graphic visualizations and analysis of data contained in. SUMMARY TIBCO is a powerful and flexible distributed in-memory data processor that solves a variety of technical challenges. The use cases presented here are only a few of the many ways that it can deliver value by supercharging your infrastructure. Download at Global Headquarters 3307 Hillview Avenue Palo Alto, CA TEL FAX TIBCO Software Inc. is a global leader in infrastructure and business intelligence software. Whether it s optimizing inventory, cross-selling products, or averting crisis before it happens, TIBCO uniquely delivers the Two-Second Advantage the ability to capture the right information at the right time and act on it preemptively for a competitive advantage. With a broad mix of innovative products and services, customers around the world trust TIBCO as their strategic technology partner. Learn more about TIBCO at , TIBCO Software Inc. All rights reserved. TIBCO, the TIBCO logo, TIBCO Software, and ActiveMatrix BusinessWorks,, TIBCO BusinessEvents, and Spotfire are trademarks or registered trademarks of TIBCO Software Inc. in the United States and/or other countries. All other product and company names and trademarks in this document are the property of their respective owners and mentioned for identification purposes only. 03/10/15