TeraScale. Amdocs White Paper January 2015

Transcription

1 TeraScale Amdocs White Paper January 2015

2 TERASCALE 2 Contents Introduction...3 Why is Big Data a Big Deal?...4 The Challenges...4 The Opportunities...5 The Challenges...6 Big Data and Analytics: A State of the Art...7 NoSQL...8 Analytics...8 Constraints...8 Is Big Data Telecom-ready?...9 Amdocs Customer Experience Solution...10 Overview...10 TeraScale Makes Big Data Telecom-grade...11 Conclusion...13

3 TERASCALE 3 Introduction The world in which we live in today is flooded with data that is being produced and consumed by billions of endpoints. Broadband access, emerging smart phones and tablets, new data services, mobile applications and cloud computing together with a seemingly infinite number of connected devices has created an unprecedented scale of data usage in telecommunication networks. Mobile and fixed data service consumption is already an increasingly growing commodity in subscribers lives; all market analysts agree on the inevitable exponential growth of exchanged data volumes that telecom operators will have to process. Service providers are currently facing a multi-dimensional challenge. On the one hand, they understand the strong potential of the insights that can be extracted from such an explosion of data within their billing and charging systems. These insights will result in additional revenue through improved market positioning, new offerings and better management of customer experience and loyalty. Providers are also aware that this goes beyond traditional products that are being offered and managed by their charging, billing and customer management systems. Moreover, they see the need for a strategic move towards analytics-driven evolution, including real-time analytics. On the other hand, however, service providers are well aware that this large amount of data must still be managed with carrier-grade quality a prerequisite for their customers with the efficiency and cost control that customers have come to expect. In this context, Big Data is seen as the key trend that will help service providers facing these challenges and contribute to monetizing the related opportunities. Big Data is commonly addressed through three dimensions the 3 V s : Volume of data, which is continuously increasing Variety of data in terms of sources and formats Velocity of data, which is the speed in which data is captured, processed and analyzed To these, some players add an important business-related dimension of Value, which includes the increasing business potential that service providers receive from the data. Big Data-related technologies are widely accepted and adopted by many industries. Moreover, main storage and data analytics solution suppliers, such as IBM, EMC, and Cloudera, are already positioning their Big Data products in the market; however, since the Big Data solutions that are currently available are only enterprise-grade and basically designed for business intelligence and heavy analytics of historical data, the question that must be answered is: Is Big Data, today, telecom-ready?.

4 TERASCALE 4 What, in fact, are the principal gaps that have to be addressed to prepare Big Data as a driver that can, simultaneously, enable service providers to effectively manage the growing volume and variety data, control costs, and confidently enter the vast ocean of new real-time analytics-based offerings in terms of Velocity and Value and data-driven decision making? When examining the current market and the products of key suppliers, one quickly sees that there are, in fact, considerable gaps. Existing product roadmaps focus on enterprises, and give little attention to user experience, loyalty, and data monetization. Amdocs plays a key role in addressing these critical gaps by adopting Big Data technologies while, at the same time, introducing cost-effective operability capabilities and providing suitable platforms and tools to meet the need for the analytics-related applications required by service providers. Amdocs CES is one of the early adopters of Big Data. Currently partnering with major Big Data solution vendors, Amdocs has built within its CES portfolio a telecom-grade Big Data platform, TeraScale, which is aimed at bringing service providers to the next level of cost-effectively managing data growth and leveraging high volume and increasing velocity to drive the business and to quickly create innovative offers well adapted to customer needs and experience. Amdocs TeraScale enables service providers efficiently transition to telecom-grade Big Data at any point and within a comprehensive migration path. Why is Big Data a Big Deal? The Challenges Big Data! is a buzzword that you surely have already heard! What, though, does it really mean, and how can telecom service providers leverage it to face the inevitable explosion of? How will Big Data help providers increase profit, stay competitive, and control costs? Is there a possibility that service providers might very well find themselves lagging well behind the market if they do not adopt Big Data-related technologies? These are the key questions addressed in this white paper. In addition, this paper discusses the main challenges faced by the industry in bringing Big Data into the telecom world, and, finally, how Amdocs CES adopts Big Data and creates from it an essential engine for growth and efficiency. Big Data is, in fact, an emerging trend in data behavior, defined as data sets that grow so large that they become difficult to work with using traditional RDBMS (Relational Database Management System) technologies and related management tools. This trend is typically characterized by an increase within the three dimensions, described earlier Volume, Variety, and Velocity.

5 TERASCALE 5 Challenges can be found at all levels of data processing, including capture, formatting, storage, analysis, sharing, search, queries, and visualization. Non-relational data models tend to be adopted for the management of Big Data because they can be easily scaled using low-cost commodity hardware, and tend to provide better performance, especially for write-intensive applications. In addition to this economical aspect, a key advantage of Big Data is its strong potential to drive business decisions and market offerings, given the valuable insights that often result from analysis of large amounts of data. By leveraging the economy of scale offered by nonrelational models, therefore, service providers can retain data for longer periods of time and increase the potential of its monetization. Hadoop ecosystem, which is a software framework that supports data-intensive distributed applications. HBase offers superior throughput, low latency, linear scalability, and high availability, all of which revolutionize data management costs. While suppliers, such as IBM, EMC, Intel, and Cloudera, are introducing NoSQL database solutions that are predominantly built on top of HBase, other vendors, such as HP, Microsoft, and SAP, specialize in analytical tools and processing of data streams. It is important to note that the vendors mentioned above are focusing on generic Big Data solutions to fit large industry segments. This leaves considerable gaps in terms of a specific service provider s needs that correspond to their environments and market dynamics. One of the main trends of non-relational models is NoSQL Not Only SQL databases that are not built primarily on tables, and generally do not use SQL for data manipulation. Big Data technologies cover both database software and analytics tools such as pattern recognition, anomaly detection, predictive modeling, and etc. HBase is one of the best-known NoSQL databases suitable for Big Data. It is a widely deployed, non-relational, distributed database (inspired by Google s BigTable), and is part of the Apache The Opportunities From the perspective of business usage, Big Data is viewed as a data-driven decision making tool that is used mainly for analytics applications. And it is. Marketing departments of various industries use Big Data to analyze customer behavior and sales and to extract from the data insights that can be used later to fine-tune existing products or define new products/packages as a means of becoming more competitive in the market.

6 TERASCALE 6 Traditionally, telecom service providers have dealt with large amounts of data for BSS and customer management activities. This data has always been managed in a carriergrade environment due to its criticality; service providers, therefore, make substantial investments to ensure high availability, guarantee data integrity, prevent data loss, enhance the manageability of the data infrastructure, and ensure fast responses to business requests. Historical data, on the other hand, is generally transferred to data warehouses for archiving and for further analytics. Today, service providers are experiencing two important paradigm shifts: the first is the explosion of usage data processed by the service provider, and the second is the criticality of analytics applications to the service provider s business. Analytics applications are no longer a complementary set of features to the operator s business; these applications have, in fact, become drivers for an enhanced user experience and to introduce revenue generating sources. Consequently, telecom operators need not only to manage properly an exponentially increasing amount of data but must also use this data in a well-organized manner to drive the business in terms of strategic moves, user experience, loyalty and new offerings. The Challenges The first challenge facing service providers is the fact that current Big Data technologies are not designed to be telecom-grade and cannot simply or seamlessly replace existing data infrastructures. This is without mentioning the technical challenges related to the NoSQL nature of Big Data. Operability and non-functional requirements, such as stability and performance essential to billing, charging and customer management are not covered by existing Big Data technologies, a serious problem that must take into consideration cost constraints. The second challenge is related to the nature of the analytics applications required in telecoms. Many of these applications require near real-time feedback from existing billing and charging systems. In addition to the ability to comprehensively analyze historical data required for making predictions and to take strategic decisions for better subscriber loyalty and experience enhancements, telecom operators, to remain competitive, must be able to quickly respond to short term situations by being responsive to customer aspirations and expectations. This poses the problem of what type of architecture the operator must consider to enable robust market responsiveness. Current Big Data solutions are far from providing clear answers to these important questions; they focus more on how to 1) host the largest amount of data on COTS hardware and storage and 2) optimize analytics applications.

7 TERASCALE 7 In this whitepaper we focus on the main question: How to make Big Data, Telecom ready?. We describe, in response, the main challenges that were already highlighted, review the expected benefits and present the solution that was adopted in the Amdocs Customer Experience Solution portfolio. For Big Data, the objective of new technologies such as NoSQL are to remove all constraints that traditional databases may encounter when dealing with large amounts of unstructured data. The main features and attributes of such solutions include: Big Data and Analytics: A State of the Art The most accepted technologies in the area of Big Data are grouped under the name of NoSQL, and share a common set of properties in terms of the key capabilities that they offer. NoSQL technology, for example, is non-relational as opposed to RDBMS (Relational Database Management Systems), which is the predominant technology for storing structured data that has been used in most businesses throughout the last thirty years. The NoSQL trend began in 2009 as a serious effort to replace the heavy and expensive relational databases with less expensive and more robust way of storing and managing data. The motivation behind this effort was the fact that Web 2.0 startups were already building their own data stores. These startups, moreover, were being increasingly driven by major players who, for years, were managing very large amounts of data using their own technologies; for example, Amazon s Dynamo and Google s BigTable. This trend was confirmed by such cloud and social network players as Cassandra for Facebook that also adopted a NoSQL approach. Reduction in data requirements, such as ACID, which increases the complexity in the management of large, dynamic data sets. High throughput, mainly in terms of write operations, the main objective of which is to avoid bottlenecks on the data collection side. Usage of commodity hardware and storage, which may involve sacrificing a degree of reliability that can be overcome by distributed replication. Horizontal Scalability, a key factor for cost control given that there is no indication that the growing increase of data that needs to be processed will be slowing down. Distribution of data storage and processing required for running parallel jobs for complex queries. Open source, which is regarded as the trend that provides the most flexibility for this solution.

8 TERASCALE 8 NoSQL The most adopted NoSQL technology is Apache open source HBase, which is built on top of an open source Hadoop framework (also from Apache). Hadoop is a framework for storing and processing large amounts of data across clusters of machines. It provides HDFS, which is a distributed file system used for reliable data storage. The processing framework provides the capability of executing a Map Reduce algorithm to process queries in parallel across multiple clusters. For reliability, data is replicated in a distributed manner. HBase is column-based and has been designed for random, real-time read/write access to data. It ensures linear scalability and offers reliably consistent reads and writes. HBase has adequate classes for backing up Hadoop Map Reduce jobs with Apache HBase tables, and exposes a Java API for client access. It also provides automatic table sharing and failover between Region Servers. HBase appears to be the most suitable for the kinds of usage data records processed in the telecommunications industry. Analytics The objective of these tools is to enable large set of analytics applications depending on the addressed segment. For the telecommunications industry, current traditional data analytics cover a set of known areas, including but not limited to churn management, subscriber profiling, proactive network monitoring, capacity planning, revenue insurance and personalized advertising. Virtually all these activities are based on legacy analysis of historical data. Constraints In addition to the vendor and technology side of the picture, there are several main constraints and factors that drive the evolution of Big Data, in particular those that impact service providers business operations. The first obvious limitation is Big Data s focus on enterprise business. This focus provides an enterprise-level quality that is not necessarily suitable for telcos. In addition, there are other dynamic parameters that demand a lot of attention not only for building a telecom-grade Big Data solution but also to make it sustainable for the future. Many tools used for Big Data are being adopted either by suppliers of Big Data storage as part of their portfolio or by new ISVs as separate solutions for advanced analytics. These tools encompass data capture, analysis and visualization based on data processing algorithms such as classification, data fusion, pattern recognition and predictive modeling.

9 TERASCALE 9 Consequently, with the exponential increase of data, other factors are becoming more important. In fact, we see a lot of challenges related to the continuous growth of data facing current Big Data vendors, including the need for high availability, manageability, short latency, stability and the ability to support heavy processing applications. Integration with billing, charging and customer management that results in value and sustainable business cases is another important issue not addressed by Big Data vendors and is a key factor to making Big Data investment profitable within a telecom environment. Is Big Data Telecom-ready? The problem being addressed here is how to enable service providers to leverage Big Data technologies in the telecom environment. As explained earlier, a storage solution from the main vendors, even combined with analytics, will not meet the providers current needs due to existing gaps. In this section we discuss those gaps in greater detail and attempt to make clear the complexity of the problems we are facing. Service providers envision that Big Data provides the ability to: Scale and effectively manage growing data volumes Because users are spending greater amounts of time consuming data and because an increasingly large number of services depend on the data itself (both new and old), the need for high availability of data is becoming critical. High performance and low latency, in conjunction with operability and management functions, are key factors. This is what service providers need to focus to achieve maximum scalability and effective data volume management. Today, HBase vendors cannot meet all these requirements with their out-of-the-box products. Consequently, the role played by BSS vendors is crucial since scalability and effective data volume management is dependent on how the BSS vendor adopts and integrates with the HBase product. Cost control is another vital issue since service providers are currently managing data with traditional RDBMS. The shift currently being taken raises clear questions regarding target architecture, including: Will the shift involve a complete transformation of data management? Is the configuration needed side-by side Is there any migration path? Control the cost of capital investment and operations Manage their business and create new services based on analytics applications

10 TERASCALE 10 These issues need to be addressed in order to achieve the second goal of controlling the cost of capital investment and operations. According to an Amdocs analysis, the cost of processing one million events in peak time is constantly dropping (at an average of 17% YoY) [EOS] as a result of such IT technological advances as stronger CPU processing power and a decline in costs associated with storage. Nevertheless, these technological advances are not enough to offset the required expansion in charging systems due to the exponential growth of network-generated events. The figure below illustrates our analysis, in which we predict, based on technological progress and the Amdocs Convergent Charging technology roadmap, a decline in the total hardware and software costs required to process the forecasted event inflow. The analysis indicates that Amdocs Convergent Charging is geared to support the predicted exponential growth while the required hardware and software costs will drop by 28% YoY. For the third goal managing the business and creating new services based on analytics applications the direction here is not to focus strictly on the analytics of archived information with heavy algorithms but, rather, to apply a combined approach where insights from both current and old data can be used, in real-time, to create value. Using the same system, service providers should be able to perform realtime analytics for immediate needs as well as perform the kinds of complex analytics required for driving the business toward the right direction. In the remaining part of this whitepaper, we show how Amdocs, as the market leader of BSS solutions (Convergent Charging, Billing and Customer Management), solves these problems and allows service providers to comfortably reach the three goals. Amdocs Customer Experience Solution Overview Amdocs Customer Experience Systems (CES) is a modular, integrated software and services portfolio that enables service providers to take advantage of new business opportunities faster, differentiate through real-time customer experiences and streamline and improve operations. The Amdocs CES platform enables service providers to bridge the gap between the network and the end user with solutions that cover customer management, digital services, revenue management, operations support systems (OSS) and network control, based on a common unified foundation. Amdocs Revenue Management is the key CES module, offering charging and billing, mediation and partner management solutions. It enables service providers to quickly create and manage complex billing, charging and collection processes to offer their customers innovative and personalized pricing models and payment services that are simple and intuitive.

11 TERASCALE 11 Within Revenue Management, Convergent Charging is the central system used for processing usage events and generating and storing rated events into a RDBMS-based usage database. It is here where the transformation for Big Data adoption starts, providing the foundation upon which all other Revenue Management and CES systems can migrate to Big Data. TeraScale Makes Big Data Telecom-grade The Amdocs solution for tackling the problem of how to enable service providers to leverage Big Data technologies in telecom environments is named TeraScale. This new architecture, at core of the Amdocs product portfolio and based on HBase technology, allows service providers to achieve the three goals discussed above. In the following sections we explain how, for each of the goals, TeraScale solves the associated problem. Ability to Scale and Effectively Manage Growing Data Volumes Since TeraScale is dependent on NoSQL solution providers, Amdocs, after a thorough analysis of offers available in the market, decided to adopt HBase as the core technology and to partner with the top suppliers of HBase systems. The partnership is driven by the ability of HBase suppliers to comply with Amdocs requirements in terms of operability, performance, integration, migration, scalability and pricing. To achieve this goal, Amdocs provides, on top of HBase a set of features that makes TeraScale operability compliant with the needs of service providers and ensures that the HBase partner s product is aligned with these features. It also adopts HBase into an architecture that provides the required performance, responsiveness and stability while minimizing the total cost of ownership and providing the means to control operational expenses. Amdocs TeraScale encompasses the following list of capabilities: Monitoring, with telecom-specific monitoring interfaces and qualities Fault Management, including troubleshooting and alarm management Performance Management, including the possibility to create customized performance sessions Lifecycle Management and Control operations Backup and Archiving, which are essential for disaster recovery and archiving of historic data Security management, including data protection and privacy

12 TERASCALE 12 Maintenance and Control Operations to achieve system manageability and safety Resource clean-up based on standard telco data models such as billing cycles Upgradability, ensuring that frequent upgrades will not cause interference or interruptions Configuration and Installation Logging and Tracing, including such telco-specific requirements as Subscriber Tracer In addition to the operability requirements, TeraScale has been designed to meet the performance, availability, stability and latency expected by telecom environments. To achieve this, Amdocs first defined architecture that allows the support of such capabilities, taking into consideration HBase s limitations. In addition, stringent performance and stability test criteria has been applied to all the selected HBase partners to ensure that the required KPIs can be met. Ability to Control the Cost of Capital Investment and Operations The solution adopted by Amdocs in the area of capital investments and operations is to reach a stage where data management systems already used for regular charging, billing and customer management functions can be consolidated with data management intended to be used for analytics. In this way, service providers enjoy significant cost reductions in both infrastructure and operations. The second solution is to disconnect real-time read/write flows and related high availability from HBase I/O flows via a write-behind approach, which ensures asynchronous coupling. This is done using Amdocs Elastic OCS [EOC], which is based on IMDG (In-Memory Data Grid) technology. Online charging latency and high availability KPIs, therefore, remain untouched and independent from the load on HBase. Such an approach allows for important savings in terms of HBase infrastructure. The third solution is related to the TeraScale migration path designed by Amdocs, which ensures a smooth transition to Big Data. The migration path brings value throughout all the stages and takes into consideration factors related to service provider pedigree adoption and the learning curve required of emerging new technologies.

13 TERASCALE 13 Ability to Manage their Business and Create New Services Based on Analytics Applications The architecture Amdocs has adopted for TeraScale aims to consolidate all data used by billing and charging, customer management and analytics applications in a single place. However, another significant aspect of the architecture is that insights uncovered by analytics applications can be fed, in real time, into online charging for innovative services related to real time interactions and customer experience enhancement. Amdocs is working with its main partners in the area of analytics applications and is building a roadmap of competitive edge analytic-driven services and processes, all of which are based on the same TeraScale platform and tightly integrated into convergent charging, billing and customer management systems. By achieving the three stated goals, service providers will profit from a number of highly valuable benefits: Service providers will no longer have to be afraid of being overloaded with increasing amounts of usage data. Instead, they will have full control of the exponential trend. Conclusion The worldwide exponential explosion of data usage is creating for service providers a major challenge and an incredible opportunity for growth. The challenge is to maintain control over the cost and quality of data management despite the extremely large volumes that need to be processed. We now have an opportunity to use this explosion of data as a driver for business decision making and as a means of generating additional sources of revenue based on analytics applications. While Big Data technologies appear to provide solutions for such challenges, for the telecom industry, many gaps need to be addressed by BSS suppliers in order to introduce Big Data as a telecom grade solution and a common data foundation for BSS and analytics. Amdocs TeraScale, at the heart of Amdocs CES, bridges these gaps and enables service providers to immediately begin a smooth transition to Big Data and analytics based upon a comprehensive migration path. Costs will be controlled through the usage of the best available technologies and the most suitable architecture. An intensive set of analytics applications and processes can be built to increase revenue and better manage the business based on data-driven decision making.

14 About Amdocs For more than 30 years, Amdocs has ensured service providers success and embraced their biggest challenges. To win in the connected world, service providers rely on Amdocs to simplify the customer experience, harness the data explosion, stay ahead with new services and improve operational efficiency. The global company uniquely combines a market-leading BSS, OSS and network control and optimization product portfolio with value-driven professional services and managed services operations. With revenue of $3.6 billion in fiscal 2014, Amdocs and its more than 22,000 employees serve customers in over 80 countries. Amdocs: Embrace Challenge, Experience Success. For more information, visit Amdocs at Amdocs has offices, development and support centers worldwide, including sites in: THE AMERICAS: ASIA PACIFIC: EUROPE, MIDDLE EAST & AFRICA: BRAZIL AUSTRALIA AUSTRIA ISRAEL SPAIN CANADA CHINA CYPRUS KAZAKHSTAN SWEDEN COSTA RICA INDIA CZECH REPUBLIC THE NETHERLANDS UNITED KINGDOM MEXICO JAPAN FRANCE POLAND UNITED ARAB EMIRATES - DUBAI UNITED STATES PHILIPPINES GERMANY RUSSIA SINGAPORE IRELAND SOUTH AFRICA TAIWAN THAILAND VIETNAM For the most up-to-date contact information for all Amdocs offices worldwide, please visit our website at Copyright 2015 Amdocs. All Rights Reserved. Reproduction or distribution other than for intended purposes is prohibited, without the prior written consent of Amdocs. Amdocs owns or has rights to use trademarks or trade names in conjunction with the sale of our products and services, including, without limitation, each of the following: Amdocs, Bridgewater Systems, ChangingWorlds, Clarify, Cramer, CES, Collabrent, DST Innovis, Ensemble, Enabler, Intelecable, Intentional Customer Experience, JacobsRimell, jnetx, MX Telecom, OpenMarket, Qpass, SigValue, Streamezzo, Stibo Graphic Software, Xacct, Aging in Place, Embrace Challenge, Experience Success, and Quality Consumption.