The Need to Know: Telcos & Big Data Analytics

Size: px
Start display at page:

Download "The Need to Know: Telcos & Big Data Analytics"


1 White Paper The Need to Know: Telcos & Big Data Analytics Prepared by Graham Finnie Chief Analyst, Heavy Reading on behalf of October 2013

2 Executive Summary As revenue from traditional sources begins to stagnate or fall, communications service providers (CSPs) are looking for innovative ways to maintain margins in the face of ever-rising demand for bandwidth. They must find new ways to refine network planning and management to minimize costs, while at the same time engaging customers in new ways to increase user satisfaction, make a positive difference to the user experience, and arrest or reverse the decline in ARPU. As our survey work has shown, most CSPs understand the importance of this for their future, and are actively seeking solutions. Yet many are paralyzed by indecision, unable to quantify the impact of new service ideas or new traffic management measures. They know they need to act, but struggle to identify how. Since CSPs directly control less and less of the applications and content environment that their customers value, escaping this impasse becomes more and more urgent. The main thing holding them back, we believe, is not a lack of business data which they have in abundance but the ability to turn that data into meaningful business intelligence that they can act upon in a timely fashion, be it real-time or otherwise. Enter analytics, which in simple terms refers to the processing, synthesis and manipulation of raw data to enable in-depth modeling and predictive analysis, allowing useful conclusions to be drawn and actions initiated. Academic research has shown that effective use of analytics can improve both productivity and profitability, and because of their unique position at the heart of a sea of data on subscriber and network activity, CSPs are in a unique position to benefit from this. To do so, we believe CSPs must tread carefully, and consider what they want to achieve when they deploy analytics tools. First, and perhaps most important, we believe that analytics is likely to be a lot more effective if separate analytics activities in different CSP departments are linked at some level, closing the loop between customer behavior, network impact and service creation/adaptation. Ideally, the aim is to provide the best possible customer experience at the least possible cost an equation that requires that all information sources are being utilized and acted upon by all the key departments. In other words, the analytics platform must be able to acquire data from many sources (in principle, any usable data source) and be able to export data as a wide range of outputs and formats suited to the many possible kinds of users inside a CSP, as well as in some cases customers themselves. In this White Paper, we look at the challenges and catalysts for deploying analytics, and set out 16 attributes which we believe a CSP analytics package ought to include. As well as being able to aggregate, synthesize and filter data from a wide range of sources, we argue that it should collate and deliver usable data in real time wherever possible, on a per-subscriber basis, and with enough granularity to enable CSPs to achieve the greatest possible basis for differentiation. There should also, we argue, be a clear chain linking information (input) to action (output); functional separation of data from software logic; ability to aggregate and anonymize data to protect privacy; the ability to generate customized reports based on ad hoc questions; predictive capabilities; automation of action where possible; open characteristics that allow the package to be extended with third-party add-ons; and ability to scale to handle very high volumes of data. HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 2

3 Most of all, perhaps, CSPs need to engage all stakeholders in the specification of analytics requirements, since this necessarily cuts across all departmental boundaries, from networks, to marketing, to finance, to operations, to customer service. Only by taking a holistic approach that enables operators to segment the market more accurately can operators meet the challenges we set out at the start of this paper arresting the decline in ARPU and thereby protecting precious cash flow well into the future. HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 3

4 The Challenges Facing CSPs For some years, analysts have been predicting that conventional CSPs would eventually face difficult choices about the kinds of services they offered as traditional sources of revenue began to dry up. That moment is now arriving for many, forcing them to face up to a potential long-term decline in revenues unless they act to address the issue. At the same time, ever-rising demand for bandwidth is making it more difficult to cut costs: many CSPs are now investing heavily in new technologies FTTX in wire line, LTE in wireless precisely in order to meet that rising demand. Yet they have little control over the big trends that are driving up demand: New bigger, better, more attractive devices to use the Internet and its applications; More and more content to share, generated by professionals, users, and social networks; The need to be connected all the time, and wirelessly; and Data mobility, with data following users wherever they go. All of these trends are certain to continue: there is no road back. So far, however, CSPs have developed few ways to get a piece of the revenue created by these trends. This danger for CSPs is that they find themselves engaged in a race to the bottom, with ever-declining revenues and margins; this is a nearinevitable outcome in competitive markets with a limited set of commodity services. As Figure 1 shows, for example, in the highly competitive French market, Orange has seen mobile ARPU decline for 14 successive quarters. Figure 1: The Challenge Facing CSPs Source: Orange HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 4

5 To maintain cash flow, CSPs must both find new ways to refine network planning and traffic management in order to minimize costs, and engage with customers in new ways that increase satisfaction (and loyalty) and arrest or even reverse the decline in ARPU. Ideally, they must do both, linking measures designed to minimize bandwidth usage to those designed to maximize customer satisfaction and engagement, taking them beyond the current "engagement" model which consists of little more than delivery of a monthly bill or pre-paid top-up. However, this prescription runs counter to the way that CSPs have traditionally developed and deployed services. Most large CSPs have reacted to the shift to a Web-centric service universe by simply modifying the long-standing utility model a simple metered service which has few distinctive features. Customers choose on the basis of brand familiarity, service availability and price; quality and service differentiation are of relatively lower importance. Yet most CSPs do recognize that they must find new sources of revenue. The real challenge is that they are often paralyzed by indecision, and unable to construct a business case for the investment required that convinces all internal stakeholders. It costs little to come up with new service ideas, but beyond simple intuition, how do CSPs know that these new services will really justify the investment required to deploy them in the network? Moreover, it's often difficult to quantify the marginal increase in revenues due to introduced new service introduction. Revenues from these new services are often hidden inside monthly fixed income and the benefit is not clear. The question is how to escape this impasse and the gap analysis shows that it can only get worse. As shown in Figure 2, while it's relatively easy to address the "known knowns," there are plenty of "known unknowns," and these are growing because CSPs no longer control the applications and device environment. For the most part they react to that environment, slowly, after the fact. Is there a better way? In the next section, we consider this question. Figure 2: Into the Unknown KNOWN KNOWNS We know traffic will continue to grow quickly We know there will be a Next Big Thing We know most new apps will be created on the Web We know users will access services on a variety of devices and networks We know video will dominate traffic KNOWN UNKNOWNS We don't know how quickly it will grow We don't know what the Next Big Thing will be We don't know who will create them We don't know what devices and networks We don't know what kind of video Source: Heavy Reading HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 5

6 Enter Analytics In the first part of this paper, we suggested that CSPs need to move beyond traditional network planning techniques and simple utility pricing models if they are to prosper in the ever-changing Web services environment. Most CSPs recognize that need, but many have been unable to find a way to meet it. We believe that a key factor holding them back is a lack of information about what is going on in the network and what subscribers are doing. If they were properly equipped with an integrated set of tools to identify and understand both network and subscriber trends and needs, in real-time wherever necessary, CSPs would be able to move with greater confidence to try out new types of service offers, fix or drop those that are not delivering, and plan more effectively for emerging applications and traffic types. Heavy Reading research has shown that the need to better understand both subscriber behavior and network behavior is now an imperative in the minds of many CSPs; in survey work, for example, we found that these needs are driving plans for deploying policy management equipment. As Figure 3 shows, in a survey conducted in 2012, CSPs said that improving understanding of subscriber behavior and improving network reporting/analysis were the first and second most important catalysts for deploying policy management platforms. In fact, more than three quarters of the sample scoring these catalysts 4 or 5 on a 1-5 scale. Figure 3: The Need to Know Catalysts for Deploying Policy Management Solution Source: Heavy Reading HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 6

7 Translating this need into reality has created a mini-investment boom in what is generally known as analytics, with many vendors adding so-called analytics capabilities to their products, including policy management solutions. Although analytics has no precise definition, at a high level it refers to the manipulation of raw data to enable useful conclusions to be drawn, and actions initiated, in areas such as business performance management, benchmarking, and predictive forecasting. Analytics solutions may include: The ability to collect or create source data, A data warehouse that stores all the source data and aggregates it in a way it can be accessed, A reporting and graphic interface Interfaces to export information to other CSP business support systems. In short, analytics is a general word for projects that can be very large, require many vendors, are very expensive and may be very hard to economically justify, even before the planned new services are weighed for potential revenues and feasibility. Whatever the definition, there is no doubting the interest or the readiness to invest, and it's worth noting that this interest has some grounding in academic research. For example, in an influential April 2011 paper called "Strength in Numbers: How Does Data-Driven Decision-making Affect Firm Performance?," Erik Brynjolfsson, Professor of Management Science at MIT Sloan School of Management, wrote: "Our research has found a shift from using intuition toward using data and analytics in making decisions. This change has been accompanied by a measurable improvement in productivity and other performance measures. Specifically, a one-standard-deviation increase toward data and analytics was correlated with about a 5 to 6 percent improvement in productivity and a slightly larger increase in profitability in those same firms. The implication for companies is that by changing the way they make decisions, they're likely to be able to outperform competitors." 1 Other researchers have found even bigger improvements. For instance, McKinsey Global Institute estimated in 2011 that "a retailer using big data to the full has the potential to increase its operating margin by more than 60 percent." 2 The rising interest has had an impact on telecommunications CSPs because of the wealth of largely unused or underutilized data that could be exploited. Broadband network browsing can provide far more insight into customers than voice and SMS data records, but for the most part CSPs make little use of it today. For instance, in a report on use of mobile data services, Allot Communications showed that data traffic was highly heterogeneous, indicating a high potential for distinct segmentation. 3 By way of example, Allot identified four categories of users, characterized respectively by how much and how often they use social networks, video streaming, peer to peer, VoIP (Skype) and other apps. Related analysis of frontier_for_innovation 3 "Segmenting the Mobile Digital Lifestyle," Allot MobileTrends Report, Feb HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 7

8 the data in this study divided the user population into those who were "personally" active and those who were "socially" active. Finally, Allot used this information to create five "digital lifestyle segments," including Info Seekers, Info Guzzlers, Social Monitors, Social Minglers, and Digital Movers & Shakers. The general principles in this analysis are shown in Figure 4. Figure 4: Categorizing Subscribers Using Usage Data Source: Allot Communications; percent of subscribers (percent of traffic) Despite the obvious value of this information, few operators are yet using big data in this way to create new service offers. However, in several recent surveys, we have detected strong CSP interest in principle in analytics. For instance, many CSPs are interested in building analytics capabilities into existing policy management and RAN congestion control solutions, as Figure 5 indicates. In interviews, one CTO with a Tier 2 European CSP said: "There's a whole new product development cycle [in policy], built on BI [business intelligence] looking at BI, seeing there is a new product opportunity, and seeing it can be done and promoted via a policy portal. And then monitoring usage of that new product using the same system, and modifying if necessary." This feedback loop design-test-modify-design is at the heart of the new analytics, and as Figure 5 shows, the majority already saw it as an essential requirement in In the next section, we consider what a next-generation analytics capability needs to include in order to meet CSP strategic objectives. HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 8

9 Figure 5: CSPs Want Policy Server Suppliers to Have Subscriber Analytics Tools In-House? Source: Heavy Reading Survey of 106 CSP executives HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 9

10 What Kind of Analytics? In this section, we will look in more detail at the constituent parts or features of an analytics solution and evaluate their usefulness. At a basic level, "analytics" is already widely used by CSPs, in both network planning/monitoring and subscriber service development. However, this legacy software has some major shortcomings that limit its utility and value, and these limitations are becoming more obvious as the network and services environment becomes more complex and fast-moving. Analytics has in our view two main objectives: To provide CSPs with actionable information that enables them to understand network traffic patterns and plan networks better; and To enable CSPs to better understand customer behavior and create service plans better tuned to customer needs. These two aims are different and are likely to be of interest to different domains within a CSP organization. Moreover, conceptually speaking, analytics formally belongs in the IT domain, but many network-side activities, like subscriber data management, are closely linked to analytics. Getting data sources from the network, and creating data warehouses to aggregate and categorize the data in a form that can be used by finance, marketing, customer service, etc., means many departments of the CSP need to work together. These are new types of projects, and may create new kinds of challenges. For example, IT is frequently asked to deliver data that it either doesn't have, or can't provide in a format that is usable. Having said that, CSPs recognize that analytics needs to fulfill more than one function, as Figure 6 shows. Figure 6: CSPs Expect to Deploy Analytics for a Range of Use Cases Source: Heavy Reading Survey of 106 CSP executives, based on those who said they planned to use vendor analytics tools (55% of total) HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 10

11 Information Must Be High-Quality, Accessible & Comprehensive The first point to make here, therefore, is that analytics is likely to be more effective if separate analytics activities in different divisions are linked at some level, closing the loop between customer behavior, network impact and service creation/adaptation. Ideally the aim is to provide the best possible customer experience at the least possible cost an equation that requires that all information sources are being utilized and acted upon by all the key departments. In other words, the analytics platforms must be able to get data from many sources and export data to many sources. The second key point to make is that the success of a CSP analytics program depends heavily on the quality and accessibility of the data used for analysis. Ideally, data must be collected both from data plane elements (such as DPI appliances) control plane elements (such as Subscriber Data Management platforms), and business systems such as BSS. And crucially, this disparate data must be consolidated in a data warehouse in a form that is accessible to all key divisions inside the CSP. In anonymous interviews with CSPs conducted by Heavy Reading, we found that there was often a strong desire to pull together data from a wide range of sources into a single tool or dashboard. For instance, the CTO of a Tier 2 CSP with multiple national operating companies offered the following wish-list, which included examples from both network planning and service development: "We want better analytics tools that can help us to refine policy rules. But we don't want a lot of [different] tools for that, we want a customer experience approach a service-centric approach I get information from this tool, and then I've got a QoE dashboard. Then you can enable marketing people to create a new type of service. Yield management is one aspect of this. Another aspect is to test devices and see how they are performing across all networks end to end before we introduce them to customers. It can also be used for market segmentation. For instance, if the user has this kind of profile, offer him this kind of device. As well as realtime info, we need at least one month's history." Another operator, a large Tier 1 with many operating companies put it more simply: "We want as much information as possible for QoE. Our strategy summarized is to turn customer information into a competitive advantage." Bell Canada, speaking at another recent industry event, said that: "Customercentric experience requires unprecedented access to all dimensions of data, spread across the network." 1 [our emphasis] For many, achieving these goals requires linking network information to persubscriber information. For example, in a November 2012 presentation at Broadband Traffic Management in London, Cyrille Joffre, then-network CTO for Cable & Wireless Monaco & Islands region, said: "Across most MNOs, Network/NOC teams use tools which serve engineering, monitoring, and troubleshooting, performance and optimization activities but do not provide per subscriber usage & analytics." [bold in original] 1 Cassio Sampaio, User Data Convergence in Policy and Real Time Charging, Policy Control event, Berlin, April 2013 HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 11

12 Joffre's suggestions for the wide possibilities for analytics (Figure 7) give a good indication of both the power and the breadth of the new analytics checklist. Figure 7: One CSP View of Analytics Use Cases POSSIBLE USE CASES FOR ANALYTICS Top 10 busiest / least busy cells with signaling /data ratio, applications breakdown and best/worst device performances Detect (and manage) radio/device attacks and port scanning Detect (and manage) tethering (devices used as modems) Detect (and manage) cell congestion Provide customer care with updated subscriber information and graphical data flow (Indoor) optimization, cell densification and remote drive tests Monetize excess capacity: promotion, turbo boost, etc Monitor (inbound and outbound) roaming experience Support devices acceptance campaign Better tailor data offerings & incentivize data usage PCC integration to dynamically amend/enrich policy rules Build service / customer centric QoE w/ control+l7 data metrics PRIMARY RECIPIENT All Operations Operations Operations Operations Engineering Sales Sales Marketing Marketing All All Source: Cable & Wireless Communications presentation Figure 8 shows the different departmental information domains and their interests in a more generic form. Figure 8: Information Domains DOMAIN OWNED INPUTS KEY INFORMATION (EXAMPLES) Networks DPI, video optimization, policy servers, HSS/HLR Congestion trends, applications trends Operations Network probes, end-user devices Security threats, congestion events BSS Billing & charging Per-package or service element usage trends, spending Customer Care CRM Per-sub QoE, subscriber value Product Development Subscriber Delivery Platform Customer behavioral trends, usage trends per app Source: Heavy Reading HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 12

13 As CSPs shift to a cloud-based everything-as-a-service (XaaS) model and LTE over the next few years, addressing this issue will become more urgent: The shift to cloud services argues even more strongly for a holistic approach to analytics. As we noted in a recent Heavy Reading report, analytics must "provide the tools to leverage distributed intelligence and profile data to meet specific XaaS use cases and the more dynamic and video-based services that LTE will enable " 1 In our interviews, other more specific requirements came to the fore. For example, there is an increasing desire for analytics to operate in real time or near-real time: In the Web era, the half-life of information gets ever-shorter, and the old data analytics cycle in which data was warehoused and took weeks to be utilized is no longer fit for this purpose. One of our interviewees, working for a large Tier 1 operator, put it this way: "To do [what we need], you need real-time network analytics, not just holding the rules, but asking, what is the network condition right now at this location? We don't have a clear view on how to do that, we are tip-toeing in." As Figure 9 shows, a great deal of the analytics information that provides the context for decision-making is essentially real-time in nature, and non-real-time information of this kind would be of limited or no utility. Figure 9: Information Needs to Be Real-Time Source: Heavy Reading 1 "SDM & Analytics: Reshaping the Services Landscape of Man & Machine": HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 13

14 An example based on a yield management approach might help to make things clearer: Network department identifies cells that have persistently low data usage (non-real-time analysis) or cells that have low usage right now (real time) Marketing identifies subscribers who might have a propensity to purchase data services Product development creates promotional "free" data service that can be used in under-utilized cells Network department identifies that target subscribers are in those cells Sales & marketing promotes new "free data" service and converts leads Network department re-examines cells to see if the new service is having an impact CRM checks satisfaction levels, impact on churn, etc. Product development refines/extends/drops free data service promotion based on this feedback HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 14

15 The New Analytics: Key Characteristics Taking these emerging requirements into account, an ideal analytics package should in our view have most or all of the following core characteristics: Ability to aggregate, synthesize and filter data from a wide range of sources [in principle, any relevant data source], including (for example) DPI, optimization software, network probes, end-user devices, HLR/HSS, OCS, etc. Intelligence devices [e.g., in 4G] will clearly play a beneficial role by enabling network CSPs to gather additional and more granular subscriber usage pattern data. A clear chain linking information [input] to action [output]. This is easy to describe, but hard to achieve. It requires that input information is formatted, filtered and presented as output when and where it is needed, throughout the organization, in a format that is usable by its intended target audience (including non-technical staff and end users). A clear chain linking networking monitoring, subscriber behavior and service creation. Although many CSPs have existing analytics capabilities in Network, IT and Product Development departments it rarely works as one. Yet the gains from doing so are likely to be substantial. Functional separation of data from software logic effectively using "backend" (BE) databases and "front-end" (FE) application service logic; this improves flexibility in building and using analytics systems. Sufficient level of detail and granularity (e.g., on applications, their use, who is using them, impact and trajectory, etc.) to that allow for the greatest possible competitive differentiation and/or greater precision in capacity planning. Information on a per-subscriber basis wherever relevant or necessary, as long as this does not violate the anonymity rule, below. This requires that analytics systems include data drawn from a wide range of sources and importantly, it means providing subscribers themselves with a subset of this data, letting them make their own decisions on what options to choose. Ability to collect, collate and deliver data in real-time or near-real-time wherever required, again in a format suitable to those who need it. Accurate identification of as much OTT applications/content as possible. Since nearly all the applications subscribers use are provided by thirdparties, it is critically important that CSPs understand trends in OTT content and applications. Ability to provide data in an aggregated, anonymized form both internally and to third parties, to increase the potential market reach of the information and to guard against privacy concerns. Ability to generate customized reports in-house in flexible formats, and to compare any variables of interest (i.e., not be dependent on vendors for pre-created and/or customized templates), and be capable of handling new requirements (e.g., M2M analysis). Equally, front-end dashboards that provide information in a form usable by marketing and other non-technical employees. Ability to conduct do-it-yourself analysis and create new reports based on ad hoc questions and not just pre-determined reports. HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 15

16 Predictive capabilities extrapolating trends in detail, and providing fast feedback on service ideas and hypotheses; ability to anticipate customer needs/experiences and proactively act upon them. Automation of action wherever feasible, including the automated delivery of options and choices that enable end users to choose new services or services features. "Open" characteristics that allow the analytics engine to draw on and connect to information/software from different vendors. Indefinite ability to scale and handle very high volumes. Among other things, this requires a very high specification database technology which will necessarily come from an IT vendor, not a network equipment vendor. These characteristics are summarized in brief in Figure 10. Figure 10: Core Characteristics of Next-Gen Analytics Platform Source: Heavy Reading HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 16

17 Best Practice in Analytics At the CTIA Executive Round Table at New Orleans in May 2012, Verizon CTO Tony Melone set out the core strategic aims of a big CSP in the following way: "We have to extract value from the destination, not just the consumer. And we're getting a lot better at extracting information from the network, and from the context of what customers are doing in order to create different business models." The penny has finally dropped with major CSPs that no "killer app" is going to emerge to save the CSPs. Instead, they must aim to provide and maintain a flexible service delivery environment that is highly personalized and configurable by each individual user. And to do that they need a new kind of analytics. The question is how best to achieve that. Drawing on the information already presented, here are seven principles that ought to inform decision-making in this area. 1. Engage all stakeholders in the specification of analytics requirements. Analytics cuts across every departmental boundary, complicating decision-making but without cross-divisional buy-in, the impact of any deployed solution will necessarily be limited 2. Ensure that it's feasible to "close the loop" between network-side information primarily used for network planning and subscriber-side information primarily used for service planning. Connecting the two ensures that both the cost and revenue sides are addressed 3. Specify a system that is flexible enough to handle diverse inputs and capable of extension to include new data sources and new kinds of actionable outputs. This means above all a system that is open and modular. 4. Equally, specify a system in which custom reports can be created to answer ad hoc questions without the need for software upgrade or development. Systems that depend on pre-created templates will have limited value in a fast-changing service environment 5. Ensure that the system can provide information in sufficiently granular detail, per-subscriber, to enable the CSP to accurately segment its markets and truly differentiate its services 6. Aim to achieve end to end reach, in particular by ensuring that wherever feasible end users and end user devices are part of the analytics chain, able to take action and make their own decisions 7. Ensure that information on network conditions and similar inputs can be collected and utilized in real-time or near real-time; not all analytics needs to be performed real-time, but it is an increasing need because of the speed of change in network behavior. HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 17

18 An Industry View From Allot Communications Allot Communications knows that just as the accuracy and granularity of monitoring data directly affects the quality and level of policy enforcement in the network so too the accuracy and granularity of source data gathered from the network affects the efficacy of any analytics effort. Numerous times, we have heard the plea for better more detailed more precise data from analytics consultants hired by the network operator. In short, source data matters. That's why Allot ClearSee Analytics focuses on providing a rich data source along with a powerful analytics system that enables different stakeholders in the operator's organization to analyze data in terms that are relevant to their domain, easy to understand, and lead to fast results. Turning Big Data into Usable Data As noted earlier in this paper, network operators do not lack business data. But the raw data they generate is big (voluminous). It comes in fast (velocity). And it has lots of different details and attributes (varied). They need to turn their big data into "usable" data. Creating a rich data source begins by leveraging the DPI technology placed inline in the operator's data network. DPI, along with control-layer systems, allows service providers to capture a rich variety of application, subscriber, and device and QoE data from the network. The goal is to transform this raw data into modeled or "usable" data stored in a purpose-built data warehouse and accessible to all stakeholders. Modeled data is typically represented in the form of intuitive business objects that are easily understood and manipulated via an analytics application. As a data source provider, Allot Communications gathers raw data from the network and processes it into a full complement of accurate raw data records detailing the usage statistics for HTTP VoIP, IM, session, subscriber, and policy activity, as well as conversation flows and more. At frequent intervals (which can be as near to real-time as the operator wishes), these raw data records are loaded into Allot's big-data warehouse located at the customer premises. This is the stage where raw data is extracted, loaded and transformed into easily accessible and understood modeled data. Various calculations on the data provide valuable metrics while optimization algorithms enable large volumes of data to be stored and easily retrieved. The open architecture of Allot's data warehouse is also designed to accept and load data feeds from other operator systems such as CRM, ERP, OCS, OSS, BSS, and others, adding new dimensions to the data that can all be analyzed together. As modeled data is loaded into Allot's data warehouse, it becomes immediately available for analysis and reporting via the Allot ClearSee Analytics application. Modeled data can also be exported to external analytics or business systems using standard file formats. Vendors such as Allot Communications understand that network operators may be in different stages of big-data projects. Some seek better source data. Many want better analytics tools. Most want both. Therefore, Allot ClearSee offers a totally modular and scalable solution for network business intelligence. HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 18

19 Figure 11: Allot Analytics Solution Source: Allot Communications Analytics From Data to Business Intelligence Allot ClearSee Analytics turns big data into meaningful business intelligence for the decision-makers in service provider organizations. This analytics toolkit lets you manipulate and process large varieties and volumes of source data with extreme efficiency. The tools also cater to network operators who may have different requirements of their analytics projects. The application includes a full complement of pre-defined reports and dashboards for analyzing data within specific domains, i.e., network, application, subscriber, device and quality of experience. The dashboards can be customized and extended as needed. Preset reports are a good way to get started in big-data projects without getting overwhelmed by the data or how to present it. The real breakthrough comes in the product's "Self-Service" module, which is somewhat like a "see-it-your-way" analytics tool. Operators aren't limited to the pre-defined set of reports or a specific way to analyze data. As new requirements arise, or new data presents itself, the Self Service approach enables all stakeholders to model their questions and find answers to help solve complex business problems. Self-Service is an interactive analytics tool. Its rich variety of data objects along with their multiple metrics and attributes are easily dragged and dropped into a visual workspace that provides instant feedback. Moreover, big data from multiple departments can be integrated to gain different perspectives and greater accuracy. HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 19

20 While Allot claims that the Self-Service module is easy enough for anyone to use, it should also be viewed as an excellent tool for experienced analysts who for the first time will have access to source data that is more varied, detailed and accurate than ever before. These professionals will be able to make custom, ad hoc queries without having to go back to the vendor for software development or upgrades. This approach helps the operator find answers to specific questions, explore possible courses of action, and often discover problems and opportunities that were not anticipated. HEAVY READING OCTOBER 2013 WHITE PAPER THE NEED TO KNOW: TELCOS & BIG DATA ANALYTICS 20