G-Cloud Service Definition Atos Big Data Strategy & Roadmap SCS
Atos Big Data Strategy & Roadmap SCS Atos Big Data Strategy & Roadmap Services combine the knowledge and expertise in the area of Big Data to support customers in defining their Big Data Strategy and Roadmap. Building a robust Big Data Strategy and Roadmap enables customers to drive better business performance by harnessing and deriving useful business insights from the vast amount of captured data, both structured and unstructured, which previously may not have been exploited for technological reasons. Successful Big Data Strategies and Roadmaps align with the goals of the business, enabling strategic, tactical, and operational decision-making. Atos believes that a Big Data Strategy and Roadmap should include consideration for a wide set of processes, technologies, and stakeholders in order to access, collect, and analyse information. The final version of Big Data Strategy should be constantly tuned and adjusted to reflect the needs of your business aligning with your overall IT strategy and Business goals. This service has been designed to be delivered agnostic of the underlying platform, and is suitable therefore for consumption by customers currently deploying public cloud, private cloud, hybrid cloud or dedicated infrastructure configurations, and by those moving towards a more cloud-based deployment. Benefits The amount of data in the present-day world has exploded. Today we get information not only from traditional (structured) data but also from a variety of semi-structured or unstructured sources, both internal and external to your business. Analysing these large data sets, identifying problems and recognising patterns is essential in order to achieve an advantage over your competitors. It also helps in the identification of new productivity, stimulating growth and innovation. The deployment of a Big Data Solution will help you to achieve the following business benefits: Collect more accurate and detailed performance information, expose variability and boost performance Unlock significant value by making information transparent and usable at much higher frequency Use collection of vast amounts of data and sophisticated predictive analysis techniques on that data to significantly improve management decisions Use collected data for basic low-frequency forecasting to high-frequency nowcasting to adjust business levers just in time Create ever-narrower segmentation of customers, allowing much more precisely tailored products or services Improve the development of the next generation of products and services ii
Establish frameworks, governance, methodologies, systems and technology to deliver improvements and value aligned with the business goals Improve focus for the business on long-term planning Improve knowledge management, allowing the best use of information Identify the right combination of people, processes, and technology. Service summary What is it? The service will help to define the Big Data IT strategy and will provide a Big Data solution architecture and a high-level implementation roadmap to support that strategy. The service will assess how closely aligned your current IT strategy is with the requirements of Big Data, and identify concrete steps to incorporating Big Data into your strategy. The focus of this service is to ensure that the Big Data architecture fits within your wider organisation landscape, ensuring that it is aligned with the existing Business Strategy, IT strategy, Information Management strategy and/or Data Warehouse strategy. Atos provide their architecture consulting and Big Data market and product expertise to deliver a Big Data architecture definition and an implementation roadmap of how to get to it ensuring that the suggested technical solution delivers specific business or technical use cases. This service is ideal for organisations keen to adopt Big Data, but not yet ready from a business or technical view point. It will provide the organisation with a roadmap towards the target architecture and transformation stages. Features of the service include: Analysis of business requirements and definition of Big Data solution to address them Assessing of business process impact Understanding what changes to the current IT landscape will be required Recommendations on skills and governance changes required for a Big Data solution Definition of the Big Data solution components Recommendations on technical products selection Delivery of a technical roadmap to move from current IT architecture to proposed Big Data architecture. How this service can be used This service can be used in many scenarios, including: Undertaking a complete assessment of the current IT landscape - processes, technology, and people i.e. current users and user profiles. This will help identify any pain points which can be addressed in future strategy documents Determining the best approach for users to access and consume information in the future (target operating model). This service will define how users will share information and knowledge in the future and the kind of collaborative environment that could be made available to them in order to make best use of the Big Data components available to them. The use of multiple technologies and techniques would be recommended if necessary iii
Defining the migration or transformation plan to bridge the gap between what the current landscape is, to what the future strategy dictates. This service will outline at a practical level high-level process issues, refresh rates for data, capacity planning, backup, recovery, archive, workflow, and security and at a strategic level the service will outline how a Big Data initiative will help the business achieve strategic, tactical, and operational goals Augmenting the team with trained and experienced architects and consultants with deep knowledge and experience of Big Data strategy. What makes us unique? The journey into the world of big data is driven by the promise of new insight of new and actionable intelligence revealed. For Atos, Big Data is the same but different. In many ways, it is a straight continuation of what Atos have been doing for years: helping our clients maximise the operational and business advantage gained from their digital assets. But it is different too. The scale of the data explosion, the growing emphasis on deriving value from unstructured data, and the massive increases in connectivity in the internet of things, all help to form a new data landscape. And in this new landscape, those who are best able to generate differentiating intelligence often in real-time will be the ones who achieve and sustain success. Atos offers relevant world-class competencies and customised solutions necessary to handle Big Data and get (more) value out of it. What s included The Atos Big Data team will work with the customer to understand the current situation, and business requirements, and then agree the strategy, principles and solution in order to achieve the desired results. Atos Big Data Strategy & Roadmap service will focus on the following activities: Analysis of Business Requirements Validation and quantification of business goals, for example reviewing current data collection mechanisms and marketing opt-ins to ensure maximum customer contactability; using advanced analytics to enhance personalisation and content strategies; ensuring customer touch points are aligned to deliver consistent messaging; improving the ability to react to behaviours in real time as the customer is engaging with the business, etc. Confirmation of the success criteria metrics Translation of business goals into specific Big Data requirements Assess Business Process Impact Analysis of business processes required for Big Data Assessment of changes needed in existing business processes iv
Handling Changes to IT Landscape Impact assessment of the existing IT landscape Analysis of data sources that will enable the Big Data strategy to establish whether data is already available or needs to be acquired internally or externally Assessment of the existing data quality, data linkage, data capture, data redundancy Assessment of Big Data hosting and deployment options Recommend People and Data Governance Requirements Evaluating Data Management policies and enhancements needed Recommendations on skills needed for Big Data solution implementation Define the Big Data solution architecture and Implementation Roadmap Provide recommendations on Product selection Deliver a technical roadmap to move from current IT architecture to proposed Big Data architecture. A Big Data Strategy & Roadmap document with the following contents: Big Data requirements Big Data metrics Big Data business process design IT landscape gap analysis Skills matrix for handling Big Data implementation Big Data governance structure Architectural components and recommendations Product recommendations Big Data solution components & required infrastructure A high level roadmap for implementation of the Big Data solution. v
vi
Contents 1. Introduction... 1 2. Service overview... 2 2.1 Services Activities... 2 2.2 Services Deliverables... 3 2.3 Inputs & Outputs... 3 2.4 Roles & Responsibilities... 4 2.5 Client Efforts... 4 3. Contractual commitment... 5 4. Timelines... 6 5. Pricing... 7 5.1 Other Professional services... 7 5.2 Termination terms... 7 6. Service management... 8 7. Service constraints... 9 8. Service levels... 10 9. Financial recompense... 11 10. Ordering and invoicing process... 12 11. Termination terms... 13 11.1 By consumers (i.e. consumption)... 13 11.2 By the Supplier (removal of the G-Cloud Service)... 13 12. Technical requirements... 14 13. Trial service... 15 14. Glossary... 16 vii
1. Introduction Atos will help to define the Big Data Strategy for your organisation, the Big Data Solution Architecture to support that strategy, and also a high-level Implementation Roadmap. This service will assess how closely aligned your current IT strategy is with the requirements of Big Data, and identifies concrete steps to incorporate Big Data into your strategy. The focus of this Atos service is to ensure that the Big Data architecture fits within your wider organisation landscape, ensuring that it is aligned with the existing Business Strategy, IT strategy, Information Management strategy and/or Data Warehouse strategy. Atos provide their architecture consulting and Big Data market and product expertise to deliver a Big Data architecture definition and an implementation roadmap ensuring that the suggested technical solution delivers a specific business use case or specific technical use case. This service is ideal for organisations that are keen to adopt Big Data, but not yet ready from a business or technical view point. Features of the service include: Analysis of business requirements and definition of Big Data solution to address them Assessment of business process impact Understanding what changes to the current IT landscape will be required Recommendations on skills and governance changes required for a Big Data solution Definition of the Big Data solution components Recommendations on technical products selection Delivery of a technical roadmap to move from current IT architecture to proposed Big Data architecture. This service has been designed to be delivered agnostic of the underlying platform, and is suitable therefore for consumption by customers currently deploying public cloud, private cloud, hybrid cloud or dedicated infrastructure configurations, and by those moving towards a more cloud-based deployment. 1
2. Service overview 2.1 Services Activities Atos Big Data team will work with the customer to understand the current situation, and business requirements, and then agree the strategy, principles and solution in order to achieve the desired results. Atos Big Data Strategy & Roadmap service will focus on the following activities: Analysis of Business Requirements Validation and quantification of business goals, for example reviewing current data collection mechanisms and marketing opt-ins to ensure maximum customer contactability; using advanced analytics to enhance personalisation and content strategies; ensuring customer touch points are aligned to deliver consistent messaging; improving the ability to react to behaviours in real time as the customer is engaging with the business, etc. Confirmation of the success criteria metrics Translation of business goals into specific Big Data requirements Assess Business Process Impact Analysis of business processes required for Big Data Assessment of changes needed in existing business processes Handling Changes to IT Landscape Impact assessment of the existing IT landscape Analysis of data sources that will enable the Big Data strategy to establish whether data is already available or needs to be acquired internally or externally Assessment of the existing data quality, data linkage, data capture, data redundancy Assessment of Big Data hosting and deployment options Recommend People and Data Governance Requirements Evaluating Data Management policies and enhancements needed Recommendations for skills needed in Big Data solution implementation Define the Big Data Solution Architecture and Implementation Roadmap Definition of a high-level Big Data solution architecture Deliver a technical roadmap to move from the current IT architecture to the proposed Big Data architecture Provide recommendations on Product selection. 2
2.2 Services Deliverables The Big Data Strategy & Roadmap documentation will include the following contents: Big Data requirements Big Data metrics Big Data business process design IT landscape gap analysis Skills matrix for handling Big Data implementation Big Data governance structure Architectural components and recommendations Product recommendations Big Data solution components & required infrastructure A high level roadmap for implementation of the Big Data solution. 2.3 Inputs & Outputs Inputs 1. Existing IT Strategy (Client) 2. Existing Architecture documentation (Client) 3. Existing Infrastructure documentation (Client) 4. As Is application architecture documents (Client) 5. As Is infrastructure architecture documents (Client) 6. Relevant data source documentation (Client) Outputs 1. Big Data Strategy & Roadmap Document (Atos) 2. Big Data Solution Architecture document (Big Data Template) (Atos) 3. Big Data Roadmap (Big Data Template) (Atos) 3
2.4 Roles & Responsibilities Roles Big Data Consultant Big Data Architect Big Data Technical Specialist Head of business departments Head of IT/BI function Business Implementers IT/BI Implementers Role Description / Responsibilities Lead the Big Data strategy activity and produce the Big Data strategy Document Provide support to the Big Data Consultant on technical areas and documentation of the strategy Document. Lead the Big Data solution architecture and roadmap activity and produce the Big Data architecture and roadmap deliverable. Provide support to the Big Data Architect to produce the Big Data architecture and roadmap deliverable Identify individuals to be available during the strategy and architecture activity to answer questions and provide information Provide answers and information related to the Big Data strategy Provide answers, information and access to system (if required) relevant for the Big Data strategy, architecture and roadmap activity Client / Atos Atos Client 2.5 Client Efforts The client will be responsible for ensuring the following: Ensuring that key individuals (client SME) are available for the period of the development of the Big Data strategy, solution architecture and implementation roadmap Providing Atos with all relevant existing As-Is strategy papers and technical documentation on data sources, architecture and infrastructure details. 4
3. Contractual commitment Atos is fully committed to protecting the Confidentiality, Integrity and Availability of consumer data that is entrusted to us. We provide accredited services to a wide range of UK government departments including the Home Office, Department of Work and Pensions, the Ministry of Defence, executive agencies, local authorities and other public sector and government regulated organisations. A number of our G-Cloud services have been formally accredited by the Pan Government Accreditor (PGA). We also serve many different private sector organisations, including financial services institutions and health services. Atos has a number of sites and facilities that are HMG accredited to process information at OFFICIAL, these sites meet or exceed the standards laid down in the Security Policy Framework (SPF). The Atos Data Centres are ISO27001 certified and PCI DSS certified. The Atos PaaS service has approval for connection to the PSN network. In all cases these facilities are hosted on-shore in the UK, managed by SC cleared staff and include secure office areas and secure data processing centres. In the financial services sector we provide services that meet FSA requirements. The PCI Security Standards Council and VISA Europe have classified Atos UK as a Level 1 Payment Services Provider, based on its aggregated level of Payment Card transactions. We have a company-wide strategic PCI DSS Compliance programme which defines methods and timescales for the achievement of compliance for all of Atos hosted customer accounts and support services, whose requirements include the processing of Payment Card information. Within the defence sector, Atos is approved to hold UK government OFFICIAL protectively marked information and has the capability and accreditation to host data and associated systems marked as SECRET or TOP SECRET. Atos uses customer networks for transmission where appropriate and is fully compliant with procedures relating to the manual handling of protectively marked information. 5
4. Timelines The service is delivered according to the specific needs of each engagement. The timelines involved in the definition of the Big Data Strategy & Roadmap depends on the functional and organisational scope and the selected Big Data use case(s). The timelines assigned below are indicative and are dependent on the client s scope. Activity Business Requirements Assessment Business Process Impact Assessment Handling Changes to IT Landscape Data Management Policies Recommendations Big Data Architecture and Roadmap Timeline between 1 to 2 weeks (5 to 10 days) between 1 to 2 weeks (5 to 10 days) between 3 to 5 weeks (15 to 25 days) between 3 to 5 weeks (15 to 25 days) between 3 to 5 weeks (15 to 25 days) 6
5. Pricing The pricing shown below identifies the primary roles which are engaged in delivering this service. All prices exclude VAT. Resource Type Big Data Consultant NB: BC Grade 6 Big Data Architect NB: SI Practice Grade 6 Big Data Technical Specialist NB: SI Practice Grade 4 Travel and Accommodation Expenses Charge Rate (per day) 1,135 850 590 See below Standards for Consultancy Day Rate: Consultant s Working Day 8 hours exclusive of travel and lunch. Working Week Monday to Friday excluding national holidays Office Hours 09:00 17:00 Monday to Friday Travel and Subsistence Included in day rate within M25. Payable at the department s standard T&S rate outside the M25. Mileage As above Professional Indemnity Insurance included in day rate All pricing excludes VAT. 5.1 Other Professional services Available as per the Atos SFIA rate card 5.2 Termination terms Please refer directly to Atos standard Terms and conditions. 7
6. Service management Where this service is purchased free-standing, a project/service management wrapper will be built in to work plans and effort estimates. This will normally include status/progress reporting. 8
7. Service constraints Please see the sections on Technical Requirements below. 9
8. Service levels Atos provides suitably trained individuals to complete the tasks necessary for this service. They can work at the client site or remotely, depending on the need for access to: Client staff Client applications & network. Our standard working hours / days are 09:00 to 17:00 Monday to Friday, excluding public & regional holidays. 10
9. Financial recompense To minimise the cost to users, Atos does not provide service credits for use of the service. All Atos services are provided on a reasonable endeavours basis. Please refer to the Atos Supplier Terms. In accordance with the guidance within the GPS G-Cloud Framework Terms and Conditions, the Customer may terminate the Call-Off Agreement at any time, without cause, by giving at least thirty (30) Working Days prior notice in writing. The Call-Off Agreement terms and conditions and the Atos Supplier Terms will define the circumstances where a refund of any pre-paid service charges may be available. 11
10. Ordering and invoicing process Ordering this product is a straightforward process. Please forward your requirements to the email address GCloud@atos.net. We will prepare a quotation and agree that quotation with you, including any volume discounts that may be applicable. Once the quotation is agreed, we will issue you with the necessary documentation (as required by the G-Cloud Framework) and ask you to provide us with a purchase order. Once we have received your purchase order, the services will be configured to the requirements agreed. If you are a new customer, additional new supplier forms may need to be completed. Invoices will be issued to you and Shared Services (quoting the purchase order number) for the services procured. On a monthly basis, we will also complete the mandated management information reports to Government Procurement Services detailing the spend that you have placed with us. Cabinet Office publish a summary of this monthly management information at: https://www.digitalmarketplace.service.gov.uk/ 12
11. Termination terms 11.1 By consumers (i.e. consumption) Termination shall be in accordance with: The G-Cloud Framework terms and conditions Any terms agreed within the Call Off Agreement under section 10.2 of the Order Form (termination without cause) where the Government Procurement Service (GPS) guidance states At least thirty (30) Working Days in accordance with Clause CO-9.2 of the Call-Off Agreement Atos Supplier Terms for this Service as listed on the Digital Marketplace. For this specific service, by default Atos ask for at least thirty (30) Working Days prior written notice of termination as per the guidance within the GPS G-Cloud Framework Terms and Conditions. 11.2 By the Supplier (removal of the G-Cloud Service) Atos commits to continue to provide the service for the duration of the Call-Off Agreement subject to the terms and conditions of the G-Cloud Framework, the Call-Off Agreement and the Atos Supplier Terms. 13
12. Technical requirements Efficient project delivery during the requirements management & specification stages - usually requires consideration of access to: The Atos network (usually via the Internet) The Internet/WWW Any shared project areas (e.g. on client or Atos network or cloud-based) Development and test environments Client legacy systems. 14
13. Trial service This service is not available on a trial basis. 15
14. Glossary Abbreviation / term Big Data Exabyte Hadoop Version Big Data is a general term used to describe the voluminous amount of unstructured and semi-structured data a company creates -- data that would take too much time and cost too much money to load into a relational database for analysis. Although Big data doesn't refer to any specific quantity, the term is often used when speaking about petabytes and exabytes of data. A primary goal for looking at big data is to discover repeatable business patterns. It s generally accepted that unstructured data, most of it located in text files, accounts for at least 80% of an organization s data. If left unmanaged, the sheer volume of unstructured data that s generated each year within an enterprise can be costly in terms of storage. Unmanaged data can also pose a liability if information cannot be located in the event of a compliance audit or lawsuit. Big data analytics is often associated with cloud computing because the analysis of large data sets in real-time requires a framework like MapReduce to distribute the work among tens, hundreds or even thousands of computers. An exabyte (EB) is a large unit of computer data storage, two to the sixtieth power bytes. The prefix exa means one billion, or one quintillion, which is a decimal term. Two to the sixtieth power is actually 1,152,921,504,606,846,976 bytes in decimal, or somewhat over a quintillion (or ten to the eighteenth power) bytes. It is common to say that an exabyte is approximately one quintillion bytes. In decimal terms, an exabyte is a billion gigabytes. An exabyte of storage could contain 50,000 years' worth of DVD-quality video Hadoop is a free, Java-based programming framework that supports the processing of large data sets in a distributed computing environment. It is part of the Apache project sponsored by the Apache Software Foundation. Hadoop makes it possible to run applications on systems with thousands of nodes involving thousands of terabytes. Its distributed file system facilitates rapid data transfer rates among nodes and allows the system to continue operating uninterrupted in case of a node failure. This approach lowers the risk of catastrophic system failure, even if a significant number of nodes become inoperative. Hadoop was inspired by Google's MapReduce, a software framework in which an application is broken down into numerous small parts. Any of these parts (also called fragments or blocks) can be run on any node in the cluster. Doug Cutting, Hadoop's creator, named the framework after his child's stuffed toy elephant. The current Apache Hadoop ecosystem consists of the Hadoop kernel, MapReduce, the Hadoop distributed file system (HDFS) and a number of related projects such as Apache Hive, HBase and Zookeeper. The Hadoop framework is 16
Abbreviation / term MapReduce Petabyte Unstructured Data Version used by major players including Google, Yahoo and IBM, largely for applications involving search engines and advertising. The preferred operating systems are Windows and Linux but Hadoop can also work with BSD and OS X. MapReduce is a software framework that allows developers to write programs that process massive amounts of unstructured data in parallel across a distributed cluster of processors or stand-alone computers. It was developed at Google for indexing Web pages and replaced their original indexing algorithms and heuristics in 2004. The framework is divided into two parts: Map a function that parcels out work to different nodes in the distributed cluster, and Reduce another function that collates the work and resolves the results into a single value. The MapReduce framework is fault-tolerant because each node in the cluster is expected to report back periodically with completed work and status updates. If a node remains silent for longer than the expected interval, a master node makes note and re-assigns the work to other nodes. MapReduce is important because it allows ordinary developers to use MapReduce library routines to create parallel programs without having to worry about programming for intracluster communication, task monitoring or failure handling. It is useful for tasks such as data mining, log file analysis, financial analysis and scientific simulations. Several implementations of MapReduce are available in a variety of programming languages, including Java, C++, Python, Perl, Ruby, and C. A petabyte is a measure of memory or storage capacity and is 2 to the 50th power bytes or, in decimal, approximately a thousand terabytes. In recently announcing how many Fibre Channel storage arrays they had sold, Sun Microsystems stated that it had shipped an aggregate of two petabytes of storage or the equivalent of 40 million four-drawer filing cabinets full of text. IBM says that it has shipped four petabytes of SSA Storage. Unstructured data is a generic label for describing any corporate information that is not in a database. Unstructured data can be textual or non-textual. Textual unstructured data is generated in media like email messages, PowerPoint presentations, Word documents, collaboration software and instant messages. Non-textual unstructured data is generated in media like JPEG images, MP3 audio files and Flash video files. If left unmanaged, the sheer volume of unstructured data that s generated each year within an enterprise can be costly in terms of storage. Unmanaged data can also pose a liability if information cannot be located in the event of a compliance audit or lawsuit. The information contained in unstructured data is not always easy to locate. It requires that data in both electronic and hard copy documents and other media be scanned so a search 17
Abbreviation / term Version application can parse out concepts based on words used in specific contexts. This is called semantic search. It is also referred to as enterprise search. In customer-facing businesses, the information contained in unstructured data can be analysed to improve customer relationship management and relationship marketing. As social media applications like Twitter and Facebook go mainstream, the growth of unstructured data is expected to far outpace the growth of structured data. According to the "IDC Enterprise Disk Storage Consumption Model" report released in Fall 2009, while transactional data is projected to grow at a compound annual growth rate (CAGR) of 21.8%, it's far outpaced by a 61.7% CAGR prediction for unstructured data. 18
19