Cloud: Cloud Computing meets Smart Ecosystems

Size: px
Start display at page:

Download "Future @ Cloud: Cloud Computing meets Smart Ecosystems"

Transcription

1 Cloud: Cloud Computing meets Smart Ecosystems Joerg Doerr, Fraunhofer IESE, Kaiserslautern, Germany Joerg.Doerr@iese.fraunhofer.de

2 Fraunhofer-Institute for Experimental Software Engineering (IESE) Leading Institute for Software Engineering Founded in 1996 in Kaiserslautern, Germany 200 employees Focus on software engineering Provide innovative and value-adding customer solutions with measurable effects Advance the state-of-the art in software and system engineering Promote the importance of empirically based software and system engineering 2

3 Fraunhofer IESE Our Competencies for innovative Systems SOFTWARE-ENABLED INNOVATIONS 3

4 Fraunhofer IESE Our Competencies SOFTWARE-ENABLED INNOVATIONS 4

5 Digital Society Business Life: Integration Enables Innovation! in Information Systems as well as in Embedded Systems 5

6 Trends and Implications New business models that did not work in the past start to work now (Apple Store, Micropayment,..) Private life pushes business life Physical objects go digital Machinery, things, living objects like plants and animals Usage of Big Data to exploit available data Uncertainty at runtime 6

7 IT Mega Trend: Integration Big Data / Data Analytics 7

8 Digital Ecosystems Software Ecosystems deliver innovations through integrated software systems are typically driven by multiple organizations at their own pace to interact with shared markets operate through the exchange of data, functions, or services with mutually influencing parts Smart Ecosystems integrate non-trivial information systems supporting business goals integrate non-trivial embedded systems supporting technical goals function as one unit to achieve a common, superior goal and share context-dependent information 8

9 Integration of IS and ES - Differences Key Goals Optimization of Business Processes Optimization of Technical Processes (sensors and actuators) Optimization of both, Business Processes & Technical Processes with Equal Rights Software Engineering IS-Driven (Information Systems 2.0) ES-Driven (Embedded Systems 2.0) ES/IS-Integration Key Qualities (Examples) may include embedded data in workflows may use information systems for data storage, e.g., in the cloud Participative Engineering: Across Organizations (sometimes with Equal Rights) Security Safety Safety & Security 9

10 Smart Ecosystems A Trend Across Domains Industry 4.0 Smart Farming V2X and C2X Smart Ecosystems ehealth eenergy 10

11 Research in Smart Ecosystems Key Challenges Diversity Big Data Uncertainty Lifecycle Management Complexity Guaranteed Qualities e.g., Safety and Security 11

12 Big Data Analysis in Smart Ecosystems Crowd Data Miner Visualization Data generation Ecosystem Simulator Global analyses, algorithmics, data fusion, analysis data base Virtual runtime environment Standardized modeling for analyses and released data Data Miner & Generator Data Miner & Generator Visualization Visualization Organization 1 Algorithmics+analyses Runtime environment Usage control Usage control Organization N Algorithmics+analyses Runtime environment Modeling Modeling Data sourcesn Data sources 12

13 Dealing with Data in Smart Ecosystems Cloud as Potential Boost for Analytics & Interoperation Data Usage Control as Key Business Enabler Moving Data to the Cloud = Moving Data to Third Parties Data Protection Challenges Data Residency (data must be kept within defined geographic borders) Data Privacy (enterprise is responsible for any breach to data) Compliance (enterprise must comply with applicable laws) Data Usage Control (data is accessed from different entities) Main concerns for critical infrastructure IT using the Cloud Security and Privacy , SECCRIT 13

14 Motivation SECCRIT in a Nutshell Challenges Analyse and evaluate cloud computing with respect to security risks in sensitive environments (i.e., critical infrastructures) Goal Development of methodologies, technologies, best practices for secure, trustworthy, high assurance and legal compliant cloud computing environments for critical infrastructure IT. Enable cloud technologies to be used for critical infrastructure IT 14

15 SECCRIT Research Focus at Fraunhofer IESE Multi-layer Policy Decision and Enforcement for Usage Control Policies Policy enforcement on different abstraction layers of the cloud (e.g., cloud infrastructure or service level) Context-aware policy enforcement mechanisms (e.g., respecting geolocation if data or service is migrated) User-friendly Policy Specification Elicitation method for security demands and mapping to machineenforceable security policies Reduction of errors and misunderstandings in policy specification 15

16 Policy Decision and Enforcement Framework: IND²UCE Dynamic framework for policy decision and enforcement Seamless integration of new components Dynamic management during runtime Powerful policy language 16

17 Policy Decision and Enforcement SECCRIT Architectural Framework (Policy-oriented View) PEP and PXP as enforcement components on different abstraction levels PDP as central decision component PIP component as additional information retrieval component for the decision making PAP as interface between stakeholders and policy framework 17

18 Enforcement in the Cloud Infrastructure Level Scenario: Enforcing Anti-Affinity Policy Scenario: Tenant A runs critical infrastructure services on different machines (VMs) on a virtual datacenter. However, the services are not allowed to share the same physical resources! Problem: If Tenant A or the cloud infrastructure operator starts migrating virtual machines (VMs) to the same physical host, both critical services run on the same physical host. VMware offers affinity rules, but allows their violation Solution: An anti-affinity policy specifies that critical VMs have to be separated. Migrating critical VMs to the same physical host results in automatically migrating the other critical service away. 18

19 Enforcement in the Cloud Infrastructure Level Scenario: Enforcing Virtual Machines Geolocation Scenario: A virtual machine hosts sensitive data and is only allowed to be operated in countries within Europe. Problem: A cloud operator might trigger the process to migrate the virtual machine to another data center outside Europe. Solution: A virtual machines geolocation policy specifies that virtual machines are only allowed to be operated in data centers within Europe. Migrating the virtual machine outside Europe will be logged and countermeasures enforced. 19

20 Enforcement in the Cloud Infrastructure Level IND²UCE for VMware VMware vsphereclient SOAP VMware vcenter Server VMware vsphere Manage VMware vsphere independent of VMware changes (except for interface changes) no disturbance of other systems only detective enforcement 20

21 Enforcement in the Service Level IND²UCE for HBase/Hadoop Cloud Databases HBase: NoSQL database inspired and modeled after Google s Bigtable 1 Hadoop: Distributed File System(HDFSTM) + Hadoop MapReduce Idea: Distribute big data into clusters MapReduce algorithm

22 Enforcement in the Service Level Scenario: Modify Data in Transit Scenario: A first level support worker is accessing person-related data for their customers. However, support worker should not have access to fields such as the concrete date of birth. Problem: The database stores the date of birth in one field and can only return the entire field or nothing. The data usage restriction could only be solved by changing the database fields accordingly. Solution: A privacy policy specifies to replace day of birth and month of birth with X. Only year of birth is visible to the first level support worker. 22

23 Enforcement in the Service Level IND²UCE for HBase/Hadoop Cloud Databases Zookeeper Ensemble Zookeeper1 Zookeeper2 Zookeeper3 HMaster2 HMaster1 Region Server Region Server HBase Task Tracker Task Tracker Job Tracker Control & Message Signals One way dependency Bi-directional dependency Name Node Secondary Name Node Data Node Map Reduce Data Node Hadoop HDFS 23

24 Takeaways Companies and Society can strongly benefit from Smart Ecosystems Opportunity and threat at the same time for companies Cloud Computing can be a significant boost for analytics and interoperability Challenges in Smart Ecosystems require guaranteed qualities Data Usage Control will be a business enabler, Security is not a showstopper Fraunhofer IESE provides strong competences for Smart Ecosystem challenges 24

25 Dr. Jörg Dörr Fraunhofer IESE

ENHANCING CLOUD SECURITY WITH CONTEXT-AWARE USAGE CONTROL POLICIES CHRISTIAN JUNG, ANDREAS EITEL, REINHARD SCHWARZ

ENHANCING CLOUD SECURITY WITH CONTEXT-AWARE USAGE CONTROL POLICIES CHRISTIAN JUNG, ANDREAS EITEL, REINHARD SCHWARZ FRAUNHOFER INSTITUTE FOR EXPERIMENTAL SOFTWARE ENGINEERING IESE ENHANCING CLOUD SECURITY WITH CONTEXT-AWARE USAGE CONTROL POLICIES CHRISTIAN JUNG, ANDREAS EITEL, REINHARD SCHWARZ Christian Jung christian.jung@iese.fraunhofer.de

More information

White paper. The Big Data Security Gap: Protecting the Hadoop Cluster

White paper. The Big Data Security Gap: Protecting the Hadoop Cluster The Big Data Security Gap: Protecting the Hadoop Cluster Introduction While the open source framework has enabled the footprint of Hadoop to logically expand, enterprise organizations face deployment and

More information

Comprehensive Analytics on the Hortonworks Data Platform

Comprehensive Analytics on the Hortonworks Data Platform Comprehensive Analytics on the Hortonworks Data Platform We do Hadoop. Page 1 Page 2 Back to 2005 Page 3 Vertical Scaling Page 4 Vertical Scaling Page 5 Vertical Scaling Page 6 Horizontal Scaling Page

More information

Big Data - Infrastructure Considerations

Big Data - Infrastructure Considerations April 2014, HAPPIEST MINDS TECHNOLOGIES Big Data - Infrastructure Considerations Author Anand Veeramani / Deepak Shivamurthy SHARING. MINDFUL. INTEGRITY. LEARNING. EXCELLENCE. SOCIAL RESPONSIBILITY. Copyright

More information

Horizontal IoT Application Development using Semantic Web Technologies

Horizontal IoT Application Development using Semantic Web Technologies Horizontal IoT Application Development using Semantic Web Technologies Soumya Kanti Datta Research Engineer Communication Systems Department Email: Soumya-Kanti.Datta@eurecom.fr Roadmap Introduction Challenges

More information

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY A PATH FOR HORIZING YOUR INNOVATIVE WORK BIG DATA HOLDS BIG PROMISE FOR SECURITY NEHA S. PAWAR, PROF. S. P. AKARTE Computer

More information

Virtualizing Apache Hadoop. June, 2012

Virtualizing Apache Hadoop. June, 2012 June, 2012 Table of Contents EXECUTIVE SUMMARY... 3 INTRODUCTION... 3 VIRTUALIZING APACHE HADOOP... 4 INTRODUCTION TO VSPHERE TM... 4 USE CASES AND ADVANTAGES OF VIRTUALIZING HADOOP... 4 MYTHS ABOUT RUNNING

More information

Big Data Analytics for Space Exploration, Entrepreneurship and Policy Opportunities. Tiffani Crawford, PhD

Big Data Analytics for Space Exploration, Entrepreneurship and Policy Opportunities. Tiffani Crawford, PhD Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities Tiffani Crawford, PhD Big Analytics Characteristics Large quantities of many data types Structured Unstructured Human Machine

More information

goberlin a Trusted Cloud Marketplace for Governmental and Commercial Services

goberlin a Trusted Cloud Marketplace for Governmental and Commercial Services goberlin a Trusted Cloud Marketplace for Governmental and Commercial Services Data Protection and Security Considerations in an egovernment Cloud in Germany Dr. Klaus-Peter Eckert Public Sector Cloud Forum

More information

IBM InfoSphere Guardium Data Activity Monitor for Hadoop-based systems

IBM InfoSphere Guardium Data Activity Monitor for Hadoop-based systems IBM InfoSphere Guardium Data Activity Monitor for Hadoop-based systems Proactively address regulatory compliance requirements and protect sensitive data in real time Highlights Monitor and audit data activity

More information

Search and Real-Time Analytics on Big Data

Search and Real-Time Analytics on Big Data Search and Real-Time Analytics on Big Data Sewook Wee, Ryan Tabora, Jason Rutherglen Accenture & Think Big Analytics Strata New York October, 2012 Big Data: data becomes your core asset. It realizes its

More information

Big Data-ready, Secure & Sovereign Cloud

Big Data-ready, Secure & Sovereign Cloud Copernicus Big Data Workshop Big Data-ready, Secure & Sovereign Cloud A Technology Enabler for Copernicus Data Innovation March 14 th, 2014 Brussels F. BOUJEMAA R&D Manager E. MICONNET - Head of Cyber

More information

Cloud Federation to Elastically Increase MapReduce Processing Resources

Cloud Federation to Elastically Increase MapReduce Processing Resources Cloud Federation to Elastically Increase MapReduce Processing Resources A.Panarello, A.Celesti, M. Villari, M. Fazio and A. Puliafito {apanarello,acelesti, mfazio, mvillari, apuliafito}@unime.it DICIEAMA,

More information

How to Hadoop Without the Worry: Protecting Big Data at Scale

How to Hadoop Without the Worry: Protecting Big Data at Scale How to Hadoop Without the Worry: Protecting Big Data at Scale SESSION ID: CDS-W06 Davi Ottenheimer Senior Director of Trust EMC Corporation @daviottenheimer Big Data Trust. Redefined Transparency Relevance

More information

Cloud Courses Description

Cloud Courses Description Courses Description 101: Fundamental Computing and Architecture Computing Concepts and Models. Data center architecture. Fundamental Architecture. Virtualization Basics. platforms: IaaS, PaaS, SaaS. deployment

More information

Cloud Courses Description

Cloud Courses Description Cloud Courses Description Cloud 101: Fundamental Cloud Computing and Architecture Cloud Computing Concepts and Models. Fundamental Cloud Architecture. Virtualization Basics. Cloud platforms: IaaS, PaaS,

More information

Modern Data Architecture for Predictive Analytics

Modern Data Architecture for Predictive Analytics Modern Data Architecture for Predictive Analytics David Smith VP Marketing and Community - Revolution Analytics John Kreisa VP Strategic Marketing- Hortonworks Hortonworks Inc. 2013 Page 1 Your Presenters

More information

A Multilevel Secure MapReduce Framework for Cross-Domain Information Sharing in the Cloud

A Multilevel Secure MapReduce Framework for Cross-Domain Information Sharing in the Cloud A Multilevel Secure MapReduce Framework for Cross-Domain Information Sharing in the Cloud Thuy D. Nguyen, Cynthia E. Irvine, Jean Khosalim Department of Computer Science Ground System Architectures Workshop

More information

Security and Cloud Computing

Security and Cloud Computing Martin Borrett, Lead Security Architect, Europe, IBM 9 th December 2010 Outline Brief Introduction to Cloud Computing Security: Grand Challenge for the Adoption of Cloud Computing IBM and Cloud Security

More information

Cloud Computing Training

Cloud Computing Training Cloud Computing Training TechAge Labs Pvt. Ltd. Address : C-46, GF, Sector 2, Noida Phone 1 : 0120-4540894 Phone 2 : 0120-6495333 TechAge Labs 2014 version 1.0 Cloud Computing Training Cloud Computing

More information

Lecture 32 Big Data. 1. Big Data problem 2. Why the excitement about big data 3. What is MapReduce 4. What is Hadoop 5. Get started with Hadoop

Lecture 32 Big Data. 1. Big Data problem 2. Why the excitement about big data 3. What is MapReduce 4. What is Hadoop 5. Get started with Hadoop Lecture 32 Big Data 1. Big Data problem 2. Why the excitement about big data 3. What is MapReduce 4. What is Hadoop 5. Get started with Hadoop 1 2 Big Data Problems Data explosion Data from users on social

More information

Problem Solving Hands-on Labware for Teaching Big Data Cybersecurity Analysis

Problem Solving Hands-on Labware for Teaching Big Data Cybersecurity Analysis , 22-24 October, 2014, San Francisco, USA Problem Solving Hands-on Labware for Teaching Big Data Cybersecurity Analysis Teng Zhao, Kai Qian, Dan Lo, Minzhe Guo, Prabir Bhattacharya, Wei Chen, and Ying

More information

Journey to the Private Cloud. Key Enabling Technologies

Journey to the Private Cloud. Key Enabling Technologies Journey to the Private Cloud Key Enabling Technologies Jeffrey Nick Chief Technology Officer Senior Vice President EMC Corporation June 2010 1 The current I/T state: Infrastructure sprawl Information explosion

More information

Big Data Analytics. Copyright 2011 EMC Corporation. All rights reserved.

Big Data Analytics. Copyright 2011 EMC Corporation. All rights reserved. Big Data Analytics 1 Priority Discussion Topics What are the most compelling business drivers behind big data analytics? Do you have or expect to have data scientists on your staff, and what will be their

More information

Data Refinery with Big Data Aspects

Data Refinery with Big Data Aspects International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data

More information

1 Publishable summary

1 Publishable summary 1 Publishable summary The 4CaaSt research project is creating an advanced Platform as a Service (PaaS). This cloud platform supports the optimized and elastic hosting of internet-scale multi-tier applications.

More information

Chukwa, Hadoop subproject, 37, 131 Cloud enabled big data, 4 Codd s 12 rules, 1 Column-oriented databases, 18, 52 Compression pattern, 83 84

Chukwa, Hadoop subproject, 37, 131 Cloud enabled big data, 4 Codd s 12 rules, 1 Column-oriented databases, 18, 52 Compression pattern, 83 84 Index A Amazon Web Services (AWS), 50, 58 Analytics engine, 21 22 Apache Kafka, 38, 131 Apache S4, 38, 131 Apache Sqoop, 37, 131 Appliance pattern, 104 105 Application architecture, big data analytics

More information

Healthcare: La sicurezza nel Cloud October 18, 2011. 2011 IBM Corporation

Healthcare: La sicurezza nel Cloud October 18, 2011. 2011 IBM Corporation Healthcare: La sicurezza nel Cloud October 18, 2011 Cloud Computing Tests The Limits Of Security Operations And Infrastructure Security and Privacy Domains People and Identity Data and Information Application

More information

HADOOP ADMINISTATION AND DEVELOPMENT TRAINING CURRICULUM

HADOOP ADMINISTATION AND DEVELOPMENT TRAINING CURRICULUM HADOOP ADMINISTATION AND DEVELOPMENT TRAINING CURRICULUM 1. Introduction 1.1 Big Data Introduction What is Big Data Data Analytics Bigdata Challenges Technologies supported by big data 1.2 Hadoop Introduction

More information

Cloud computing research activities at. Jordan University of Science and Technology. Yaser Jararweh. Irbid, Jordan yijararweh@just.edu.

Cloud computing research activities at. Jordan University of Science and Technology. Yaser Jararweh. Irbid, Jordan yijararweh@just.edu. Cloud computing research activities at Jordan University of Science and Technology Yaser Jararweh Jordan University of Science and Technology Irbid, Jordan yijararweh@just.edu.jo 29/08/2013 Yaser Jararweh:

More information

Big Data Storage Architecture Design in Cloud Computing

Big Data Storage Architecture Design in Cloud Computing Big Data Storage Architecture Design in Cloud Computing Xuebin Chen 1, Shi Wang 1( ), Yanyan Dong 1, and Xu Wang 2 1 College of Science, North China University of Science and Technology, Tangshan, Hebei,

More information

Hadoop as a Service. VMware vcloud Automation Center & Big Data Extension

Hadoop as a Service. VMware vcloud Automation Center & Big Data Extension Hadoop as a Service VMware vcloud Automation Center & Big Data Extension Table of Contents 1. Introduction... 2 1.1 How it works... 2 2. System Pre-requisites... 2 3. Set up... 2 3.1 Request the Service

More information

Securing the Cloud with IBM Security Systems. IBM Security Systems. 2012 IBM Corporation. 2012 2012 IBM IBM Corporation Corporation

Securing the Cloud with IBM Security Systems. IBM Security Systems. 2012 IBM Corporation. 2012 2012 IBM IBM Corporation Corporation Securing the Cloud with IBM Security Systems 1 2012 2012 IBM IBM Corporation Corporation IBM Point of View: Cloud can be made secure for business As with most new technology paradigms, security concerns

More information

The Future of Data Management

The Future of Data Management The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

More information

Transforming the Telecoms Business using Big Data and Analytics

Transforming the Telecoms Business using Big Data and Analytics Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe

More information

Presenting Mongoose A New Approach to Traffic Capture (patent pending) presented by Ron McLeod and Ashraf Abu Sharekh January 2013

Presenting Mongoose A New Approach to Traffic Capture (patent pending) presented by Ron McLeod and Ashraf Abu Sharekh January 2013 Presenting Mongoose A New Approach to Traffic Capture (patent pending) presented by Ron McLeod and Ashraf Abu Sharekh January 2013 Outline Genesis - why we built it, where and when did the idea begin Issues

More information

Green Software Services From requirements to Business Models

Green Software Services From requirements to Business Models Green Software Services From requirements to Business Models Schahram Dustdar TU Wien Austria Distributed Systems Group dsg.tuwien.ac.at Smart Evolution People, Services,Things Smart Buildings and Energy

More information

Strengthen security with intelligent identity and access management

Strengthen security with intelligent identity and access management Strengthen security with intelligent identity and access management IBM Security solutions help safeguard user access, boost compliance and mitigate insider threats Highlights Enable business managers

More information

Huawei Technologies ERC Position Statement: Towards a Future Internet Public Private Partnership

Huawei Technologies ERC Position Statement: Towards a Future Internet Public Private Partnership Huawei Technologies ERC Position Statement: Towards a Future Internet Public Private Partnership Kostas Pentikousis, Mirko Schramm, and Cornel Pampu Huawei Technologies European Research Centre Carnotstrasse

More information

Open Source in Financial Services: Meet the challenges of new business models and disruption

Open Source in Financial Services: Meet the challenges of new business models and disruption Open Source in Financial Services: Meet the challenges of new business models and disruption Speakers Vamsi Chemitiganti, General Manager Financial Services, Hortonworks Josh West, Senior Solutions Architect,

More information

#TalendSandbox for Big Data

#TalendSandbox for Big Data Evalua&on von Apache Hadoop mit der #TalendSandbox for Big Data Julien Clarysse @whatdoesdatado @talend 2015 Talend Inc. 1 Connecting the Data-Driven Enterprise 2 Talend Overview Founded in 2006 BRAND

More information

HDP Hadoop From concept to deployment.

HDP Hadoop From concept to deployment. HDP Hadoop From concept to deployment. Ankur Gupta Senior Solutions Engineer Rackspace: Page 41 27 th Jan 2015 Where are you in your Hadoop Journey? A. Researching our options B. Currently evaluating some

More information

How to avoid building a data swamp

How to avoid building a data swamp How to avoid building a data swamp Case studies in Hadoop data management and governance Mark Donsky, Product Management, Cloudera Naren Korenu, Engineering, Cloudera 1 Abstract DELETE How can you make

More information

Hadoop IST 734 SS CHUNG

Hadoop IST 734 SS CHUNG Hadoop IST 734 SS CHUNG Introduction What is Big Data?? Bulk Amount Unstructured Lots of Applications which need to handle huge amount of data (in terms of 500+ TB per day) If a regular machine need to

More information

SECURING YOUR MODERN DATA CENTER WITH CHECK POINT

SECURING YOUR MODERN DATA CENTER WITH CHECK POINT SECURING YOUR MODERN DATA CENTER WITH CHECK POINT Javier Hijas Security Architect Check Point Europe 1 Agenda 1 2 3 4 What Questions is a modern / Answers datacenter Datacenter protection evolution Security

More information

Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray VMware

Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray VMware Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray ware 2 Agenda The Hadoop Journey Why Virtualize Hadoop? Elasticity and Scalability Performance Tests Storage Reference

More information

Application and practice of parallel cloud computing in ISP. Guangzhou Institute of China Telecom Zhilan Huang 2011-10

Application and practice of parallel cloud computing in ISP. Guangzhou Institute of China Telecom Zhilan Huang 2011-10 Application and practice of parallel cloud computing in ISP Guangzhou Institute of China Telecom Zhilan Huang 2011-10 Outline Mass data management problem Applications of parallel cloud computing in ISPs

More information

Scaling Big Data Mining Infrastructure: The Smart Protection Network Experience

Scaling Big Data Mining Infrastructure: The Smart Protection Network Experience Scaling Big Data Mining Infrastructure: The Smart Protection Network Experience 黃 振 修 (Chris Huang) SPN 主 動 式 雲 端 截 毒 技 術 架 構 師 About Me SPN 主 動 式 雲 端 截 毒 技 術 架 構 師 SPN Hadoop 基 礎 運 算 架 構 師 Hadoop in Taiwan

More information

Introduction to Big data. Why Big data? Case Studies. Introduction to Hadoop. Understanding Features of Hadoop. Hadoop Architecture.

Introduction to Big data. Why Big data? Case Studies. Introduction to Hadoop. Understanding Features of Hadoop. Hadoop Architecture. Big Data Hadoop Administration and Developer Course This course is designed to understand and implement the concepts of Big data and Hadoop. This will cover right from setting up Hadoop environment in

More information

Big Data Explained. An introduction to Big Data Science.

Big Data Explained. An introduction to Big Data Science. Big Data Explained An introduction to Big Data Science. 1 Presentation Agenda What is Big Data Why learn Big Data Who is it for How to start learning Big Data When to learn it Objective and Benefits of

More information

Assignment # 1 (Cloud Computing Security)

Assignment # 1 (Cloud Computing Security) Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual

More information

Oracle Big Data SQL Technical Update

Oracle Big Data SQL Technical Update Oracle Big Data SQL Technical Update Jean-Pierre Dijcks Oracle Redwood City, CA, USA Keywords: Big Data, Hadoop, NoSQL Databases, Relational Databases, SQL, Security, Performance Introduction This technical

More information

How we keep harmful apps out of Google Play and keep your Android device safe

How we keep harmful apps out of Google Play and keep your Android device safe How we keep harmful apps out of Google Play and keep your Android device safe February 2016 Bad apps create bad experiences, so we work hard to keep them off your device and out of Google Play. In 2015,

More information

Extending the Enterprise Data Warehouse with Hadoop Robert Lancaster. Nov 7, 2012

Extending the Enterprise Data Warehouse with Hadoop Robert Lancaster. Nov 7, 2012 Extending the Enterprise Data Warehouse with Hadoop Robert Lancaster Nov 7, 2012 Who I Am Robert Lancaster Solutions Architect, Hotel Supply Team rlancaster@orbitz.com @rob1lancaster Organizer of Chicago

More information

Trustworthiness of Big Data

Trustworthiness of Big Data Trustworthiness of Big Data International Journal of Computer Applications (0975 8887) Akhil Mittal Technical Test Lead Infosys Limited ABSTRACT Big data refers to large datasets that are challenging to

More information

BIG DATA TRENDS AND TECHNOLOGIES

BIG DATA TRENDS AND TECHNOLOGIES BIG DATA TRENDS AND TECHNOLOGIES THE WORLD OF DATA IS CHANGING Cloud WHAT IS BIG DATA? Big data are datasets that grow so large that they become awkward to work with using onhand database management tools.

More information

Peers Techno log ies Pv t. L td. HADOOP

Peers Techno log ies Pv t. L td. HADOOP Page 1 Peers Techno log ies Pv t. L td. Course Brochure Overview Hadoop is a Open Source from Apache, which provides reliable storage and faster process by using the Hadoop distibution file system and

More information

Delivery date: 18 October 2014

Delivery date: 18 October 2014 Genomic and Clinical Data Sharing Policy Questions with Technology and Security Implications: Consensus s from the Data Safe Havens Task Team Delivery date: 18 October 2014 When the Security Working Group

More information

Industry 4.0 and Big Data

Industry 4.0 and Big Data Industry 4.0 and Big Data Marek Obitko, mobitko@ra.rockwell.com Senior Research Engineer 03/25/2015 PUBLIC PUBLIC - 5058-CO900H 2 Background Joint work with Czech Institute of Informatics, Robotics and

More information

Introduction to Hadoop. New York Oracle User Group Vikas Sawhney

Introduction to Hadoop. New York Oracle User Group Vikas Sawhney Introduction to Hadoop New York Oracle User Group Vikas Sawhney GENERAL AGENDA Driving Factors behind BIG-DATA NOSQL Database 2014 Database Landscape Hadoop Architecture Map/Reduce Hadoop Eco-system Hadoop

More information

The Power of Pentaho and Hadoop in Action. Demonstrating MapReduce Performance at Scale

The Power of Pentaho and Hadoop in Action. Demonstrating MapReduce Performance at Scale The Power of Pentaho and Hadoop in Action Demonstrating MapReduce Performance at Scale Introduction Over the last few years, Big Data has gone from a tech buzzword to a value generator for many organizations.

More information

Virtualized Hadoop. A Dell Hadoop Whitepaper. By Joey Jablonski. A Dell Hadoop Whitepaper

Virtualized Hadoop. A Dell Hadoop Whitepaper. By Joey Jablonski. A Dell Hadoop Whitepaper Virtualized Hadoop A Dell Hadoop Whitepaper By Joey Jablonski A Dell Hadoop Whitepaper Introduction to Virtualized Hadoop Hadoop has become a standard within many organizations and data centers for its

More information

Cloud-Security: Show-Stopper or Enabling Technology?

Cloud-Security: Show-Stopper or Enabling Technology? Cloud-Security: Show-Stopper or Enabling Technology? Fraunhofer Institute for Secure Information Technology (SIT) Technische Universität München Open Grid Forum, 16.3,. 2010, Munich Overview 1. Cloud Characteristics

More information

Getting Started with SandStorm NoSQL Benchmark

Getting Started with SandStorm NoSQL Benchmark Getting Started with SandStorm NoSQL Benchmark SandStorm is an enterprise performance testing tool for web, mobile, cloud and big data applications. It provides a framework for benchmarking NoSQL, Hadoop,

More information

You should have a working knowledge of the Microsoft Windows platform. A basic knowledge of programming is helpful but not required.

You should have a working knowledge of the Microsoft Windows platform. A basic knowledge of programming is helpful but not required. What is this course about? This course is an overview of Big Data tools and technologies. It establishes a strong working knowledge of the concepts, techniques, and products associated with Big Data. Attendees

More information

FRAUNHOFER INSTITUTE FOR EXPERIMENTAL SOFTWARE ENGINEERING IESE VARIATION MANAGEMENT: USER EXPERIENCE FOR EFFICIENCY IN PROVIDING SOLUTIONS

FRAUNHOFER INSTITUTE FOR EXPERIMENTAL SOFTWARE ENGINEERING IESE VARIATION MANAGEMENT: USER EXPERIENCE FOR EFFICIENCY IN PROVIDING SOLUTIONS FRAUNHOFER INSTITUTE FOR EXPERIMENTAL SOFTWARE ENGINEERING IESE VARIATION MANAGEMENT: USER EXPERIENCE FOR EFFICIENCY IN PROVIDING BUSINESS CUSTOMIZED APPLICATIONS SOLUTIONS 2 Do you need to develop variation-rich

More information

Open Source for Cloud Infrastructure

Open Source for Cloud Infrastructure Open Source for Cloud Infrastructure June 29, 2012 Jackson He General Manager, Intel APAC R&D Ltd. Cloud is Here and Expanding More users, more devices, more data & traffic, expanding usages >3B 15B Connected

More information

How To Run Apa Hadoop 1.0 On Vsphere Tmt On A Hyperconverged Network On A Virtualized Cluster On A Vspplace Tmter (Vmware) Vspheon Tm (

How To Run Apa Hadoop 1.0 On Vsphere Tmt On A Hyperconverged Network On A Virtualized Cluster On A Vspplace Tmter (Vmware) Vspheon Tm ( Apache Hadoop 1.0 High Availability Solution on VMware vsphere TM Reference Architecture TECHNICAL WHITE PAPER v 1.0 June 2012 Table of Contents Executive Summary... 3 Introduction... 3 Terminology...

More information

Extend your analytic capabilities with SAP Predictive Analysis

Extend your analytic capabilities with SAP Predictive Analysis September 9 11, 2013 Anaheim, California Extend your analytic capabilities with SAP Predictive Analysis Charles Gadalla Learning Points Advanced analytics strategy at SAP Simplifying predictive analytics

More information

Application Development. A Paradigm Shift

Application Development. A Paradigm Shift Application Development for the Cloud: A Paradigm Shift Ramesh Rangachar Intelsat t 2012 by Intelsat. t Published by The Aerospace Corporation with permission. New 2007 Template - 1 Motivation for the

More information

Dominik Wagenknecht Accenture

Dominik Wagenknecht Accenture Dominik Wagenknecht Accenture Improving Mainframe Performance with Hadoop October 17, 2014 Organizers General Partner Top Media Partner Media Partner Supporters About me Dominik Wagenknecht Accenture Vienna

More information

Secure Cloud Computing for Critical Infrastructures

Secure Cloud Computing for Critical Infrastructures SEcure Cloud computing for CRitical Infrastructure IT Secure Cloud Computing for Critical Infrastructures Aleksandar Hudic and Christian Wagner AIT Austrian Institute of Technology AIT Austrian Institute

More information

VMware vsphere: Fast Track [V5.0]

VMware vsphere: Fast Track [V5.0] VMware vsphere: Fast Track [V5.0] Experience the ultimate in vsphere 5 skills-building and VCP exam-preparation training. In this intensive, extended-hours course, you will focus on installing, configuring,

More information

Can You Really Support Business Continuity Requirements?

Can You Really Support Business Continuity Requirements? Can You Really Support Business Continuity Requirements? Intelligent N+X Redundancy, Placement Affinities, & Future Proofing in the Virtualized Data Center BACKGROUND One of the significant gains provided

More information

Cloud Computing and Big Data What Technical Writers Need to Know

Cloud Computing and Big Data What Technical Writers Need to Know Cloud Computing and Big Data What Technical Writers Need to Know Greg Olson, Senior Director Black Duck Software For the Society of Technical Writers Berkeley Chapter Black Duck 2014 Agenda Introduction

More information

VMware vcenter Configuration Manager for Virtual Infrastructure Management [V5.X]

VMware vcenter Configuration Manager for Virtual Infrastructure Management [V5.X] VMWARE COURSE OUTLINE Visit Our Website to Enroll Now Www.ITBigBang.Com/IT-Training VMware vcenter Configuration Manager for Virtual Infrastructure Management [V5.X] Course Title VMware vcenter Configuration

More information

Introduction to Cloud Computing

Introduction to Cloud Computing Introduction to Cloud Computing Cloud Computing I (intro) 15 319, spring 2010 2 nd Lecture, Jan 14 th Majd F. Sakr Lecture Motivation General overview on cloud computing What is cloud computing Services

More information

VMware vcenter Site Recovery Manager 5 Technical

VMware vcenter Site Recovery Manager 5 Technical VMware vcenter Site Recovery Manager 5 Technical Raj Jethnani, VCP 4/5, VCAP4-DCA / DCD Systems Engineer VMware, Inc. rjethnani@vmware.com @rajtech 2009 VMware Inc. All rights reserved Agenda Simplifying

More information

Advanced Data Management and Analytics for Automated Demand Response (ADR) based on NoSQL

Advanced Data Management and Analytics for Automated Demand Response (ADR) based on NoSQL Advanced Data Management and Analytics for Automated Demand Response (ADR) based on NoSQL Bert Taube btaube@versant.com Versant Corporation www.versant.com Rolf Bienert Rolf@openadr.org OpenADR Alliance

More information

CloudCenter Full Lifecycle Management. An application-defined approach to deploying and managing applications in any datacenter or cloud environment

CloudCenter Full Lifecycle Management. An application-defined approach to deploying and managing applications in any datacenter or cloud environment CloudCenter Full Lifecycle Management An application-defined approach to deploying and managing applications in any datacenter or cloud environment CloudCenter Full Lifecycle Management Page 2 Table of

More information

The Next Generation Data Centers: SPECS and The 3 rd Platform.

The Next Generation Data Centers: SPECS and The 3 rd Platform. The Next Generation Data Centers: SPECS and The 3 rd Platform. Dr. Silvio La Porta Senior Research Scientist EMC Research Europe Dr. Said Tabet Senior Technology Strategist Corporate CTO Office, EMC 1

More information

Big Data at Cloud Scale

Big Data at Cloud Scale Big Data at Cloud Scale Pushing the limits of flexible & powerful analytics Copyright 2015 Pentaho Corporation. Redistribution permitted. All trademarks are the property of their respective owners. For

More information

Preface Introduction

Preface Introduction Preface Introduction Cloud computing is revolutionizing all aspects of technologies to provide scalability, flexibility and cost-effectiveness. It has become a challenge to ensure the security of cloud

More information

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON Overview * Introduction * Multiple faces of Big Data * Challenges of Big Data * Cloud Computing

More information

ESS event: Big Data in Official Statistics. Antonino Virgillito, Istat

ESS event: Big Data in Official Statistics. Antonino Virgillito, Istat ESS event: Big Data in Official Statistics Antonino Virgillito, Istat v erbi v is 1 About me Head of Unit Web and BI Technologies, IT Directorate of Istat Project manager and technical coordinator of Web

More information

The 4 Pillars of Technosoft s Big Data Practice

The 4 Pillars of Technosoft s Big Data Practice beyond possible Big Use End-user applications Big Analytics Visualisation tools Big Analytical tools Big management systems The 4 Pillars of Technosoft s Big Practice Overview Businesses have long managed

More information

Upcoming Announcements

Upcoming Announcements Enterprise Hadoop Enterprise Hadoop Jeff Markham Technical Director, APAC jmarkham@hortonworks.com Page 1 Upcoming Announcements April 2 Hortonworks Platform 2.1 A continued focus on innovation within

More information

Cloud and DataCenter Systems: `Fast Data -> Online Management

Cloud and DataCenter Systems: `Fast Data -> Online Management Cloud and DataCenter Systems: `Fast Data -> Online Management Karsten Schwan, Greg Eisenhauer, Ada Gavrilovska, Hrishikesh Amur, Liting Hu, Chengwei Wang, Junwei Li, Motivation Motivation Microsales, Product

More information

Open source large scale distributed data management with Google s MapReduce and Bigtable

Open source large scale distributed data management with Google s MapReduce and Bigtable Open source large scale distributed data management with Google s MapReduce and Bigtable Ioannis Konstantinou Email: ikons@cslab.ece.ntua.gr Web: http://www.cslab.ntua.gr/~ikons Computing Systems Laboratory

More information

Mitra Innovation Leverages WSO2's Open Source Middleware to Build BIM Exchange Platform

Mitra Innovation Leverages WSO2's Open Source Middleware to Build BIM Exchange Platform Mitra Innovation Leverages WSO2's Open Source Middleware to Build BIM Exchange Platform May 2015 Contents 1. Introduction... 3 2. What is BIM... 3 2.1. History of BIM... 3 2.2. Why Implement BIM... 4 2.3.

More information

Massive Cloud Auditing using Data Mining on Hadoop

Massive Cloud Auditing using Data Mining on Hadoop Massive Cloud Auditing using Data Mining on Hadoop Prof. Sachin Shetty CyberBAT Team, AFRL/RIGD AFRL VFRP Tennessee State University Outline Massive Cloud Auditing Traffic Characterization Distributed

More information

Workshop on Hadoop with Big Data

Workshop on Hadoop with Big Data Workshop on Hadoop with Big Data Hadoop? Apache Hadoop is an open source framework for distributed storage and processing of large sets of data on commodity hardware. Hadoop enables businesses to quickly

More information

Adobe Deploys Hadoop as a Service on VMware vsphere

Adobe Deploys Hadoop as a Service on VMware vsphere Adobe Deploys Hadoop as a Service A TECHNICAL CASE STUDY APRIL 2015 Table of Contents A Technical Case Study.... 3 Background... 3 Why Virtualize Hadoop on vsphere?.... 3 The Adobe Marketing Cloud and

More information

This Symposium brought to you by www.ttcus.com

This Symposium brought to you by www.ttcus.com This Symposium brought to you by www.ttcus.com Linkedin/Group: Technology Training Corporation @Techtrain Technology Training Corporation www.ttcus.com Big Data Analytics as a Service (BDAaaS) Big Data

More information

Hadoop Ecosystem Overview. CMSC 491 Hadoop-Based Distributed Computing Spring 2015 Adam Shook

Hadoop Ecosystem Overview. CMSC 491 Hadoop-Based Distributed Computing Spring 2015 Adam Shook Hadoop Ecosystem Overview CMSC 491 Hadoop-Based Distributed Computing Spring 2015 Adam Shook Agenda Introduce Hadoop projects to prepare you for your group work Intimate detail will be provided in future

More information

Has been into training Big Data Hadoop and MongoDB from more than a year now

Has been into training Big Data Hadoop and MongoDB from more than a year now NAME NAMIT EXECUTIVE SUMMARY EXPERTISE DELIVERIES Around 10+ years of experience on Big Data Technologies such as Hadoop and MongoDB, Java, Python, Big Data Analytics, System Integration and Consulting

More information

A ROAD MAP FOR GEOSPATIAL INFORMATION SYSTEM APPLICATIONS ON VBLOCK INFRASTRUCTURE PLATFORMS

A ROAD MAP FOR GEOSPATIAL INFORMATION SYSTEM APPLICATIONS ON VBLOCK INFRASTRUCTURE PLATFORMS A ROAD MAP FOR GEOSPATIAL INFORMATION SYSTEM APPLICATIONS ON VBLOCK INFRASTRUCTURE PLATFORMS June 2011 WHITE PAPER 2011 VCE Company LLC, All rights reserved. 1 Table of Contents Executive Overview... 3

More information

WHAT S NEW IN SAS 9.4

WHAT S NEW IN SAS 9.4 WHAT S NEW IN SAS 9.4 PLATFORM, HPA & SAS GRID COMPUTING MICHAEL GODDARD CHIEF ARCHITECT SAS INSTITUTE, NEW ZEALAND SAS 9.4 WHAT S NEW IN THE PLATFORM Platform update SAS Grid Computing update Hadoop support

More information

Data-intensive HPC: opportunities and challenges. Patrick Valduriez

Data-intensive HPC: opportunities and challenges. Patrick Valduriez Data-intensive HPC: opportunities and challenges Patrick Valduriez Big Data Landscape Multi-$billion market! Big data = Hadoop = MapReduce? No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard,

More information

Big Data: A Storage Systems Perspective Muthukumar Murugan Ph.D. HP Storage Division

Big Data: A Storage Systems Perspective Muthukumar Murugan Ph.D. HP Storage Division Big Data: A Storage Systems Perspective Muthukumar Murugan Ph.D. HP Storage Division In this talk Big data storage: Current trends Issues with current storage options Evolution of storage to support big

More information