An approach to monitoring, data analytics, and decision support for levee supervision
|
|
- Maurice Walker
- 8 years ago
- Views:
Transcription
1 An approach to monitoring, data analytics, and decision support for levee supervision M. Bubak, B. Baliś, D. Harężlak, M. Kasztelnik, P. Nowakowski, T. Bartyński, T. Gubala, M. Malawski, M. Pawlik, B. Wilk AGH University of Science and Technology Department of Computer Science Krakow, Poland ISMOP: a computerized levee monitoring system Comprehensive research project on monitoring and assessment of levees which comprises: Construction of an artificial levee Design and construction of sensors for levee instrumentation Design and development of a sensor communication infrastructure Optimal collection and transmission of sensor data Levee modeling and simulation Comparison of simulated and real levee behavior Central System: software platforms for execution management, data management, visualization and decision support
2 ISMOP Consortium Department of Computer Science AGH Department of Hydrogeology and Engineering Geology AGH Department of Geoinformatics and Applied Computer Science AGH NeoSentio, Kraków Sweco Hydroprojekt Kraków in collaboration with the Czernichów Community Project leader: Prof. Krzysztof Zieliński ISMOP: target users Main goal: support the decision making for Flood protection Levee maintenance Target users: National and regional flood protection agencies Local authorities (levee maintenance)
3 ISMOP central system Visualization & decision support Execution management Data management Monitoring & decision support high level workflow Stand by mode Monitoring data collection (low frequency) Initialon line analysis (trends, deviations in sensor readings) Presentation of external info: weather prediction, flood wave prediction, etc. Threat assessment mode Increased frequency of sensor data collection Resourceintensive threat level evaluation Alert mode Prediction of levee behavior Notification of authorities
4 Visualization & decision support: selected challenges Interoperability with external systems (e.g. ISOK, regional flood protection agencies) Visualization of relevant information to effectively support the decision making process Adaptability to other domains (e.g. monitoring of communication infrastructure) Leveragingopen standards (OGC, INSPIRE) for data & metadata models Solution: research in progress Open domain agnostic design (metadata and public APIs design are crucial) Visualization & decision support system
5 Execution management: selected challenges Scale up to 100s 1000s kilometers of levees Monitored area divided into sections Managed by multiple instances of a Monitoring Application, dynamically deployed on demand Highly variable resource demands: from very low in standby mode to high in threat assessment mode Dynamic provisioning of resources from private or public clouds Autoscaling algorithms and policies Execution and Provisioning Platform (EXP) Composite App Execution Platform App model Provisioning platform Optimizer & Scheduler Scaling rules Data access (DAP) Events Enact HyperFlow Enactment Engine Execute App state Autoscaler Initial deployment Reconfigure app Provisioner Start/Stop/Reconfigure measurements Cloud Input data Executor Trigger app execution Monitoring
6 Data management: selected challenges Diverse data sets (spatial, time series, binary, metadata) and data usage patterns Multiple data stores and models to address diverse needs Data intensive processing Threat level evaluation scenario: up to 130 GB of data to search per 1km of a levee Big data infrastructure Map Reduce data search Example of data intensive analysis: threat level evaluation Real time sensor data Decision support Pre simulated scenarios (up to 130 GB of data / 1km of levee) Data intensive search Ranking of matching scenarios Global levee state & behavior prediction
7 Data Access & Analytics Platform (DAP) From sensor network communication infrastructure Sensor data collection Apache Flume propagation propagation DB interfaces Time Series DB GIS database HDFS Namenode Storage node 2nd NN Storage node Offline data analytics Storage node synchronization Online data access API Hive Pig Data access To application services Previous experience: UrbanFlood Online Flood Early Warning System Authorities Science SS S SS S Control Centre Control Centre Public
8 Conclusion ISMOP: comprehensive solution for levee monitoring & decision support Currently at the initial stage of research ISMOP Central System: visualization & decision support, execution management, data management
An approach to monitoring, data analytics, and decision support for levee supervision
An approach to monitoring, data analytics, and decision support for levee supervision M. Bubak, B. Baliś, D. Harężlak, M. Kasztelnik, P. Nowakowski, T. Bartyński, T. Gubala, M. Malawski, M. Pawlik, B.
More informationGridSpace2 Towards Science-as-a-Service Model
Polish Roadmap towards Domain-Specific Infrastructure for Supporting Computational Science in European Research Area GridSpace2 Towards Science-as-a-Service Model Eryk Ciepiela, Bartosz Wilk, Daniel Harężlak,
More informationBig Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies
Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies Big Data: Global Digital Data Growth Growing leaps and bounds by 40+% Year over Year! 2009 =.8 Zetabytes =.08
More informationDataNet Flexible Metadata Overlay over File Resources
1 DataNet Flexible Metadata Overlay over File Resources Daniel Harężlak 1, Marek Kasztelnik 1, Maciej Pawlik 1, Bartosz Wilk 1, Marian Bubak 1,2 1 ACC Cyfronet AGH, 2 AGH University of Science and Technology,
More informationSecuring the Big Data Ecosystem
Securing the Big Data Ecosystem SESSION ID: STU-T07A Davi Ottenheimer Senior Director of Trust, EMC @daviottenheimer COWS NOT PETS ( ) (xx) /-------\/ / * ---- ^^ ^^ Systematic Treatment of Illness Easily
More informationA Platform for Collaborative e-science Applications. Marian Bubak ICS / Cyfronet AGH Krakow, PL bubak@agh.edu.pl
A Platform for Collaborative e-science Applications Marian Bubak ICS / Cyfronet AGH Krakow, PL bubak@agh.edu.pl Outline Motivation Idea of an experiment Virtual laboratory Examples of experiments Summary
More informationExperiments with Complex Scientific Applications on Hybrid Cloud Infrastructures
Experiments with Complex Scientific Applications on Hybrid Cloud Infrastructures Maciej'Malawski 1,2,'Piotr'Nowakowski 1,'Tomasz'Gubała 1,'Marek'Kasztelnik 1,' Marian'Bubak 1,2,'Rafael'Ferreira'da'Silva
More informationAUTOMATIC PROXY GENERATION AND LOAD-BALANCING-BASED DYNAMIC CHOICE OF SERVICES
Computer Science 13 (3) 2012 http://dx.doi.org/10.7494/csci.2012.13.3.45 Jarosław Dąbrowski Sebastian Feduniak Bartosz Baliś Tomasz Bartyński Włodzimierz Funika AUTOMATIC PROXY GENERATION AND LOAD-BALANCING-BASED
More informationMr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo
Sensor Network Messaging Service Hive/Hadoop Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo Contents 1 Introduction 2 What & Why Sensor Network
More informationDeploying Hadoop with Manager
Deploying Hadoop with Manager SUSE Big Data Made Easier Peter Linnell / Sales Engineer plinnell@suse.com Alejandro Bonilla / Sales Engineer abonilla@suse.com 2 Hadoop Core Components 3 Typical Hadoop Distribution
More informationDistributed Cloud Environment for PL-Grid Applications
Distributed Environment for PL-Grid Applications Piotr Nowakowski, Tomasz Bartyński, Tomasz Gubała, Daniel Harężlak, Marek Kasztelnik, J. Meizner, P. Suder, M. Bubak ACC CYFRONET AGH KUKDM 2015 Zakopane,
More informationJFlooder - Application performance testing with QoS assurance
JFlooder - Application performance testing with QoS assurance Tomasz Duszka 1, Andrzej Gorecki 1, Jakub Janczak 1, Adam Nowaczyk 1 and Dominik Radziszowski 1 Institute of Computer Science, AGH UST, al.
More informationSavanna Hadoop on. OpenStack. Savanna Technical Lead
Savanna Hadoop on OpenStack Sergey Lukjanov Savanna Technical Lead Mirantis, 2013 Agenda Savanna Overview Savanna Use Cases Roadmap & Current Status Architecture & Features Overview Hadoop vs. Virtualization
More informationCertified Big Data and Apache Hadoop Developer VS-1221
Certified Big Data and Apache Hadoop Developer VS-1221 Certified Big Data and Apache Hadoop Developer Certification Code VS-1221 Vskills certification for Big Data and Apache Hadoop Developer Certification
More informationLecture 32 Big Data. 1. Big Data problem 2. Why the excitement about big data 3. What is MapReduce 4. What is Hadoop 5. Get started with Hadoop
Lecture 32 Big Data 1. Big Data problem 2. Why the excitement about big data 3. What is MapReduce 4. What is Hadoop 5. Get started with Hadoop 1 2 Big Data Problems Data explosion Data from users on social
More informationA standards-based open source processing chain for ocean modeling in the GEOSS Architecture Implementation Pilot Phase 8 (AIP-8)
NATO Science & Technology Organization Centre for Maritime Research and Experimentation (STO-CMRE) Viale San Bartolomeo, 400 19126 La Spezia, Italy A standards-based open source processing chain for ocean
More informationControl-M As an Application Management Platform
Control-M As an Application Management Platform Zagreb Lipanj 2015 Vedran Vesel, Imaves Control-M stručnjak Legal Notice The information contained in this presentation is the confidential information of
More informationAnkush Cluster Manager - Hadoop2 Technology User Guide
Ankush Cluster Manager - Hadoop2 Technology User Guide Ankush User Manual 1.5 Ankush User s Guide for Hadoop2, Version 1.5 This manual, and the accompanying software and other documentation, is protected
More informationAn Industrial Perspective on the Hadoop Ecosystem. Eldar Khalilov Pavel Valov
An Industrial Perspective on the Hadoop Ecosystem Eldar Khalilov Pavel Valov agenda 03.12.2015 2 agenda Introduction 03.12.2015 2 agenda Introduction Research goals 03.12.2015 2 agenda Introduction Research
More informationCapitalize on Big Data for Competitive Advantage with Bedrock TM, an integrated Management Platform for Hadoop Data Lakes
Capitalize on Big Data for Competitive Advantage with Bedrock TM, an integrated Management Platform for Hadoop Data Lakes Highly competitive enterprises are increasingly finding ways to maximize and accelerate
More informationModern Data Architecture for Predictive Analytics
Modern Data Architecture for Predictive Analytics David Smith VP Marketing and Community - Revolution Analytics John Kreisa VP Strategic Marketing- Hortonworks Hortonworks Inc. 2013 Page 1 Your Presenters
More informationData Lake In Action: Real-time, Closed Looped Analytics On Hadoop
1 Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop 2 Pivotal s Full Approach It s More Than Just Hadoop Pivotal Data Labs 3 Why Pivotal Exists First Movers Solve the Big Data Utility Gap
More informationHPC technology and future architecture
HPC technology and future architecture Visual Analysis for Extremely Large-Scale Scientific Computing KGT2 Internal Meeting INRIA France Benoit Lange benoit.lange@inria.fr Toàn Nguyên toan.nguyen@inria.fr
More informationHorizontal IoT Application Development using Semantic Web Technologies
Horizontal IoT Application Development using Semantic Web Technologies Soumya Kanti Datta Research Engineer Communication Systems Department Email: Soumya-Kanti.Datta@eurecom.fr Roadmap Introduction Challenges
More informationCollaborative Big Data Analytics. Copyright 2012 EMC Corporation. All rights reserved.
Collaborative Big Data Analytics 1 Big Data Is Less About Size, And More About Freedom TechCrunch!!!!!!!!! Total data: bigger than big data 451 Group Findings: Big Data Is More Extreme Than Volume Gartner!!!!!!!!!!!!!!!
More informationAN INTEGRATED SOLUTION FOR MANAGING EXPLORATION DATA
www.wipro.com AN INTEGRATED SOLUTION FOR MANAGING EXPLORATION DATA Sandipan Chakraborti, Senior Architect ENU Table of Contents 03... Introduction 04... Lack of Holistic View 06... Figure 1: Conceptual
More informationNASA s Big Data Challenges in Climate Science
NASA s Big Data Challenges in Climate Science Tsengdar Lee, Ph.D. High-end Computing Program Manager NASA Headquarters Presented at IEEE Big Data 2014 Workshop October 29, 2014 1 2 7-km GEOS-5 Nature Run
More informationITG Software Engineering
Introduction to Cloudera Course ID: Page 1 Last Updated 12/15/2014 Introduction to Cloudera Course : This 5 day course introduces the student to the Hadoop architecture, file system, and the Hadoop Ecosystem.
More informationPivotal HD Enterprise
PRODUCT DOCUMENTATION Pivotal HD Enterprise Version 1.1.1 Release Notes Rev: A02 2014 GoPivotal, Inc. Table of Contents 1 Welcome to Pivotal HD Enterprise 4 2 PHD Components 5 2.1 Core Apache Stack 5 2.2
More informationData-intensive HPC: opportunities and challenges. Patrick Valduriez
Data-intensive HPC: opportunities and challenges Patrick Valduriez Big Data Landscape Multi-$billion market! Big data = Hadoop = MapReduce? No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard,
More informationCloud services in PL-Grid and EGI Infrastructures
1 Cloud services in PL-Grid and EGI Infrastructures J. Meizner, M. Radecki, M. Pawlik, T. Szepieniec ACK Cyfronet AGH Cracow Grid Workshop 2012, Kraków, 22.10.2012 Overview 2 Different types of Compute
More informationInvestigating Hadoop for Large Spatiotemporal Processing Tasks
Investigating Hadoop for Large Spatiotemporal Processing Tasks David Strohschein dstrohschein@cga.harvard.edu Stephen Mcdonald stephenmcdonald@cga.harvard.edu Benjamin Lewis blewis@cga.harvard.edu Weihe
More informationUrbanFlood Monitoring, rich browsers and cloud service technologies for an online EWS hosting platform Work Package 6 D6.4 version 1.0, 29 Nov.
UrbanFlood Monitoring, rich browsers and cloud service technologies for an online EWS hosting platform Work Package 6 D6.4 version 1.0, 29 Nov. 2010 November 2010 URBAN FLOOD A project funded under the
More informationCAPTURING & PROCESSING REAL-TIME DATA ON AWS
CAPTURING & PROCESSING REAL-TIME DATA ON AWS @ 2015 Amazon.com, Inc. and Its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent
More informationData Governance in the Hadoop Data Lake. Kiran Kamreddy May 2015
Data Governance in the Hadoop Data Lake Kiran Kamreddy May 2015 One Data Lake: Many Definitions A centralized repository of raw data into which many data-producing streams flow and from which downstream
More informationFuture @ Cloud: Cloud Computing meets Smart Ecosystems
Future @ Cloud: Cloud Computing meets Smart Ecosystems Joerg Doerr, Fraunhofer IESE, Kaiserslautern, Germany Joerg.Doerr@iese.fraunhofer.de Fraunhofer-Institute for Experimental Software Engineering (IESE)
More informationSimplifying Big Data Analytics: Unifying Batch and Stream Processing. John Fanelli,! VP Product! In-Memory Compute Summit! June 30, 2015!!
Simplifying Big Data Analytics: Unifying Batch and Stream Processing John Fanelli,! VP Product! In-Memory Compute Summit! June 30, 2015!! Streaming Analy.cs S S S Scale- up Database Data And Compute Grid
More informationLeveraging Cloud-Based Mapping Solutions
Leveraging Cloud-Based Mapping Solutions GeoAlberta October 28, 2014 Laura Kerssens Safe Software Agenda To the Cloud Using Basic Services Cloud Applications Web Services Cloud-Hosted Databases Real-time
More informationHadoop 101. Lars George. NoSQL- Ma4ers, Cologne April 26, 2013
Hadoop 101 Lars George NoSQL- Ma4ers, Cologne April 26, 2013 1 What s Ahead? Overview of Apache Hadoop (and related tools) What it is Why it s relevant How it works No prior experience needed Feel free
More informationBringing Big Data to People
Bringing Big Data to People Microsoft s modern data platform SQL Server 2014 Analytics Platform System Microsoft Azure HDInsight Data Platform Everyone should have access to the data they need. Process
More informationCraig McWilliams Craig Burrell. Bringing Smarter, Safer Transport to NZ
Craig McWilliams Craig Burrell Bringing Smarter, Safer Transport to NZ World Class Transport. Smarter, Stronger, Safer. Bringing Smarter Safer Transport to NZ Craig Burrell Infrastructure Advisory Director
More informationHow To Scale A Server Farm
Basics of Cloud Computing Lecture 3 Scaling Applications on the Cloud Satish Srirama Outline Scaling Information Systems Scaling Enterprise Applications in the Cloud Auto Scaling 25/02/2014 Satish Srirama
More informationCloudera Administrator Training for Apache Hadoop
Cloudera Administrator Training for Apache Hadoop Duration: 4 Days Course Code: GK3901 Overview: In this hands-on course, you will be introduced to the basics of Hadoop, Hadoop Distributed File System
More informationE6893 Big Data Analytics Lecture 2: Big Data Analytics Platforms
E6893 Big Data Analytics Lecture 2: Big Data Analytics Platforms Ching-Yung Lin, Ph.D. Adjunct Professor, Dept. of Electrical Engineering and Computer Science Mgr., Dept. of Network Science and Big Data
More informationBig Data Analytics Platform @ Nokia
Big Data Analytics Platform @ Nokia 1 Selecting the Right Tool for the Right Workload Yekesa Kosuru Nokia Location & Commerce Strata + Hadoop World NY - Oct 25, 2012 Agenda Big Data Analytics Platform
More informationDevelopment and Execution of Collaborative Application on the ViroLab Virtual Laboratory
Development and Execution of Collaborative Application on the ViroLab Virtual Laboratory Marek Kasztelnik 3, Tomasz Guba la 2,3, Maciej Malawski 1, and Marian Bubak 1,3 1 Institute of Computer Science
More informationSAP MRS Multiresource Scheduling Info session - 2013. Atul Wakankar May 2013
SAP MRS Multiresource Scheduling Info session - 2013 Atul Wakankar May 2013 MRS: Foundation for the End-to-end Scheduling Process Resource Management for various Industries and different Scenarios Oil
More informationBIG DATA TRENDS AND TECHNOLOGIES
BIG DATA TRENDS AND TECHNOLOGIES THE WORLD OF DATA IS CHANGING Cloud WHAT IS BIG DATA? Big data are datasets that grow so large that they become awkward to work with using onhand database management tools.
More informationCloud Cruiser and Azure Public Rate Card API Integration
Cloud Cruiser and Azure Public Rate Card API Integration In this article: Introduction Azure Rate Card API Cloud Cruiser s Interface to Azure Rate Card API Import Data from the Azure Rate Card API Defining
More informationAgenda. Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback #EMCVIPR
1 Agenda Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback 2 A World of Connected Devices Need a new data management architecture for Internet of Things 21% the % of
More informationPRIVACY AWARE ACCESS CONTROL FOR CLOUD-BASED DATA PLATFORMS
www.openi-ict.eu Open-Source, Web-Based, Framework for Integrating Applications with Social Media Services and Personal Cloudlets PRIVACY AWARE ACCESS CONTROL FOR CLOUD-BASED DATA PLATFORMS Open-Source,
More informationArcGIS Enterprise Systems: Designing, Testing and Monitoring
Federal GIS Conference February 9 10, 2015 Washington, DC ArcGIS Enterprise Systems: Designing, Testing and Monitoring Jim VanOstenbridge, jvanostenbridge@esri.com> Martin Hamann, MHamann@esri.com Andrew
More informationOpen Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud)
Open Cloud System (Integration of Eucalyptus, Hadoop and into deployment of University Private Cloud) Thinn Thu Naing University of Computer Studies, Yangon 25 th October 2011 Open Cloud System University
More informationComprehensive Analytics on the Hortonworks Data Platform
Comprehensive Analytics on the Hortonworks Data Platform We do Hadoop. Page 1 Page 2 Back to 2005 Page 3 Vertical Scaling Page 4 Vertical Scaling Page 5 Vertical Scaling Page 6 Horizontal Scaling Page
More informationFrom Big Data to Smart Data Thomas Hahn
Siemens Future Forum @ HANNOVER MESSE 2014 From Big to Smart Hannover Messe 2014 The Evolution of Big Digital data ~ 1960 warehousing ~1986 ~1993 Big data analytics Mining ~2015 Stream processing Digital
More informationCLOUD MANAGED SERVICES FRAMEWORK E-BOOK
CLOUD MANAGED SERVICES FRAMEWORK E-BOOK TABLE OF CONTENTS 1 Introduction 2 2 Operational Insight 3 3 Cloud Management Process Control 4 4 Infrastructure, Application & Data Security 5 5 Continuous Improvement
More informationPro Apache Hadoop. Second Edition. Sameer Wadkar. Madhu Siddalingaiah
Pro Apache Hadoop Second Edition Sameer Wadkar Madhu Siddalingaiah Contents J About the Authors About the Technical Reviewer Acknowledgments Introduction xix xxi xxiii xxv Chapter 1: Motivation for Big
More informationPROPOSAL To Develop an Enterprise Scale Disease Modeling Web Portal For Ascel Bio Updated March 2015
Enterprise Scale Disease Modeling Web Portal PROPOSAL To Develop an Enterprise Scale Disease Modeling Web Portal For Ascel Bio Updated March 2015 i Last Updated: 5/8/2015 4:13 PM3/5/2015 10:00 AM Enterprise
More informationA Brief Introduction to Apache Tez
A Brief Introduction to Apache Tez Introduction It is a fact that data is basically the new currency of the modern business world. Companies that effectively maximize the value of their data (extract value
More informationThe Way to SOA Concept, Architectural Components and Organization
The Way to SOA Concept, Architectural Components and Organization Eric Scholz Director Product Management Software AG Seite 1 Goals of business and IT Business Goals Increase business agility Support new
More informationBig Data Big Data/Data Analytics & Software Development
Big Data Big Data/Data Analytics & Software Development Danairat T. danairat@gmail.com, 081-559-1446 1 Agenda Big Data Overview Business Cases and Benefits Hadoop Technology Architecture Big Data Development
More informationQsoft Inc www.qsoft-inc.com
Big Data & Hadoop Qsoft Inc www.qsoft-inc.com Course Topics 1 2 3 4 5 6 Week 1: Introduction to Big Data, Hadoop Architecture and HDFS Week 2: Setting up Hadoop Cluster Week 3: MapReduce Part 1 Week 4:
More informationBIG DATA AND ANALYTICS
BIG DATA AND ANALYTICS Björn Bjurling, bgb@sics.se Daniel Gillblad, dgi@sics.se Anders Holst, aho@sics.se Swedish Institute of Computer Science AGENDA What is big data and analytics? and why one must bother
More informationHadoop Development & BI- 0 to 100
Development Master the Data Analysis tools like Pig and hive Data Science Hadoop Development & BI- 0 to 100 Build a recommendation engine Hadoop Development - 0 to 100 HADOOP SCHOOL OF TRAINING Basics
More informationESS event: Big Data in Official Statistics. Antonino Virgillito, Istat
ESS event: Big Data in Official Statistics Antonino Virgillito, Istat v erbi v is 1 About me Head of Unit Web and BI Technologies, IT Directorate of Istat Project manager and technical coordinator of Web
More informationINTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY
INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY A PATH FOR HORIZING YOUR INNOVATIVE WORK BIG DATA HOLDS BIG PROMISE FOR SECURITY NEHA S. PAWAR, PROF. S. P. AKARTE Computer
More informationFactories of the Future Horizon 2020: LEIT ICT WP2016-17. FoF-11-2016: Digital Automation
Factories of the Future Horizon 2020: LEIT ICT WP2016-17 FoF-11-2016: Digital Automation Directorate General for Communication Networks, Content and Technology FoF PPP in Work Programme 2016-2017 FoF-11-2016:
More informationENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE
ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE Hadoop Storage-as-a-Service ABSTRACT This White Paper illustrates how EMC Elastic Cloud Storage (ECS ) can be used to streamline the Hadoop data analytics
More informationCloud-based Infrastructures. Serving INSPIRE needs
Cloud-based Infrastructures Serving INSPIRE needs INSPIRE Conference 2014 Workshop Sessions Benoit BAURENS, AKKA Technologies (F) Claudio LUCCHESE, CNR (I) June 16th, 2014 This content by the InGeoCloudS
More informationAngelos Tzotsos IMIS Athena, Scientific & Technical Manager OSGeo Charter Member Copernicus Big Data Workshop, March 2014
Angelos Tzotsos IMIS Athena, Scientific & Technical Manager OSGeo Charter Member Copernicus Big Data Workshop, March 2014 Genesis of the project Consortium established on active research, commercial, and
More informationAzure Data Lake Analytics
Azure Data Lake Analytics Compose and orchestrate data services at scale Fully managed service to support orchestration of data movement and processing Connect to relational or non-relational data
More informationOpen source Google-style large scale data analysis with Hadoop
Open source Google-style large scale data analysis with Hadoop Ioannis Konstantinou Email: ikons@cslab.ece.ntua.gr Web: http://www.cslab.ntua.gr/~ikons Computing Systems Laboratory School of Electrical
More informationCase Study : 3 different hadoop cluster deployments
Case Study : 3 different hadoop cluster deployments Lee moon soo moon@nflabs.com HDFS as a Storage Last 4 years, our HDFS clusters, stored Customer 1500 TB+ data safely served 375,000 TB+ data to customer
More informationHortonworks CISC Innovation day
Hortonworks CISC Innovation day Simon gregory sgregory@hortonworks.com Here was the ask Hortonworks' data reposition - how this works and the types of data you work with. 1: Data Types & Value. What have
More informationChase Wu New Jersey Ins0tute of Technology
CS 698: Special Topics in Big Data Chapter 4. Big Data Analytics Platforms Chase Wu New Jersey Ins0tute of Technology Some of the slides have been provided through the courtesy of Dr. Ching-Yung Lin at
More informationANALYTICS IN BIG DATA ERA
ANALYTICS IN BIG DATA ERA ANALYTICS TECHNOLOGY AND ARCHITECTURE TO MANAGE VELOCITY AND VARIETY, DISCOVER RELATIONSHIPS AND CLASSIFY HUGE AMOUNT OF DATA MAURIZIO SALUSTI SAS Copyr i g ht 2012, SAS Ins titut
More informationEnabling Access to Environmental. on the Web
ENVISION Overview Enabling Access to Environmental Models, Data, and Services on the Web Dumitru Roman dumitru.roman@sintef.no Bucharest, November 13, 2012 http://www.envision-project.eu/ ENVISION Motivation
More informationDeploying Predictive Analytics Solutions in the Rail Industry and Seeing a Return on Investment
Robert Morris, Ph.D. Co- Founder & Chief Science Officer Predikto, Inc. Deploying Predictive Analytics Solutions in the Rail Industry and Seeing a Return on Investment SHOWING BUSINESS VALUE FROM DEPLOYED
More informationCOMP9321 Web Application Engineering
COMP9321 Web Application Engineering Semester 2, 2015 Dr. Amin Beheshti Service Oriented Computing Group, CSE, UNSW Australia Week 11 (Part II) http://webapps.cse.unsw.edu.au/webcms2/course/index.php?cid=2411
More informationReal-time Monitoring Platform to Improve the Drinking Water Network Efficiency
Real-time Monitoring Platform to Improve the Drinking Water Network Efficiency SUMMARY Recent advances in real-time water sensing technology have enabled new opportunities for improved assessment and management
More informationBig Data: Making Sense of it all!
Big Data: Making Sense of it all! Jamie Engesser E-mail : jamie@hortonworks.com Page 1 Data Driven Business? Facts not Intuition! Data driven decisions are better decisions its as simple as that. Using
More informationApplication of Petri Nets to Evaluation of Grid Applications Efficiency
Application of Petri Nets to Evaluation of Grid Applications Efficiency Wojciech Rząsa 1, Marian Bubak 2,3 (1) Rzeszow University of Technology, Poland (2) Science AGH, Krakow, Poland (3) ACC Cyfronet,
More informationHadoop Introduction. Olivier Renault Solution Engineer - Hortonworks
Hadoop Introduction Olivier Renault Solution Engineer - Hortonworks Hortonworks A Brief History of Apache Hadoop Apache Project Established Yahoo! begins to Operate at scale Hortonworks Data Platform 2013
More informationHDP Hadoop From concept to deployment.
HDP Hadoop From concept to deployment. Ankur Gupta Senior Solutions Engineer Rackspace: Page 41 27 th Jan 2015 Where are you in your Hadoop Journey? A. Researching our options B. Currently evaluating some
More informationData processing goes big
Test report: Integration Big Data Edition Data processing goes big Dr. Götz Güttich Integration is a powerful set of tools to access, transform, move and synchronize data. With more than 450 connectors,
More informationBig Data Applications in Disaster Management
Taiwan 2012 Bridging Big Data Infrastructure Workshop Case Studies and Needs: Disaster Mitigation Whey-Fone Tsai National Center for High-performance Computing National Applied Research Laboratories Hsinchu,
More informationXpoLog Competitive Comparison Sheet
XpoLog Competitive Comparison Sheet New frontier in big log data analysis and application intelligence Technical white paper May 2015 XpoLog, a data analysis and management platform for applications' IT
More informationApache Hadoop: The Big Data Refinery
Architecting the Future of Big Data Whitepaper Apache Hadoop: The Big Data Refinery Introduction Big data has become an extremely popular term, due to the well-documented explosion in the amount of data
More informationInformation Builders Mission & Value Proposition
Value 10/06/2015 2015 MapR Technologies 2015 MapR Technologies 1 Information Builders Mission & Value Proposition Economies of Scale & Increasing Returns (Note: Not to be confused with diminishing returns
More informationA common interface for multi-rule-engine distributed systems
A common interface for multi-rule-engine distributed systems Pierre de Leusse, Bartosz Kwolek and Krzysztof Zieliński Distributed System Research Group, AGH University of Science and Technology Krakow,
More informationOracle s Big Data solutions. Roger Wullschleger. <Insert Picture Here>
s Big Data solutions Roger Wullschleger DBTA Workshop on Big Data, Cloud Data Management and NoSQL 10. October 2012, Stade de Suisse, Berne 1 The following is intended to outline
More informationBig Data and Analytics: A Conceptual Overview. Mike Park Erik Hoel
Big Data and Analytics: A Conceptual Overview Mike Park Erik Hoel In this technical workshop This presentation is for anyone that uses ArcGIS and is interested in analyzing large amounts of data We will
More informationAgile Infrastructure Update Monitoring
Agile Infrastructure Update Monitoring Pedro Andrade IT/GT 6 th July 2012 IT Technical Forum CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it Overview Introduction Motivation, Challenge,
More informationBig Data Spatial Analytics An Introduction
2013 Esri International User Conference July 8 12, 2013 San Diego, California Technical Workshop Big Data Spatial Analytics An Introduction Marwa Mabrouk Mansour Raad Esri iu UC2013. Technical Workshop
More informationData Analyst Program- 0 to 100
Development Data Analyst Program- 0 to 100 Master the Data Analysis tools like Pig and hive Data Science Build a recommendation engine 1 Data Analyst Program- 0 to 100 HADOOP SCHOOL OF TRAINING Basics
More informationProviding drivers with actionable intelligence can minimize accidents, reduce driver claims and increase your bottom line. Equip motorists with the
Providing drivers with actionable intelligence can minimize accidents, reduce driver claims and increase your bottom line. Equip motorists with the ability to make informed decisions based on reliable,
More informationBig Data Course Highlights
Big Data Course Highlights The Big Data course will start with the basics of Linux which are required to get started with Big Data and then slowly progress from some of the basics of Hadoop/Big Data (like
More informationAdobe s Story of Integrating Hadoop and SAP HANA with SAP Data Services
Orange County Convention Center Orlando, Florida June 3-5, 2014 Adobe s Story of Integrating Hadoop and SAP HANA with SAP Data Services Kevin Davis, Senior Data Warehouse Engineer, Adobe Hemant Puranik,
More informationInternational Journal of Advancements in Research & Technology, Volume 3, Issue 2, February-2014 10 ISSN 2278-7763
International Journal of Advancements in Research & Technology, Volume 3, Issue 2, February-2014 10 A Discussion on Testing Hadoop Applications Sevuga Perumal Chidambaram ABSTRACT The purpose of analysing
More informationTECHNOLOGY WHITE PAPER Jun 2012
TECHNOLOGY WHITE PAPER Jun 2012 Technology Stack C# Windows Server 2008 PHP Amazon Web Services (AWS) Route 53 Elastic Load Balancing (ELB) Elastic Compute Cloud (EC2) Amazon RDS Amazon S3 Elasticache
More information