Research of Smart Distribution Network Big Data Model
|
|
|
- Rudolf Gilmore
- 10 years ago
- Views:
Transcription
1 Research of Smart Distribution Network Big Data Model Guangyi LIU Yang YU Feng GAO Wendong ZHU China Electric Power Stanford Smart Grid Research Institute Smart Grid Research Institute Research Institute University North America North America ABSTRACT In this research, we tease out three rules of the big data for the electricity sector and induce a systematic criteria framework for building and evaluating data models for supporting the electricity-system big data. On the basis of the three rules, we induce a criteria system, which contains five types of criteria, to building a data model or evaluate the performance of an existing data model to facilitate the electricity-system big data. We then build a big-data model for Chinese electricity system according to the criteria system. In order to satisfy the requirements of the criteria, we choose CIM model to unify the protocol of data collection and communication. We also explore a data-classification policy to functionally structure the data and maintain their spatial and sequential relations. Finally, we suggest a framework to integrate the database into a platform by using multiple data-storage and data-communication technologies. INTRODUCTION Even though the current development of data-mining technologies provides opportunities for big-data analysis for the power system, the lack of a utility unified data model blocks the efficient data integration and module deployment [1]. This research therefore builds an experimental big-data model to explore the mechanism for framing the big data to support the real-time operation and adaptive innovation of the smart distribution network [3]. The big-data use for the smart distribution network faces to a sequence of challenges differing from its use in other system; therefore the experimental data model is designed aiming to explore the solutions for these challenges. One main challenge is that the data gathered from the smart distribution network has complicate logic structures, multiple sources, and large volumes [4]. Therefore, any single database technology is insufficient to support the big data analysis in the electricity system. In addition, because of the lack of understanding for use of the data, the data model must comprehensively and smartly keep all the information about facilities, market participants, and relations connecting them [3]. Furthermore, the data model must support the development of the intellectual and interactive visualized operation system [3]. 2 CRITERIA SYSTEM FOR BUILDING AND EVALUATING AN ELECTRIC SYSTEM DATA MODEL 2.1Three rules of building a big data system for electric power system In this section we tease out three rules for building a big data system for electric power system, which is supported by an electric system data model. Therefore, an effective electric system data model must be designed to comprehensively facilitate the functionalization of the three rules. The three rules for building a big data system for electric power system are 1. Comprehensive: the system should appropriately store, process, and communicate all available data and information and integrate all available tools and interfaces for data analyses 2. Adaptive and dynamic: the system should be adaptively organized, dynamic innovated, and function-based structured to support all kinds of dynamic data process and analyses. 3. Distributed and multi-layered: the system should be multi-layered; and each layer or knot need to be, to some extent, intelligent, such that the data and information can be effectively processed and analyzed in appropriate layer or knot on the system. The three rules are different from the rules for a big-data system in other field because of the electricity systems particular physical characteristics and operational requirements. We summarize these three rules by embedded the characteristic of a big data system into considerations of electric system operation. In particularly, we would like to emphasize that the three rules listed above are for the building of a smart-grid system. A smart-grid system, which is a decentralized system equipped with intelligent processing and communicating facilities at multi layers in the system, is an integrated platform to support the utilization of integrated distributed renewable-energy sources, smart meters, and the technologies of demand-side response. CIRED /5
2 Functionally be structured Effectively convert information value to profit Adaptive & Dynamic Distributed & Multi-layered Deal with big-data sets and a big set of small datasets Comprehensive Support the smartgrid technologies Capture the sequential and spatial structure Figure 1 Three Rules and five criteria 2.2Requirements for an effective electricitysystem data model According to the three rules of building a big-data system for electric power system, we correspondingly induce a criteria system for an effective data model, which is used to collect sample, storage data, and support data analysis under a big-data environment in an electricity system. The criteria system contains five types of criteria. In figure 1, we describe the relation between the three rules and their corresponding criteria types. According to the criteria system, we not only could evaluate the effectiveness of a data model, but also design a new framework Capture the sequential and spatial structure The electricity-system data has heterogeneous data forms, which are determined by the sampling frequency, and contains complicated relations, which reflect the hierarchical network structure of the electric system. The electricity-system big data is highly time-scaled heterogeneous. The frequency for data sampling varies from one time per microsecond to one time per month or year. For example, the data captured by the Phasor Management Unit (PMU) is sampled per millisecond. In contrast, the data used for the resource-capacity planning only include the monthly or yearly information. The large variation of sampling sequence not only causes the big difference of information included in the data, but also requires different recording methods. For instance, the data from PMU is stream data. Therefore, the datarecording system must timely tease out the information from the data and determine how to deal with the data. In contrast, the data for resource-capacity planning does not need to be highly frequently analyzed but must include a broad set of information, such as demographic change and economical growth forecast. In addition to the data-form heterogeneity caused by the time sequential, the other characteristic of the big data for electric system is that the data include information about complicated spatial relation. The data collected by smart meter is centralized into the data center at each node in the distributed networks and then submitted to the system-control center. All data not only include the situation at the position where the data is collected, but also contain the information about how demand, supply, and power flow perform with the constraints of the grid topology. Therefore, we suggest the first requirement for an effective electric power system data model as follows. Type 1 criteria: An effective big data model for electric power system should have an adaptive data-sampling protocol according to the data s time-sequential characteristic and contain the information about the spatial relation between different data sources in well-organized data storage framework Deal with big-data sets and a big set of small datasets The electric power system data is included in different databases. A few of the sub systems have very huge database. Each of these databases contains big data sets, for example, the smart-meter data. Many other sub systems have databases with limited size. However, the number of these small databases is large enough, such that the set of these databases can be considered as big data. Then, we have the following requirement Type 2 criteria: An effective data model should have ability to parallel process multiple big-data datasets and effectively and timely connect multiple small datasets among a large number of datasets Support the smart-grid technologies CIRED /5
3 The big data for electric power system is the foundation of the smart-grid creation. A smart grid system is built upon the technologies of interactive devices, internet, data analytics, and distributed control. The data model must effectively connect with these technologies and support the target functions of operating a smart-grid system. The basic functions of a smart-grid system include the energy management at the terminal devices, the adaptive response to the market signal from the agents on the distributed networks, and the distributed communication and control for the electricity dispatch and frequency regulation [3][6]. In order to support the energy management at the terminal devices and the adaptive response from the agents on the distributed network, the electricity-system data model need the ability to analyze high-frequency stream data and differentiate the information to be submitted from raw data timely treated. In order to support the distributed process and control, the data model should be designed to associate with the optimal distributed signal-communication algorithm and distributed control system. Type3 criteria: An effective data model for the big data in an electricity system should support the functions of converting terminal device to management platform, distributed processing and controlling, and adaptively distributed-side response Functionally be structured In contrast with the current data model, the electricitysystem big-data model need to be structured to support the function of system operation by integrating data and information from multi-sources. The current data model is separated by collection systems because the marginal benefit of functionally integrating databases cannot balance its costs. Then, the same data can be collected by two systems and then stored in different databases. However, the value of functionally integrating databases in a big data system is sufficiently large. Furthermore, the collection-system-based data model significantly weakens the ability of using new information from big data to benefit the smart-grid building and operation. The challenges of creating a function-based integrated data model include not only the effective storing data but also integrating data with different formations and types. For instance, the demand-response management needs stream data collecting from smart meter and also pointwise data from feeder monitors. In some cases, the data model needs to support the processing of effectively integrating the raw data with some processed information from data processors and computers on other layers of the system. Type 4 criteria: An effective electricity big data model should be functionally structured such that Data is effectively collected and adaptively connected according to the possible functions. Data with different types and formations can be appropriately sorted for the utilization Effectively support the function of convert information value to profit The big-data model for an electricity system must have ability to convert the information value to profit. The bigdata technologies reveal a great amount of information, for instance individual consumer s hourly load data and high frequent PMU data. These pieces of data include valuable information for improving the whole-system efficiency. Therefore, the acquisition of these data and information provides the opportunity to the whole system to gain profit from reducing the costs caused by information incompleteness and asymmetry. However, because of the large size of the database and sparse information matrix, the data-model design highly affects the effectiveness of converting potential information value to profit. The data model must appropriately support multi tools for profit mining. There is a sequence of tools of mining the value of the information contained in the electricitysystem big data. For example, the information about consumers behaviors collected from smarter meter must have appropriate tools to analyses. Simultaneously, a tool is needed to match the consumers with the distributed renewable-energy capacities. Therefore, the data model must support multi tools. The data model also needs to support the new market designs and mechanisms that utilizing big data to improve the social efficiency. For instance, the big data made the trade inside a distributed system is available. However, a new market mechanism and a dispatch protocol need to be designed and implemented. The data model is a foundation of the new mechanisms and protocols built upon big data. Type 5 criteria: An effective electricity big data model should Effectively support analyses and processing CIRED /5
4 tools for converting potential value contained in the data into profit. Effectively support the new market designs and management protocols built upon the electricitysystem big data 3. DATA MODEL FOR THE ELECTRICITY- SYSTEM BIG DATA We in the last section systematically build a criteria framework for evaluating and building a data model for an electricity-system big-data model. With considering the length of the paper, we do not include the example of evaluating a current data model by our criteria. We in the section provide an example to build a big data model for Chinese electricity system. We would like to emphasize that the data model need to particularly design for each individual electricity system because the operation protocols are different by electricity systems. However, our criteria framework can be generalized to any electricity systems. In this section, we first select technologies used to frame the data model. Then, we build a policy to functionally classify and store data according to spatial and sequential relations of data. At last, we state how to create the integrated platform of the data model according to knowledge-management theory. 3.1 Technologies used for data-model building From criteria type 1 to type 4, we argue that the core issue of the big-data model data is effective data structure and connection. Therefore, the unified data model integrates the data according to the relational model based on the Common Information Model (CIM) [4]. We choose the CIM model because The CIM model provides a common language for data collection from different devices located in different layers of the system. Furthermore, the CIM has a uniform standard and protocol for data communication and classifies and connects data. The CIM model is simultaneously able to functionally connect data and information and appropriately maintain the spatial and sequential relations in the real world [4]. 3.2 Data classification and functional organization In order to functionally structure the data, we summarize the data groups, which are in the Chinese electricity system and grouped according to the geographic collecting position, data formation, and colleting frequency. This information reflects the spatial and sequential relations of the data, as well as the characteristics inside a dataset. Then, we map the classes to their functions in the smart-grid system. According to the mapping, we reclassify the data. We in Table 1 show this classification method. Table 1Function-based structure for the electricity-system big data model CIRED /5
5 Each cell of the Table 1 represents a data class. We reclassify data in this way to effectively reflect the information of data s functions, maintain the data s spatial and sequential relations, and minimize the efforts to record these pieces of information. From the table, we show that our classification is not restricted by the sources of the data collection. Instead, the data, which is used for the same types of tasks, are grouped together. Furthermore, the data is organized in a hierarchic protocol. These hierarchic structures support the multi-layered adaptive intelligent structure of a smart grid system Knowledge management and data storage The unified data model is built upon a technology portfolio of state-of-the-art computing. In the experimental platform, a data-storage-and-processing system based on the Hadoop technology supports an integration platform and an exploring platform. The Hadoop system provides an environment for storing semi-structure and non-structure data, and exploring the optimal solution for the data framing [2]. The integration platform, which is built upon the Data Warehouse technology, is for the instant inquiry and the decision making for the data analysis and the module development. The exploring platform, which is based on the paralleling processing technology, is established for visualized analysis and data mining. The modules for data analysis, inquiry and decision-making are integrated into the two platforms [5]. The unified data model also provides a research environment to examine the optimal technology portfolio to achieve cost-efficiently and stably data management and analysis. A self-reinforce learning process on the basis of the RM theory makes the data mode adaptive and self-innovative. A policy classifying the data will be established according to the uses and the structures of the data. According to their uses, the data will be classified to cold, warm, and hot data. The data will simultaneously be grouped as structured, unstructured, and stream data according to their logic structures. The storage protocol is determined by the classification of the data. supporting the electricity-system big data. The three rules, which include comprehensive, adaptive and dynamic, and distributed and multilayered, are the basic principles for the smart-grid building and data-mode framing. On the basis of the three rules, we induce a criteria system, which contains five types of criteria, to building a data model or evaluate the performance of a exist data model to facilitate the electricity-system big data. We then build a big-data model for Chinese electricity system according to the criteria system.in order to satisfy the requirements of the criteria, we choose CIM model to unify the protocol of data collection and communication. We also explore a data-classification policy to functionally structure the data and maintain their spatial and sequential relations. Finally, we suggest a framework to integrate the database into a platform by using multiple data-storage and data-communication technologies. Reference [1] K.Bakshi, 2012, "Considerations for big data: Architecture and approach." In Aerospace Conference, 2012 IEEE, pp [2] P.Zikopoulos & C. Eaton, 2011, Understanding big data: Analytics for enterprise class hadoop and streaming data, McGraw-Hill Osborne Media. [3] D.Agrawal,S. Das, & A.El Abbadi,2011, "Big data and cloud computing: current state and future opportunities", Proceedings the 14th International Conference on Extending Database Technology ACM, [4] P.C.Chen, T. DOKIC, & M.KEZUNOVIC 2014, "The Use of Big Data for Outage Management in Distribution Systems" Proceedings International Conference on Electricity Distribution (CIRED) Workshop. [5] NoSQL Architecture Web Site: e/ [6] A. Jacobs, 2009, The pathologies of big data. Communications of the ACM, Vol.52 no.8, CONCLUSION In this research, we tease out three rules of the big data for the electricity sector and induce a systematic criteria framework for building and evaluating data models for CIRED /5
Big Data Storage Architecture Design in Cloud Computing
Big Data Storage Architecture Design in Cloud Computing Xuebin Chen 1, Shi Wang 1( ), Yanyan Dong 1, and Xu Wang 2 1 College of Science, North China University of Science and Technology, Tangshan, Hebei,
Volume 3, Issue 6, June 2015 International Journal of Advance Research in Computer Science and Management Studies
Volume 3, Issue 6, June 2015 International Journal of Advance Research in Computer Science and Management Studies Research Article / Survey Paper / Case Study Available online at: www.ijarcsms.com Image
Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control
Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control EP/K006487/1 UK PI: Prof Gareth Taylor (BU) China PI: Prof Yong-Hua Song (THU) Consortium UK Members: Brunel University
Client Overview. Engagement Situation. Key Requirements
Client Overview Our client is one of the leading providers of business intelligence systems for customers especially in BFSI space that needs intensive data analysis of huge amounts of data for their decision
UPS battery remote monitoring system in cloud computing
, pp.11-15 http://dx.doi.org/10.14257/astl.2014.53.03 UPS battery remote monitoring system in cloud computing Shiwei Li, Haiying Wang, Qi Fan School of Automation, Harbin University of Science and Technology
Indian Journal of Science The International Journal for Science ISSN 2319 7730 EISSN 2319 7749 2016 Discovery Publication. All Rights Reserved
Indian Journal of Science The International Journal for Science ISSN 2319 7730 EISSN 2319 7749 2016 Discovery Publication. All Rights Reserved Perspective Big Data Framework for Healthcare using Hadoop
Associate Professor, Department of CSE, Shri Vishnu Engineering College for Women, Andhra Pradesh, India 2
Volume 6, Issue 3, March 2016 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Special Issue
TRENDS IN THE DEVELOPMENT OF BUSINESS INTELLIGENCE SYSTEMS
9 8 TRENDS IN THE DEVELOPMENT OF BUSINESS INTELLIGENCE SYSTEMS Assist. Prof. Latinka Todoranova Econ Lit C 810 Information technology is a highly dynamic field of research. As part of it, business intelligence
International Journal of Engineering Research ISSN: 2348-4039 & Management Technology November-2015 Volume 2, Issue-6
International Journal of Engineering Research ISSN: 2348-4039 & Management Technology Email: [email protected] November-2015 Volume 2, Issue-6 www.ijermt.org Modeling Big Data Characteristics for Discovering
The 4 Pillars of Technosoft s Big Data Practice
beyond possible Big Use End-user applications Big Analytics Visualisation tools Big Analytical tools Big management systems The 4 Pillars of Technosoft s Big Practice Overview Businesses have long managed
LV4MV: A CONCEPT FOR OPTIMAL POWER FLOW MANAGEMENT IN DISTRIBUTION GRIDS, USING DER FLEXIBILITY
LV4MV: A CONCEPT FOR OPTIMAL POWER FLOW MANAGEMENT IN DISTRIBUTION GRIDS, USING DER FLEXIBILITY Emmanuelle VANET Gaspard LEBEL Raphaël CAIRE G2Elab France G2Elab France G2Elab - France [email protected]
CONNECTING DATA WITH BUSINESS
CONNECTING DATA WITH BUSINESS Big Data and Data Science consulting Business Value through Data Knowledge Synergic Partners is a specialized Big Data, Data Science and Data Engineering consultancy firm
INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY
INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY A PATH FOR HORIZING YOUR INNOVATIVE WORK A REVIEW ON HIGH PERFORMANCE DATA STORAGE ARCHITECTURE OF BIGDATA USING HDFS MS.
A CIM-Based Framework for Utility Big Data Analytics
A CIM-Based Framework for Utility Big Data Analytics Jun Zhu John Baranowski James Shen Power Info LLC Andrew Ford Albert Electrical PJM Interconnect LLC System Operator Overview Opportunities & Challenges
BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON
BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON Overview * Introduction * Multiple faces of Big Data * Challenges of Big Data * Cloud Computing
CONCEPTUAL MODEL OF MULTI-AGENT BUSINESS COLLABORATION BASED ON CLOUD WORKFLOW
CONCEPTUAL MODEL OF MULTI-AGENT BUSINESS COLLABORATION BASED ON CLOUD WORKFLOW 1 XINQIN GAO, 2 MINGSHUN YANG, 3 YONG LIU, 4 XIAOLI HOU School of Mechanical and Precision Instrument Engineering, Xi'an University
Hadoop Technology for Flow Analysis of the Internet Traffic
Hadoop Technology for Flow Analysis of the Internet Traffic Rakshitha Kiran P PG Scholar, Dept. of C.S, Shree Devi Institute of Technology, Mangalore, Karnataka, India ABSTRACT: Flow analysis of the internet
SeaCloudDM: Massive Heterogeneous Sensor Data Management in the Internet of Things
SeaCloudDM: Massive Heterogeneous Sensor Data Management in the Internet of Things Jiajie Xu Institute of Software, Chinese Academy of Sciences (ISCAS) 2012-05-15 Outline 1. Challenges in IoT Data Management
CHAPTER SIX DATA. Business Intelligence. 2011 The McGraw-Hill Companies, All Rights Reserved
CHAPTER SIX DATA Business Intelligence 2011 The McGraw-Hill Companies, All Rights Reserved 2 CHAPTER OVERVIEW SECTION 6.1 Data, Information, Databases The Business Benefits of High-Quality Information
Evolution of Information System
Information Systems Classification Evolution of Information System The first business application of computers (in the mid- 1950s) performed repetitive, high-volume, transaction-computing tasks. The computers
Big Data and Advanced Analytics Technologies for the Smart Grid
1 Big Data and Advanced Analytics Technologies for the Smart Grid Arnie de Castro, PhD SAS Institute IEEE PES 2014 General Meeting July 27-31, 2014 Panel Session: Using Smart Grid Data to Improve Planning,
Data Mining to Characterize Signatures of Impending System Events or Performance from PMU Measurements
Data Mining to Characterize Signatures of Impending System Events or Performance from PMU Measurements Final Project Report Power Systems Engineering Research Center Empowering Minds to Engineer the Future
Information Services for Smart Grids
Smart Grid and Renewable Energy, 2009, 8 12 Published Online September 2009 (http://www.scirp.org/journal/sgre/). ABSTRACT Interconnected and integrated electrical power systems, by their very dynamic
A Conceptual Approach to Data Visualization for User Interface Design of Smart Grid Operation Tools
A Conceptual Approach to Data Visualization for User Interface Design of Smart Grid Operation Tools Dong-Joo Kang and Sunju Park Yonsei University [email protected], [email protected] Abstract
locuz.com Big Data Services
locuz.com Big Data Services Big Data At Locuz, we help the enterprise move from being a data-limited to a data-driven one, thereby enabling smarter, faster decisions that result in better business outcome.
Synchronized real time data: a new foundation for the Electric Power Grid.
Synchronized real time data: a new foundation for the Electric Power Grid. Pat Kennedy and Chuck Wells Conjecture: Synchronized GPS based data time stamping, high data sampling rates, phasor measurements
Wide Area Monitoring, Control, and Protection
Wide Area Monitoring, Control, and Protection Course Map Acronyms Wide Area Monitoring Systems (WAMS) Wide Area Monitoring Control Systems (WAMCS) Wide Area Monitoring Protection and Control Systems (WAMPACS)
BIG DATA IN SUPPLY CHAIN MANAGEMENT: AN EXPLORATORY STUDY
Gheorghe MILITARU Politehnica University of Bucharest, Romania Massimo POLLIFRONI University of Turin, Italy Alexandra IOANID Politehnica University of Bucharest, Romania BIG DATA IN SUPPLY CHAIN MANAGEMENT:
Formal Methods for Preserving Privacy for Big Data Extraction Software
Formal Methods for Preserving Privacy for Big Data Extraction Software M. Brian Blake and Iman Saleh Abstract University of Miami, Coral Gables, FL Given the inexpensive nature and increasing availability
Building a Scalable Big Data Infrastructure for Dynamic Workflows
Building a Scalable Big Data Infrastructure for Dynamic Workflows INTRODUCTION Organizations of all types and sizes are looking to big data to help them make faster, more intelligent decisions. Many efforts
ORGANIZATIONAL KNOWLEDGE MAPPING BASED ON LIBRARY INFORMATION SYSTEM
ORGANIZATIONAL KNOWLEDGE MAPPING BASED ON LIBRARY INFORMATION SYSTEM IRANDOC CASE STUDY Ammar Jalalimanesh a,*, Elaheh Homayounvala a a Information engineering department, Iranian Research Institute for
Big Data and Analytics: Getting Started with ArcGIS. Mike Park Erik Hoel
Big Data and Analytics: Getting Started with ArcGIS Mike Park Erik Hoel Agenda Overview of big data Distributed computation User experience Data management Big data What is it? Big Data is a loosely defined
Smart Grid Demonstration Lessons & Opportunities to Turn Data into Value
Smart Grid Demonstration Lessons & Opportunities to Turn Data into Value Matt Wakefield Senior Program Manager Berlin, Germany December 3rd, 2012 EPRI Smart Grid Demonstration Projects Integration of Distributed
Data-intensive HPC: opportunities and challenges. Patrick Valduriez
Data-intensive HPC: opportunities and challenges Patrick Valduriez Big Data Landscape Multi-$billion market! Big data = Hadoop = MapReduce? No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard,
Enabling the SmartGrid through Cloud Computing
Enabling the SmartGrid through Cloud Computing April 2012 Creating Value, Delivering Results 2012 eglobaltech Incorporated. Tech, Inc. All rights reserved. 1 Overall Objective To deliver electricity from
An Agent-Based Concept for Problem Management Systems to Enhance Reliability
An Agent-Based Concept for Problem Management Systems to Enhance Reliability H. Wang, N. Jazdi, P. Goehner A defective component in an industrial automation system affects only a limited number of sub
5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014
5 Keys to Unlocking the Big Data Analytics Puzzle Anurag Tandon Director, Product Marketing March 26, 2014 1 A Little About Us A global footprint. A proven innovator. A leader in enterprise analytics for
Next Generation Grid Data Architecture & Analytics Required for the Future Grid
& Analytics Required for the Future Grid Arjun Shankar and Russell Robertson Team: Lin Zhu, Frank Liu, Jim Nutaro, Yilu Liu, and Tom King 1 2 2 Project Purpose PURPOSE To foster open collaboration on issues,
Smart Grid Data Management Challenges
Smart Grid Data Management Challenges Marie-Luce PICARD EDF R&D [email protected] 16 Novembre 2010 Outline 1. Smart grids : What? Where? What for? 2. A road map for Smart Grids functionalities and
Lluis Belanche + Alfredo Vellido. Intelligent Data Analysis and Data Mining
Lluis Belanche + Alfredo Vellido Intelligent Data Analysis and Data Mining a.k.a. Data Mining II Office 319, Omega, BCN EET, office 107, TR 2, Terrassa [email protected] skype, gtalk: avellido Tels.:
Pentaho High-Performance Big Data Reference Configurations using Cisco Unified Computing System
Pentaho High-Performance Big Data Reference Configurations using Cisco Unified Computing System By Jake Cornelius Senior Vice President of Products Pentaho June 1, 2012 Pentaho Delivers High-Performance
Understanding traffic flow
White Paper A Real-time Data Hub For Smarter City Applications Intelligent Transportation Innovation for Real-time Traffic Flow Analytics with Dynamic Congestion Management 2 Understanding traffic flow
I. TODAY S UTILITY INFRASTRUCTURE vs. FUTURE USE CASES...1 II. MARKET & PLATFORM REQUIREMENTS...2
www.vitria.com TABLE OF CONTENTS I. TODAY S UTILITY INFRASTRUCTURE vs. FUTURE USE CASES...1 II. MARKET & PLATFORM REQUIREMENTS...2 III. COMPLEMENTING UTILITY IT ARCHITECTURES WITH THE VITRIA PLATFORM FOR
Memory Database Application in the Processing of Huge Amounts of Data Daqiang Xiao 1, Qi Qian 2, Jianhua Yang 3, Guang Chen 4
5th International Conference on Advanced Materials and Computer Science (ICAMCS 2016) Memory Database Application in the Processing of Huge Amounts of Data Daqiang Xiao 1, Qi Qian 2, Jianhua Yang 3, Guang
Keywords Big Data, NoSQL, Relational Databases, Decision Making using Big Data, Hadoop
Volume 4, Issue 1, January 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Transitioning
Towards Cloud Factory Simulation. Abstract
Towards Cloud Factory Simulation 第 十 八 屆 決 策 分 析 研 討 會 Toly Chen Department of Industrial Engineering and Systems Management, Feng Chia University *[email protected] Abstract An important and practical
Design of Electric Energy Acquisition System on Hadoop
, pp.47-54 http://dx.doi.org/10.14257/ijgdc.2015.8.5.04 Design of Electric Energy Acquisition System on Hadoop Yi Wu 1 and Jianjun Zhou 2 1 School of Information Science and Technology, Heilongjiang University
The Key Technology Research of Virtual Laboratory based On Cloud Computing Ling Zhang
International Conference on Advances in Mechanical Engineering and Industrial Informatics (AMEII 2015) The Key Technology Research of Virtual Laboratory based On Cloud Computing Ling Zhang Nanjing Communications
Dong-Joo Kang* Dong-Kyun Kang** Balho H. Kim***
Visualization Issues of Mass Data for Efficient HMI Design on Control System in Electric Power Industry Visualization in Computerized Operation & Simulation Tools Dong-Joo Kang* Dong-Kyun Kang** Balho
Business Challenges and Research Directions of Management Analytics in the Big Data Era
Business Challenges and Research Directions of Management Analytics in the Big Data Era Abstract Big data analytics have been embraced as a disruptive technology that will reshape business intelligence,
Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges
Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges Prerita Gupta Research Scholar, DAV College, Chandigarh Dr. Harmunish Taneja Department of Computer Science and
Data Refinery with Big Data Aspects
International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data
CHAPTER FIVE RESULT ANALYSIS
CHAPTER FIVE RESULT ANALYSIS 5.1 Chapter Introduction 5.2 Discussion of Results 5.3 Performance Comparisons 5.4 Chapter Summary 61 5.1 Chapter Introduction This chapter outlines the results obtained from
Opportunities to Overcome Key Challenges
The Electricity Transmission System Opportunities to Overcome Key Challenges Summary Results of Breakout Group Discussions Electricity Transmission Workshop Double Tree Crystal City, Arlington, Virginia
IEEE 2015-2016 JAVA TITLES
ECWAY ECHNOLGIES IEEE 2015-2016 JAVA TITLES BE, B.TECH, ME, M.TECH, MSC, MCA PROJECTS Abstract: Introduction: Literature Survey: System Analysis: Existing System: Disadvantages: Proposed System: Advantages:
Dr. John E. Kelly III Senior Vice President, Director of Research. Differentiating IBM: Research
Dr. John E. Kelly III Senior Vice President, Director of Research Differentiating IBM: Research IBM Research Priorities Impact on IBM and the Marketplace Globalization and Leverage Balanced Research Agenda
Business Intelligence and Decision Support Systems
Chapter 12 Business Intelligence and Decision Support Systems Information Technology For Management 7 th Edition Turban & Volonino Based on lecture slides by L. Beaubien, Providence College John Wiley
Map-Parallel Scheduling (mps) using Hadoop environment for job scheduler and time span for Multicore Processors
Map-Parallel Scheduling (mps) using Hadoop environment for job scheduler and time span for Sudarsanam P Abstract G. Singaravel Parallel computing is an base mechanism for data process with scheduling task,
Research and realization of Resource Cloud Encapsulation in Cloud Manufacturing
www.ijcsi.org 579 Research and realization of Resource Cloud Encapsulation in Cloud Manufacturing Zhang Ming 1, Hu Chunyang 2 1 Department of Teaching and Practicing, Guilin University of Electronic Technology
An Integrated Big Data & Analytics Infrastructure June 14, 2012 Robert Stackowiak, VP Oracle ESG Data Systems Architecture
An Integrated Big Data & Analytics Infrastructure June 14, 2012 Robert Stackowiak, VP ESG Data Systems Architecture Big Data & Analytics as a Service Components Unstructured Data / Sparse Data of Value
Log Mining Based on Hadoop s Map and Reduce Technique
Log Mining Based on Hadoop s Map and Reduce Technique ABSTRACT: Anuja Pandit Department of Computer Science, [email protected] Amruta Deshpande Department of Computer Science, [email protected]
How To Manage Assets In Utilities
Preparing for the Future: How Asset Management Will Evolve in the Age of Smart Grid By: Jeff Meyers, Smart Grid Solutions Telvent Utilities Group Table of Contents Introduction 3 The State of Asset Management
Research on IT Architecture of Heterogeneous Big Data
Journal of Applied Science and Engineering, Vol. 18, No. 2, pp. 135 142 (2015) DOI: 10.6180/jase.2015.18.2.05 Research on IT Architecture of Heterogeneous Big Data Yun Liu*, Qi Wang and Hai-Qiang Chen
Big Data Are You Ready? Jorge Plascencia Solution Architect Manager
Big Data Are You Ready? Jorge Plascencia Solution Architect Manager Big Data: The Datafication Of Everything Thoughts Devices Processes Thoughts Things Processes Run the Business Organize data to do something
http://www.paper.edu.cn
5 10 15 20 25 30 35 A platform for massive railway information data storage # SHAN Xu 1, WANG Genying 1, LIU Lin 2** (1. Key Laboratory of Communication and Information Systems, Beijing Municipal Commission
Digital Modernization of Oilfields Digital Oilfield to Intelligent Oilfield. Karamay Hongyou Software Co., Ltd.
Digital Modernization of Oilfields Digital Oilfield to Intelligent Oilfield Karamay Hongyou Software Co., Ltd. Professional technologies and information technology enhance the development of the oil industry.
IoT Analytics: Four Key Essentials and Four Target Industries
IoT Analytics: Four Key Essentials and Four Target Industries 1 Introduction Analysts and IT personnel across all industries seek technology to better engage and manage data generated by the Internet of
Information Systems and Technologies in Organizations
Information Systems and Technologies in Organizations Information System One that collects, processes, stores, analyzes, and disseminates information for a specific purpose Is school register an information
Chapter Managing Knowledge in the Digital Firm
Chapter Managing Knowledge in the Digital Firm Essay Questions: 1. What is knowledge management? Briefly outline the knowledge management chain. 2. Identify the three major types of knowledge management
BUILDING A SCALABLE BIG DATA INFRASTRUCTURE FOR DYNAMIC WORKFLOWS
BUILDING A SCALABLE BIG DATA INFRASTRUCTURE FOR DYNAMIC WORKFLOWS ESSENTIALS Executive Summary Big Data is placing new demands on IT infrastructures. The challenge is how to meet growing performance demands
ORACLE UTILITIES ANALYTICS
ORACLE UTILITIES ANALYTICS TRANSFORMING COMPLEX DATA INTO BUSINESS VALUE UTILITIES FOCUS ON ANALYTICS Aging infrastructure. Escalating customer expectations. Demand growth. The challenges are many. And
Three steps to put Predictive Analytics to Work
Three steps to put Predictive Analytics to Work The most powerful examples of analytic success use Decision Management to deploy analytic insight in day to day operations helping organizations make more
Understanding Web personalization with Web Usage Mining and its Application: Recommender System
Understanding Web personalization with Web Usage Mining and its Application: Recommender System Manoj Swami 1, Prof. Manasi Kulkarni 2 1 M.Tech (Computer-NIMS), VJTI, Mumbai. 2 Department of Computer Technology,
Master big data to optimize the oil and gas lifecycle
Viewpoint paper Master big data to optimize the oil and gas lifecycle Information management and analytics (IM&A) helps move decisions from reactive to predictive Table of contents 4 Getting a handle on
SURVEY REPORT DATA SCIENCE SOCIETY 2014
SURVEY REPORT DATA SCIENCE SOCIETY 2014 TABLE OF CONTENTS Contents About the Initiative 1 Report Summary 2 Participants Info 3 Participants Expertise 6 Suggested Discussion Topics 7 Selected Responses
International Conference on Advances in Energy, Environment and Chemical Engineering (AEECE-2015)
International Conference on Advances in Energy, Environment and Chemical Engineering (AEECE-2015) An Optimal Reactive Power Planning Software for Urban Network Qiang Sun 1, a, Xue Wang 1,b, Yingting Ni
Applied research on data mining platform for weather forecast based on cloud storage
Applied research on data mining platform for weather forecast based on cloud storage Haiyan Song¹, Leixiao Li 2* and Yuhong Fan 3* 1 Department of Software Engineering t, Inner Mongolia Electronic Information
A Risk Management System Framework for New Product Development (NPD)
2011 International Conference on Economics and Finance Research IPEDR vol.4 (2011) (2011) IACSIT Press, Singapore A Risk Management System Framework for New Product Development (NPD) Seonmuk Park, Jongseong
Cloud Storage Solution for WSN Based on Internet Innovation Union
Cloud Storage Solution for WSN Based on Internet Innovation Union Tongrang Fan 1, Xuan Zhang 1, Feng Gao 1 1 School of Information Science and Technology, Shijiazhuang Tiedao University, Shijiazhuang,
Available online at www.sciencedirect.com Available online at www.sciencedirect.com. Advanced in Control Engineering and Information Science
Available online at www.sciencedirect.com Available online at www.sciencedirect.com Procedia Procedia Engineering Engineering 00 (2011) 15 (2011) 000 000 1822 1826 Procedia Engineering www.elsevier.com/locate/procedia
Towards Privacy aware Big Data analytics
Towards Privacy aware Big Data analytics Pietro Colombo, Barbara Carminati, and Elena Ferrari Department of Theoretical and Applied Sciences, University of Insubria, Via Mazzini 5, 21100 - Varese, Italy
Figure 1: Architecture of a cloud services model for a digital education resource management system.
World Transactions on Engineering and Technology Education Vol.13, No.3, 2015 2015 WIETE Cloud service model for the management and sharing of massive amounts of digital education resources Binwen Huang
Microsoft Big Data. Solution Brief
Microsoft Big Data Solution Brief Contents Introduction... 2 The Microsoft Big Data Solution... 3 Key Benefits... 3 Immersive Insight, Wherever You Are... 3 Connecting with the World s Data... 3 Any Data,
Scaling Objectivity Database Performance with Panasas Scale-Out NAS Storage
White Paper Scaling Objectivity Database Performance with Panasas Scale-Out NAS Storage A Benchmark Report August 211 Background Objectivity/DB uses a powerful distributed processing architecture to manage
Image Analytics on Big Data In Motion Implementation of Image Analytics CCL in Apache Kafka and Storm
Image Analytics on Big Data In Motion Implementation of Image Analytics CCL in Apache Kafka and Storm Lokesh Babu Rao 1 C. Elayaraja 2 1PG Student, Dept. of ECE, Dhaanish Ahmed College of Engineering,
An Experimental Approach Towards Big Data for Analyzing Memory Utilization on a Hadoop cluster using HDFS and MapReduce.
An Experimental Approach Towards Big Data for Analyzing Memory Utilization on a Hadoop cluster using HDFS and MapReduce. Amrit Pal Stdt, Dept of Computer Engineering and Application, National Institute
International Journal of Innovative Research in Computer and Communication Engineering
FP Tree Algorithm and Approaches in Big Data T.Rathika 1, J.Senthil Murugan 2 Assistant Professor, Department of CSE, SRM University, Ramapuram Campus, Chennai, Tamil Nadu,India 1 Assistant Professor,
Evolution of Control for the Smart Transmission Grid
Evolution of Control for the Smart Transmission Grid Anjan Bose Washington State University Pullman, WA SCE Smart Grid Research Symposium California Institute Of Technology October, 2011 The Past (before
Toward standards for dynamics in future electric energy systems The basis for plug and play industry paradigm
Toward standards for dynamics in future electric energy systems The basis for plug and play industry paradigm Marija Ilić [email protected] Electric Energy Systems Group (EESG) http://www.eesg.ece.cmu.edu/,
Preparing for the Future: How Asset Management Will Evolve in the Age of the Smart Grid
Preparing for the Future: How Asset Management Will Evolve in the Age of the Smart Grid Executive summary Most utilities struggle to organize information about their distribution network assets. Operations,
Big Data and Analytics in Government
Big Data and Analytics in Government Nov 29, 2012 Mark Johnson Director, Engineered Systems Program 2 Agenda What Big Data Is Government Big Data Use Cases Building a Complete Information Solution Conclusion
Problems to store, transfer and process the Big Data 6/2/2016 GIANG TRAN - [email protected] 1
Problems to store, transfer and process the Big Data COURSE: COMPUTING CLUSTERS, GRIDS, AND CLOUDS LECTURER: ANDREY SHEVEL ITMO UNIVERSITY SAINT PETERSBURG 6/2/2016 GIANG TRAN - [email protected]
Keywords: Dynamic Load Balancing, Process Migration, Load Indices, Threshold Level, Response Time, Process Age.
Volume 3, Issue 10, October 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Load Measurement
