UPENET (UPGRADE European NETwork) CEPIS NEWS

CEPIS UPGRADE is the European Journal for the Informatics Professional, published bimonthly at <http://cepis.org/upgrade> Publisher CEPIS UPGRADE is published by CEPIS (Council of European Professional Informatics Societies, <http://www. cepis.org/>), in cooperation with the Spanish CEPIS society ATI (Asociación de Técnicos de Informática, <http:// www.ati.es/>) and its journal Novática CEPIS UPGRADE monographs are published jointly with Novática, that publishes them in Spanish (full version printed; summary, abstracts and some articles online) CEPIS UPGRADE was created in October 2000 by CEPIS and was first published by Novática and INFORMATIK/INFORMATIQUE, bimonthly journal of SVI/FSI (Swiss Federation of Professional Informatics Societies) CEPIS UPGRADE is the anchor point for UPENET (UPGRADE European NETwork), the network of CEPIS member societies publications, that currently includes the following ones: inforewiew, magazine from the Serbian CEPIS society JISA Informatica, journal from the Slovenian CEPIS society SDI Informatik-Spektrum, journal published by Springer Verlag on behalf of the CEPIS societies GI, Germany, and SI, Switzerland ITNOW, magazine published by Oxford University Press on behalf of the British CEPIS society BCS Mondo Digitale, digital journal from the Italian CEPIS society AICA Novática, journal from the Spanish CEPIS society ATI OCG Journal, journal from the Austrian CEPIS society OCG Pliroforiki, journal from the Cyprus CEPIS society CCS Tölvumál, journal from the Icelandic CEPIS society ISIP Editorial TeamEditorial Team Chief Editor: Llorenç Pagés-Casas Deputy Chief Editor: Rafael Fernández Calvo Associate Editor: Fiona Fanning Editorial Board Prof. Vasile Baltac, CEPIS President Prof. Wolffried Stucky, CEPIS Former President Prof. Nello Scarabottolo, CEPIS President Elect Luis Fernández-Sanz, ATI (Spain) Llorenç Pagés-Casas, ATI (Spain) François Louis Nicolet, SI (Switzerland) Roberto Carniel, ALSI Tecnoteca (Italy) UPENET Advisory Board Dubravka Dukic (inforeview, Serbia) Matjaz Gams (Informatica, Slovenia) Hermann Engesser (Informatik-Spektrum, Germany and Switzerland) Brian Runciman (ITNOW, United Kingdom) Franco Filippazzi (Mondo Digitale, Italy) Llorenç Pagés-Casas (Novática, Spain) Veith Risak (OCG Journal, Austria) Panicos Masouras (Pliroforiki, Cyprus) Thorvardur Kári Ólafsson (Tölvumál, Iceland) Rafael Fernández Calvo (Coordination) English Language Editors: Mike Andersson, David Cash, Arthur Cook, Tracey Darch, Laura Davies, Nick Dunn, Rodney Fennemore, Hilary Green, Roger Harris, Jim Holder, Pat Moody. Cover page designed by Concha Arias-Pérez "Luminous Recharge" / ATI 2011 Layout Design: François Louis Nicolet Composition: Jorge Llácer-Gil de Ramales Editorial correspondence: Llorenç Pagés-Casas <pages@ati.es> Advertising correspondence: <info@cepis.org> Subscriptions If you wish to subscribe to CEPIS UPGRADE please send an email to info@cepis.org with Subscribe to UPGRADE as the subject of the email or follow the link Subscribe to UPGRADE at <http://www.cepis.org/upgrade> Copyright Novática 2011 (for the monograph) CEPIS 2011 (for the sections Editorial, UPENET and CEPIS News) All rights reserved under otherwise stated. Abstracting is permitted with credit to the source. For copying, reprint, or republication permission, contact the Editorial Team The opinions expressed by the authors are their exclusive responsibility ISSN 1684-5285 Monograph of next issue (December 2011) "Risk Management" Vol. XII, issue No. 4, October 2011 Monograph Green ICT: Trends and Challenges (published jointly with Novática*) Guest Editors: Juan-Carlos López-López, Giovanna Sissa, and Lasse Natvig 2 Presentation. Green ICT: The Information Society s Commitment for Environmental Sustainability Juan-Carlos López-López, Giovanna Sissa, and Lasse Natvig 6 CEPIS Green ICT Survey Examining Green ICT Awareness in Organisations: Initial Findings Carol-Ann Kogelman on behalf of the CEPIS Green ICT Task Force 11 The Five Most Neglected Issues in "Green IT" Lorenz M. Hilty and Wolfgang Lohmann 16 Utility Computing: Green Opportunities and Risks Giovanna Sissa 22 Good, Bad, and Beautiful Software In Search of Green Software Quality Factors Juha Taina 28 Towards the Virtual Power Grid: Large Scale Modeling and Simulation of Power Grids Peter Feldmann, Jinjun Xiong, and David Kung 41 Artificial Intelligence Techniques for Smart Grid Applications María-José Santofimia-Romero, Xavier del Toro-García, and Juan-Carlos López-López 49 Green Computing: Saving Energy by Throttling, Simplicity and Parallelization Lasse Natvig and Alexandru C. Iordan 59 Towards Sustainable Solutions for European Cloud Computing Kien Le, Thu D. Nguyen, Íñigo Goiri, Ricardo Bianchini, Jordi Guitart-Fernández, and Jordi Torres-Viñals 67 A State-of-the-Art on Energy Efficiency in Today s Datacentres: Researcher s Contributions and Practical Approaches Marina Zapater-Sancho, Patricia Arroba-García, José-Manuel Moya-Fernández, and Zorana Bankovic UPENET (UPGRADE European NETwork) 75 From Mondo Digitale (AICA, Italy) IT for Education IT in Schools. A European Project for Teachers Training Pierfranco Ravotto and Giovanni Fulantelli CEPIS NEWS 81 Selected CEPIS News Fiona Fanning * This monograph will be also published in Spanish (full version printed; summary, abstracts, and some articles online) by Novática, journal of the Spanish CEPIS society ATI (Asociación de Técnicos de Informática) at <http://www.ati.es/novatica/>.

Presentation Green ICT: The Information Society s Commitment for Environmental Sustainability Juan-Carlos López-López, Giovanna Sissa, and Lasse Natvig ICT and Sustainability A growing social awareness of environmental problems and the rational use of the planet s resources has caused Public Administrations to show an increasing concern for these issues and, in consequence, to deal with economical and social development from this new perspective. Thus, one of the pillars of the Information Society strategy of the European Union (i2010 Lisbon Strategy) is precisely the application of Information and Communication Technologies (ICT) to improve the quality of life, and to foster environmental care and sustainable development. The ICT sector has been and will be a basic element in the generation of wealth, not just in the sense of allowing new business models, but by offering new services which foster innovation in other more "traditional" sectors. Besides its role as a support for future economical growth, this sector shows a great ability to contribute with innovative solutions towards a productive model based on sustainable development. Several reports have already shown the capacity of ICT to help reduce CO 2 emissions, either by decreasing the carbon footprint of the own sector activity, or by helping, by means of applications and services, to efficiently manage energy resources in other critical sectors, such as industry, power, transport and others. Thus, ICT help reduce the environmental impact of human activity, by working on both the design of new hardware devices and systems (low power devices and systems, efficient solar cells, sensors ), and the development of new software technologies, applications and services (information management, communications, artificial intelligence, etc.). This monograph reviews how ICT can play multiple roles towards improving environmental sustainability. Managing the ICT Sector s own Activity According to a Gartner report (see the "Useful References" box), the total footprint of the ICT sector in 2007 was about 2% of the estimated total emissions from human activity released that year. This amount is expected to reach more than 6% in 2020. This means that one of the targets of this sector regarding environmental sustainability should be the reduction of its own environmental costs, arising from the use of equipment and ICT solutions. Efforts should be concentrated on two main areas: The Guest Editors Juan-Carlos López-López received the MS and PhD degrees in Telecommunication (Electrical) Engineering from the Universidad Politécnica de Madrid, Spain, in 1985 and 1989, respectively. From September 1990 to August 1992, he was a Visiting Scientist in the Dept. t of Electrical and Computer Engineering at Carnegie Mellon University, Pittsburgh, PA USA. His research activities center on embedded system design, distributed computing and advanced communication services. From 1989 to 1999, he has been an Associate Professor of the Dept. of Electrical Engineering at the Universidad Politécnica de Madrid. Currently, Dr. López is a Professor of Computer Architecture at the Universidad de Castilla-La Mancha, Spain, where he served as Dean of the School of Computer Science from 2000 to 2008. He has been member of different panels of the Spanish National Science Foundation and the Spanish Ministry of Education and Science, regarding Information Technologies research programs. He is member of the IEEE and the ACM. <juancarlos.lopez@uclm.es> Giovanna Sissa was awarded a degree in Physics in 1978. She has been involved in ICT since 1980, as Strategic Consultant for Industries and Public Administrations, for the identification of emerging technologies and technological trends in informatics. She is a member of the Italian Register of Technological Innovation Experts (Ministry of Economical Development) and a member of the Scientific Committee of the Piedmont Regional System for Research and Innovation. In 2000 she founded, and managed until 2009, the Osservatorio Tecnologico of the Italian Ministry of Education. Engaged in research on ICT and sustainability, she has written books and papers about the environmental impact of ICT. As a PhD student of Informatics at the Università degli Studi di Milano her research focus is on ICT-driven societal behavioral changes and their effect on environmental sustainability. <sissa.giovanna@ gmail.com> Lasse Natvig received the MS and Dr. Ing. degrees in Computer Science from the Norwegian Institute of Technology, NTNU, in 1982 and 1991, respectively. He is currently a Professor in Computer Architecture at the Norwegian University of Science and Technology (NTNU), Trondheim, Norway. His main research interests are computer architecture, parallel processing, multi-core systems, system-level computer simulation, memory architectures and green computing. Dr.Natvig is a full member of HiPEAC2 Webpage: <http://www.idi.ntnu.no/people/lasse>. <Lasse@computer.org> 2 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

This monograph intends to review some of the issues related to the green side of ICT, but also tries to encourage the scientific and professional ICT community to play an active role in the Green ICT challenge Energy consumption of ICT infrastructures (data centres, computers, communication equipment, etc.). Disposal and recycling of ICT equipment (electrical and electronic devices, materials and components of diverse nature, such as plastic, magnetic, or chemical). ICT Solutions to manage other Sectors Activities The use of ICT for intelligent resource management has been well known for decades. Different types of applications have already shown ICT potential, and it is time now to extend their capabilities to other fields in which several studies have evaluated their impact as really promising (see the Smart 2020 report). In fact, these fields constitute priority areas for the R&D programs of various administrations, providing solutions for the most energy intensive sectors. Among them, it is worth mentioning: Smart cities/buildings: utilities (electricity, water, etc.) management, early detection and, potentially, self-healing of failures and breakdowns, etc. Smart transportation of goods and people: logistics, positioning and routes (reduction of fossil combustion greenhouse gases), and electric vehicles. Smart grids: grid operation and management (load balancing, active demand management), Wide Area Monitoring and Control (WAMC). ICT as a Driving Force towards a more Sustainable Social Model The potential of ICT to offer new ways of interaction between people and their environment as well as new mechanisms to ease the management of the production processes (industrial, innovation ) is unquestionable. This can bring about a reshaping of the social model, incorporating ways to address the growing environmental concern, and thereby reinforcing the information society s commitment to sustainable development. In this sense, one of ICT s main contributions to the development of both social and industrial and corporate activities is based on the elimination of unnecessary displacements, offering the so-called solutions "by substitution". That is, ICT allow the replacement of physical movements by information movements, thereby reducing the consumption of energy and materials. This has led to services such as telecommuting, distance education and remote learning, telemedicine, teleconferencing, electronic commerce and business, remote assistance for the elderly and disabled, and electronic government. On the other hand, the so-called dematerialization enables material intensive physical products and services to be replaced by their virtual equivalents: e-billing or e-health services, for example, avoid the production of paper tickets or X-ray films. These ICT-based opportunities offer important savings in energy and material and, therefore, environmental improvements, but they impact enormously on people s relationships both with one another and with the social environment, leading to new individual and collective behaviours. On the other hand, information systems help to efficiently manage production and business processes, allowing the automation of information processing, information that nowadays also includes energy and environmental parameters that can be useful in decision-making processes. Green ICT Challenges As we have seen, ICT can be pivotal towards a more sustainable world, increasing environmental awareness and fostering greener behaviours. In this sense, the role played by ICT is important. But not just ICT-related skills are required. Both the reduction of the environmental impact of the ICT sector itself or the use of ICT applications to enable green behaviours in other sectors, also involve environmental skills together with a better awareness of the environmental implications of people behaviours. Moreover, ICT professionals should be conscious of the Green ICT opportunities, but also of their potential risks. A green ICT strategy has to start by increasing public knowledge about ICT and their effects on the environment, and by supporting environmental-related ICT skills and education. If computer scientists and professionals have to go green, a new awareness and culture has to be encouraged. This is the key to really fostering Green ICT. This monograph, thus, intends to review some of the issues related to the green side of ICT, but also tries to encourage the scientific and professional ICT community to play an active role in the Green ICT challenge. One of the pillars of the Information Society strategy of the European Union is the application of ICT to improve the quality of life, and to foster environmental care and sustainable development Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 3

ICT allow the replacement of physical movements by information movements, thereby reducing the consumption of energy and materials In this Monograph The monograph starts with the paper "CEPIS Green ICT Survey Examining Green ICT Awareness in Organizations: Initial Findings" which describes the initial results of a survey carried out by the CEPIS Green ICT Task Force. It includes over three dozen comprehensive questions about "green awareness" in ICT companies and has gathered more than 300 responses. The final results, which will be compiled into a pan-european report, depict how ICT Managers in different countries implement energy efficient methods in their organizations. A focus on sustainability from a broader point view is given in "The Five Most Neglected Issues in Green IT". The authors focus on five issues concerning the "greening" of ICT which are usually overlooked and that in the long term, they argue, will be essential for the sustainability of ICT. Consequently it will be necessary to understand the ICT life cycle better, so as to be able to support a green strategy for the sector. In "Utility Computing: Green Opportunities and Risks", the author puts the accent on green potentials and risks of computing for the environment. The view is taken from the software developers side, alluding to fact that they have great power but a great responsibility, as a result of the possibility of available computing resources being used like a utility (like gas or water). The contribution "Good, Bad, and Beautiful Software - In Search for Green Software Quality Factors" goes deeper into the software life cycle approach, with a visionary and specific accent on the role played by software in a "green software engineering" perspective. The author looks for some "green software quality factors" that enable some metrics for green software to be established. The Smart Grid is currently one of the main fields of application which demonstrates the green capabilities of ICT. The paper "Towards the Virtual Power Grid: Large Scale Modelling and Simulation of Power Grids" reviews the major disciplines that need to be evolved and converged to meet the challenges of future energy grids. The authors propose a model of distribution network that increases the efficiency of power production, control and management. The core of this model is a virtual power grid (VPG) which is able to provide complete real-time visibility of the grid at a full dynamic system level. Focusing once again on the Smart Grid, the paper "Artificial Intelligence Techniques for Smart Grid Applications" shows how one of the main computing fields offers an efficient way to wisely manage the energy distribution network. Both artificial intelligence and computational intelligence techniques are being applied to address different issues such as the real-time management of the dynamic behaviour of the electricity network, by integrating renewable energy sources or by providing fault tolerance and selfhealing. In the paper "Green Computing: Saving Energy by Throttling, Simplicity and Parallelization", the authors start by giving a broad overview of the techniques used in making computers more energy efficient in all market segments from embedded systems to supercomputers. They introduce energy-saving techniques in hardware, continue at the operating system level and end by describing techniques at the application level. They explain how parallelization in both HW and SW can help reduce the energy consumption, and how throttling and simplicity can give the same effect. The paper "Towards Sustainable Solutions for European Cloud Computing" discusses different issues related with the emerging Cloud Computing paradigm. Besides issues such as the privacy and availability of data, the efficient use of energy resources poses new research challenges. In particular, the authors centre the discussion on how to wisely manage applications execution according to the dynamic behaviour of renewable energy sources. They propose scheduling applications so as to maximize green energy consumption while meeting job deadlines. Finally, the dramatically increasing energy consumption of datacentres is also addressed by the paper "A Stateof-the-Art on Energy Efficiency in Today s Datacentres: Researcher s Contributions and Practical Approaches". The authors explain how the concept of energy efficiency has become an increasingly important design issue. The paper also addresses the importance of reliability, how it interacts with energy efficiency and how cooling turns out to be another important concern. Finally, the gap existing between research contributions in this field and the products offered by industry is highlighted. Acknowledgements We would like to thank all the authors for their willingness to contribute to this special issue. We really appreciate the quality of their work and their efforts to get them in on time. We must also mention the UPGRADE Editorial Team, in particular the Chief Editor, Llorenç Pagés-Casas, for his help and support (and patience) in bringing out this issue. ICT professionals should be conscious of the Green ICT opportunities, but also of their potential risks 4 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

Useful References on "Green ICT" In addition to the materials referenced by the authors in their articles, we offer the following ones for those who wish to dig deeper into the topics covered by the monograph. Books, Articles, and Reports S. Alberts. "Energy-efficient algorithms". Communications of the ACM, Vol. 53 No. 5, 2010. S. Collier. "Ten Steps to a Smarter Grid". IEEE Industry Applications Magazine 16.2 : 62-68, 2010. M. Ebbers, A. Galea, M. Schaefer, M. Tu Duy Khiem. "The Green Data Center: Steps for the Journey", IBM Redpaper REDP-4413, 2010. <http://ibm.com/redbooks>. L. Erdmann, L. Hilty. "Scenario Analysis: Exploring the Macroeconomic Impacts of Information and Communication Technologies on Greenhouse Gas Emissions". Journal of Industrial Ecology, 14: 826 843, 2010. Gartner Inc. (2008), "Green IT: The new Industry Shockwave". Presentation at Symposium/ITXPO conference, 2008. <http://www.gartner.com/it/page.jsp?id=503867>. Greenpeace International. "How dirty is your data?". Greenpeace International, Amsterdam, The Netherlands, 2011. M. Hashmi. "Survey of smart grid concepts worldwide". VTT, Espoo, VTT Working Papers, 2011. S. Kaxiras, M. Martonosi. "Computer Architecture Techniques for Power-Efficiency". Synthesis Lectures on Computer Architecture, Mark D. Hill Series Editor, 2008. R. Kuehr, E. Williams. "Computers and the Environment: Understanding and Managing their Impacts". Kluwer Academic Publisher, Dordrect, The Netherland, 2003. J. Laudon. "Performance/Watt: the new server focus", ACM SIGARCH Computer Architecture News, Volume 33, Issue 4, 2005. E. Masanet, H.S. Matthews. "Exploring Environmental Applications and Benefits of Information and Communication Technology". Journal of Industrial Ecology, 14: 687 691 j.1530-9290.2010.00285.x, 2010. S. Murugesan. "Harnessing Green IT", Principles and Practices, IT Professional, Vol.10, 1, pp. 24-33, 2008. OECD. "Toward green ICT Strategies: Accessing Policies and programmes on ICT and the Environment". OECD Publishing, Paris, 2009. OECD. "Measuring the relationship between ICT and the Environment". OECD Publishing, Paris, 2009. OECD. "Greener and smarter ICTs the environment and Climate Change". OECD Publishing, Paris, 2010. OECD. "Technology Roadmap: Smart Grids, IEA Technology Roadmaps". OECD Publishing, Paris, 2011. P. Ranganathan. "Recipe for Efficiency: Principles of Power-Aware Computing". Communications of the ACM, Vol. 53, No. 4, 2010. The Climate Group. "Smart2020: Enabling the low carbon economy in the information age", 2008. <http://www. theclimategroup.org/_assets/files/smart2020report.pdf>. Some Web Sites Worth Visiting ClimateSavers. Started by Google and Intel in 2007, the Climate Savers Computing Initiative is a not-for-profit group of eco-conscious consumers, businesses and conservation organizations. Their goal is to promote the development, deployment and adoption of smart technologies that can both improve the efficiency of a computer s power delivery and reduce the energy consumed when the computer is in an inactive state. <http://www.climatesavers computing.org/>. ERCIM on Green ICT. ERCIM news, special Theme Towards Green ICT. <http://ercim-news.ercim.eu/en79>. Green500. Rankings of the most energy-efficient supercomputers in the world. <http://www.green500.org/>. MINECC. EU 7FP FP7: FET Proactive Initiative: Minimising Energy Consumption of Computing to the Limit (MINECC). <http://cordis.europa.eu/fp7/ict/fet-proactive/ minecc_en.html>. Mont Blanc Project. Is a newly started EU project with the objectives to develop a fully energy-efficient HPC prototype using low-power commercially available embedded technology. <http://www.montblanc-project.eu/objectives>. Smart Grids in Europe. The European Technology Platform for Electricity Networks of the Future, also called SmartGrids ETP, is the key European forum for the crystallization of policy and technology research and development pathways for the smart grids sector, as well as the linking glue between EU-level related initiatives. <http://www. smartgrids.eu/>. Smart Grid in the USA. Web site of the US Department of Energy that gathers all the information on federal initiatives supporting the development of technologies, policies and projects transforming the electric power industry. <http:/ /www.smartgrid.gov/>. Smart Grid News. <http://www.smartgridnews.com/>. The Green Grid. The Green Grid is a not-for-profit, open industry consortium of end-users, policy-makers, technology providers, facility architects, and utility companies collaborating to improve the resource efficiency of data centres and business computing ecosystems. <http://www. thegreengrid.org/>. ZeroPower. The goal of this project is to create a coordination activity among consortia involved in "Toward Zero- Power ICT" research projects (FET proactive call FP7-ICT- 2009-5, Objective 8.6) and communities of scientists interested in energy harvesting and low power, energy efficient ICT. <http://www.zero-power.eu>. Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 5

CEPIS Green ICT Survey Examining Green ICT Awareness in Organisations: Initial Findings Carol-Ann Kogelman on behalf of the CEPIS Green ICT Task Force Green ICT is an increasingly important issue for many organisations across Europe today. This paper presents the initial findings of research to assess how ICT Managers in different countries implement energy efficient methods in their organisations. The CEPIS Green ICT Task Force is conducting this work and these preliminary results are based on 300 responses received so far. Keywords: CEPIS, Digital Agenda for Europe Green ICT, Energy Efficiency, Europe 2020, Green ICT Task Force, ICT, ICT Manager, Sustainability. 1 Introduction In recent years, being green has become synonymous with the far-reaching possibilities of ICT in achieving energy-efficiency for the ICT sector itself, and in all other dayto-day activities of citizens, organisations, and government administrations, to name but a few. ICT has a key role in enabling energy efficiency in most areas, including the reduction of its own sector s carbon emissions. Organisations are coming under increasing scrutiny for how they are using energy efficient methods in the day-to-day running of the business, and whether energy is being wasted through various ICT-related processes. The European Commission has embarked on a number of activities to address the enabling role that the ICT sector can play in diminishing the high carbon emissions of the ICT sector itself, and in various other sectors to become more energy efficient. For example as part of the Europe 2020 strategy with the Digital Agenda for Europe initiative, two specific actions 1 are focused at analysing and managing the energy consumption of the ICT sector and other major emitting sectors. In addition to these highly effective actions, the European Commission is also currently working on developing guides to calculate the environmental footprint of products 2 and companies 3 in general. The European Commission s Europe 2020 strategy has targeted three key areas for sustainable growth: 1. 20% increase in energy efficiency 2. 20% reduction of greenhouse gas emissions 3. increase the share of renewables by 20% Organisations in particular have a responsibility in ensuring the energy efficient use of their ICT products, processes and services as much as possible. Since ICT Managers of organisations are usually responsible for the management, installation and maintenance of ICT hardware and 1 See <http://ec.europa.eu/information_society/newsroom/cf/pillar. cfm?pillar_id=49&pillar=ict%20for%20social%20challenges>. 2 See <http://ec.europa.eu/environment/eussd/product_ footprint. htm>. 3 See <http://ec.europa.eu/environment/eussd/corporate_ footprint.htm>. Author The CEPIS Green ICT Task Force carries out the strategic objectives of CEPIS around Green ICT, including but not limited to promoting the concept of Green ICT across Europe and contributing to the protection of the environment through the creating and disseminating of good practices. The Task Force is composed of a group of experts from various CEPIS Member Societies across Europe and led by Manolis Labovas from the Greek Member Society, Hellenic Professionals Informatics Society (HePIS). Members of the group include Matei Dimitriu from Asociatia Pentru Tehnologia Informatiei si Comunicatii in Romania; Francisco Esteve and Luis Fernández-Sanz from Asociación de Técnicos de Informática, ATI, in Spain; Laura Georg from Swiss Informatics Society, Panagiotis Georgiadis from HePIS; Peter Lawless from the Irish Computer Society; Marco Mevius from German Informatics; Volker Schanz from Informations technische Gesellschaft im Verband der Elektrotechnik Elektronik Informationstechnik in Germany; Giovanna Sissa from Associazione Italiana per l Informatica ed il Calcolo Automatico, AICA, in Italy; Arjan Van Dijk from Vereniging van Register Informatica in The Netherlands; Chris Wallace from British Computer Society; and Brian Warrington from Computer Society Malta. The Green ICT Task Force has its own LinkedIn group, at <http://www.linkedin.com/groups?mostpopular=&gid= 3899686&trk=myg_ugrp_ovr>. This paper has been authored by Carol-Ann Kogelman from the CEPIS Secretariat on behalf of the Green ICT Task Force. Contact <carolann.kogelman@cepis.org> software, they are ideal candidates for assessing whether a culture of energy efficiency exists in European organisations. The Council of European Professional Informatics This paper presents the preliminary results of the survey conducted by the CEPIS Green ICT Task Force 6 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

Since ICT Managers of organisations are usually responsible for the management of ICT HW and SW, they are ideal candidates for assessing whether a culture of energy efficiency exists in European organisations Figure 1: Implementation of Green IT Practices. Societies, CEPIS, in 2011 through its Green ICT Task Force launched the CEPIS Green ICT Survey. The CEPIS Green ICT Survey is aimed at ICT Managers, to examine awareness regarding energy consumption & energy efficiency of ICT equipment & policies within organisations. The survey s final results will be compiled into a pan-european report depicting how ICT Managers in different countries implement energy efficient methods in their organisations. This can provide useful business information that many ICT managers may find important, in order to see how their organisation s ICT usage & green ICT practices compare to organisations in other countries. Members of the CEPIS Green ICT Task Force brought together their expertise on the topic of Green ICT and created this survey that includes over three dozen comprehensive questions. The Task Force Members represent over 10 different countries in Europe, including Germany, Greece, Ireland, Italy, Malta, Romania, Spain, Switzerland, The Netherlands, and UK. Since launching the survey, the Task Force aims to achieve at least 50 respondents in each of their own countries. Such a result will achieve a fair and balanced sample of responses to create a comparative analysis of Green ICT awareness in organisations across Europe today. The data collection phase for this research is still ongoing and so far the survey has reached a substantial number of countries through Task Force Members efforts. Here below we provide you with the initial findings so far. The survey is still open, and after reading these exciting results we encourage you to contribute to this ground-breaking research project by taking the survey at the following link: <http://www.surveymonkey.com/s/ CEPISGreen ICTSurvey>. Over 300 survey respondents so far have provided information on a range of topics by answering questions in the survey such as "Does your organisation implement green IT practices?" to simply "Do you use recycled printer supplies/cartridges?". However the Green ICT Task Force is open to more participation for this research to be Figure 2: Procedures for Removal of Old Computers. The European Commission has embarked on a number of activities to address the enabling role that the ICT sector can play in diminishing the high carbon emissions of the ICT sector itself Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 7

Over a third of organisations in Europe do not implement green IT practices, the most prominent reason given is that there is no official legislation in their countries enforcing green IT practices there is no pressure from management or customers to follow such practices. (see Figure 1.) Figure 3: Printers connected to LAN and shared with Other Users. of greater value. Below you may find an overview of the findings we have compiled so far during this on-going research process. 2 Initial Findings of the CEPIS Green ICT Survey 2.1 Energy Managers/Officers An energy manager or officer plans, regulates and monitors energy use in an organisation. Of the approximately 300 responses that have been received so far, almost three quarters do not have a person dedicated to this task for their organisation. 2.3 Disposal & Use of ICT Products Over half of organisations who responded to the survey do actually dispose of their old computers with electronics recycling companies. And just over a fifth of respondents donate their old computers to charitable organisations. (see Figure 2.) Over two thirds of organisations do use recycled printer supplies/cartridges. Further almost half of respondents return their used ink cartridges to companies that recycle. It appears that many organisations make an effort to use recycled products, but also to recycle used products themselves. From initial responses it appears that organisations are aware of using printers in an energy efficient manner. Over half of respondents answered that 75 100% of the printers used in their organisations are LAN connected and shared with other users (see Figure 3). Just over a third of the 2.2 How Green is your Organisation? It appears that from the responses, just over a third of organisations in Europe do not implement green IT practices, the most prominent reason given is that there is no official legislation in their countries enforcing green IT practices. The second most given reason is that Figure 4: Restrictions to Use of Printers. Less than one fifth of organisations actually monitor how employees reduce their energy consumption 8 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

printers also have a double-sided print functionality, but only two fifths of those have the double-sided printing option as the default option. Interestingly over half of respondents confirmed that there are no printing restriction policies within their organisations. With those organisations that do have printing restrictions in place, almost half stated that the reduction measure is that "There is a specific number of users that have the right to print". Some other reduction measures from organisations included avoiding printing as much as possible, and using common sense when printing. (see Figure 4.) 2.4 Employees Energy Consumption During the survey ICT Managers, (the main target audience for this survey) are asked to determine the level of awareness of other employees within their organisations towards energy Figure 6: Frequency of Energy Consumption Measurement. efficiency. The responses so far show that there is an even split with regard to whether or not computer users have been informed/ trained in energy consumption reduction procedures. In any case just under half of all respondents agree that their employees have been trained in some way. Less than one fifth of organisations actually monitor how employees reduce their energy consumption (see Figure 5). Of those organisations that monitor employees, around 40% noticed that the energy consumption was reduced after three months. Yet two thirds did not publish these results. Within their own departments, ICT Managers responded that over two-thirds Figure 5: Monitoring Reduction of Energy Consumption by Employees. do not monitor the energy consumption in any of their areas such as data centres, or even within the whole IT department itself. Significantly, overall energy consumption is measured within most European organisations only on a monthly basis. However over a third of organisations seem to never measure the energy consumption and a significant proportion of respondents did not know if energy consumption was measured or not. (see Figure 6.) ICT Managers decision making processes in purchasing products are also investigated during the survey. From the responses we have received so far, two thirds of respondents take into account the product s energy consumption before deciding to purchase. They also rate knowing the level of energy consumption of a product as important and very important. Yet almost half do not take into consideration whether the product is made of recycled materials, but those that do, rate this value as important. (see Figure 7.) Figure 7: Importance given to Knowing the Level of Energy Consumption. Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 9

3 Conclusion We have described the initial results of the CEPIS Green ICT Survey. The Green ICT Task Force has been heavily involved in disseminating and continuing to disseminate the survey link to ICT Managers within their respective societies. If you are interested in seeing the final outcome of this pioneering research project, we encourage you to check our Green ICT Survey page 4 in future. The Green ICT Task Force also has its own LinkedIn group 5, where regular updates will appear about upcoming activities, events, and the progress of this research. We invite you to take the CEPIS Green ICT Survey at <http://www.surveymonkey. com/s/cepisgreenictsurvey>! Annex: Invitation to Digital Trends 2011 Green ICT and Cloud Computing CEPIS and the Hellenic Professionals Informatics Society (HePIS), the Greek CEPIS Member Society, are cohosting Digital Trends 2011 in Athens, on 5 December 2011. This is the first Forum of its kind in Greece and has been established to create dialogue on the contribution of the ICT sector to economic growth, increased productivity, and the advancement of a creative digital culture. In particular the conference aims to focus on how the ICT sector contributes to the adoption of Green and Cloud practices, and the strengthening of the overall role of professionals within the business community. Digital Trends 2011 will focus on the business dimension of Cloud Computing and Green ICT through offering ICT and business professionals a useful guide to the introduction of an organisation to a new business environment. Issues to be discussed at this event include relating to Green ICT include: How are green practices being implemented? What are the benefits of Green IT? Why are Green IT practices not implemented in many organisations? We invite you to visit the Digital Trends 2011 Website at <http://www.digital trends.gr/>. 4 See <http://cepis.org/index.jsp?p=1152&n= 2667>. 5 See <http://www.linkedin.com/groups?most Popular=&gid= 3899686&trk=myg_ugrp_ovr>. 10 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

The Five Most Neglected Issues in "Green IT" Lorenz M. Hilty and Wolfgang Lohmann Many studies in Green IT/Green ICT have been published, focusing on the energy consumption of ICT or the role of ICT as an enabler of energy efficiency. In this article, we argue that such an approach is too narrow, and that a broader perspective is needed to utilize the potential of ICT to make our lives more sustainable. Keywords: Critical Infrastructure, Energy Efficiency, Energy Prices, Green ICT, Green IT, Green Software, Life Cycle Assessment, Rebound Effects, Scarce Metals, Sustainability. 1 Introduction The notion of Green IT has become hype after the publication of Gartner s report "Green IT: a new industry shock wave" [1]. Many projects have been conducted and seminal studies published by industry associations (e.g. The Climate Group), NGOs (e.g. WWF[2]) and international organizations (e.g. OECD[3]). Most of the existing studies focus on the impacts of ICT on CO 2 emissions, either related to the power consumption of ICT or to the role of ICT as an enabling technology for conserving energy in various fields. However, the Green IT/ICT discussion is still lacking awareness of systemic interrelations among the technological, economic and environmental aspects of ICT. This results in a neglect of five issues which we discuss below. We argue that these issues will be essential in the long term, i.e. addressing and resolving them may decide the sustainability of ICT. 2 Five Challenging Issues 2.1 A Lack in the Transparency of Energy Costs in ICT Services It is unusual for a provider to bill customers for ICT services with accurate accounting for hardware utilization and resulting energy use; instead, there is almost no correlation between the price of a service and the energy cost it creates. ICT services are often cross-subsidized, which may create misdirected incentives. For example, telecommunications service providers usually charge high fees for SMS and attract customers with low flat rates for Internet access. We are not arguing here against cross-subsidization for dogmatic reasons, but when the prices of ICT services and their actual energy and material costs diverge too much, that is definitely a problem. The example of e-mail spam makes this clear. According to estimates spam e-mail caused in 2008 worldwide an energy consumption of 33 TWh [4]. This roughly matches the total power generation of Bangladesh for its more than 150 million people. If the senders of spam had to bear the energy costs themselves, there would presumably be no more spam in circulation. Adequate allocation of energy costs would be the first Authors Lorenz M. Hilty is professor of Informatics at the University of Zurich, Switzerland, and Head of the interdisciplinary "Informatics and Sustainability" Research Group, which is shared between the University of Zurich and Empa, the Swiss Federal Laboratories for Materials Science and Technology. Dr. Hilty s research interests include the assessment of Information and Communication Technologies (ICT) with regard to sustainability, ICT applications in the contexts of energy and the environment, as well as basic methods and principles that lead to sustainable solutions. He is the author of more than 50 publications in the fields of Environmental Informatics, ICT and sustainable development, and Green ICT. <hilty@ifi. uzh.ch> Wolfgang Lohmann is a researcher at the interdisciplinary "Informatics and Sustainability" Research Group, which is shared between the University of Zurich, Switzerland, and Empa, the Swiss Federal Laboratories for Materials Science and Technology. In addition to ICT applications in the context of energy consumption and environment, Dr. Lohmann s research interests include the role of software and its design, architecture and implementation in the context of sustainability. <wolfgang.lohmann@empa.ch> issue for an economist to think of in a "Green IT" context. However, only in exceptional cases has the allocation of energy cost been mentioned in the Green IT literature thus far. One exception is a report by the German Ministry of the Environment on best-practices in energy-efficient data centres [5], mentioning that the Dutch co-location data centre EvoSwitch bills energy costs to the customers. The Green IT/ICT discussion is still lacking awareness of systemic interrelations among the technological, economic and environmental aspects of ICT Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 11

According to estimates spam e-mail caused in 2008 worldwide an energy consumption of 33 TWh 2.2 The Material Demand of ICT Hardware Production The variety of materials contained in ICT hardware makes recycling difficult and less efficient. Digital ICT is the first technology claiming more than half of the periodic table of the elements. For example, 57-60 chemical elements are used to build a microprocessor today; in the 1980ies, a microprocessor contained only 12 elements [6]. Memory components, peripheral devices and external storage media are also increasing in material complexity. Miniaturization and integration work against efforts to close material loops by recycling electronic waste. Some metals are contained in very small concentrations (such as indium in flat screens) and could therefore only be recovered in centralized industrial processes as far as recovery is profitable at all, both in economic and energetic terms. If not recovered, these resources are dissipated and therefore irreversibly lost. The combination of highly toxic and highly valuable materials in digital electronics adds to the challenges of recycling, which are not only of a technical nature, but also involve trade-offs among environmental, occupational health and economic objectives. By focusing on the reduction of CO 2 emissions caused by power generation from fossil fuels, the Green ICT de- Technology (A: servers, B: network, C: end-user devices; D: embedded) Energy Consumption Enabling Effect on Energy Efficiency A1: servers outside data centres high Medium A2: corporate data centres for in-house services high High A3: data centres of ICT service providers high High B1: terrestrial and marine communication: optic fibre cables & copper cables B2: wireless communication: GSM, WiFi, 3G antennas Low medium Medium Medium B3: wireless communication: telecom satellites low Medium B4: supporting Internet infrastructure: routers, DNS servers C1: personal computing devices: desktops, laptops, netbooks C2: home telecommunication devices: landline phones C3: mobile telecommunication devices: cellular phones high high medium medium Medium Medium Low Medium C4: TV sets, set-top boxes high Low C5: portable media (music and/or video) players, e-books medium Low C6: digital cameras medium Low C7: peripherals (scanners, printers, etc.) medium Low D: embedded ICT high High Table 1: Results of an Expert Survey on the Effects of Different Types of ICT on Energy Consumption [11]. 12 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

The variety of materials contained in ICT hardware makes recycling difficult and less efficient bate tends to underestimate the supply risks and resulting geopolitical and ecological problems following from ever increasing hardware churn rates combined with miniaturization and integration. The demand for rare metals is growing fast: For the elements gallium, indium, iridium, palladium, rhenium, rhodium and ruthenium, over 80 percent of the quantities extracted since 1900 were mined in the past 30 years [7]. There will be no really Green ICT until we learn to reverse the trends towards higher material complexity and shorter service lives of ICT hardware. 2.3 Insufficient Understanding of the ICT Life Cycle Not all ICT products are the same in terms of production, use and end-of-life treatment. For some ICT products (such as servers or set-top boxes) it is essential to reduce the power consumption during use, because the use phase comprises the largest share in their total life cycle impact; for others it is more important to optimize their design for recyclability or to avoid negative effects during end-of-life treatment. For example, RFID chips and small embedded ICT products entering the waste stream can affect established recycling processes, such as paper, metals, glass or plastics recycling [8][9] or textile recycling [10]. Even if the focus is restricted to energy aspects, different types of ICT have different life cycle profiles. Table 1 shows the result of an expert survey in which the experts were asked to give a rough estimate of the energy consumption of each type of ICT and the energy efficiency created by the enabling effect of the same type of ICT [11]. There is much more variation among ICT types if material aspects and typical life spans are included as well. Given these differences, becoming "green" may have a different meaning in each case. Many Life Cycle Assessment, LCA, studies have been done in the ICT sector: for PCs [12][13]; mobile phone networks [14][15], screens [16] or lighting technologies [17], but the standardized methodology of LCA and the inventory databases that LCA relies on [18] have not received much attention in Green IT/ICT studies thus far. If ICT is viewed as an enabling technology to improve or be substituted for processes in other sectors (manufacturing, transport, housing, energy), the effects in the target sector called second-order effects must also be evaluated from a life-cycle perspective. This leads to the approach of linking life cycles (introduced by Hilty [19] and cited in OECD [3: 15f] to assess the net environmental impacts of an ICT application. ICT applications can have effects on the design, production, use or end-of-life treatment of non-ict products. The vision of "Greening through IT" [20] can only become a reality if methods for systematic assessment of these effects are in place, including optimization, substitution and induction effects [19]. 2.4 Rebound Effects and the Role of Software Progress in the efficiency of producing a good or service means that the same output can be provided using less input. By increasing efficiency, input factors (e.g. energy input) can in principle be saved in absolute terms. However, what we usually see in practice is that these savings are soon balanced out or even overcompensated for by an increase in demand for the output, because the output is getting cheaper in terms of money or time. Even if the demand for this specific output does not increase, saved income can be spent on other goods or services, the production of which requires additional input. These effects are known as rebound effects [21]. Saving resources such as energy by improving the efficiency with which the resource is used therefore is not as straightforward as it may appear to be from a technical perspective. From an economic perspective, the situation is more complex, because the dynamics of markets has to be taken into account to predict the outcome. Efficiency is a necessary, but not a sufficient condition for saving resources. Not surprisingly, many rebound effects have been observed in the ICT field, such as the "IT productivity paradox", "featurism", "software bloat", the "miniaturization paradox" and growth of organizational bureaucracy (for details see [22][19: Chapter 4]. Rebound effects are usually categorized as "systemic" or third-order effects, together with structural change effects such as structural dematerialization [23][24], which would hopefully reduce the overall material intensity of the economy. A recent comparison of the 10 most influential studies assessing the effects of ICT on CO 2 emissions revealed that only a few of them even tried to take rebound effects into account. The five studies published in 2007 or later (i.e. after the beginning of the Green IT hype) almost completely ignored systemic effects, whereas the older studies showed more ambition to account for these effects [25]. Software development plays a specific role in creating rebound effects. The usual response of software engineers (and users as well) to an increase in the processing power and storage capacity available at a given price is to capture Different types of ICT have different life cycle profiles Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 13

Software development plays a specific role in creating rebound effects more of the same. One example is the development of computer games using a combination of fast action and graphics. Each new gain in performance is greedily translated into increasing the reality of the scenes. New patches delivered to increase the resolution of textures and improved graphics can even slow down the speed of the game [26]. Gaining a better understanding the role of software in inducing or avoiding rebound effects is essential. This could lead to "green" software design principles. 2.5 ICT as a Critical Infrastructure Organizations and facilities that are important for the public good are called critical infrastructures if their failure or impairment would have serious effects on public safety or other consequences. The more dependent we make ourselves on ICT networks, the more they become critical infrastructures for us. The problem of controlling the increasing complexity of ICT systems paired with their criticality presents an emerging risk for society. System failure and intrusion into critical infrastructures may have severe implications that are not only economic, but also social and environmental in nature. Arguments to use the precautionary principle in the information society apply here [27]. Can ICT become "green" before we have done our homework in the discipline that E. W. Dijkstra called "how to avoid unmastered complexity"? [28: p. 107]. As software engineers, as designers of information systems and communications networks, we are creating part of society s infrastructure; we should therefore take this warning of one of the greatest computer scientists seriously: "Because we are dealing with artefacts, all unmastered complexity is of our own making; there is no one else to blame and so we had better learn how not to introduce the complexity in the first place." [29]. 3 Conclusion and Outlook We have argued that the Green IT/ICT discussion is neglecting some aspects of the relationship between ICT products and the economic, social and ecological system in which they are embedded: markets, material resources, life cycles of products and their interactions, systemic effects (in particular rebound effects), as well as the uncontrolled complexity emerging with ICT infrastructures. The solution to this problem is to take a broader view a systemic perspective. Truly inter- or transdisciplinary research is needed to establish a deeper understanding of the multifaceted relationships between ICT, society and nature. The aim of this research is to provide comprehensive conceptual frameworks, methodologies of integrated assessment and dynamic models which will help to utilize the potential of ICT to make our lives more sustainable. References [1] S. Mingay. "Green IT: The New Industry Shock Wave". Gartner Inc., 2007. Available at <http://download. microsoft.com/download/e/f/9/ef9672a8-592c- 4FA2-A3BF-528E93DF44EA/VirtualizationPublic Safety_GreenITWhitepaper.pdf>. [Accessed Oct. 31, 2011.] [2] WWF. "The potential global CO2 reductions from ICT use Identifying and assessing the opportunities to reduce the first billion tonnes of CO2". WWF Sweden, 2008. Available at <http://www.wwf.se/source. php/.../identifying_the_1st_ billion_tonnes_ict.pdf >. [Accessed Oct. 31, 2011.] [3] OECD. "Greener and Smarter ICTs, the Environment and Climate Change". Report to the Working Party on the Information Economy (WPIE), OECD, September 2010. Available at <http:// www.oecd.org/dataoecd/27/ 12/45983022.pdf >. [Accessed Oct. 31, 2011.] [4] McAfee, ICF. "The Carbon Footprint of Email Spam Report", 2009. Available at <http://www.mcafee.com/ us/resources/reports/rp-carbonfootprint2009.pdf>. [Accessed Oct. 31, 2011.] [5] K. Fichter, C. Clausen, M. Eimertenbrink. "Energieeffiziente Rechenzentren. Best-Practice- Beispiele aus Europa, USA und Asien" (in German). Hrsg. Bundesministerium fu r Umwelt, Naturschutz und Reaktorsicherheit (BMU), Berlin 2008. [6] National Research Council. "Minerals, Critical Minerals, and the U.S. Economy". The National Academies Press, 2008. [7] P. Wäger, D. Lang, R. Bleischwitz, C. Hagelüken, S. Meissner, A. Reller, D. Wittmer. "Rare metals Raw materials for technologies of the future". SATW, Swiss Academy of Engineering Sciences, 2010. [8] Ph. Kräuchi, P. Wäger, M. Eugster, G. Grossmann, L.M. Hilty. "End-of-life Impacts of Pervasive Computing". IEEE Technology and Society Magazine 24 (1) 2005, pp. 45-53. The more dependent we make ourselves on ICT networks, the more they become critical infrastructures for us 14 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

pp. 45-53. [9] P. Wäger, M. Eugster, L.M. Hilty, C. Som. "Smart Labels in Municipal Solid Waste a Case for the Precautionary Principle?". Environmental Impact Assessment Review 25 (5) 2005, pp. 567-586. [10] A.R. Köhler, L.M. Hilty, C. Bakker. "Prospective Impacts of Electronic Textiles on Recycling and Disposal". Journal of Industrial Ecology 15 (4) 2011, pp. 496-511. [11] V.Coroama, L.M. Hilty. "Energy Consumed vs. Energy Saved by ICT A Closer Look". In: Wohlgemuth,V., Page, B., Voigt, K. (Eds.): Environmental Informatics and Industrial Environmental Protection: Concepts, Methods and Tools, 23rd International Conference on Informatics for Environmental Protection, Berlin 2009. [12] J. von Gleiber et al. "The enviromental impacts of mobile computing A case study with HP". Wuppertal Institute, April, 2003. [13] M. Eugster, R. Hischier, D. Huabo. "Key Environmental Impacts of the Chinese EEE-Industry A Life Cycle Assessment Study". Empa and Tsinghua University, St. Gallen and Bejing, 2007. [14] W. Scharnhorst, L.M. Hilty, O. Jolliet. "Life Cycle Assessment of Second Generation (2G) and Third Generation (3G) Mobile Phone Networks". Environment International 5 (32) 2006, pp. 656-675. [15] M.F.Emmenegger, R. Frischknecht, M. Stutz, et al."life Cycle Assessment of the Mobile Communication System UMTS Towards Eco-Efficient Systems". International Journal of Life Cycle Assessment, 11 2006, pp. 265-276. [16] R. Hischier, I. Baudin. "LCA study of a plasma television device". The International Journal of Life Cycle Assessment (2010), Volume: 15, Issue: 5, Publisher: Springer, pp. 428-438. [17] T. Welz, R. Hischier, L.M. Hilty. "Environmental impacts of lighting technologies Life cycle assessment and sensitivity analysis". Environmental Impact Assessment Review 31 (3) 2011, pp. 334-343. [18] R. Frischknecht, N. Jungbluth, H.J. Althaus, et al." The Ecoinvent Database: Overview and Methodological Framework". International Journal of Life Cycle Assessment, 10 2005, pp. 3-9. [19] L.M. Hilty. "Information Technology and Sustainability". Essays on the Relationship between ICT and Sustainable Development. Books on Demand, Norderstedt 2008. [20] B. Tomlinson. "Greening Through IT: Information Technology for Environmental Sustainability". Cambridge, MA: The MIT Press, 2010. [21] M. Binswanger." Technological Progress and Sustainable Development: What About the Rebound Effect?". Ecological Economics 36 2001, pp. 119-132. [22] L.M. Hilty, A. Köhler, F. von Schéele, R. Zah, T. Ruddy. "Rebound Effects of Progress in Information Technology. Poiesis & Praxis". International Journal of Technology Assessment and Ethics of Science, 1 (4) 2006, pp. 19-38. [23] E. Heiskanen, M. Halme, M. Jalas, A. Kärnä, R. Lovio. "Dematerialisation: The Potential of ICT and Services". Ministry of the Environment of Finland, Helsinki, 2001. [24] L.M. Hilty, T.F. Ruddy. "Sustainable Development and ICT Interpreted in a Natural Science Context: the Resulting Research Questions for the Social Sciences". Information, Communication & Society 13 (1) 2010, pp. 7-22. [25] L. Erdmann, L.M. Hilty. "Scenario Analysis: Exploring the Macroeconomic Impacts of Information and Communication Technologies on Greenhouse Gas Emissions". Journal of Industrial Ecology 14 (5) 2010, pp. 824-841. [26] N. Ernst, P. Steinlechner. "Crysis 2 und DirectX-11: Operation Tessellation ist weitgehend gelungen". In Golem.de. (in German). Available at <http://www. golem.de/1106/84525.html>. [Accessed June 28th, 2011.] [27] C. Som, L.M. Hilty, T. Ruddy. "The Precautionary Principle in the Information Society". Human and Ecological Risk Assessment, 10 (5) 2004, pp. 787-799. [28] E.W. Dijkstra. "Selected Writings on Computing: A Personal Perspective". New York: Springer-Verlag, 1982. [29] E.W. Dijkstra. "The next fifty years. Manuscript, 1996". Available at <http://www.cs.utexas.edu/~ewd/ ewd12xx/ewd1243.pdf>. [Accessed Sep. 25, 2011.] Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 15

Utility Computing: Green Opportunities and Risks Giovanna Sissa A visionary idea of computing since the early 60s has been that of utility. Cloud computing finally looks to be the implementation of such an idea. While this paradigm is providing many opportunities for the development of the software sector, concerns about its environmental impact are also being raised. This paper focuses on the green potential of clouds and how they have to be deployed for different user levels, highlighting the related environmental risks. The trend shows clearly how cloud computing is turning computing into a pay-per-use model, one in which quality of service requirements will need to be expanded to include green requirements. Green computing has to take into consideration new opportunities and new issues for the environment, not only focusing on the energy use phase but also on all phases of the life cycle for any service provided in the cloud. The awareness of users and developers is the first step to realizing the green potential of the cloud. Keywords: Cloud Computing, Green ICT, Life Cycle, Quality Of Service Requirements, Reuse, Utility Computing. 1 Back to the Future: Computing as Utility In the 60s computers were as big and expensive as they were difficult to use and maintain. Computational centers had to have human operators as an interface between users and the computer. Users wrote their programs on a set of punch cards and in order to run them they had to contact the computer center operator to give him the packaged cards and pay for computation time. The model was pay-per-use of the computing resource. One of the stronger ideas underlying the development of computing has always been that computing should be a utility, like water, electricity, gas, or telephony. To became true, this dream would have needed the availability of computing everywhere. At that time, there was no possibility of computing joining the ranks of other kinds of utilities, because of the lack of a "pipeline" for computing resources. But the computing model evolved in the opposite direction: towards individual availability, at home or at the office, of the computer itself, i.e. the personal computer. In the PC paradigm the user has become the owner of computing capability, which he or she manages. With the Internet it soon became clear that something was changing. As early as 1969, Leonard Kleinrock [1], one of the chief scientists at ARPANET, said "As of now, computer networks are still in their infancy, but as they grow up and become sophisticated, we will probably see the spread of computer utilities which, like present electric and telephone utilities, will service individual homes and offices across the country". The pipeline issue could be solved. The vision of computing utilities based on a "service provisioning model" anticipated the cloud computing era, in which computing services are readily available on demand, just like other utilities, and users need to pay providers only when they access them. 2 Cloud Computing Opportunities In the ICT sector, cloud computing is one of the most Author Giovanna Sissa was awarded a degree in Physics in 1978. She has been involved in ICT since 1980, as Strategic Consultant for Industries and Public Administrations, for the identification of emerging technologies and technological trends in informatics. She is a member of the Italian Register of Technological Innovation Experts (Ministry of Economical Development) and a member of the Scientific Committee of the Piedmont Regional System for Research and Innovation. In 2000 she founded, and managed until 2009, the "Osservatorio Tecnologico" of the Italian Ministry of Education. Engaged in research on ICT and sustainability, she has written books and papers about the environmental impact of ICT. As a PhD student of Informatics at the Università degli Studi di Milano her research focus is on ICT-driven societal behavioral changes and their effect on environmental sustainability. <sissa.giovanna@ gmail.com> searched terms. There are a great many definitions, but none which is fully accepted by the scientific community as a whole. The NIST (National Institute of Standards and Technology) definition is very broad: "Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics, three service models, and four deployment models" [2]. In September 2011Wikipedia defines cloud computing as follows: "Cloud computing is the delivery of computing as a service rather than a product, whereby shared resources, software and information are provided to computers and other devices as a utility (like the electricity grid) over a network (typically the Internet)". Cloud computing delivers infrastructure, platform and software applications as a service, which are made available to consumers as subscription-based services under the pay-per-use model. 16 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

This paper focuses on the green potential of clouds and how they have to be deployed for different user levels, highlighting the related environmental risks And within each layer of abstraction there are myriad opportunities for defining the services that can be offered [3]. Users can access and deploy applications from anywhere in the world, on demand, and at a competitive cost depending on their quality of service requirements. QoS requirements are specified to users via Service Level Agreements (SLA). The need to manage multiple applications in a datacenter creates the challenge of on-demand resource provisioning and allocation in response to time-varying workloads. This feature, called elasticity, is one of the five cited by NIST: "Capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out, and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time" [2]. In other words, cloud computing refers to both the applications, delivered as services over the Internet, and the hardware and systems software in the datacenters that provide those services. The datacenter hardware and software is what we will call a Cloud. When a Cloud is made available in a pay-per-use to the general public, we call it a Public Cloud; the service being sold is Utility Computing [4]. If cloud computing is finally the implementation of the old idea of "computing as a utility" [5], what are the implications arising from it? The answer depends on whoever is posing the question. The meaning of cloud computing is different for different people, depending on their use of the cloud. For application user it is the delivery of computing, storage and application over the Internet from centralized datacenters. For Internet application developers it is an Internet-scale software development platform and runtime environment. For infrastructure providers it is the massive distributed datacenter infrastructure connected by IP network [6]. The cloud has been a boon for the companies hosting it. Developers no longer need to invest heavily or go to the trouble of building and maintaining complex IT infrastructures. Developers with innovative ideas for new Internet services no longer require large capital outlays in hardware to deploy their service. Thus the computing world is rapidly transforming towards the development of software for millions to consume as a service rather than to run on individual computers [5]. The network is the platform for all computing, where everything we thought of as a computer yesterday is just a device that connects to the internet [7]. If cloud computing represents plenty of opportunities for different kind of users, what opportunity does it represent for the environment? What does the implementation of utility computing mean from an environmental point of view? Does it represent a major opportunity? Or are there also some risks concerning sustainability? Is cloud computing green computing? 3 Green Computing Green computing is an umbrella term that includes the dimensions of environmental sustainability, the economics green it 1.00 Figure 1: Green IT in Google Trends. Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 17

green it 1.00 cloud computing 1.20 Figure 2: Comparison between Green IT and Cloud Computing in Google Trends. of energy efficiency, and the total cost of ownership, which includes the cost of disposal and recycling. Green ICT benefits the environment by improving energy efficiency, lowering greenhouse gas emissions, using less harmful materials, and encouraging reuse and recycling [8]. Green design, green manufacturing, green use, green disposal are complementary paths of green ICT. Only by focusing on these four fronts can we achieve total environmental sustainability from the IT side and make IT greener throughout its entire lifecycle. Corporations and their IT departments are recognizing the impact of their "carbon footprint" on the environment, and if we make a Google Trends search we can see how the popularity of the term is growing. A query on "green IT" (Figure 1) shows a growth of the interest. It s interesting to make a comparison between the term green IT and the term cloud computing (Figure 2). Given the growing importance of cloud computing, the question is not whether it is green as it is now, but how it can became really green. The focus will be on the potential green role played by cloud computing as an implementation of utility computing. Strongly driven by the hardware producers, green computing supplies a huge offer of green ICT devices and products. But, since the computing paradigm has shifted towards cloud computing, i.e. utility computing, the green challenge of ICT will be played out more and more on such a paradigm. Before going into the green potential of cloud computing in greater depth, we have to remember some basic environmental sustainability principles related to the ICT sector and then try and apply them to cloud computing. 4 Cloud Computing and First Order Effects In 2007 the total footprint of the ICT sector was 830 MtCO2 emissions, about 2% of the estimated total emissions from human activity released that year [9]. Adjustment to the Smart 2020 report have been suggested by environmental organizations [10], highlighting the scale of ICT s estimated energy consumption, and providing a new analysis of the projected growth in energy consumption of the Internet and cloud computing for the coming decade, particularly as driven by datacenters. Each stage of a computer s life cycle, from its production, throughout its use, and on to its disposal, increases carbon dioxide emissions and the impact on the environment. The total electrical energy consumption by servers, computers, monitors, data communications equipment, and cooling systems for datacenters is steadily increasing. Data Center now produces more carbon emissions than both Argentina and the Netherland [2]. Google, Microsoft and Yahoo are building their datacenter near the Columbia river, to exploit cheap and reliable hydroelectric power. There is a trend emerging to build data farms in cold regions, like Iceland, to decrease cooling power needs and price. In other Given the growing importance of cloud computing, the question is not whether it is green as it is now, but how it can became really green 18 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

What is the hypothetical footprint of a startup that may have chosen to build its own datacenter versus using cloud computing? words, there are a lot of nested relationships between ICT and the environment. ICT devices are becoming more and more compact and energy efficient and green computing is responsible for such improvements. New generation IT systems provide more computing power per unit of energy but, despite this, they are actually responsible for an overall increase in energy consumption. The demand for ICT is increasing even faster than the energy efficiency of ICT devices [11]. This is because users are taking and using the increased computing power offered regardless of its effect on sustainability. Moreover cloud computing is changing the way we quantify the direct effects of ICT, adding some additional issues about its measurability. The shift toward cloud computing looks, in principle, to be more environmentally friendly compared to traditional datacenter operational/deployment models. The rule of thumb says that a higher consolidation/optimization of the infrastructure will make it possible to conserve energy. But if cloud computing can enable green, and it could be a great way to reduce the carbon footprint, we have to be able to demonstrate it. And to demonstrate something you have to quantify it. The emission factor, the rate for converting kilowatthours into units of carbon dioxide emissions, is the basis for any evaluation of the direct impact of ICT. This rate varies from country to country and from region to region because it depends on the source from which electric power is produced [12]. Power sources can have dramatically different CO2 footprints, say coal vs. wind or solar. Industry adopted metrics (PUE, DCIE) take into consideration the efficiency of datacenter infrastructure relative to energy demand, but not to the overall resource impact or even the amount of energy needed for a particular computing activity. Metrics like PUE are valuable in helping datacenter operators to benchmark the design and efficiency of their facilities by providing an objective metric that drives effort to improve facility efficiency. Recent efforts have been made to develop additional resource-based metrics that speak to the carbon intensity (CUE) and water utilization (WUE) of a datacenter All ICT-based services will increasingly be delivered on the cloud. When an ICT-based service is provided, it is responsible for a given amount of CO2 emissions. The challenge, from a green perspective, is to be able to quantify the per-unit energy consumption, and more generally, the perunit carbon emissions. In particular the challenge is to quantify a service when it is delivered on the cloud. Even as a rough estimate, the entire life cycle of the whole system providing a given service should be studied, in order to assess the environmental impact of producing one functional unit of the service. While it is quite straightforward to compare the CO2 emissions of a new generation tablet with those of a desktop computer, it is far from straightforward to compare the emission equivalence of a computing activity delivered traditionally or by the cloud. In other words we have to be able to quantify the impact in terms of CO2 emission equivalent of an ICT-based service delivered on the cloud. By definition clouds are promising to provide services to users without reference to the infrastructure on which these are hosted. As consumers rely on cloud providers for their computing needs, they have to require that a specific QoS (Quality of Service) will be maintained by their providers, in order to meet their objectives and sustain their operations [5]. While it is clear that there are critical parameters such as time, cost, reliability and trust/security, equally important are the parameters linked with the green performance of the cloud. If we measure software quality with software quality factors which describe how software behaves in its system, from a green perspective we need new green quality factors. In particular we need green cloud computing factors allowing a uniform way to measure the supposed gain in efficiency allowed by the cloud. 5 Cloud Computing: Environmental Issues and Challenges Cloud computing with increasingly pervasive front-end client devices interacting with back-end datacenters will cause an enormous escalation of energy usage. To address this problem, datacenter resources need to be managed in an energy-efficient manner to drive Green Cloud computing. In particular, cloud resources need to be allocated not only to satisfy QoS but also to reduce energy usage [13]. In order to test the green performance of the cloud we have to be able to answer such questions as: What is the hypothetical footprint of a startup that may have chosen to build its own datacenter versus using cloud computing? Running the numbers about how green a particular usage scenario actually is becomes more complicated than showing green credentials. Moving on from the why in cloud computing to the how, claims regarding the green Apart from a lack of transparency in the quantification of energy consumption by cloud providers, some other environmental risks can be envisaged Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 19

credentials of cloud computing need to be clearly answered, motivated and calculated in order to substantiate those claims. Common sense says that reducing the number of hardware components and replacing them with remote cloud computing systems reduces energy costs for running hardware and cooling as well as reducing the carbon footprint, while higher DC consolidation / optimization will conserve energy. The IT industry points to cloud computing as the new, green model for our IT infrastructure needs, but few companies provide data that would allow us to objectively evaluate these claims. And quantifications may not be comparable, because different cloud computing provides different service features and has incompatible starting assumptions. Some concerns are also emerging within the cloud computing community [14][15]. We now have the ability to run our applications on thousands of servers, whereas previously this was not even possible. So we can potentially use several years worth of energy in literally a few hours, while previously this was not even an option. So in direct contrast, hypothetically we are using more resources, not less. On the flip side, if we bought those thousand servers and had them running (underutilized), the power usage would be significantly higher. You may use 80% less energy per unit, but you would have 1000% more capacity, which at the end of the day means you are using more energy, not less. Apart from a lack of transparency in the quantification of energy consumption by cloud providers, some other environmental risks can be envisaged. That is because cloud computing encourages behavior that may not be very green [16]. The availability of cheap resources may encourage poor optimization. The ability and ease of access to a massively abundant cloud computing resource will drive that behavior on the server. It will be cheaper to add 10 more web servers than to profile, optimize, regression test and deploy the code base. Cloud computing allows things that may never have been processed before to be processed without an impact on performance, for example selecting a very large set of data for analysis (because you can literally process the data in an hour where previously it could take days). If the cloud lowers the cost of providing services, it is possible to provide services that only generate a few pennies per transaction. While generally considered a benefit of the cloud, one has to question where the value of the end product is worth its environment cost. Another risk then is providing low value products and services. The spread of mobile ICT is changing how we communicate, relate and manage our daily lives at an astounding speed. In 2011 the world will create a staggering 1.8 zettabytes 1 of digital information [17]. Think about the rate of increase in the number of people performing some sort 1 1 zettabyte = 1 trillion gigabyte = 10 21 bytes. of computation (for example, the hundred million members of Facebook all uploading photographs) and the rate of increase in the amount of data to be manipulated (consider a five megapixel camera built into everyone s phone,). All the while, in the cloud, processors will be running algorithms while constantly making adjustments as they dynamically navigate the trade-off between data size, connection speed, and client performance (as, for example, processor and screen resolution). The question is, are we more environmentally friendly doing all of this on a shared cloud or at our own datacenters? Since the cloud allows our digital consumption to be largely invisible (and sometimes free of charge), we may fail to recognize that the information we receive actually devours more and more electricity. The more compute cycles are available, the more we will use. Awareness from developers is a precondition for a green behavior. If cloud computing represents an extraordinary opportunity for developers, never seen before, able to decrease or fully eliminate the entry level in the application or services delivering on the Net, for the final user it is a new way of using the computer. Power-users, as well as simple-users are shifting from a computer-centered to an Internetcentered style. Consumers now need nothing but a personal computer and internet access to fulfill most of their computing needs. Personal applications are becoming available via Web, Google Docs being the best known example, an "Office Suite" in the Cloud, accessible anywhere, from any computer with a net connection and a decent browser. It is no longer mandatory to install the application on a personal computer! Public awareness of climate change is increasing and the Cloud can be a good opportunity to achieve a greener ICT, in a broader sense, just by starting from end-user behavior. For example, by reducing the obsolescence rate of end-user devices, which are responsible for the major environmental problem called e-waste. The cloud allows device-independence. It is possible to reuse the PC as a thin always-on client and to access our data and application everywhere. Old PCs can easily be turned into thin clients, further reducing equipment costs - the costs of operating system licenses and upgrades are reduced or even eliminated with open source software solutions [18]. Cloud computing can give end-users the opportunity to be free from a specific access device, traditionally their own PC, and to shift their working environment onto Awareness and responsible behaviors are a background condition to achieve sustainable and green cloud computing 20 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

a virtualized desktop. The critical infrastructure is broadband, not computing power. 6 Conclusion Cloud computing is inherently green. To move to cloud computing appears to be more environmentally friendly compared with traditional datacenter operational/deployment models. Many companies have been able to do away with the need for physical infrastructure and thus reduce their energy footprint. Thus, in some ways cloud computing can enable green, and could be a great way to reduce the carbon footprint. There are many advantages to this approach for both customers (lower cost, less complexity) and service providers (economies of scale). But there is also some risk for the environment as well. Awareness and responsible behaviors are a background condition to achieve sustainable and green cloud computing. References [1] L. Kleinrock. "A vision for the Internet". ST Journal of Research 2(1) (2005), pp. 4-5 [2] NIST Special Publication 800-145, "The NIST Definition of Cloud Computing (Draft)", January 2011. [3] Sun Microsystems. "Take your business to a higher level A cloud computing primer", 2009. [4] M. Armbrust et al. "Above the Clouds: A Berkeley View of Cloud Computing". Technical Report No. UCB/ EECS-2009-28, University of California at Berkley, USA, Feb. 10, 2009. [5] R. Buyya, C.S. Yeo, S., Venugopal, J. Broberg, I. Brandic. "Cloud computing end emerging IT platform: Vision, hype, and reality for delivering computing as the 5th utility". Future generation Computer System, 25, pp 599-616, 2009. [6] G. Lin, D. Fu, J. Zhu, G. Dasmalchi. "Cloud Computing: IT as a service". IT Pro March/April 2009. [7] T. O Reilly. "What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software", 2005. <http://oreilly.com/web2/archive/what-is-web- 20.html>. [8] S. Murugesan. "Harnessing Green IT: Principles and Practices". IT Professional, Vol.10, 1, pp. 24-33, 2008. [9] Gesi Group. "Smart2020", 2008. Available at <http:// www.smart2020.org/publications/>. [10] Greenpeace International. "How dirty is your data?". Greenpeace International, Amsterdam, The Netherlands, 2011. Available at <http://www.greenpeace.org/ international/en/publications/reports/how-dirty-isyour-data/>. [11] L.M. Hilty, A. Köhle, F. Schéele, R. Zah, T.F. Ruddy: "Rebound effects of progress in information technology: Poiesis & Praxis". International Journal of Technology Assessment and Ethics of Science 4, 1, pp. 19-38, 2006. [12] V. Coroama, L.M. Hilty. "Energy Consumed vs. Energy Saved by ICT - A Closer Look". In: Wohlgemuth, V., Page, B., Voigt, K. (Eds.): Environmental Informatics and Industrial Environmental Protection: Concepts, Methods and Tools, 23rd International Conference on Informatics for Environmental Protection, pp. 353-361, ISBN 978-3-8322-8397-1, 2009. [13] R. Buyya, A. Beloglazov, J. Abawajy. "Energy-Efficient Management of Data Center Resources for Cloud Computing: A Vision, Architectural Elements, and Open Challenges". Proceedings of the 2010 International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA 2010), Las Vegas, USA, July 12-15, 2010. [14] R. Coven. "Is Cloud Computing Actually Environmentally Friendly". Available at <http://www.elasticvapor. com/2009/12/is-cloud-computing-actually.html>. [15] S. Munro. "Environmentally Unfriendly Side Effects of Cloud Computing". <http://consultingblogs. emc.com/simonmunro/archive/2010/01/12/environmentally-unfriendly-side-effects-of-cloud-computing.aspx>. [16] J. Colgan. "Environmental Effects of Cloud Computing". Available at <http://www.xuropa.com/blog/2010/ 01/19/environmental-effects-of-cloud-computing/>. [17] EMC 2 website. <http://www.emc.com/leadership/ programs/digital-universe.htm>. [18] G. Sissa. "Green Software". UPGRADE, Vol XI, No 3, June 2010, pp. 53-63. Available at <http:// www.cepis.org/upgrade/files/cepisupgrade/2010/docs/ 20100713125017_upenet-iii-2010.pdf>. Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 21

Good, Bad, and Beautiful Software In Search of Green Software Quality Factors Juha Taina New emerging areas of software development and usage are green software engineering and software engineering for the planet. In green software engineering, software is developed, executed, and maintained in as environment friendly a way as possible. In software engineering for the planet, software actively helps in the fight against climate change. Currently, we lack good factors for estimating how good green software is. We consider this problem on three levels. First, a developer s approach is to calculate the carbon footprint of software development, delivery, and maintenance. These factors are relatively easy to estimate but they lack an execution phase. Second, an engineer s approach is to count how much resource software uses relative to its tasks. These factors are good for green software engineering but not enough for software engineering for the planet. Finally, a planet s approach is to calculate how much software offers or how much damage it does to its environment in its life-time. Together these would give an ultimate solution for the green software factor dilemma but unfortunately it is not at all clear how to define metrics for such factors. Keywords: Green Metrics, Green Software, Software Engineering for the Planet. 1 Introduction Software can and will help in our pursuit to sustainable development and reduction in carbon dioxide emissions. It will play an ever increasing role in controlling systems, optimising algorithms and generating alternatives for carbon intensive processes. Software systems are present in practically every aspect of the world around us. This implies that software is present in a myriad of ways. We measure software quality with software quality factors that describe how software behaves in its system. In the future, we will need new factors, green quality factors, to define how software supports sustainable development and our fight against climate change. From the sustainable development and climate change point of view, good software helps to reduce greenhouse gases, waste, and resource requirements while bad software increases them. Sometimes software can be downright ugly: badly written, difficult to use, and resource intensive. On the other hand, we can have elegantly beautiful software that not only reduces resource use but also affects how people see sustainable development. In order to understand how good, bad, or beautiful software is, we need relevant metrics. Once we have such metrics we can compare software either with other software or against an absolute scale. Unfortunately, it is not at all clear what sort of metrics should be used. It is even arguable whether software has a role in sustainable development or how large the role is. In principle it is easy to say how green a software system is. First we take the initial state of a system domain Author Juha Taina is a university lecturer at University of Helsinki, Faculty of Science, Finland. He is an experienced teacher and researcher in software engineering. His current research interests include green software, green software engineering, sustainable development and software engineering education. <taina@ cs.helsinki.fi> without the system and calculate the resources required in the domain. Then we re-calculate the required resources after we have installed the software system. If the new sum is smaller than the old one, the software system supports sustainable development. We can compare two software systems in the same way. For instance, let us have a teleconferencing software system that allows people to use fast network connections for on-line meetings. The original domain would require everyone to travel to their meetings which would of course require resources. The new domain would cut travelling but perhaps add some extra resource requirements such as network usage. However, the net effect would most probably cut carbon emissions and required resources. The system would help in sustainable development. The scenario gets more complex when we talk about software alone. Software is a necessary component of a software system, but so is hardware. It is not at all clear how to define how much of the resources saved is due to good software and how much is hardware-related. In this paper we give a classification for green quality factors and define green metrics. Due to the inexact nature of the field, our definitions are neither complete nor abso- Software can and will help in our pursuit to sustainable development and reduction in carbon dioxide emissions 22 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

A larger field called Software Engineering for the Planet (SEP) addresses issues and questions on where software and software engineering can help sustainable development and a carbon-free environment lute. It is possible to define a different set of green quality factors, however, as far as we know this paper is the first attempt to define them. Perhaps in a few years we will have a large set of well-defined green quality factors from which we can pick the best ones. The rest of the paper is organised as follows. Section 2 introduces green software and green software engineering. Section 3 presents our framework for green quality factors. Section 4 introduces green quality factors and metrics that are related to software development. Section 5 introduces factors and metrics related to software execution. Section 6 introduces factors and metrics related to software effects in sustainable development. Section 7 summarises and concludes the paper. 2 Background When people think about green software, most of them are thinking about Green IT. In short, it is the study and practice of using computing resources efficiently [1]. Green IT is a huge field that includes everything in a software system life cycle from hardware manufacturing to minimising disposable waste. Resource usage is not a small matter in IT. For instance, the TOP500 s leading supercomputer, the K computer, consumes enough energy to power nearly 10,000 homes and costs $10 million a year to operate [2]. Each PC in use generates about a ton of carbon dioxide a year [3]. Green IT addresses the research and practice of how to reduce carbon emissions and other resource requirements of software systems. It is an important field where green software plays a remarkable role, yet "only" 2% of global carbon emissions are directly due to IT systems [4]. Green software can help to indirectly reduce carbon emissions, waste, and resource usage. A larger field called Software Engineering for the Planet (SEP) [5] addresses issues and questions on where software and software engineering can help sustainable development and a carbon-free environment. While Green IT belongs to SEP, it includes other areas as well. In a recent workshop, the participants defined that SEP consists of at least the following [6]: 1. Software support for green education. Use software to support education and general knowledge about sustainable development and climate change. 2. Green metrics and decision support. Software processes and tools for environmental-friendly software design, implementation, usage, and disposal. 3. Lower IT energy consumption. Software to support or be part of Green IT. 4. Support for better climate and environment models. Let environmental scientists do their research with better software. The need to greatly reduce carbon emissions and eventually become a carbon-neutral sustainable society is a difficult task that needs cooperation from all research fields. Software engineering is no exception. 3 Green Quality Software quality is defined via a set of software quality factors. If software fulfils the defined factors, it is considered a quality product. A software quality factor defines its relevant requirements. If software fulfils the requirements, it fulfils the defining software quality factor. Several models of software quality factors have been suggested in software engineering literature. Galin [7] lists some of the most common classic models while new models are constantly emerging. For instance, in a recent article Naumann et al. defined sustainable-related green metrics based on the direct and indirect effects of software [8]. Any defined software quality metric is related to one or more quality factors. Hence, in order to define a new quality metric one first needs to decide what quality factors its measurements will model. While the list of defined quality factors is comprehensive, they all lack factors for software greenness and sustainability. We need factors to define how environment friendly software is and how it will support sustainable development. We call such factors green software quality factors or simply green factors. A green factor defines properties that green software must fulfil. It needs one or more green metrics which measures the factor fulfilment in software. For instance, if we have a green factor stillness (moving requires resources) we could have a metric steps to calculate how many steps a software developer needs to take during a software development phase. The resulting number of steps needs to be interpreted in order to decide how the development process has fulfilled the stillness factor. Any green factor needs to fulfil one or more green requirements. The higher-level the requirements we have, the higher-level the green factors need to be defined. We require green software to fulfil three abstract requirements: 1. The required software engineering processes of software development, maintenance, and disposal must save resources and reduce waste. 2. Software execution must save resources and reduce Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 23

waste. 3. Software must support sustainable development. These most abstract requirements give us three abstract green factors: 1. Feasibility. How resource efficient it is to develop, maintain, and discontinue software. 2. Efficiency. How resource efficient it is to execute software. 3. Sustainability. How software supports sustainable development. The required resources can be material, energy, or human resources. For instance, time is a very important resource that should not be wasted. Green software is developed in the minimum possible time but not any faster. Resource efficiency minimizes waste. Hence, green software minimises the waste produced both in development and execution. For instance, carbon dioxide is a typical waste that must be minimised. Software is always part of a software system. One aspect of sustainability is how software helps its system to reach its goal. Green software supports its system at maximum efficiency. For instance, the goal of an eshop system is to sell products efficiently via the web. Good eshop software supports this by offering a positive user experience. It is important to notice the difference between green software and a green software system. We can have green software in a non-green software system and vice versa. For instance, a coal power plant software system is not green but optimising software that minimises carbon dioxide emissions from the coal power plant software system is green. It supports sustainable development when it minimises emissions and it supports its system to reach the goal of producing maximum energy with minimum waste. 4 Feasibility The software life cycle begins when the decision to build software is made. Green software is built with green methods. Feasibility defines how projects and processes that develop, maintain, and discontinue software follow sustainable development. Feasible software is built with sustainable processes. Its software engineering processes support sustainable development, minimise resource requirements and produce minimal waste. The emphasis on feasibility is in software engineering processes instead of final products. Fortunately, we know how to measure processes and have good factors and metrics for it. We can easily use the same metrics as in regular industry. Our feasibility factors are related to human behaviour instead of software execution. For instance, it is possible to calculate how many kilometres an average developer needs to travel during the development phase. The result is a measurable absolute value that can be interpreted to required resources and produced waste. In an earlier paper, we calculated development carbon footprints to analyse feasibility [9]. It is relatively straightforward to calculate required resources for various software life cycle steps such as development, maintenance, and disposal. It is relatively easy to calculate how many human resources are needed to manage software. Since feasibility is an easy factor to measure, it is a practical factor to compare software. In other words, we can use feasibility to advertise our software and especially our software development and management processes: "Our software is good because feasibility measurements show optimal development processes." Here are a few good feasibility metrics. They are based on traditional project and process metrics. 1. A Carbon Footprint (CF) defines how much carbon dioxide a software development, management, or maintenance phase will emit. This is the most important green metric and eventually all green factors should be calculated with it. In the case of the feasibility factor, its values are relatively easy to calculate. In the context of other factors, it may give more ambiguous results. 2. Energy defines how much energy is consumed during the development phases. This metric does not care how energy is produced. The CF metric does that. 3. Travel defines how much travelling time is required during the development phases. This is an important metric because travelling takes time and requires resources. The less travelling that is required during the development phases, the better the feasibility. This is true in cases when travelling is seemingly emission-free, like when walking or using a bicycle. 4. Waste defines how much resources are consumed in activities that create no visible value to software end users. Waste can be physical, energy, or process waste. Process waste defines operations in a process that require resources but do not produce anything valuable. For instance, idle development times and waiting queues are waste because they do not create any value. The CF metric is the most important of the listed metrics. It is a direct metric that can be used as a basis for various indirect metrics. Is software A more feasible than software B? Calculate functionality/cf. Based on our earlier paper it appears that "travelling rules" in development resource requirement analysis [9]. Feasible software is built with sustainable processes. Its software engineering processes support sustainable development, minimise resource requirements and produce minimal waste 24 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

Fortunately travelling is one of the easiest areas to reduce. A lot of travelling can be replaced with conference meetings using software, online discussions, and shared development screens. 5 Efficiency Good software does not waste resources whether from its system or from its users. Bad software steals time and increases waste. Efficiency defines how software behaves when it comes to saving resources and avoiding waste. We define the following factors and metrics for efficiency. With these factors it is possible to estimate how software reduces waste. 1. CPU-intensity. How many CPU cycles software consumes. A good metric is cycle count. 2. Memory usage. How much memory is used and how. A good metric is main memory consumption. 3. Peripheral intensity. How much peripheral equipment is used. A good metric is peripheral usage time that can be further translated to carbon dioxide emissions or other waste metrics. 4. Idleness. How much software is idle. A good metric is idle time. This factor and metric are relevant to only certain types of software systems such as virtual servers. main memory. In case of main memory, it not only matters how much data is stored but also what kind of data is stored. For instance, in a very low-energy environment it is useful to minimise memory consumption and reset unused bytes to nil. The result is a minimal number of set bits in main memory. We save energy because it is more efficient to refresh a reset bit than a set bit. High CPU-intensity, peripheral intensity, and main memory usage are justified if they are for a good cause. Software can consume resources as long as it helps its software system to reach its goal efficiently. This is especially true of CPU intensity. Consumed CPU cycles are usually useful. Idle cycles, on the other hand, are never useful. Any idle cycle of a CPU is a wasted resource. It uses energy and creates waste but does not give anything back. A typical home desktop or even an IT server can be 90% idle [1]. Such a system is like a plane with 90% empty seats. Fortunately Green IT has already acknowledged the problem and with proper IT-server solutions it is possible to minimise idle cycles. So far we have analysed the resource requirements for inputs to software. They are the requirements for software to execute. However, in efficiency the resource requirements Even if we are not yet able to define good absolute metrics for all factors, we think that recognising green software factors is already an important contribution 5. Reflectivity. How software indirectly affects its domain. Good reflectivity metrics need more research. The most intuitive software resource is a CPU cycle. Each executed cycle requires energy and hence has a measurable waste value. We can measure it and translate it to any environment based metric such as the amount of carbon emissions. We call this CPU-intensity. It defines how many CPU cycles software consumes. While CPU-intensity is a possible metric for efficiency, it is a bit limited and difficult to calculate. Software requires more than just CPU cycles. It needs main memory and peripherals in order to execute. Hence, we need metrics for main memory and peripheral requirements. The peripheral requirements for software are easier to estimate. We can calculate how many requests software generates for peripherals and what resources are required. A simple metric is to calculate energy requirements for a peripheral and partition the measures to all software that requires services from it. We call this peripheral intensity. It defines how much peripheral equipment software requires. The main memory requirements are more difficult to estimate. We need an attribute to evaluate how much memory is in use and how much it required resources. We call this main memory consumption. It defines how software uses for outputs from software are also meaningful: how much software usage indirectly affects required resources. A typical output of desktop software is a report. If the report is viewed on a display, its resources are included in the peripheral resources. If it is printed, paper and ink usage should be included. If it is sent to several people, the resource requirements to process the output should be included for all receivers. We call this reflectivity: how much software affects others and how. Reflectivity is perhaps the most important factor in efficiency. The amount of generated waste can be devastating. For instance, consider reflectivity of spamming software. There is one reflectivity aspect that may not be as obvious as the others. On laptop and desktop computers in particular, a special type of software has a nasty indirect way to increase waste without being CPU-intensive: memory resident software. It does not use much resources but when it does it is always at a critical time: when the host computer is turned on. They slow the computer start-up because that is when they are automatically loaded. The slower the start-up, the higher the probability that end users would rather leave the computer on than turn it off. This directly weakens software efficiency. Hopefully current trend to forwards flash drives and optimised hibernation algorithms Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 25

allows computers to hibernate more efficiently. According to Lo et al., the rebooting of a hibernated system with a flash drive can be about one second [10]. Such reboot times would effectively nullify this aspect. These factors and their metrics are not as intuitive as the feasibility software metrics. The metrics are difficult or almost impossible to calculate, yet it is clear that green software needs to execute in a green way. The less resources software consumes and wastes directly or indirectly, the greener is its execution. 6 Sustainability So far we have discussed green software factors that are directly or indirectly related to software itself. However, perhaps apart from reflectivity, such factors do not tell much about the possible environmental effects of software. We would like to have factors with easily measurable metrics that would answer the following questions: 1. How fit for purpose software is for its system? 2. How software supports its system waste reduction in the system domain? 3. What is the value of software in sustainable development? We can analyse the requirements from two points of view: 1. Software as part of a software system, and 2. Software as a stand-alone product. From the software system point of view software is an indivisible component. In that case sustainability can be defined only within a software system. This in turn implies that sustainability is a relative factor. We cannot compare sustainability of two pieces of software unless they are in similar software systems. While reducing sustainability to similar systems is a serious simplification, it gives us simple tools to analyse sustainability within a software system. We can define sustainability as having the following factors: 1. Fit for purpose defines how software helps its system to reach its goal. 2. Reduction defines how software supports its system in waste reduction. 3. Beauty defines the value of software in sustainable development. All these factors can be measured similarly inside a software system: 1. Select a reference system. 2. Calculate whatever metric you want with the reference system. 3. Calculate the same metric with the software system. 4. Compare results. For instance, if we have software from a car brake system that saves 10% of braking energy, the reduction of this software is 10%. If the estimated maximum energy saving of the braking system is 20%, software fit for purpose is - 10%. If we have software that increases global awareness of climate change by 15%, software beauty is 15%. All these values are relative to the software system where software is executed. Unfortunately, it is not possible to have absolute metrics with these factors. The factors are subjective and depend on the system where software is executed and the domain where the system is used. In order to measure these factors, we need to be domain-specific. For each system, we need to define a reference system with which to compare. Then we can calculate how much the new system saves when compared with the old one. The analysis becomes more complicated when we compare software in different systems. In fact, software as a stand-alone product might get unintuitive results from the previous factors. For example, in an earlier paper we had software that helped to reduce carbon plant emissions by 10% [11]. The fit for purpose of the software is good since it obviously helps the plant to produce energy more efficiently than with a reference system (without the software). The reduction of the software is good since it helps the plant to reduce carbon emissions. The beauty of the software is good since by reducing carbon emissions it supports sustainable development. Hence, we can say that the carbon plant software is green. In the same paper, we had an example of software that controls a pack of windmills. The software does its work but not very efficiently. Because of this, the windmills do not work at maximum efficiency. Obviously, the fit for purpose of the software could be better. The windmills are not as efficient as they could be. However, the fit for purpose of the windmill depends on the reference system. If the software helps the windmills to create more energy (but not at maximum level), the software is purposeful. The reduction of software is probably neutral since it neither helps to reduce emissions nor helps to cut them. (A windmill has carbon dioxide emissions from physical components and maintenance.) The beauty of the software is not good since software does not support sustainable development (but the software system indeed does). With our definitions, we can say that windmill software is not green. The result may be unintuitive at first, because there is an underlying assumption that windmills are always more environment friendly than coal power plants. However, here software systems differ from each other. With our definitions, we can only compare coal power plant software with similar software and windmill software with similar software. Our power plant software is green because it cuts emissions compared with a reference power plant without control software. If we want factors with more intuitive metrics, we need to measure software as a stand-alone product. With abstract factors, such metrics would also be abstract. Software can be anything and exist everywhere. As a minimum, we need to define a problem domain where software is executed before we can define sustainability. For instance, we can compare windmill software and power plant software since their common domain is energy production. We cannot compare car brake software with eshop software since they do not have a common problem domain. Nevertheless it is unclear 26 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

currently how to get absolute beauty measurements of software as a stand-alone product. 7 Conclusion In this paper we have defined green software via green software factors. Of the defined factors, feasibility is the most intuitive and easiest to measure. It addresses software development projects, processes, and methods. Efficiency is relatively intuitive but may not be easy to measure. The interpretation of efficiency metrics is not straightforward. Sustainability is the most abstract factor and the most difficult to measure. We can get relative measures by comparing software within a software system but that may give us unintuitive results. We wish to measure software sustainability without a software system but currently we lack metrics to do it. Even if we are not yet able to define good absolute metrics for all factors, we think that recognising green software factors is already an important contribution. It matters how we design, implement, manage, and discontinue software in sustainable development. It matters what purpose software serves directly and indirectly. It matters how software supports sustainable development and waste reduction. We have tools and techniques to create and execute software that can be truly beautiful. Workshop on Software Research and Climate Change. Orlando, FL, 2009. <http://www.cs.toronto.edu/wsrcc/ Taina-WSRCC-1.pdf>. References [1] J. Lamb. "The greening of IT. How companies can make a difference for the environment". IBM Press, 2009. [2] T. Geller. "Supercomputing s exaflop target". Communications of the ACM 54,8 (2011), pp. 16-18. [3] S. Murugesan. "Harnessing green IT: principles and practices". IT Professional 10,1 (2008), pp. 24-33. [4] Gartner Newsroom. <http://www.gartner.com/it/ page.jsp?id=503867>. [5] Software engineering for the planet, <http://groups. google.com/group/se-for-the-planet>. [6] First International Workshop on Software Engineering and Climate Change. Orlando, FL, 2009, <http:// www.cs.toronto.edu/wsrcc/wsrcc1/index.html>. [7] D. Galin. "Software quality assurance: from theory to implementation". Pearson Education Limited, 2004. [8] S. Naumann et al. "The GREENSOFT model: a reference model for green and sustainable software and its engineering". Sustainable Computing: Informatics and Systems, 2011. [9] J. Taina. "How green is your software?". Proceedings of the First International Conference of Software Business (ICSOB 2010). Jyväskylä, Finland, 2010, pp. 151-162. [10] Shi-wu Lo, Wei-shiuan Tsai, Jeng-gang Lin, Guanshiung Cheng. "Swap-before-hibernate: a time efficient method to suspend an OS to a flash drive". Proceedings of the 2010 ACM Symposium on Applied Computing (SAC 10). ACM, New York, NY, USA, pp. 201-205. [11] J. Taina, P. Pohjalainen. "In search for green metrics". Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 27

Towards the Virtual Power Grid: Large Scale Modeling and Simulation of Power Grids Peter Feldmann, Jinjun Xiong, and David Kung This paper discusses the smart grid operation software, tools, and methodologies necessary to support the evolving transmission network. The network will integrate large percentages of highly variable and intermittent sources of renewable energy and will need to transport more electricity over wider areas and longer distances. The use of the dynamic models for monitoring, analysis, wide area protection, and control applications with and without a "human in the loop" will contribute to fulfilling the smart grid requirement to run a much more complex network closer to its physical limits with increased efficiency, reliability, and security. This paper first surveys the major disciplines that must evolve and converge for the realization of this vision. They include: high resolution, real-time electrical sensing, fast, secure and reliable communication, processing, and storage of real-time data, dynamic modeling and simulation, stability analysis and state estimation. Next it formulates the concept for the leap from the current static steady state approach to full dynamic modeling of the network. At the center of this concept, is the virtual power grid (VPG), a fully computable, detailed, dynamic model of a large power system interconnect (at the system of differential equations level), validated in real-time, and available in real-time for advanced applications ranging from control room visualization, operation support, and ultimately automatic wide area protection and network control. Keywords: Dynamic Simulation, Dynamic Stability Analysis, Dynamic State Estimation, High Performance Computing, Kalman Filters Smart Grid, Numerical Methods, Parallel Computing, Power Grid Simulation, Scientific Computing, State Estimation. 1 Introduction An image of the future power grid emerges from the US Department of Energy report, "20% Wind Energy by 2030" [1], and from the analysis of the ambitious California s Renewable Portfolio Standard (RPS) goals of 20% Authors Peter Feldmann received the BSc degree, summa cum laude, in Computer Engineering, in 1983 and the MSc degree in Electrical Engineering in 1987, both from the Technion, Israel, and the PhD degree in 1991 from Carnegie Mellon, USA. Dr. Feldmann began his engineering career in Zoran Microelectronics in Haifa, Israel, designing digital signal processors. Between 1991 and 2000, he was Distinguished Member of Technical Staff at Bell Labs in Murray Hill, NJ, USA, in the Design Principles Research Department. Subsequently, he served two years as Vice President, VLSI and Integrated Electro-optics at Celight, a fiber-optic communications start-up. Currently he is a Research Staff Member at the IBM T.J. Watson Research Center, USA. His research interests include analysis, design, and optimization methods for integrated electronic circuits and communication systems, timing, power and noise analysis in digital VLSI circuits, integrated electro-optics, simulation and optimization for the Smart Grid. Peter Feldmann authored over 80 papers and over 25 patents. He is a Fellow of the IEEE. <feldmann@watson.ibm.com> Jinjun Xiong has been a Research Staff Member at the IBM Thomas J. Watson Research Center, USA, since 2006, working in areas of electronic design automation, and smarter grid and smarter energy. Dr Xiong has published more than 60 technical papers in refereed international conferences and journals. He has also filed more than 20 U.S. and world-wide patents. He was the PI for the U.S. Department of Energy (DOE) Project, Request for Information on Computation Needs for the Next- Generation Electric Grid from 2010 to 2011. The project produced a special report, titled "Framework for large-scale modeling and simulation of electricity systems for planning, monitoring, and secure operations of next generation electricity grids." He is a member of the IBM s smart energy research team and works on the Pacific Northwest Smart Grid Demonstration Project, a DOE Smart Grid Demonstration project. His area of research is large-scale grid simulation and optimization. <jinjun@us.ibm.com> David Kung received a BA from U.C. Berkeley, USA, MA from Harvard, USA, and a PhD from Stanford University, USA (all in Physics). Dr. Kung joined the Advanced Simulation group in IBM Research in 1986 and worked on a massively parallel simulation engine. He moved on to the Logic Synthesis group and contributed to IBM s BooleDozer logic synthesis system. He became the manager of the Logic Synthesis group in 1999 and led the development of the Placement Driven Synthesis tool. He became the Senior Manager of the Design Automation Department in 2004 and is responsible for the Design Automation Strategy for IBM Research. He has won 4 Outstanding Technical Achievement Awards and one Corporate Award, and he is a member of the IBM Academy of Technology. He is the chair of the IEEE Design Automation Technical Committee, and has served on executive and technical program committees of major design automation conferences and workshops. <kung@us.ibm.com> 28 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

This paper discusses the smart grid operation software, tools, and methodologies necessary to support the evolving transmission network penetration by 2010 and 33% by 2020 [2]. These reports, and numerous others (for example, the Midwest ISO study of the impact of achieving a wind energy penetration of 10% in the region by 2020, with 20% in Minnesota [3]) have focused attention on the needs of the future bulk power systems to successfully integrate these levels of renewable energy. Several common threads emerge from these studies. As the penetration of renewables increases, the transmission capabilities of the grid need to be considerably expanded. Increased transmission capacity is needed both in order to provide access to the renewable generation facilities, often built at considerable distances from the main energy consumers, and also to compensate for the variability and intermittency of such sources from reserves drawn over a wider area. The new transmission capability will include new and emerging technologies such as High-voltage Direct Current (HVDC) transmission, storage, superconducting devices, and power flow control devices. New capabilities are required to address fast ramping, dynamic response, generator unit coordination, excess capacity, and minimum load. One emerging consensus of these studies is that the conventional "build" solutions alone prove inadequate or are expensive as a means of achieving the desired objectives. In addition to the infrastructure deployment, the increased capacity must also come from more efficient utilization through better tools and methodologies, i.e., the "smart" aspect of the grid. Cited examples of "smart" technologies are Real-time Systems Operation, and Planning and Uncertainty Analysis Tools. Real-time Systems Operation implies the use of advanced sensors, data communications, analytical software, and computer visualization methodologies to enable system operators, and, ultimately, automated system control to improve the performance of the transmission system over a wide area. Planning and Uncertainty Analysis Tools include dynamic modeling of new renewable generators, improved dynamic modeling of system loads, new probabilistic forecasting methods, and extreme event analysis. The ability to push the limits of the transmission system depends on how well stable operation can be maintained against disturbances [5]. Efficient infrastructure utilization necessarily means operating the grid closer to the safety limits. This, in turn, makes knowing exactly where those limits are and how much operating margin remains increasingly important [4]. In other words, there is a need for increased grid situational awareness and visibility for efficient, stable, secure, and reliable operation of the grid. The potential efficiency gain from precise situational awareness is enormous. According to [2], the presently deployed, imprecise situation monitoring leads to conservative static operating limits, often set by 20-50% or more below the physical limits as an engineered safety factor. Recognizing this fact, the Federal Government directed over half of the $3.4 billion smart grid related stimulus funds towards deploying and enhancing the grid visibility infrastructure: PMUs, smart meters, sensors, and so on for providing the necessary raw data. The next challenge is to transform all this data being made available into tools, insights, and capabilities that make a difference in the operation of the grid. The remainder of this paper will focus on the evolution of situational awareness tools for the Transmission system in the foreseeable future. The special report on smart systems, "It s a smart world" published in the 6 November 2010 issue of The Economist magazine, quotes from David Gelernter s book, Mirror Worlds [9]: "What if there were two worlds, the real one and its digital reflection? The real one is strewn with sensors, picking up everything from movement to smell. If a door opens in the real world so does its virtual equivalent. If the temperature in the room with the open door falls below a certain level, the digital world automatically turns on the heat." Today, the 1990s vision is slowly materializing as more and more "mirror worlds" are implemented as "smart systems." The power industry has started a few decades before the publishing of Mirror Worlds to implement its own version of "mirrors." They are the State Estimators currently running within Energy Management Systems (EMS). While extraordinarily useful, they are far from being high quality mirrors. They offer only a foggy, delayed, low-resolution image of the real world, and "do not turn on the heat" automatically. Instead, they just hint to very experienced human operators, trained to recognize the subtlest message, that it might be a good idea to do so. However, the power grid mirrors are constantly improving. One of the most important developments brought by the smart grid to the generation and transmission systems is the increasing deployment of phase-measurement units (PMUs) together with the associated networking and dataprocessing infrastructure. The broad coverage of the grid with these instruments will enable within the next decade the advancement from the current static steady state situational awareness to the complete real-time visibility of the power grid at the full dynamic system level. We call this dynamic level "mirror" of the real grid, the "virtual power grid," which is illustrated in Figure 1. At the core of the virtual power grid or VPG are detailed models, both steady state models and dynamic mod- Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 29

Figure 1: Conceptual View of the Virtual Power Grid. els, of various network components and subsystems, which are further assembled to form even larger system models including large-scale systems of systems. The detailed models are represented through a large system of algebraic equations and differential equations, whose composition and related model parameters will be continuously updated through real-time hardware measurements (such as PMUs) and validated through simulation. Dynamic and steady state estimation are performed based on the permanently up-to-date topology of the network and their hardware measured states, both of which are maintained in the Mass Digital Storage Management, (MDSM). The simulation of the regional interconnect model is performed in real-time, and is reconciled in real-time with the stream of network measurement snapshots (mainly from PMUs) also provided through MDSM via a Kalman-filter type state-estimation process. The result of the reconciliation is a validated dynamic state and models of the network and tracking adjustments to the various component models and their parameters. A number of monitoring, decision support, and visualization applications are running in parallel with the VPG and project a high-fidelity image of the real grid state to human operators and to automatic control modules. The detailed state visibility offered by the VPG, combined with massive computing power in control centers, will make possible a suite of applications that are almost unimaginable today. They range from sophisticated network monitoring for stability, security, reliability, wide area protection, prevention of cascading failures, ability to pre-test by detailed dynamic simulation of all management and control operations, to real-time contingency planning. They will evolve in the longer term all the way to almost full automatic control of the grid, relegating human intervention to only the most sensitive and complex operations. The "mirror" will be clear with high resolution, will give its reflection immediately, and at some point may be trusted to "turn on the heat" for us. The potential benefits from dynamic system level ob- 30 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

In addition to the infrastructure deployment, the increased capacity must also come from more efficient utilization through better tools and methodologies, i.e., the smart aspect of the grid servability for the power industry and society in general will be enormous: significantly more efficient operation, lower margins with no loss of reliability, increased ability to integrate the less-predictable and fluctuating renewable sources of energy, to activate and take advantage of demand response capabilities that are currently built into the smart grid, the elimination of blackouts, and many more. The evolution of the smart grid to full dynamic network observability will be a gradual one. It will consist of the evolution and merger of several existing technologies: (1) the current Energy Management Systems (EMS) presently based on updating a quasi-static steady state model of the network based on sensor measurements through state estimation, (2) high coverage with high resolution sensors, such as PMUs, and a fast, high-bandwidth networking infrastructure, and (3) dynamic modeling and simulation, dynamic stability analysis, dynamic state estimation, and dynamic model validation. The various forms of dynamic analysis are currently used mainly off-line for design and studies, such as the introduction of new devices into the network, analysis of network failures, and the understanding of events such as lightning strikes, short circuits, etc. Presently, the state of the art of power system software and the computing resources available to network operators do not permit these analyses to be performed in real-time. Therefore, achieving full dynamic observability will also require (4) high performance, parallel, and hybrid computing systems, and the adaptation of the grid analysis algorithms to these systems. The preconditions that make such a development possible in the next decade are falling into place. The power grids are getting equipped with PMUs at an accelerated pace. The networking infrastructure, while still in the process of solving the all-important data security challenges, is on track to deliver the very high data rates, and low latencies required to transmit frequent measurements from the large numbers of collection points to the control centers. Powerful, highly parallel, multithreaded, high performance, and hybrid computing platforms with massive data-streaming capability become widely available and affordable to the power industry. Finally, active research is taking place in novel parallel algorithms for dynamic state-estimation, transient simulation, stability analysis, and automatic control, which, running on high performance computing platforms, will achieve the required real-time performance and reliability. While it relies on technology yet to be deployed, the power systems "world mirror" could become a reality within a decade and, at that time, provide almost complete continuous visibility into the dynamic state of the real power grid. It will serve as computational infrastructure for a wide range of monitoring, planning, operational, and control applications, which together will contribute to the realization of the promise of the smart grid. The following sections survey the state-of-the-art in power network sensor deployment, dynamic modeling and simulation of the generation and transmission system, and its dynamic state estimation and stability analysis. We then propose a concept for the Virtual Power Grid (VPG), the power industry s version of a world mirror. 2 Real-time Instrumentation of Power Grids The first piece of the VPG puzzle is the instrumentation of the Smart Grid with sensors. For the transmission and generation network, this means mainly deployment and interconnection of PMUs. In the U.S., the goal of developing the wide area measurement system (WAMS) is to improve power system reliability, visibility, and controllability, a mission assigned to the North American Synchrophasor Initiative (NASPI) formed in 2007 by the U.S. Department of Energy (DOE) and the North American Electric Reliability Corporation (NERC), along with electric utilities and other organizations [10]. A WAMS network consists of many components, including PMUs, phasor data concentrators (PDCs), remote terminal units (RTUs), and intelligent electronic devices (IEDs) [11]. It is designed to enhance the operator s realtime situational awareness. It is an infrastructure that complements the grid s SCADA system. A PMU is a three-phase device that measures magnitude and phase angles (phasors) of currents and voltages based on digital sampling of alternating current (AC) waveforms. These measurements are time-stamped according to a common time reference, typically the global positioning system (GPS) time signal. Time stamping allows all phasors from all locations to be synchronized and, together, to provide a precise and consistent sequence of snapshots of the entire grid. Because PMUs are time stamped with a GPS signal, they are also called synchrophasor measurement units. While most PMUs now sample at 30 measurements per second, several phasor projects are contemplating applications that will require PMU sampling speeds of 60 per second, and some envision a future standard sampling rate at 120 per second (the Nyquist rate for AC power systems in North America). The IEEE C37.118 standard allows for a variety of sampling rates ranging from 10 to 120 per second, with some PMUs capable of even higher sampling Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 31

rates. In contrast, the conventional SCADA system working in today s EMS programs has a sampling rate of one every 4 seconds. Moreover, SCADA data are not time-synchronized and not shared widely across the grid. Thus the existing SCADA systems deployed by most grid operators cannot give grid operators real-time, wide area visibility into what is happening across a region. A PDC receives and time-synchronizes phasor data from multiple PMUs to produce a real-time time-aligned output data stream. A PDC can exchange phasor data with PDCs at other locations. Through use of multiple PDCs, multiple layers of concentration can be implemented within an individual synchrophasor data system. Besides measurement devices, communication mechanisms and data storage are also critical to a WAMS. A communication network is required to transport the digital information from the PMUs to the location where the data will be used. Communication is typically provided through a private wide-area network (WAN), but it can be any digital transport system that offers acceptable security, reliability, availability, and quality of service. NASPInet provides the functional requirements for a synchrophasor system architecture, and it includes a mechanism for flexible, fast, vendor-agnostic, and secure communication of phasor measurements from data collection points to various levels of PDCs and phasor application use points. Systems to store synchrophasor data and to make them conveniently available for post-real-time analysis can be integrated with the PDC or can be standalone data historians. A good reference for the status of PMU deployment in the U.S. and throughout the world, initial results, applications, and research was well summarized in the October 5-6, 2010, NASPI Work Group Meeting, in Arlington, VA [56]. Currently, China is the clear leader in the industry, with more than 1,000 units already deployed according to Arun Phadke, an IEEE Smart Grid Expert and author of reference text on synchrophasors [57]. The U.S. currently has about 250 units in place and expects to reach the 1000 PMU level in next several years [58]. PMU data can be analyzed even in the absence of a network model. The collections of time series, proper analytic methods, and visualization techniques can provide a very useful diagnosis of the state of the grid. Using a medical analogy, the "model-less" analysis is equivalent to diagnosis based just on external symptoms. In contrast, in the rest of this paper, we will focus on the more elaborate, ambitious, computationally expensive, and model based analysis analogous to anatomical and radiological diagnosis. Dynamic modeling will provide the "anatomy", and dynamic state estimation will be our "CT-scanner". 3 Dynamic Power System Modeling and Analysis In current practice, the most widely used method for power grid analysis in numerous applications throughout the power industry is power flow analysis. Power flow is based on a quasi-static assumption and the individual component s steady-state models are combined into a complete system model to determine the steady-state behavior of the entire grid. In most cases, models of transmission systems in power flow analysis assume a balanced operation of the three phases and represent only positive sequence quantities. Positive sequence [33] is an efficient way to represent a balanced multiphase system that can be analyzed in a manner similar to single phase circuits. In power flow analysis, each of the components (transmission elements, generators, and load) is represented by its steady-state model. For transmission lines, transformers, and shunt capacitors/reactors, model development is accomplished by an accurate calculation of the impedances, ratings, and other parameters that will be incorporated into the full steady-state network model. Steady-state models for Flexible A.C. Transmission System (FACTS) and high voltage direct current (HVDC) vary with the device type being modeled and the operating mode of the device. For generation, steady-state models represent real and reactive power capability and voltage control at either the generator terminal bus or a nearby high-voltage bus. Load is typically represented as constant real and reactive power; constant current and constant impedance loads are also sometimes represented in steady-state models. As stated in the introduction, the quasi-static power flow representation does not explicitly model the time varying nature of the grid and has only limited predictive capability. Nevertheless, current EMS systems present the operators in control rooms with the quasi-static state of the network, which, combined with their decades long experience of network management, allows them to make the necessary decisions. However, as network complexity increases and the need to operate it closer to capacity limits grows, simulation and modeling must take network dynamics into account. One type of dynamic power system analysis is the Electromagnetic Transients Program (EMTP) [15][16]. EMTP does transient analysis by assembling the system of differential equations to model the power network and to numerically integrate the initial-value problem in time. EMTP typically uses a modified nodal analysis formulation of the network. This is similar to the widely known SPICE (Simulation Program with Integrated Circuit Emphasis) [17] program used in the field of electronic circuit designs. The descendant of the original EMTP program is freely distrib- At the core of the virtual power grid or VPG are detailed models, both steady state models and dynamic models, of various network components and subsystems 32 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

The first piece of the VPG puzzle is the instrumentation of the Smart Grid with sensors uted as Alternative Transients Program (ATP) from www.emtp.org. EMTP is mainly used in off-line studies of very fast (in comparison to the 50-60Hz fundamental frequency) phenomena, which may cause over voltages, over currents, abnormal waveforms, or electromechanical transients. Typical EMTP studies are lightning over voltage studies, switching transients and faults, machine modeling, transient stability, motor startup, shaft torsional oscillations, transformer and shunt reactor/capacitor switching, power electronic applications, circuit breaker duty cycle (electric arc), current chopping, FACTS devices modeling, harmonic analysis, network resonances, and protective device testing. The dynamics of the power network extend over a wide range of time-scales. The simulation step size of EMTP is of the order of tens of microseconds, but can even be smaller depending on the type of electromagnetic phenomenon being studied. Such time-steps would make it computationally expensive to analyze phenomena with large time constant associated with the dynamics of power plants, such as generators and turbines. For analyzing the large time-constant (slower) phenomena, the concept of generalized averaging method, also referred to as "dynamic phasors" approach, was proposed in [21] to model power electronics based equipment. The main idea behind this method is to represent the periodical or nearly periodical system quantities not by their instantaneous values but by their time varying Fourier coefficients. The variations of the time varying Fourier coefficients are much slower than the original instantaneous values. The application of this method was then extended to model FACTS devices [22] and increased the accuracy of the fundamental frequency phasor models. The same approach has also been applied to model electrical machines under unbalanced conditions [23]. One of the key components of a dynamic simulator is the models that represent the dynamics of components. For stability studies, the characteristics of concern typically have time constants in the range of a few tens of milliseconds to many seconds. Thus, in this context, the dynamics models are represented by differential equations that capture various behaviors of a power plant, including excitation systems, governors, turbines, their controls, certain components of loads, power electronic transmission devices (such as FACTS and HVDC), and, for some studies, on-load tap changers, power line communication (PLC) controls on shunt devices, remedial action schemes, and other similar control devices. The dynamic models also need to capture the dependence of transmission network impedance parameters to frequency variations. A major challenge is the dynamic modeling of the load, which is also the least accurate component of the power system model. The load represents an aggregate of the total electrical load in the system at any given time. The load ranges from simple light bulbs to large industrial facilities. Historically, the load has been represented in dynamics studies with a static ZIP model, which consists of a combination of constant impedance (Z), constant current (I), and constant power (P) elements. Typically, the real power of a load is modeled as a constant current and the reactive power is modeled as a constant impedance. A process known as load "conversion" transforms the (usually) constant power load models in a steady-state power flow model into the selected composition for a dynamics model prior to any dynamics simulations. However, static load models are increasingly viewed as inadequate for representing loads in dynamics studies, particularly with increased penetration of air conditioning and power electronics. Dynamic load models are needed to model many crucial dynamic phenomena of the loads. Motor load models are both dynamic and readily available, but newer load models are needed to represent certain phenomena such as air-conditioner motor stalling. The generation of a system-wide model of the load first needs an estimate of load composition. The aggregated static/dynamic model can be obtained from a component based approach using information on customer types and categories at substations. Such work needs to be repeated at multiple times of the year (for example, summer peak, winter peak, fall, and light load). The recent deployment of PMUs synchrophasor data will help better characterize and validate dynamic models [18][19]. WECC s Load Modeling Task Force is completing a multi-year effort on developing and implementing a composite load model. That model has reproduced (in principle) historic events, and will now be used for voltage stability assessments. Synchrophasor data are invaluable for understanding and modeling loads in power system studies. When specific loads can be identified on the system, PMUs installed at sub-transmission levels can collect data on those loads responses to actual frequency events, and use these data to improve load modeling. Southern California Edison has also successfully used phasor data from load locations for load model calibration and tuning [20]. Another smart grid element, which will become very instrumental in increasing the accuracy of load models, is smart meters that are currently being deployed in millions at customer premises. At present, dynamic simulation and stability analysis are mainly used offline in design activities to identify potential issues on newly built systems, and have successfully explained and devised appropriate mitigation of transient problems experienced on operational systems. They Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 33

have also been successfully used to devise guidelines for maintaining stability during operation. Because of their high computational complexity, dynamic simulations are rarely used in real-time or control applications. Research into new algorithms that can fully take advantage a high-performance hybrid computing platform is needed to bring the speed of dynamic simulation to "faster than real-time" and thus realize the full potential of dynamic simulation into the operation of the power grid. The VPG concept as will be discussed later may hold keys to address these needs. 4 Stability Analysis With Dynamic Modeling One of the major applications of dynamic modeling of the power network is stability analysis. Dynamic stability analysis is becoming even more important in the modern grid because of the need to transport energy over long distances and to integrate new technologies and large intermittent generation capabilities (such as on and offshore wind farms and solar plants), and the presence of FACTS controllers. Efficient operation of the grid closer to its limits requires accurate dynamic modeling and stability margin evaluation. The theory of stability analysis is a very mature and wellstudied topic in dynamical systems in general [24]. Its specific application to power systems is quite mature, and [25] represents a comprehensive reference in this regard. In today s practice, the dynamic stability problems consist of analyzing critical scenarios such as short-circuit, loss of mechanical power, loss of electrical supply, load fluctuations, predicting network reactions to these disturbances, and recommending the appropriate operating measures such as type of protection device, relay setting, and load shedding. Because of the lack of validated accurate models, and also the high computational complexity, stability analysis is seldom used in real-time operation. One important category of stability analysis is smallsignal stability. Small-signal stability analysis determines the power system s ability to maintain proper operation and synchronism when subjected to "small" disturbances. From a mathematical point of view, a disturbance is considered to be small if the differential equations that describe the behavior of the power system can be linearized about the current operating point for the purpose of the analysis. A typical small-signal stability problem is the insufficient damping of system oscillations. Small-signal stability analysis reduces to eigen-analysis of the properly formulated linearization of the generally nonlinear system of differential equation describing the power network [12:chapter 25 th ]. The computational cost of small-signal stability depends on the number of state variables needed to model the system. The cost is moderate for small problems but increases rapidly as the number of state variables grows. Large systems require specialized techniques as discussed in the sequel that focused on a subset of eigenvalues/vectors. Small-signal stability problems are of two flavors, local and global. The local problems, called local plant mode oscillations, analyze rotor angle oscillations of a single generator or a single plant against the rest of the power system. Another category of problems are the inter-machine or inter-plant mode oscillations associated with oscillations between the rotors of a few generators close to each other. Analysis of local small-signal stability problems requires the detailed representation of a small portion of the complete interconnected power system, while the rest of the system representation may be appropriately simplified by the use of simple models, such as system equivalents or model-order reduction techniques [26]. Global small-signal stability problems are caused by interactions among large groups of generators and have widespread effects. They involve oscillations of a group of generators in one area swinging against a group of generators in another area. Such oscillations are called inter-area mode oscillations. Analysis of inter-area oscillations in a large interconnected power system requires a detailed modeling of the entire system as described in the previous section with hundreds of thousands or more state variables. For such large systems, special techniques have been developed that focus on evaluating a selected subset of eigenvalues associated with the complete system response. Several well-established global small-signal stability programs exist. The AESOPS algorithm [27] uses a novel frequency response approach to calculate the eigenvalues associated with the rotor angle modes. The selective modal analysis (SMA) approach [28] computes eigenvalues associated with selected modes of interest by using special techniques to identify variables that are relevant to the selected modes, and then constructing a reduced-order model that involves only the relevant variables. The program for Eigen-value Analysis of Large Systems (PEALS) [29] uses two of these techniques: the AESOPS algorithm and the modified Arnoldi method. These two methods have been found to be efficient and reliable, and they complement each other in meeting the requirements of small-signal stability analysis of large complex power systems. In recent years, there has been significant progress in the development of parallel algorithms for large-scale eigenvalue problems [30][31][32] targeting high perform- The collections of time series, proper analytic methods, and visualization techniques can provide a very useful diagnosis of the state of the grid 34 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

ance and hybrid computing platforms. A major research effort is needed for the adoption of these techniques to the power system stability problems, to facilitate faster or even real-time speed solutions of the even larger problems that may occur during network operation. Not all power system stability problems fall into the small-signal category. The power system must maintain synchronous operation even in the presence of a large disturbance, such as the loss of generation or a transmission facility, or a significant change in load. The intermittent nature of renewable generation facilities is also prone to cause instabilities. The system response to such phenomena consists of large excursions in power flows, voltages, currents, and angles. Therefore, the local linearization of the system differential equations employed by the small-signal stability analysis is no longer valid. For these situations, stability must be assessed through transient simulation methods as described in the previous section. Additionally, some of the analyzed events depart from the assumption of balanced operation of the three phases, which could be analyzed in a manner similar to single phase circuits. The transient stability analysis of these problems must use equations for all three phases. The method of symmetrical components [33] is typically used to formulate the unbalanced system equations. The first symmetrical component is the positive sequence, the only one needed when the system is balanced. An alternative way to analyze transient stability is by the so-called "direct methods" which are special forms of Lyapunov s second method [34][35]. These methods assess stability or calculate stability margins directly, without performing transient simulation. The application of these methods to practical systems remains limited because of the difficulty of including detailed generator models into the energy function. Therefore, the best approach today seems to be a hybrid one where the computation of the energy function is incorporated in the transient simulation [36]. This way, transient simulations can also compute stability margins. Presently, stability analysis for both small signal and transient is considered computationally difficult, and is thus rarely solved in real-time for immediate operational purposes. However, with the increased information of the power grid, extensive instrumentation for real-time measurements such as those provided by PMUs, and the harnessing of massive computational power of hybrid, high performance computing platforms, and of new parallel algorithms, the stability analysis of the dynamic network is bound to move increasingly into the real-time running of the power system, where it can deliver precise, predictive stability monitoring, a priori verification of operator actions, and ultimately participation of automatic control and wide area protection. The role of the VPG as described at the end of this paper will provide the up-to-date measurement validated dynamic model of the grid, from which stability analysis can evaluate the margins, detect dangerous developments, and eventually initiate preventive action. 5 Dynamic State Estimation In the present, state estimation is mainly used within EMS, and its main purpose is to generate a real-time network model. This model is determined from data provided through the SCADA network which provides topological information (such as the status of breakers and switches), electrical measurements (such as power flow, power injection, and voltage magnitudes), and setting data (such as transformer tap positions, and switchable capacitor bank values). These raw data are processed by the state estimator in order to filter measurement noise and detect errors. However, this real-time model is based on the quasi-static mathematical representation of the interconnected power network, i.e., the power flow model that assumes slow and steady changes in the system. Therefore, the state needs continuous updating. Though a new data snapshot arrives every few seconds, because of the high computational complexity, state estimation is performed only every few minutes or at major events. The state estimator solution represents the best estimate of the system state based on the available measurements and on the assumed system model. Faster state estimation in EMS systems today use "tracking state estimation" that updates the states for the next instant of time with new set of measurement data obtained for that instant without fully running the static state estimation algorithm. This allows continuous monitoring of the system without excessive usage of the computing resources. Hence tracking state estimation plays an important role in the energy management system. References [37] through [46] describe various methods for static state estimation. A particularly interesting recent work [7] attempts to speed-up static state-estimation using high performance computing (16 Cray MTA-2 parallel processors) and adapted algorithms (shared memory conjugate gradient) to perform state estimation of the 14000 bus Western Interconnect system. They achieved parallel speed-up about 10 times with 16 processors. Further algorithmic tuning demonstrates that real time static state estimation at the SCADA speed is achievable in the immediate future with high performance computing platforms. The advent of PMUs led to a paradigm shift in the state estimation process [47]. With this technology, the capability of directly measuring the quasi-static state of the power system has become a reality. PMUs measure voltages at network buses and the currents in transmission lines and One type of dynamic power system analysis is the Electromagnetic Transients Program (EMTP) Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 35

Efficient operation of the grid closer to its limits requires accurate dynamic modeling and stability margin evaluation transformers. With sufficient PMU coverage, quasi-static state estimation may not even be necessary. In fact, it has been found that by installing PMUs at about one-third of the number of system buses with voltage and current measurements, it is possible to determine the complete system state vector. At present, however, PMU coverage is far from sufficient for direct measurement. This situation may change in the next few years. In the interim, systems with a few or completely without PMUs still need to rely on state estimation for calculating phase angles, and determining the power flow for operational and market purposes. State estimation simulations take longer to predict phase angles compared to real-time measurements of phase angles from PMUs. Simulations would need to be 100 times faster to match the real-time measurements from PMUs whose sampling rate is at least 30 times per second. Provided there is no bad data from measurements and network topology, state estimation simulations could give good results. Even without providing full coverage, accurate voltage and current phasor measurements from PMUs will enhance the accuracy of state estimators. Direct measurement of voltage phases at buses by PMUs further reduces the size of the matrices in state estimation models and improves the convergence speed and precision of state estimation [48][49]. Both tracking and direct PMU measurements of state operate on a quasi-static representation of the grid, which does not include any explicit physical modeling of the time varying nature of the system. Even in this mode, the quasistatic state is very useful for clearing the energy market, (determining how much energy was produced or consumed by whom), and, when combined with proper visualization, provides a good image of the network for the experienced operator. In order to increase the predictive power of the models, we need to move to an entirely different paradigm - dynamic state estimation - i.e., estimating the state of a dynamic simulation model that fully describes the time varying nature of the system and allows to project the behavior of the power system to the future. Such prediction capability is a key for detecting and preventing instabilities, predicting outages, and supporting automatic control. The dynamic state estimator was introduced in [50] and a considerable amount of research has been performed to analyze its capabilities and to reduce its computational complexity. The dynamic state estimator assumes the existence of a predictor function (in general nonlinear) that, given the present state of the system, can produce the state of the system at the next time-point of interest. Such a function would emerge naturally if the power system would be modeled at the level of physically based dynamic differential equations. However, the majority of the dynamic state estimation literature does not assume the existence of such a detailed model. Instead, it imposes a simplified parameterized predictor function (most often linear) and a dynamic state/parameter estimator based on the Kalman-Bucy [51] filter. The filter yields estimates of a state vector and the parameters of the function. It is also assumed that an initial value of the parameter vector is given along with its respective covariance matrix. Reviews of developments in dynamic state estimation for power systems and hierarchical state estimation were presented in [52][53]. In practical power grid interconnects, the idealized assumption of the Kalman filter theory tends to break down and significant research has been done to develop robust algorithms that can be used in real-life situations. Reference [54] presents an in-depth survey of these techniques. The emergence of PMUs has so far affected mainly the accuracy of dynamic state estimation, but has not generated a paradigm shift. The paper [55] demonstrates that PMU based dynamic state estimation reduces the estimation errors. As expected, the coverage and location of the PMUs are important to obtain a better estimate. For power systems applications, the dynamic state estimators have been introduced to improve on the accuracy and robustness of the tracking estimators. However, since they do not rely on physically based dynamic equations of the power system, the parameterized models, even used and fine-tuned by the state estimator, have very limited predictive capability and can only provide limited insight on important metrics of the grid such as stability. The addition of PMUs does improve the quality of the state estimation but does not by itself change its predictive quality. On the other hand, the present state of the art does not permit real-time, dynamic simulation with PMU based dynamic state estimation. For that to happen, significant research needs to be invested in scalable algorithms that can take full advantage of powerful multi-threaded, multi-processor, and hybrid high performance computing platforms. In fact, the Pacific Northwest National Laboratory group, who performed the static state estimation on a parallel computing platform [7], has also performed a study for dynamic state estimation [8], and demonstrated the feasibility to track the 30 samples per second phasor measurement updates for a small-scale power system with 3 machines and 9 busses. The core of the VPG as described in detail in the next section is in fact such an advanced state estimator that augments, corrects, and validates the real-time dynamic simulation models of the grid. It not only predicts the correct state as a combination of the simulation result and the meas- 36 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

urement based snapshot, but also provides a continuous correction stream to the network models and their parameter values. Significant research in state-estimation and simulation algorithms, computing platforms, and network architecture is necessary to achieve the required capability and performance. 6 The Decade Goal: The Virtual Power Grid The key to unlock the true potential of quality, widearea, and dense coverage sensor data (synchrophasors, topology, and device settings) is robust, real-time state estimation in a higher dimension than before, i.e., dynamic model based state estimation. We call this dynamic model the Virtual Power Grid (VPG). It was discussed that even traditional quasi-static (power flow) state estimation can benefit from the knowledge of the synchronized angle data provided by PMUs, and can generate a network solution based on both synchrophasors and conventional measurements simultaneously using the redundant measurements to enable better solution accuracy [60][61]. However, with good network coverage, high sampling rate, time synchronization provided by the PMUs, fast networking, and massive computational power, it will become possible to perform state estimation for a fully dynamic model of the power grid, which in turn will provide the starting point to a wide range of revolutionary applications. The VPG model consists of the combined network differential equations of generators, transmission components, transformers, and aggregated loads at the most detailed level. For example, the generator plant models must include rotor dynamics, governors, excitation control, and various mechanical and digital controllers that constitute the machines. All transmission devices such as lines and transformers also need to be represented at the differential equation level. The system needs to be aware of the full network topology including the current state of breakers and other similar devices down to substation level. The aggregated loads at substations are modeled as compositions of load types such as resistive, compressor (air-conditioning and heat pump) electronic loads, and single and tri-phased. Since their true proportion can only be estimated, these models need to be supplemented with adequately placed PMUs installed at subtransmission levels that collect data on those loads responses to actual frequency events, and use these data to improve load modeling. Similarly, PMUs must be installed at wind and solar generation collector points to provide instantaneous data on renewable source variability. Information from the smart meters deployed throughout the distribution network can also improve the accuracy of load modeling significantly. Similarly, precise high-resolution weather information and forecast can be incorporated in the VPG to increase the accuracy and predictive power of the models. The full value of the VPG model will be realized when run in real-time that is, the system of differential equations is solved at the same rate as the evolution of the real power grid (one virtual second is solved in one real second). Since we are not aware of a similar work in the literature, we attempt to estimate the computational effort at the order of magnitude level. We assume that the real-time dynamic simulation will be performed in terms of dynamic phasors [21] and not at the full EMTP level. The expected time-step in this formulation will be 10-2 s. An Eastern Interconnect Network-sized problem at around 30,000 buses can easily generate 10 6 state-variables. Furthermore, we estimate that we can reach a state solution at a cost of 10 4 operations per state variable and per time point. A rough estimate of the necessary computation power is therefore 10 12 operations/second just for the open-loop simulation of the Eastern Interconnect. However, the simulation of the VPG is not just a simple open-loop simulation problem. It also needs to incorporate the continuous stream of PMU and network state information data, and to perform dynamic state estimation whenever a synchrophasor snapshot becomes available in a time window small enough to be compatible with real-time operation. The mechanics of combining the open-loop simulated state with the measurement data stream require sophisticated algorithms that are essentially variants of the Kalman filter [65][66], including the particle filter, the extended Kalman filter (EKF), or the unscented Kalman filter (UKF). The combination of simulated and measured data results in not only a robust measurement validated state, but also continuous adjustment and fine-tuning of all the model parameters. The assumptions for implementing such algorithms are even more speculative than the open-loop simulation. Given that the expected full state-estimation needs to be performed only about once every second and its total cost could be 10 2 times higher than computing a time step, the total cost of running the VPG may reach an order of teraflops that are well within the reach a high performance computer with 10 4 nodes. Achieving real-time simulation based on regional or interconnect level network models is, nevertheless, a considerable challenge that will require research into high performance computer configurations and novel algorithms that fully exploit their speed and parallelism. Nevertheless, such a challenge is fully attainable within this decade. The result of the combined dynamic simulation and state estimation offers complete visibility at the dynamic level The result of the combined dynamic simulation and state estimation offers complete visibility at the dynamic level of the grid state Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 37

A number of open research problems need to be addressed in order to make the VPG a practical reality of the grid state, and, through continuous parameter adjustment, in a living, continuously validated, computable model: the VPG. The virtual grid s real-time simulation results can be analyzed, mined, visualized in the control room for all kinds of network metrics: power oscillations, oscillation mode shapes voltage stability indications, and system angular stress. The VPG is permanently available in memory, and serves as a starting point to a wide range of simulations and analyses needed for alarms, stability margin calculations, evaluation of operation decisions, planning, and so on. The computational resources required to perform the various derived applications could match or exceed those of the VPG itself. A number of open research problems need to be addressed in order to make the VPG a practical reality. Following is our attempt to list some of the most important ones: Formulation of the dynamic simulation model of an Interconnect sized power grid in terms of dynamic phasors, and optimization of a parallel integration algorithm capable of simulating the full grid in real-time or faster than real-time speed. Development of algorithms for physical model-based dynamic state estimation using streaming measurement (mainly PMU) data. The algorithms need to be robust to detect topology changes and to track and back-propagate model parameter values. This is a vast topic by itself with a number of subtasks. Optimal placement of PMUs for robustness, to compensate the least accurate models in the grid (such as load at substations), and to stabilize and decouple the open-loop simulation equations Choice of the most scalable Kalman-like filter algorithm (for example, Monte Carlo-based) and map it onto a high performance computing platform Practical, robust, dynamic stability estimation, tracking, and monitoring algorithms with instantaneous response time on high performance computing platforms. Initially, the VPG, when combined with powerful analytic and visualization techniques, will serve as an order of magnitude more precise tool for the existing human-in-theloop paradigm. Gradually and in time, the VPG will evolve toward supporting automated control of physical system actions, including generator balancing energy and reactive power production, demand response and storage, reaction to various events for intelligent protection and operations, maintenance decision support, and to effectively manage and maintain generation-load balance. The full realization of the VPG and its applications will depend on the massive deployment of network sensing technology, research and advances in methodologies and algorithms for modeling, simulation, state estimation, result analysis, and visualization, and full exploitation of fast secure networking and high performance, hybrid, and distributed computing engines. However, the potential enormous benefits to the industry and society, such as increased efficiency of power production, better and fuller exploitation of the existing grid infrastructure, increased safety and reliability, reduction of emissions of greenhouse gasses and pollution, and elimination of cascading failures and blackouts, fully justify such efforts to establish the VPG. References [1] "20% Wind Energy by 2030: Increasing Wind Energy s Contribution to U.S. Electricity Supply". Available electronically at <http://www.osti.gov/bridge>. [2] M. Brown et al. "Transmission Technology Research for Renewable Integration". California Institute for Energy and Environment, University of California, Sep 2008. Available from: <http://uc-ciee.org/piertrans/documents/ TransmissionTechnologyResearchforRenewables FinalReport. pdf>. [3] Midwest ISO (Independent Transmission System Operator). Midwest ISO Transmission, 2006. [4] J. Hauer, T. Overbye, J. Dagle, S. Widergren. "Advanced Transmission Technologies". Available from <http://certs.lbl.gov/ntgs/issue-6.pdf>. [5] A. Bose. "Power System Stability: New Opportunities for Control". Chapter in Stability and Control of Dynamical Systems and Applications, Derong Liu and Panos J. Antsaklis, editors, Birkhäuser (Boston), 2003. Available from <http://gridstat.eecs.wsu.edu/bose- GridComms-Overview-Chapter.pdf>. [6] J. Nieplocha, D. Chavarria-Miranda, V. Tipparaju, Zhenyu Huang. "Parallel WLS State Estimator on shared Memory Computers". Proceedings of 8th International Power Engineering Conference, Singapore, 3-6 Dec, 2007. [7] Zhenyu Huang, J. Nieplocha. "Transforming Power Grid Operations via High Performance Computing". IEEE Power Engineering society General Meeting, Pittsburgh, PA, 2008. [8] Zhenyu Huang, K. Schneider, J. Nieplocha. "Feasibility Studies in Applying Kalman Filter Techniques to Power Systems Dynamic State Estimation," Proceedings of 8th International Power Engineering Conference, Singapore, 3-6 Dec, 2007. [9] D. Gelernte. "Mirror Worlds: or the Day Software Puts the Universe in a Shoebox... How It Will Happen and What It Will Mean". Oxford University Press, 1992. 38 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

[10] The NASPI Vision statement. <http://www.naspi.org/ vision.stm>. [11] M. Hadley, J. McBride, T. Edgar, L. O Neil, J. Johnson. "Securing Wide Area Measurement Systems". Pacific Northwest National Laboratory Technical Report PNNL-17116, June 2007. [12] B. Gou. "Generalized Integer Linear Programming Formulation for Optimal PMU Placement". IEEE Transactions on Power Systems, Vol 23, No 3, August 2008, pp. 1099-1104. [13] N. Abbasy, H. Ismail. "A Unified Approach for the Optimal PMU Location for Power System State Estimation". IEEE Transactions on Power Systems, Vol 24, No 2, May 2009, pp. 806-813. [14] R. Sodhi, S. Srivastava, S. Singh. "Optimal PMU Placement to Ensure System Observability Under Contingency", Power & Energy Society General Meeting, July, 2009, pp. 1-6. [15] Electric Power Research Institute, EMTP Development Coordination Group, "EPRI EL-6412-L: Electromagnetic Transients Program Rule Book, Version 2". [16] H. W. Dommel. "Electromagnetic Transients Program reference manual (EMTP Theory Book)". Bonneville Power Administration, August 1986. [17] A. Vladimirescu and S. Liu. "The Simulation of MOS Integrated Circuits Using SPICE2". ERL Memo No. ERL M80/7, Electronics Research Laboratory, University of California, Berkeley, October 1980. [18] Dmitry Kosterev, Anatoliy Meklin, John Undrill, Bernard Lesieutre, William Price, David Chassin, Richard Bravo, Steve Yang. "Load Modeling in Power System Studies: WECC Progress Update". Presented at 2008 Power Engineering General Meeting, Pittsburgh, PA, July 2008. [19] Zhenyu Huang, M. Kosterev, R. Guttromson, and T. Nguyen. "Model validation with hybrid dynamic simulation". Power Engineering Society General Meeting, October 2006. [20] Z. Huang, R. Guttromson, and J. Hauer. "Large-Scale Hybrid Dynamic Simulation Employing Field Measurements". In: Proceedings of the IEEE Power Engineering Society General Meeting 2004, Denver, USA, June 5-10, 2004. [21] S. Sanders, J. Noworolski, X. Liu, and G. Verghese. "Generalized averaging method for power conversion circuits". Power Electronics, IEEE Transactions on, vol. 6, no. 2, pp. 251 259, 1991. [22] A. M. Stankovic, P. Mattavelli, V. Caliskan, and G. C. Verghese. "Modeling and analysis of FACTS devices with dynamic phasors". In Power Engineering Society Winter Meeting, 2000. IEEE, vol. 2, 2000, p. 1440. [23] A. Stankovic, B. Lesieutre, and T. Aydin. "Applications of generalized averaging to synchronous and induction machines". In North American Power Symposium, Boston, 1996, pp. 277 282. [24] V. I. Arnold, "Mathematical methods of classical mechanics". Springer-Verlag, 1982. ISBN 0-387-96890-3. [25] Prabha Kundur, "Power System Stability and control". The EPRI Power System Engineering Series, McGraw- Hill, Inc., 1994. [26] N. Singh, R. Prasad, H. O. Gupta. "Reduction of power system model using Balanced Realization, Routh and Pade approximation methods". Int. J. Model. Simul. Pp. 57-63, 2008, ACTA Press. [27] R.T. Byerly, R.J. Bennon, D.E. Sherman. "Eigenvalue Analysis of Synchronizing Power Flow Oscillations, Power Apparatus and Systems". IEEE Transactions on, Jan. 1982, Vol. PAS-101 Issue:1, pp. 235 243. [28] J.L. Sancha, I.J. Perez-Arriaga. "Selective modal analysis of power system oscillatory instability". Power Systems, IEEE Transactions on, May 1988 Vol. 3 Issue:2, pp. 429 438. [29] P.W. Sauer, C. Rajagopalan, M.A. Pai. "An explanation and generalization of the AESOPS and PEALS algorithms [power system models]". Power Systems, IEEE Transactions on, Feb 1991, Vol. 6, Issue: 1, pp. 293 299. [30] "PDSYEVR. ScaLAPACK s parallel MRRR algorithm for the symmetric eigenvalue problem". D. Antonelli, C. Vomel, Lapack working note 168, (2005). [31] I.J. Bush, A.G. Sunderland, and G. Pringle, "Using Scalable Eigensolvers on HPCx: Case Studies," HPCx Technical Report HPCxTR0510, 2005. [32] A.G. Sunderland and E. Breitmoser. "An Overview of Eigensolvers for HPCx". HPCx Technical Report HPCxTR0312, 2003. [33] C. L. Fortescue. "Method of Symmetrical Co-Ordinates Applied to the Solution of Polyphase Networks". Presented at the 34 th annual convention of the AIEE (American Institute of Electrical Engineers) in Atlantic City, N.J. on 28 July 1918. [34] P.C. Magnusson. "Transient Energy Method of Calculating Stability". AIEE Trans, Vol 66 pp. 747-755, 1947. [35] A.A. Fouad, V. Vittal. "Power Systems Transient Stability Analysis using the Transient Energy Function Method". Prentice Hall, 1992. [36] C.K. Tang, C.E. Graham, M. El-Kady, R.T.H. Alden. "Transient stability index from conventional time domain simulation". Power Systems, IEEE Transactions on, Aug 1994, Vol. 9 Issue 3, pp. 1524 1530. [37] F. C. Schweppe, J. Wildes, and D. Rom. "Power system static state estimation". Power Syst. Eng. Group, M.I.T. Rep. 10, Nov. 1968. [38] F. C. Schweppe and J. Wildes. "Power system staticstate estimation, part I: Exact mode". IEEE Trans. Pourer App. Syst., vol. PAS-89, pp. 120-125, Jan. 1970. [39] F. C. Schweppe and D. B. Rom. "Power system staticstate estimation, part 11: Approximate model". IEEE Trans. Power Apparatus and Systems, Vol. PAS-89, pp. 125-130, Jan. 1970. [40] D. M. Falco, P. A. Cooke, A.Brameller. "Power System Tracking State Estimation And Bad Data Processing". IEEE Transactions on Power Apparatus and Sys- Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 39

tems, Vol. PAS-101, No. 2 February 1982. [41] E. Handschin. "Real-Time Data Processing Using State Estimation in Electric Power Systems". In Real-Time Control of Electric Power Systems, E. Handschin, Ed., Elsevier, Amsterdam, 1972. [42] R. D. Massiello and F. C. Schweppe. "A Tracking Static State Estimator". IEEE Trans. PAS, March/April 1971, pp 1025-1033. [43] E. Handschin, F. C. Schweppe, J. Kohlas, A. Fiechter. "Bad Data Analysis for Power System State Estimation". IEEE Trans. Power App. Syst., vol. P AS-94, pp.329-337, Mar/Apr.1975. [44] W. W. Kotiuga. "Development of a Least Absolute Value Power System Tracking State Estimator". IEEE Transactions on Power Apparatus and Systems, Vol. PAS-104, No. 5, May 1985. [45] S. C. Tripathy, S. Chohan. "On Line Tracking State- Estimation in Power Systems". IE (I) Journal-EL, 1992. [46] A. K. Sinha, L. Roy, H. N. P. Srivaetava. "A New and Fast Tracking State Estimator for Multi terminal DC/ AC Power Systems". TENCON 89. Fourth IEEE Region 10 International Conference, pp. 949-952. [47] S. Horowitz, A.Phadke, B.Renz. "The Future of Power Transmission". Power and Energy Magazine, IEEE, March-April 2010, Volume 8, Issue 2, pp. 34 40. [48] F. Chen, X. Han, Z. Pan, L. Han. "State Estimation Model and Algorithm Including PMU". DRPT2008 April, 2008, Nanjing, China, pp.1097-1102. [49] H. Zhao. "A New State Estimation Model of Utilizing PMU Measurements", 2006 International Conference on Power System Technology, pp.1-5. [50] A. S. Debs and Robert. E. Larson. "A Dynamic Estimator for tracking the state of a Power System". IEEE Transactions on Power Apparatus and Systems, Vol. PAS 89, NO.7, September-October, 1970. [51] R. S. Bucy. "Lectures on Discrete Time Filtering". Berlin, Germany: Springer-Verlag, 1994. [52] P. Rousseaux, Th. Van Cutsem, and T. E. Dy Liacco. "Whither dynamic state estimation". Int. J. Elect. Power, vol. 12, no. 2, pp. 105 116, Apr. 1990. [53] I. W. Slutsker, S. Mokhtari, and K. A. Clements. "Real time recursive parameter estimation in energy management sys- tems". IEEE Trans. Power Syst., vol. 11, pp. 1393 1399, Aug. 1998. [54] A. Jain, N.R. Shivakumar. "Power system tracking and dynamic state estimation". Power Systems Conference and Exposition, 2009. PSCE 09. IEEE/PES, 2009, pp. 1 8. [55] A. Jain and N.R. Shivakumar. "Impact of PMUs in dynamic State Estimation of Power Systems". Proceedings of the North American Power Symposium (NAPS), Calgary, Canada, September 28-30, 2008. [56] October 5-6, 2010, NASPI Work Group Meeting, Arlington, VA, meeting theme: Recovery Act Project Update. <http://www.naspi.org/meetings/workgroup/ workgroup.stm>. [57] A.G. Phadke and J.S. Thorp. "Synchronized Phasor Measurements and their Applications". ISBN 978-0- 387-76535-8. [58] October 7-8, 2009, NASPI Work Group Meeting in Chattanooga, Tennessee, <http://www.naspi.org/meetings/archive/meetingarchive.stm>. [59] A. Johnson. "Syncro-phasor based voltage and var management". Power Systems Conference, PSC 09. Clemson, SC, 10-13 March 2009, pp. 1 7. [60] NYPA Enhanced state estimation work with EPRI and Siemens - Hirsch P. "NYPA investigates a next generation phasor-assisted state estimator at its energy control center". Innovations with EPRI Technology, No. 1011164, Jan. 2005. [61] A. Ghassemian, B. Fardanesh. "Phasor assisted state estimation for NYS transmission system - Implementation & testing". Power Systems Conference and Exposition, 2009. PSCE 09. IEEE/PES, pp.1-8, 15-18 March 2009. [62] SDG&E. "Integrating phasor data into state estimator to better manage congestion". Available from <http:/ /www.naspi.org/meetings/workgroup/2008_ march/ session_two/sdge_enhance_ state_estimation_ kondragunta.pdf>. [63] TVA On-line state estimator using PMUs. Available from <http://www.naspi.org/meetings/workgroup/ 2008_june/vendor/areva_activities_ synchrophasor_ giri.pdf>. [64] D. Chaniotis, and M. A. Pai. "Model Reduction in Power Systems Using Krylov Subspace Methods". IEEE Transactions on Power Systems, Vol. 20, No. 2, May 2005. [65] G. Evensen. "The Ensemble Kalman Filter: theoretical formulation and practical implementation". Ocean Dynamics (2003) 53: 343-367. [66] M. S. Arulampalam, S. Maskell, N. Gordon, T. Clapp. "A Tutorial on Particle Filters for Online Nonlinear/ Non-gGaussian Bayesian Tracking". IEEE Transactions on Signal Processing, Vol 50, No. 2, Feb 2002. [67] S. Zhong and A. Abur. "Combined state estimation and measurement calibration". IEEE transactions on Power Systems, Vol. 20, No. 1, Feb 2005 pp. 458-465. 40 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

Artificial Intelligence Techniques for Smart Grid Applications María-José Santofimia-Romero, Xavier del Toro-García, and Juan-Carlos López-López The growing interest that the smart grid is attracting and its multidisciplinary nature motivate the need for solutions coming from different fields of knowledge. Due to the complexity, and heterogeneity of the smart grid and the high volume of information to be processed, artificial intelligence techniques and computational intelligence appear to be some of the enabling technologies for its future development and success. The aim of this article is to review the current state of the art of the most relevant artificial intelligence techniques applied to the different issues that arise in the smart grid development. This work is therefore devoted to summarize the most relevant challenges addressed by the smart grid technologies and how intelligence systems can contribute to their achievement. Keywords: Artificial Intelligence, Computational Intelligence, Dynamic Grid Management, Smart Grid. 1 Introduction The structure of traditional electrical grids comprises different stages in the energy supply process. The first stage consists of the power generation that takes place in large power plants. In the second stage the energy is transported to the areas where it will be consumed. Finally, after being adequately transformed, energy is delivered to the end user in the distribution stage. This last stage in particular has experienced many changes in recent years with the progressive introduction of new players such as distributed generation units (mainly wind and solar farms and co-generation plants), the expected growth of storage systems and the future introduction of the required infrastructure to recharge electrical vehicles (see Figure 1). These new players are bringing new possibilities and more flexibility in the way energy has been traditionally managed. Nevertheless, the resulting system requires the introduction of new advanced technologies to cope with the increased complexity. In the most recent decades, the world has experienced a very significant increase in energy consumption and a generalized concern about future energy problems and sustainability has arisen. This situation has led governments and the scientific community to look for solutions that allow an efficient, reliable and responsible use of energy, appealing to an optimized and more flexible conception of the electrical grid. This new paradigm is known as the smart grid. Despite the wide spectrum of technologies encompassed which makes it unfeasible to provide a simple and unique definition, it is quite common to consider the smart grid as the framework that integrates all the advanced control and information technologies to monitor and manage power generation and distribution. There are many common aspects between the conception of the smart grid and those principles that enabled the Internet, as it is known nowadays [1]. Indeed, it is quite usual to conceive smart grid Authors María-José Santofimia-Romero received the degree of Technical Engineer in Computer Science in 2001 from the Universidad de Córdoba, Spain; Master s degree on Computer Security from the University of Glamorgan, Wales UK, in 2003; and the degree of Engineer in Computer Science in 2006 from the Universidad de Castilla-La Mancha, Spain. She is currently working towards her PhD as a member of the Computer Architecture and Networks Research Group (ARCO) at the Universidad de Castilla-La Mancha. <mariajose.santofimia@uclm.es> Xavier del Toro-García received the degrees of Technical Engineer on Industrial Electronics and Engineer in Automatic Control and Industrial Electronics in 1999 and 2002, respectively, from the Universitat Politècnica de Catalunya, Spain. He received a PhD degree from the University of Glamorgan, UK, in 2008. From September 2005 until October 2006 he was a Marie Curie Research Fellow at Politecnico di Bari (Italy). Since 2008 he has been working as a researcher at the Universidad de Castilla-La Mancha, Spain. His research interests include power electronics, renewable energy sources, energy storage systems and power quality. <xavier.deltoro@uclm.es> Juan-Carlos López-López received the MS and PhD degrees in Telecommunication (Electrical) Engineering from the Universidad Politécnica de Madrid, Spain, in 1985 and 1989, respectively. From September 1990 to August 1992, he was a Visiting Scientist in the Dept. t of Electrical and Computer Engineering at Carnegie Mellon University, Pittsburgh, PA USA. His research activities center on embedded system design, distributed computing and advanced communication services. From 1989 to 1999, he has been an Associate Professor of the Dept. of Electrical Engineering at the Universidad Politécnica de Madrid. Currently, Dr. López is a Professor of Computer Architecture at the Universidad de Castilla-La Mancha, Spain, where he served as Dean of the School of Computer Science from 2000 to 2008. He has been member of different panels of the Spanish National Science Foundation and the Spanish Ministry of Education and Science, regarding Information Technologies research programs. He is member of the IEEE and the ACM. <juancarlos.lopez@uclm.es> Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 41

Figure 1: Electrical Grid Structure and Evolution. technologies as traditional grids enhanced with information and communication technologies providing an efficient, safe and reliable use of electricity. For a more in depth view of the smart grid, some of the concrete goals that smart grid technologies aim to achieve are listed below: To provide prompt response to changing conditions in the electricity network, To foresee electricity network behavior (consumption peak demands, faults, etc.), To improve the power quality that finally gets to the clients, To provide security guarantees (privacy, prevention from attacks or deliberate disruptions, etc.), To provide fault-tolerance and self-healing capabilities, This work is devoted to summarize the most relevant challenges addressed by the smart grid technologies and how intelligence systems can contribute to their achievement To integrate different and distributed renewable energy sources. Some of these challenges mainly result from the need to mitigate the impact that grid failures and power quality disturbances have on industrial and domestic customers. Furthermore, besides the economical concerns, reducing carbon emissions as a step towards sustainable development seems to be of major interest for the driving forces of the smart grid development. In this sense, the use of renewable energies is gaining relevance as technological improvements are being achieved. Nevertheless, there are still several major drawbacks which are preventing them from being massively deployed. Renewable energies, such as solar power and wind power, are not distributable power sources that can quickly respond to the grid operator s energy demands due to their intermittent nature. Moreover, the amount of energy that can be generated cannot even be predicted and scheduled even though considerable improvements have been achieved in short-term production forecasts. These important drawbacks can only be overcome by integrating alternatives in the generation mix, interconnecting large grids, an improved scheduling of the generation and consumption times and the introduction of energy storage systems. Another important fact to be considered is that future smart grids will also need to allocate power to electrical vehicles to allow sustainability in terms of mobility. The electrical infrastructure required for a large deployment of electrical vehicles will have a big impact on the electrical infrastructure and the consumption profile. Nevertheless, it will bring new opportunities since a bidirectional energy 42 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

flow in the vehicle battery is possible, the so-called Vehicle-to-Grid (V2G) concept, and electrical vehicles can therefore be seen as a distributed storage infrastructure that can contribute to the stability of the grid. Considering the complex scenario previously described the smart grid challenges can be summarized into three main groups [2], namely: a) Technological challenges, b) Economic challenges, c) Regulatory challenges. Technological challenges basically address the attainment of distributed communication strategies with optimized latency and bandwidth, advanced control systems, reliable fault tolerance management techniques, efficient massive data processing methods and new energy storage devices. Regarding the economic challenges, new business models arise from a new way of conceiving the upcoming energy market. For example, active demand response strategies help in reducing peaks of consumption in the power system by temporarily changing and shifting the consumption patterns followed by users, either by increasing or decreasing their consumption at certain times of the day. Finally, the regulatory challenges are related to the establishment of standards that, at different levels, specify the basis for interoperability required to make smart grids feasible. Despite the highly diverse nature of the aforementioned challenges, they share a common set of features that need to be considered as the starting point to propose solutions based on computational systems. Such systems should be capable of dealing with evolving, uncertain, variable and complex scenarios. In order to do so, as stated in [3], computational systems need the capability to understand the ongoing situations, make decisions, and re-evaluate the situations to determine whether further actions are required. This article presents a survey of the wide spectrum of computational intelligence systems that are contributing to address the existing challenges of future smart grids. Section 2 reviews the different smart grid technology areas distinguishing between those that can be considered mature enough to be deployed nowadays and those in an early stage of development. Sections 3 and 4 analyze the role that the different computational strategies, agglutinated under the perspective of Artificial Intelligence, are playing in developing smart grid systems. Finally, Section 5 summarizes the most relevant ideas presented in this article. 2 Smart Grid Technologies In order to evolve towards a smarter grid, research and development efforts must be concentrated in the following key technology areas [4]: Wide-area monitoring and control. Wide-area monitoring and control systems (WAMCS) assume the responsibility for preventing and mitigating the possible disturbances that might strike the grid. In order to do so, WAMC systems perform advanced operations intended to identify the presence of instabilities, aid the integration of renewable sources, or improve and increase the transmission capabilities. WAMC systems consist in the centralized processing of data that have been collected from distributed sources in order to evaluate the state of the grid. The main functions of a WAMC system are carried out in three different stages, namely: data acquisition, data delivery, and data processing [5]. Information and communication technology integration. An essential aspect of the smart grid is the need for real-time information exchange that has to be based on some sort of communication infrastructure able to support the integration of distributed and heterogeneous devices. Integration of distributed renewable generation systems. Distributed generation consists in the integration of many small power generation sources at the distribution level. When those sources of generation come from renewable sources, additional challenges arise due to the unpredictability and controllability issues associated with the resource availability. Nowadays, distributed generation based on renewable energies is quickly gaining presence in some countries, in which stability and power allocation problems are arising. In a long term, research efforts start to be noticeable in the field of energy storage systems, as a mean to alleviate the drawbacks associated to the existing decoupling between energy generation and consumption. Transmission enhancement applications. The electricity transmission stage is responsible for migrating the bulk power obtained from the generation plants to the distribution substations. Current research efforts are being addressed to increase the transmission capacity by resorting to different applications such as Flexible AC transmission systems (FACTS), high-voltage DC (HVDC), hightemperature superconductors (HTS), or dynamic line rating (DLR). Distribution grid management. The idea behind the technological solutions intended to support the distribution grid management is that of providing communication improvements throughout the different components of the distribution system. The main applications of the technologies provided for distribution grid management are intended to perform tasks such as load balancing, optimization, fault detection, recovery, etc. Advanced metering infrastructure. Those technologies encompassed under the umbrella of advanced metering systems are intended to provide enhanced functionalities other In the most recent decades, the world has experienced a very significant increase in energy consumption and a generalized concern about future energy problems and sustainability has arisen Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 43

It is quite usual to conceive smart grid technologies as traditional grids enhanced with information and communication technologies providing an efficient, safe and reliable use of electricity than just metering or counting. Probably, the most well-known application of the advanced metering infrastructure is that of the dynamic pricing, intended to help consumers to reduce their bills by adjusting their consumption to those periods outside the peak demand times. In order to do that, an advanced metering system is composed of three components: the smart meter, the communication utility, and the meter data management application [6]. It should be noted that the distributed character of the components comprising the advanced metering infrastructure brings to light a growing concern for it is the need for standards that helps to overcome interoperability problems. Electric vehicle charging infrastructure. One of the main tenets of the smart grid is that of using a more sustainable conception of the power generation and consumption process. In order to do so, it is a priority to minimize the emission of green-house gases and electric vehicles provide a solution in response to this concern. Moreover, the role electric vehicles can play in the smart grid is also relevant with regard to its capability to work as a storage unit. Customer-side systems. Customer-side systems encompass those applications that are intended to make more efficient use of electricity as well as a cost reduction for the customer. Energy management systems, storage devices, smart appliances and small generation systems (i.e. solar panels) fall into this category. This section has summarized the most relevant aspects of the different technologies that are encompassed under the umbrella of the smart grid. Each of these technologies, simultaneously, is articulated by a wide set of applications, some of which have been succinctly mentioned. The next section focuses on those specific applications, and how computational intelligence can contribute to narrowing the existing gap between contemporary grids and the envisaged smart grid. 3 Artificial Intelligence and the Smart Grid In light of the most common activities undertaken by smart grid systems, it can be stated that there is an association between the smarter grid conception and the Artificial Intelligence paradigm. Indeed, rather than appealing to the Artificial Intelligence paradigm in all its aspects, the focus may be directed upon distributed intelligence efforts [7]. The main challenge to be tackled in the smart grid comes from the vast amount of information involved in it. In contrast to traditional grids, in which the consumption metering information was only retrieved monthly, smart grids present a new scenario in which all the interconnected nodes are gathering information about many different matters, and not only consumption (i.e. real-time prices, peak loads, network status, power quality issues, etc.) [8]. In this sense, one of the main challenges for computational intelligence is how to intelligently manage such an amount of information so that conclusions and inferences can be drawn to support the decision making process. This challenge is being mainly addressed from the perspective of Complex Event Processing (CEP) techniques. CEP techniques address event filtering to seek for relevant patterns. However, the selected events need to be semantically enriched in some way so that they can lead to an understanding of the ongoing situation. CEP systems need to be complemented with more sophisticated techniques that support the understanding process. In this sense, one of the possible approaches is the use of Qualitative Reasoning. In [9] an example of this approach employs a qualitative behavioral model of the electric grid, in which some power quality issues such as voltage dips and reactive power are modeled in order to anticipate their possible evolution and negative effects. Problems related to power quality provide an interesting field of application for monitoring and diagnosis tasks. The cited work [9] tries to bridge the gap that leads to self-sufficient systems, capable of anticipating and reacting to power faults from simple data gathering. In order to so, the proposed approach provides a characterization of the power quality problem, presenting a qualitative behavioral model of the grid dynamics. This model supports a multi-agent system in charge of anticipating and reacting to power faults and disturbances. In the same way, more complex reasoning systems can also enhance smart grid technologies. Those reasoning systems based on large-scale knowledge base of common sense, such as Cyc [10], Scone [11], or ConceptNet [12], can provide advanced capabilities to assist and enrich Supervision, Control and Data Acquisition (SCADA) systems. Achievements in the context-awareness field can be easily extrapolated to fulfill certain valuable functionalities of the smart grid. For example, Dynamic Line Rating technology is in charge of maximizing the distribution lines by dynamically reconfiguring the line capacity of the different grid sections, taking into account external conditions such as the weather. In this sense, further knowledge about the surrounding context can lead to a more efficient and sophisticated use of the distribution line. Additionally, Active Demand Management technology addresses the problem of providing the right amount of electrical power at the right location and time. This task involves some sort of load balancing, in which the Artificial Intelligence planning and knowledge-based techniques can contribute with important improvements [13]. Nevertheless, information management is not the only 44 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

The use of renewable energies is gaining relevance as technological improvements are being achieved but there are still several major drawbacks which are preventing them from being massively deployed requirement for smart grids that benefits from the strengths of artificial intelligence techniques. On the contrary, several smart grid technologies resort to a wide variety of intelligent solutions to tackle uncertainty and unpredictability. For example, the Active Network Management technology can resort to intelligent agents so as to address the automation of activities such as voltage and frequency control, reactive power control, fault detection and fault ride-through, or self-healing. Provided that this technology requires a communication infrastructure to support the SCADA system, achievements in intelligent distributed systems can be of a great help. In this regard, the distributed systems theory can contribute with algorithms, communication approaches, or consistency and replication techniques, among other possibilities. In particular, Multi-Agent Systems (MAS) can be considered as a sort of intelligent distributed systems that can find a very suitable field of application in the smart grid due to the distributed and heterogeneous nature of the problem [14]. As it is illustrated and discussed in [15] and [16], MAS have been successfully used in a wide range of power engineering applications. 4 Computational Intelligence and the Smart Grid The previous section has surveyed the most relevant approaches of Artificial Intelligence that have been applied to the smart grid in order to tackle the existing challenges. This section continues this survey paying special attention to those techniques that have been specifically devised to address dynamic and stochastic problems. There are two main approaches when it comes to dealing with scenarios that are unpredictable or uncertain, the artificial intelligence and computational intelligence techniques (see Figure 2). Despite the fact that at first glance, both approaches might seem to be equivalent, artificial intelligence and computational intelligence techniques differ in the way they approach complex problems. Artificial intelligence techniques adopt some sort of goal-oriented approach, in the sense that problems tend to be grounded in thorough descriptions of the world and associations can be established between the problem to be solved and the actions that can help in achieving the desired state. However, this approach for complex problems is not very suitable for situations in which stochastic processes interfere. The openness of these scenarios is better addressed by means of computational intelligence techniques, comprising well-known approaches such as evolutionary computation, fuzzy logic, or artificial neural networks. The common aspect of these approaches is that solutions, rather than being provided as a result of previous knowledge stating actions attached to goals, they resort to stochastic processes consisting in iterative cycles of generation and evaluation. Traditional artificial intelligence techniques experience difficulties when several goals are in conflict with each other. In contrast, computational intelligence techniques perform well under such circumstances. The role that computational intelligence can play in the smart grid environment is therefore grounded in its capability to enable intelligent behaviors under uncertainty conditions [17]. This section is therefore devoted to reviewing the most successful techniques as well as the potential challenges that those techniques will be able to address. The main contributions of computational intelligence techniques to the field of smart grids are identified in [18]. The most interesting feature of these techniques is their ability to anticipate relevant information that assists the decision making process. Additionally, these methods provide the means to control the grid in a reliable and rapid manner. Artificial Neural Networks (ANNs) consist in the replication of the operation of biological neural systems [19]. In this sense, a neural network comprises a collection of interconnected nodes that are nothing else but processing units that have associated two values, an input and a weight [20]. The most characteristic feature of the neural networks is that rather than being programmed to perform certain tasks, they can be trained to identify certain data patterns. During the training phase the neural network is provided with input data and targets to learn the recognition of certain patterns. Nonetheless, their main advantage also turns out to be their main drawback since these methods require from a large amount of input data that needs to be representative enough for generalizing behavioral patterns [21]. The work in [20] surveys some of the most relevant applications of ANN techniques to the field of energy systems. These applications range from a wide variety of purposes such as, modeling solar energy heat-up response [22], prediction of the global solar irradiance [23], adaptive critic design [24], or even for security issues as reviewed in [25]. The idea behind these applications is based on learning how system performances can be related to certain input values, for instance, how weather conditions (solar or wind) determine the energy output that can be expected [26]. Voltage stability monitoring [27] applications are among the most successful contributions of ANNs to the smart grid technologies. In this sense, the work in [28] presents an innovative method to estimate the voltage stability load index using synchrophasor measurements of voltage Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 45

Figure 2: Artificial Intelligence and Computational Intelligence Techniques and their Contribution to the Advanced Features of the Smart Grid. The regulatory challenges are related to the establishment of standards that, at different levels, specify the basis for interoperability required to make smart grids feasible Several smart grid technologies resort to a wide variety of intelligent solutions to tackle uncertainty and unpredictability; for example, the Active Network Management magnitudes and angles. Wide area monitoring applications do also benefit from the potential of the ANN-based methods. The work in [29] describes the implementation of a system intended to identify the dynamics of the non-linear power system. Neural networks have demonstrated their capability to detect in real time the changing dynamics of power systems. Whenever such changes are identified, an additional ANN can be employed to generate the appropriate control signals that minimize those negative effects [30]. Evolutionary algorithms (EA) [31], and more specifically genetic algorithms, have gained great relevance due to their capability to successfully address optimization problems with a relatively low demand of computational resources. Genetic algorithms are inspired in the evolutionary principle of natural selection [32] and they consists in the encoding of a set of plausible solutions, as though they were the initial population, out of which the fittest members are favored to create the next generation of solutions. This methodology is intended to remove, in recursive iterations, those solutions that are considered poor. Additionally, the work in [32] provides a list of the different applications of genetic algorithms in the field of the energy systems. A specific example of an application can be found in [33], which resorts to a genetic algorithm method with a twofold aim: firstly, a genetic algorithm is used to generate a feasible solution, constrained to the desired load convergence and secondly, a genetic algorithm is used to optimize the obtained solution. The work in [34] also resorts to a genetic algorithm approach for efficiency enhancement, demonstrating that those fuzzy controllers implemented by means of genetic algorithms obtained optimum results (both for entire and discrete time intervals). Finally, the last approach considered in this review of computational intelligence methods is Fuzzy Logic (FL) [35]. There are many processes in the smart grid that in- 46 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

The role that computational intelligence can play in the smart grid environment is grounded in its capability to enable intelligent behaviors under uncertainty conditions volve decision-making tasks, for example, deciding how to allocate renewable energy production, or when to consume on the light of the energy price market evolution. The fact that FL methods attempt to provide solutions to control problems based on approximations, resorting to a system representation that rather than using conventional analytical and numerical calculations, uses simple labels to quantify inputs and simple rules based on "IF-THEN" statements. FLs have demonstrated outstanding performance for decisionmaking processes with estimated values under uncertainty conditions [20]. Additionally, FL methods have also been implemented in a long list of energy system applications, such as supervision and planning tasks in the presence of renewable energy sources as the work presented in [36, 37]. 5 Conclusions The growing concerns about the environmental impact that energy corruption is having on the planet are leading to a new conception of power systems, in which renewable energies are increasingly integrated into the production cycle and also in which energy efficiency and security should be maximized. This work has paid special attention to how artificial and computational intelligence can contribute to the achievement of smarter grids. To this end, this work has reviewed the main smart grid technologies and the current and potential impact that intelligent approaches have upon them. It can be concluded that knowledge-engineering approaches can successfully address the problem of managing the vast amount of information that will be generated in future smart grids. Additionally, distributed intelligence techniques can provide the means to monitor and manage all the different stages involved in the whole process. Multiagent system approaches have demonstrated their capability to cooperate and articulate responses to the distributed data coming from the different sources of information. Finally, the role of computational intelligence or soft computing techniques, can greatly contribute to the optimization and control process involved in the smart grid. Acknowledgements This work has been partially funded by the Spanish Center for Technological and Industrial Development (CDTI) - Ministry of Science and Innovation, under the project "ENERGOS: Technologies for the automated and intelligent management of the future power grid" (CEN-20091048, CENIT program). References [1] U.S. Department of Energy. "The smart grid: An introduction". [Online], 2008. Available at <http:// www.oe.energy.gov/smartgridintroduction.htm>. [2] M. Hashmi. "Survey of smart grid concepts worldwide". VTT, Espoo. VTT Working Papers, 2011. [3] G.K. Venayagamoorthy. "Dynamic, Stochastic, Computational, and Scalable Technologies for Smart Grids". Computational Intelligence Magazine, IEEE, vol.6, no. 3, pp. 22-35, Aug. 2011. [4] OECD. "Technology Roadmap: Smart Grids". IEA Technology Roadmaps, OECD Publishing, 2011. [5] M. Zima, M. Larsson, P. Korba, C. Rehtanz, G. Andersson. "Design Aspects for Wide-Area Monitoring and Control Systems". Proceedings of the IEEE In Proceedings of the IEEE, Vol. 93, No. 5. (May 2005), pp. 980-996. [6] R. Hornby, C. Salamone, S. Perry, D. White, K. Takahashi. "Advanced Metering Infrastructure Implications for Residential Customers in New Jersey". Ex. UWUA-4, July 8, 2008. [7] Collier, Steven. "Ten Steps to a Smarter Grid". IEEE Industry Applications Magazine 16.2 (2010) : pp. 62-68. [8] Y. Simmhan, S. Aman, B. Cao, M. Giakkoupis, A. Kumbhare, Q. Zhou, D. Paul, C. Fern, A. Sharma, V. Prasanna. "An informatics approach to demand response optimization in smart grids". Technical report, Computer Science Dept., USC, 2011. [9] M.J. Santofimia, X. del Toro, P. Roncero-Sánchez, F. Moya, M.A. Martínez, J.C. López. "A qualitative agent-based approach to power quality monitoring and diagnosis". Integrated Computer-Aided Engineering, 17:305-319, December 2010. [10] D. Lenat, M. Prakash, M. Shepherd. "CYC: Using common sense knowledge to overcome brittleness and knowledge acquisition bottlenecks". Artificial Intelligence Magazine, 6:65 85, January 1986. [11] S. E. Fahlman. The Scone knowledge-base project, 2010. [Online]: Available at <http://www.cs.cmu.edu/ sef/scone/>. [Retrieved on February 28th, 2010]. [12] H. Liu, P. Singh. "Conceptnet: A practical commonsense reasoning toolkit". BT Technology Journal, 22:211 226, 2004. [13] Y. Oualmakran, J. Meléndez, S. Herraiz, M. López- Perea, E. González. "Survey on knowledge based methods to assist fault restoration in power distribution networks". International Conference on Renewable Energies and Power Quality (ICREPQ 11). Las Palmas (Spain) 2011. [14] G. Weiss. "Multiagent systems: a modern approach to distributed artificial intelligence". MIT Press, Cambridge, MA, USA, 1999. [15] S.D.J. McArthur, E.M. Davidson, V.M. Catterson, A.L. Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 47

Dimeas, N.D. Hatziargyriou, F. Ponci, T. Funabashi. "Multi-Agent Systems for Power Engineering Applications Part I: Concepts, Approaches and Technical Challenges". IEEE Transactions on Power Systems, Vol. 22, no. 4, pp. 1743-1752, Nov. 2007. [16] S.D.J. McArthur, E. M. Davidson, V.M. Catterson, A.L. Dimeas, N.D. Hatziargyriou, F. Ponci, T. Funabashi. "Multi-Agent Systems for Power Engineering Applications Part II: Technologies, Standards, and Tools for Building Multi-agent Systems". IEEE Transactions on Power Systems, Vol. 22, no. 4, pp. 1753-1759, Nov. 2007. [17] G.K. Venayagamoorthy. "Potentials and Promises of Computational Intelligence for Smart Grids", IEEE Power General Society General Meeting, Calgary, CA, July 26-30, 2009. [18] R. Harley, J. Liang. "Computational Intelligence in Smart Grids. "IEEE Symposium Series on Computational Intelligence (SSCI) 2011 - CIASG, April 11-15, 2011. [19] S. Haykin. "Neural Networks: A Comprehensive Foundation". Macmillan, New York, 1994. [20] S. Kalogirou, K. Metaxiotis, and A. Mellit. "Artificial Intelligence Techniques for Modern Energy Applications". Intelligent Information Systems and Knowledge Management for Energy: Applications for Decision Support, Usage, and Environmental Protection. IGI Global, 2010, pp. 1-39. [21] G. Zini, S. Pedrazzi, P. Tartarini, K. Gopalakrishnan, S. Khaitan, S. Kalogirou. "Use of Soft Computing Techniques in Renewable Energy Hydrogen Hybrid Systems". Soft Computing in Green and Renewable Energy Systems, Studies in Fuzziness and Soft Computing, Springer Berlin, pp. 37-64, vol. 269, 2011. [22] S. Kalogirou, C. Neocleous, and C. Schizas. "Artificial neural networks for modelling the starting up of a solar steam generator". Applied Energy, 60, pp. 89 100, 1998. [23] P.L. Zervas, H. Sarimvies, J.A. Palyvos, N.G.C. Markatos. "Model-based optimal control of a hybrid power generation system consisting of photovoltaic arrays and fuel cells", Journal of Power Source 181, pp. 327 338, 2008. [24] P.J. Werbos. "Approximate Dynamic Programming for Real Time Control and Neural Modelling". In White DA and Sofge DA (Eds.), Handbook of Intelligent Control, Van Nostrand Reinhold, New York, 1992, ISBN 0-442-30857-4, pp. 493-525. [25] Y. Mansour, E. Vaahedi, M.A. El-Sharkawi. "Dynamic security contingency screening and ranking using neural networks". IEEE Trans. Power Syst., vol. 8, no. 4, pp. 942 950, July 1997. [26] D. Riley, G.K. Venayagamoorthy. "Characterization and modeling of a grid connected photovoltaic system using a recurrent neural network". In Proc. IEEE Int. Joint Conf. Neural Networks, San Jose, CA, July 31 Aug. 5, 2011. [27] G.K. Venayagamoorthy. "Dynamic, Stochastic, Computational, and Scalable Technologies for Smart Grids". Computational Intelligence Magazine, IEEE, vol. 6, no. 3, pp. 22-35, Aug. 2011. [28] K.J. Makasa, G.K. Venayagamoorthy. "On-line voltage stability load index estimation based on PMU measurements". In Proc. IEEE Power and Energy Society, Detroit, MI, July, pp. 24-28, 2011. [29] W. Qiao, G.K. Venayagamoorthy, R.G. Harley. "Optimal Wide-Area Monitoring and Non-Linear Adaptive Coordinating Control of a Power System with Wind Farm Integration and Multiple FACTS Devices". Neural Networks, Vol. 21, Issues 2-3, March/April 2008, pp. 466-475. [30] Z. Vale, G.K. Venayagamoorthy, J. Ferreira and H. Morais. "Computational Intelligence Applications for Future Power Systems". Intelligent Systems, Control And Automation: Science and Engineering, Springer Netherlands, pp. 176-193, vol. 46, 2011. [31] T. Bäck. "Evolutionary algorithms in theory and practice". Oxford University Press, Oxford, 1996. [32] A. Pacheco-Vega. "Soft Computing Applications in Thermal Energy Systems". Soft Computing in Green and Renewable Energy Systems (Studies in Fuzziness and Soft Computing), pp. 1-35, Vol. 269, Springer- Verlag, Berlin, Germany, 2011. [33] M.S. Osman, M.A. Abo-Sinna, A.A. Mousa. "A solution to the optimal power flow using genetic algorithm". Applied Mathematics and Computation 155(2), pp. 391-405, 2004. [34] G. Jahedi, M.M. Ardehali. "Genetic algorithm-based fuzzy-pid control methodologies for enhancement of energy efficiency of a dynamic energy system". Energy Convers. and Manage. 52(1), pp. 725 732, 2011. [35] L.A. Zadeh. "Fuzzy logic, neural networks and soft computing". Communications of the ACM 37 (1994) 77-84. [36] V. Courtecuisse, J. Sprooten, B. Robyns, M. Petit, B. Francois, J. Deuse. "A methodology to design a fuzzy logic based supervision of hybrid renewable energy systems". Mathematics and Computers in Simulation, 81(2), 208 224 (2008). [37] Y.F. Li, Y.P. Li, G.H. Huang, X. Chen. "Energy and environmental systems planning under uncertainty-an inexact fuzzy-stochastic programming approach". Applied Energy, 87(10), 3189 3211 (2010). 48 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

Green Computing: Saving Energy by Throttling, Simplicity and Parallelization Lasse Natvig and Alexandru C. Iordan The aim of this article is to give an overview of several techniques that are used in making computers more energy efficient. All segments of the ICT market are facing this challenge, starting with the smallest embedded systems, through mobile devices and laptops, to workstations and supercomputers. We start by giving a broad overview of energy saving techniques in hardware, continue at the operating system level, system software and finally techniques that can be used at the application level. Techniques for low-power electronics such as different levels of sleep modes, architectural techniques like multi-cores and heterogeneous processing, the role of customization and accelerators as well as reconfigurable hardware are outlined. We describe how parallelization in both HW and SW can help reduce the energy consumption, and discuss several examples of how throttling (slowing down) and simplicity give the same effect. The authors have developed an experimental framework for exploring these issues, and the paper ends by presenting a few recent results showing how parallelism expressed with Task Based Programming, TBP, can save energy. Keywords: Energy Awareness, Energy Efficiency, Green Computing, Multi-Core, Power Management. 1 Introduction The need for saving energy has become a top priority in almost all segments of the ICT market. In sensor network, tiny computers report periodically measurements over a time frame of 10 years or more. In such situations it is impossible or too costly to replace batteries. Most users of mobile phones and laptops have the experience of running out of battery too fast. Ordinary desktop computers could produce less heat, less noise from cooling fans and be cheaper to operate if they were to consume less energy. In High Performance Computing, HPC, the need for power efficiency has become even more important and is now a critical design factor and operational requirement. In almost all cases, reducing the energy consumption leads to a lower performance a classical and challenging situation of conflicting goals. This paper gives an introduction to green computing, focusing on what can be done in hardware and systems software to save energy when doing computations on computers of all kinds and sizes. This is a relatively narrow focus compared to other papers that include the environmental impact of the whole life cycle of computers and related equipment, such as the manufacturing process and disposal and recycling. These very important aspects, definitely within Green IT or Green ICT, are covered in other papers of this CEPIS UPGRADE Oct. 2011 Special Issue on Green ICT. As indicated in the title, we will explain three general technological principles that have the potential to save energy in computations. Throttling, i.e. slowing down a processor or core to a lower frequency is a widespread technique today, but must be balanced against the disadvantage of lost performance. Simplicity is perhaps our most important design principle and it has self-evident effects on en- Authors Lasse Natvig received the MS and Dr. Ing. degrees in Computer Science from the Norwegian Institute of Technology, NTNU, in 1982 and 1991, respectively. He is currently a Professor in Computer Architecture at the Norwegian University of Science and Technology (NTNU), Trondheim, Norway. His main research interests are computer architecture, parallel processing, multi-core systems, system-level computer simulation, memory architectures and green computing. Dr. Natvig is a full member of HiPEAC2 Webpage: <http://www.idi.ntnu.no/people/lasse>. <Lasse@computer.org> Alexandru C. Iordan received his BSc and MS degrees from the Politehnica Universitatea of Bucharest, Romania. He is currently a PhD candidate at the Norwegian University of Science and Technology, focusing on energy efficient methods for multi-core programming. <iordan@idi.ntnu.no> ergy consumption. However the need for product portability and longevity has pushed towards increasingly complex ICT systems with lots of abstraction layers. Green computing gives increased focus to simplicity and a need to re-evaluate the design process that leads to all this complexity. Parallelism, i.e. distribution of computation on multiple processors or cores, might not look at first sight like a power saving technique since it generally requires some overhead and most often increased complexity. However, as made evident by the rapid and dominating technological shift to multi-core processors, it has become the main design factor in microprocessor development to save energy. Green computing is an interdisciplinary, large and rapidly increasing area. In this overview the aim is to introduce the main concepts within the focus of the paper, as well as provide motivation for further studies within the area. Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 49

Figure 1: Selected Aspects of Green Computing. 2 Green Computing A Subset of Green ICT We will focus on a subset of green ICT, namely those techniques that are used to reduce the energy needed to execute a computer program. The techniques we discuss are outlined in Figure 1. There is also an abundance of software applications that are green in the sense that they help the community to save energy. Well-known examples are intelligent control of combustion engines, routing of transportation to minimize travel distance and teleconferencing. Such applications are not covered in this overview. In Section 2.1 and 2.2, we present a high level view of the main hardware techniques. Our main emphasis is at the architectural level since it is of paramount importance to have a holistic view of the interplay of both HW and SW when designing an energy efficient system. Section 2.3 discusses techniques such as processor throttling and load balancing that are controlled at the operating system level. These are normally invisible to the application programmer. Section 2.4 introduces energy aware algorithms and data-structures. Implicitly, good traditional design principles such as aiming at simplicity have helped produce such algorithms. However, the recent increased interest in green computing Clock frequency (speed): f Supply voltage: V Dynamic power consumption: P dynamic Static power consumption: P static Physical capacitance: C Activity factor: a Execution time: T Performance: Perf = 1/T Energy Delay Product ([20]): EDP (Eq. 1) P dynamic acv 2 f (Eq. 2) Power = P dynamic + P static (Eq. 3) Energy = Power T (Eq. 4) EDP = Perf 2 /Power Figure 2: Microprocessor Speed, Power, and Energy Simplified Definitions and Metrics. has stimulated new research in the area. Also, research into data-structures has gained renewed interest in the multicore era. This can be explained by the need to make the data-structures more efficient for sharing in multithreaded programs. Also, the large on-chip shared caches of a modern multi-core processor offer new possibilities to implement such data sharing in a more energy efficient way. 2.1 Low Power Design and HW Energy Saving Techniques There is a large interest in improving the energy efficiency of a chip s logical gates. These efforts include basic research to find new materials that can be used to do computations and research in electronics as how to operate transistors at lower voltage levels, as in near-threshold and subthreshold designs. This technology will reduce the possible speed, i.e. increased propagation delay, and therefore give a reduced clock frequency. It is predicted that near threshold computing can reduce energy requirements by 10 to 100 times or even more in future systems [1]. Within digital design we have a rich set of circuit and logic level techniques extensively surveyed in the paper Power Reduction Techniques For Multiprocessor Systems by Venkatachalam and Franz [2]. To mention just a few: making transistor smaller, reordering transistors in a circuit, half-frequency clocks synchronizing events using both edges of the clock, logic gate restructuring, technology mapping where the components are selected from a library to meet energy constraints, and the use of low-power flipflops. Placed slightly higher in the abstraction levels is low power control logic design. An example can be to implement a finite state machine (FSM) so that the switching activity in the processor is minimized, or to decompose the FSM into smaller sub-fsms that can be deactivated when not in use. The need for saving energy has become a top priority in almost all segments of the ICTmarket 50 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

The aim of this article is to give an overview of several techniques that are used in making computers more energy efficient 2.1.1 Dynamic Voltage and Frequency Scaling, DVFS Another important technique is dynamic voltage scaling and frequency scaling, DVFS. We list it here since it requires special hardware features, but it is normally controlled by software. Dynamic Voltage Scaling has also been called undervolting, and is used to lower the supply voltage of a processor when the workload is getting so small that the processor can reduce its speed (frequency) and still have a performance that is sufficiently high to meet the systems requirements. Reducing the frequency also makes it possible to reduce the supply voltage (V) since the gates can use longer time to switch. Both result in lower dynamic power consumption (see Figure 2). DVFS is used in most modern laptop processors where it is controlled by the operating system to save energy under light load. Much of the research on DVFS has focused on singlecore processors, and similar techniques have been used to other components such as main memory, RAM, and hard disks [3]. Today, this kind of power management is also available in multi-core processors, also called chip multiprocessors, CMP. A simple example of such a processor is illustrated in Figure 3. Using such a CMP, where each core is operating at f/4, the same performance as a single-core processor running at frequency f can be achieved. Also, since the supply voltage V can be reduced when the frequency f is reduced, eq. 1 in Figure 2 tells us that we can get a cubic reduction in power used per core. In general, parallelizing a computation by executing it on many slow cores instead of one fast core makes it possible to do the same amount of work equally fast or faster and still use less energy. This is one of the main reasons for the rapid paradigm shift in the microprocessor market from single- to multi-cores. The underlying assumption is of course that the overhead incurred by the parallelization is not too large. When using DVFS in a multi-core context the most straight-forward approach is to use one "knob" to control the whole chip, i.e. to reduce the frequency of all the cores at the same rate. However, improved energy efficiency can be achieved if the individual cores can be controlled separately or even turned off [4]. In mainstream Intel multicores, a technique called Turbo Boost Technology, TBT, is used to allow adjustments of core frequency at runtime [5]. Considering the number of active cores, estimated power requirements and CPU temperature, TBT can determine a maximum frequency for a core. The frequency can be incremented in steps of 133 MHz to give a boost in performance while still maintaining the power envelope. To save energy, it is possible to power down cores when they are idle. Other multi-core vendors offer similar techniques. In the Intel "Single-chip Cloud computer", 48 cores are integrated on a single chip and fine-grained power management is made possible by the 8 voltage islands and 28 frequency islands defined on the chip. The supply voltage can be scaled from 1.3 to 0V in 6.25mV steps. Voltage islands can be set to 0.7V for idle cores, a value that is a safe voltage for state retention, or completely collapsed to 0V, if retention is unnecessary [6]. State retention is closely linked to the concept of sleep modes covered in the next paragraph. 2.1.2 Sleep Modes There are many real time applications where a task has to be completed within a given time frame. Often a performance guarantee for that deadline is of importance, but being faster has little or no value. In such cases, it is tempting to use DVFS or similar techniques to slow down the computation and save energy. However, if much of the system can be put into sleep mode after the execution has fin- Figure 3: Quad-core Processor Example. PU = Processing Unit. L1 and L2 caches are explained in Section 2.2. Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 51

Figure 4: Alternative Designs, Simple Example. ished, it might turn out to be more energy efficient to combine a fast and power-hungry period of execution with the use of sleep modes. This is illustrated in Figure 4 where case A (red) is one design doing a given computational task in 2 ms using 3 watt while case B (green) is the same task slowed down to 6 ms but using 1 watt. We assume that the task is done repeatedly, and must be completed within 6 ms when it occurs. In the example above, the two alternatives would give the same energy consumption if we can assume that the unit can be completely turned off while not executing. However, if the unit has to retain some state during the 4 ms of time saved, the total energy consumption will be higher. But, on the other side, if some peripheral units can be turned off when the unit is not executing, the fastest alternative will be most energy efficient. This latter effect often motivates embedded system designers to aim at "unnecessarily fast" execution to save energy. DVFS and other slow down techniques (see also Section 2.2) must therefore be balanced against the use of sleep-modes. The situation is in practice much more complex. First of all, the energy consumed during execution will normally vary and not be constant as we assumed in Figure 4. Further, a computer will have different components that can be put into different levels of sleep. As an example, the EFM32 energy friendly microcontrollers from Energy Micro use 5 energy modes: run, sleep, deep sleep, stop and shutoff mode [7]. More details about different kinds of sleep modes can be found as a part of the large ACPI Specification [8]. ACPI is short for Advanced Configuration and Power Interface and describes a common industrial interface enabling operating system directed configuration and power management. Sleep modes are also known as standby modes or suspend modes. 2.2 Parallelism and Other Architectural Techniques The architecture sub-tree of Figure 1 refers to a vast field of research. It contains many different subfields each of which is a large research area. Some examples are CPU design (also called micro-architecture), memory systems (which includes among others cache hierarchies, memory compression and 3D stacking technology) and system interconnections. In Figure 1, we used only two broad classes: multi-cores and customization. There are also architectural techniques that apply to single-cores. Kontorinis et al. have described a highly adaptive processor design using a tabledriven approach that is used to configure the core s elements allowing tradeoffs between execution time and energy consumption. The dynamic configuration is based on run-time resource demands and user-defined performance. Bhattacharjee and Martonosi have investigated how custom design of Translation Lookaside Buffers, TLB, can improve overall performance of a CMP. At first sight, this topic appears to be related only to performance and not to energy consumption. However, "multi-core aware" TLBs will improve performance for the same energy budget, which leads to an improvement in energy-efficiency. 2.2.1Multi-core Processors A simple homogeneous multi-core architecture was sketched in Figure 3. As explained in the previous section there is a cubic relation between processor speed (clock frequency), supply voltage and power. 2 cores running at f/ 2 can do a computation in the same time T seconds as one core running at f. In a very simplistic view, considering only dynamic power, and assuming that supply voltage can be scaled down to V/2 when frequency is set to f/2, these two cores will use 1/8 of the power each, and the total power consumption will be 1/4 for 2 cores doing the same work equally fast. In reality, it is not that simple, and some of the energy savings are lost. Normally, quite a bit of overhead is involved when transforming an application into a parallel program. This depends on the parallelism available in the application, the programming tools used and the underlying architecture. There are some, so-called embarrassingly parallel applications that can be executed on a large number of cores with very little coordination and communication between the cores, i.e. having a large computation/communication ratio. However, very often lots of communication are involved in the parallelisation, and a higher degree of parallelism will reduce the computation/communication ratio to a point We describe how parallelization in both HW and SW can help reduce the energy consumption, and discuss several examples of how throttling (slowing down) and simplicity give the same effect 52 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

Green computing is an interdisciplinary, large and rapidly increasing area where the parallelisation gives no further speedup. When energy-efficiency is taken into account in addition to execution time, it becomes even more important to avoid too much overhead. This explains the development of different throttling techniques in addition to processor slowdown via DVFS. These can restrict or reduce the degree of thread level parallelism, TLP, as in Feedback-Driven threading, FDT [9] and Dynamic Concurrency Throttling, DCT [3]. A related technique is Fetch Throttling that refers to the ability to control the number of instructions a processor fetches each cycle. This will reduce the amount of instruction level parallelism, ILP, exploited by a processor. Throttling is not a new technique in computer architecture. It was used as early as in 1980s in the Manchester Dataflow Machine, MDM where an architectural technique called Throttle was designed to control the parallelism [10]. The dataflow computing paradigm allows for very fine-grained parallelism and applications could easily express large amounts of parallelism that gave the MDM excessive use of storage, hence it had to be throttled. A somewhat similar situation is so called thrashing in operating systems, a situation that occurs when so many processes have to be context-switched that this overhead drastically reduces the processors ability to do computations. The simplistic multi-core in Figure 3 also shows another aspect of crucial importance for energy efficiency in all contemporary computers. For several decades we have seen a growing performance gap between processor and memory (also called the memory wall) the speed of processors (core) has increased much faster than the speed of memory access. A direct result is the need for memory hierarchy, including several levels of fast and small memory (caches). Today s multi-cores have at least two levels of cache: in the example in Figure 3 there is a level 1 (L1) cache in each core and one shared level 2 (L2). These caches are beneficial for energy saving in two main ways. All memory access that can be kept locally, i.e. on-chip, will save huge amounts of both time and energy. Note also that the offchip memory bandwidth is expected to become an increasingly severe bottleneck. As a result, many new programming languages have keywords that can be used by programmers to specify data locality. A second advantage of cache hierarchies in multi-core systems is that the on-chip shared cache gives a fast way for the processes on the different cores to communicate. This can be used by the programmer as traditional shared memory programming, which is considered to be much easier than the alternative of message passing. Both these parallel programming paradigms have been used on multiprocessors for decades. For the future, we believe the advent of multi- and many-cores and large on-chip shared caches will foster great improvements in both per chip performance and energy efficiency. 2.2.2 Amdahl s Law and Asymmetric Multi-cores If we generalize the homogeneous multi-core architecture of Figure 3 to a chip with n equally powerful processor cores it will become an ideal execution vehicle for an application that can undergo an ideal parallelization into n equally sized computational tasks. However, as stated by Amdahl s law, the part of an application that is serial, i.e. cannot be parallelized, will severely limit the maximum speed-up that can be obtained by parallelisation. A direct consequence is the increasing interest in asymmetric multicores where at least one of the cores is more powerful, and well suited to "crunch" the serial part of the computation. The effect of such an architectural choice and the more general discussion about few fat cores vs. many thin cores is elegantly discussed in the paper Amdahl s Law In the Multicore Era by Hill & Marty [11]. A related study with focus on energy-efficiency is the paper Maximizing Power Efficiency with Asymmetric Multicore Systems by Fedorova et al. [12]. 2.2.3 Customization and Heterogeneity A multi-core processor extended to include a single core with enhanced computational power or asymmetric multicores are in the category of heterogeneous multi-cores. This is a large class of very diverse microprocessor products and many of them contain very different processors on the same chip with different architectures and even programmed in different languages. Generally, this makes programming of such chips much more complex. Nevertheless, the reason for its popularity is the benefits of customization. Most applications consist of different phases or subtasks that can be executed more efficiently by a special purpose processor or some dedicated HW unit. This is even truer for energy-efficiency, as witnessed by a large amount of different customized MPSoC, Multiprocessor System-on-Chip, chips used in embedded systems. 2.2.4 The Green500 List Also among supercomputers, when it comes to energy efficiency, heterogeneous multi-cores are dominating today. This is evident from the recent June 2011 Green500 ranking list of power efficient computers, <http:// www.green500. org/lists/2011/06/top/list.php>. The list resembles the more famous top500.org list of the most powerful supercomputers, but Green500 use performance/watt as ranking criteria. IBM supercomputers lead the Green500 s latest list of the world s most energy efficient high-performance com- Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 53

Figure 5: PowerXcell 8i, Simplified View. puters. The two Blue Gene/Q prototypes use a 18 core processor that is based on a modified version of the Power A2 chip, but one running at a lower speed 1.6GHz versus 2.3GHz. 16 cores are used for computing and 1 core for running the operating system. The 18th core is used as a spare. The University of Nagasaki built a hybrid cluster called DEGIMA with Intel Core i5 processors matched to ATI Radeon graphics controllers all linked together with 40Gb/sec InfiniBand switches. Its components (7,600 cores total) are relatively inexpensive and their energy efficiency gave DEGIMA the number three ranking on the Green 500 list. The fourth most energy-efficient supercomputer is the Tsubame 2.0 hybrid built by Tokyo Institute of Technology, comprising HP ProLiant servers and Nvidia Tesla GPU coprocessors. Another IBM hybrid the PLX Cluster at the Cineca/ SCS research consortium in Italy, made up of 274 idataplex servers using six-core Xeon 5600 processors and Nvidia M2070 GPU coprocessors is occupying the fifth position. The most powerful supercomputer (as ranked in the Top500), the K supercomputer built by Fujitsu for the Japanese government, is ranked as the sixth most energy efficient machine. This is a monolithic parallel machine, using the eightcore Sparc64-VIIIfx processors. Other machines at the top of the Green500 list include a set of rather special supercomputers, the QS22 blade servers, based on IBM s Cell PowerXCell 8i chips and a 3D torus interconnect. Figure 5 shows a simplistic view of the architecture that is the 2 nd generation product of the Cell Broadband Engine architecture developed by Sony, Toshiba and IBM. The processor consists of a quite standard main processor based on the Power PC architecture, and 8 specialised cores called synergistic processing units (SPU). These have a local store (LS) and a DMA engine for fast memory transfers. By explicit use of these two resources the programmer can achieve high power efficiency, but the price paid is the increase in software development complexity. A further step toward using specialized or customized HW units is reconfigurability as offered by the successful FPGA, Field Programmable Gate Array, technology. FPGA is hardware that can be reconfigured during run-time and is often called programmable HW. A FPGA chip can be configured to be ideal for executing one phase of an application, and then be reconfigured to be well suited for a later phase with different computational demands. Zain et al implemented an autofocus algorithm used in synthetic aperture radar systems in a FPGA as a case study. They showed that two different parallel implementations using many cores running on much lower clock frequency could achieve both better performance and highly improved energy efficiency [13]. Again, the price paid for excellent energy efficiency results is more difficult programming, although the researchers raised the abstraction level of the FPGA programming by using a language called Occam-pi. Thus, good energy efficiency and high programmability can be conflicting goals, and we will sketch a few more of these challenging trade-offs in Section 4. 2.3 Power Management and Techniques at the OS Level Fortunately, software tools and systems have been developed that can improve energy efficiency without the end user or programmer being aware of it. There are also tools giving good help during energy efficiency engineering. It is predicted that near threshold computing can reduce energy requirements by 10 to 100 times or even more in future systems 54 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

Another important technique is dynamic voltage scaling and frequency scaling, DVFS. It requires special hardware features, but it is normally controlled by software Several of these techniques control the amount of parallelism dynamically. Suleman et al have developed a framework called feedback-driven threading, FDT, that dynamically controls the number of threads using runtime information. FDT can be used to implement Synchronization-Aware Threading, SAT, controlling the number of threads depending on the amount of synchronization. Further, it can form the basis for Bandwidth-Aware Threading, BAT, that predicts how many threads we can execute before the off-chip bus gets saturated. Both these techniques, and their combined use, can reduce the execution time and power consumption with up to more than 70% [9]. Singh et al have developed a power aware thread scheduler that can be used to ensure that the system executes within a maximum power envelope [14]. Both the above mentioned techniques exploit the performance counters of modern microprocessors giving low level programmers deep insight into processor state and dynamic behaviour. Bertran et al have shown that such models can be used to not only model power consumption with good accuracy, but also to provide per component power consumption [15]. Bhattacharjee and Martonosi propose and evaluate simple but effective methods for predicting critical threads in parallel applications. It is shown by the above mentioned research that accurate predictors can be built using performance counters. If a system can accurately find the critical, or slowest, threads of a parallel program, this information can be used for load rebalancing, energy optimization, and capacity management on constrained resources. A lot of research has been done in power management tools used at the Operating System or Run Time System level. Dynamic power management policies perform better than static power management and it has been shown that a global view of the activity of all the cores in the system can gives the most optimal control of the power consumption [4]. Borkar and Chien outline a hypothetical heterogeneous processor consisting of a few large cores and many small cores, where the supply voltage and frequency of each core is controlled individually. This fine-grained power management improves energy-efficiency without burdening the application programmer, i.e. controlled by the run time system. A recent example of an innovative tool that helps in developing energy efficient software is the energyaware profiler developed by Energy Micro for use with their energy friendly microcontrollers [7]. This tool plots a graph of the use of current during execution, and by clicking on a power-peak the developer is automatically directed to the source code causing the peak. We expect that similar tools will be provided for larger processors in the future. 2.4 Energy Aware Algorithms and Data Structures A recent paper by Susanne Albers [16] presents a survey of algorithmic techniques to solve problems in energy management. The goal of these algorithms is to reduce energy consumption while minimizing compromise to service. One of their major challenges is that the power management problem normally is a so called online problem meaning that at any time the system is not aware of future events. A power down operation typically uses very little energy, but since a power up operation will consume some extra energy it might be a wrong decision to power down a unit being idle if the idle-time appears to be very short. The challenge becomes even larger when there are more than two power states. The same survey paper also covers dynamic speed scaling as made possible by DVFS described in Section 2.1. This gives the OS scheduler a more challenging job but also more options for saving energy. Further, scheduling of jobs with deadlines, trying to minimize processor temperature or the more classical goal of minimal response time all give different algorithmic challenges. We agree with Albers that speed scaling techniques in multiprocessors and multicore based computers will become increasingly important. In Section 3, we present some early experiments in a related area. The future will show new ways of expressing parallel algorithms and new ways for the corresponding application to interact with the scheduling and power management algorithm in the underlying OS and RTS. Also in the area of algorithms the value of simplicity must never be forgotten. Traditional algorithmic studies use asymptotical analysis and complexity classes. As an example, a O(log n) sorting algorithm is considered to have a better performance than a O(log 2 n) algorithm (n is the number of items to sort). However, as exemplified by a study of Cole s parallel merge sort, complex algorithms have large complexity constants, so a O(log 2 n) algorithm might be faster than a O(log n) algorithm for all practical problem sizes. The term "simplicity" has several interpretations. In the example above, it was synonymous with low complexity constants. Another interpretation is simple to understand. Again using sorting as example, the old odd-even transposition sort is perhaps the easiest to understand parallel algorithm, however being O(n 2 ) it can also be said to be a brute force algorithm where simplicity in the algorithm is achieved by adding redundant operations. In this paper, simplicity refers to low implementation complexity or little parallelisation overhead. The value of simplicity is well known among engineers and known as "principles" such as Occam s razor and the KISS principle ("Keep It Simple Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 55

Stupid"). The latter has inspired the paper The KILL Rule for Multicore. Here, "Kill" stands for Kill If Less than Linear, and describes an approach for power-efficient multi-core design. A recent paper by Nir Shavit explains how the advent of multi-cores gives a need for new research in data structures [17]. To improve parallel performance we should try to reduce the serial fraction that may be caused by using a traditional shared data structure that allows access to only one core at a time. New research in lock-free data structures and relaxation methods can be used to implement slightly more complex but concurrent data structures that in many cases can reduce this serial fraction. This can give higher speed up and better performance, and if the added complexity is not too high, it can improve the energy efficiency. In addition, the new multi-cores also have very large on-chip shared caches. Since the increased cost both in time and energy. for memory accesses that miss in this last-level cache (LLC). might be as much as 100 times larger or more, there is great importance in keeping the data on-chip. We believe this fact also provides motivation for research into new energy-efficient multi-core data structures. 3 Case Study: Energy-aware Task Based Programming To build competence in how to save energy by parallelization we have developed an experimental framework combining an architecture simulator with full system simulation with a power and area estimation tool. We studied the impact of load-balancing on energy-efficiency in parallel executions and we identified several issues that limit the energy-efficiency of a system as the number of cores grows. For our experiments we used a Task Based Programming, TBP, approach to parallelize our test benchmarks. The base concept of the TBP model is that the programmer should identify and annotate pieces of code (tasks) which can be executed concurrently with other tasks, while the complexity of the hardware is covered by an abstraction. Each application was organized as a set of computational units (called tasks) that were scheduled across different cores. The experimental framework we put in place for this study combines an architectural simulator and a power estimation tool. We used GEM5 [18] to perform full-system simulations of several multi-core platforms. The simulations were run on the Kongull compute cluster made available by the NTNU HPC Division (http://docs.notur.no/ntnu). The performance statistics gathered from GEM5 were used as input for a power, area and timing estimation tool called McPAT [19]. This setup allowed us to record the behaviour (for both performance and energy consumption) for all key components of the system: cores, cache hierarchy, main memory etc. We simulated our test benchmarks on systems with 2 - Figure 6: Energy consumed by running 6 Test Applications on Different Numbers of Cores. 56 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

Also among supercomputers, when it comes to energy efficiency, heterogeneous multi-cores are dominating today 8 cores and for various levels of work imbalance. For our measurements we used the Energy-Delay-Product, EDP, in order to ensure that a complete profile of the application (one that includes performance and energy consumption) is created. For each core count we recorded the load balancing characteristics with the best EDP value. Figure 6 shows the energy consumption of the most power efficient execution for number of cores ranging from 1 to 8 for 6 different benchmark applications. We see that all applications reach minimum energy consumption for 2, 3 or 4 cores. This reflects that our benchmarks are all relatively small a necessary choice for avoiding our detailed simulations to becoming too lengthy. The "bathtub" curves are due to two issues we observed: parallelization overhead and stalls caused by the synchronization portions in the code. All these negative effects grow together with the core count and at a certain point the decrease in energy consumption stops. Fib and SparseLU perform better than the other benchmarks when scaling up the core count because they require less task synchronization. Fib uses a recursive approach to generate a very extensive task tree. This ensures that every core will find work every time it needs it. SparseLU generates far less tasks, but they require less synchronization so the cores are stalled a very small amount of time during the execution. Our experiments show a significant potential for energyefficiency improvements of parallel executions on multicores compared to the serial version of the same application on a single-core system. In all, we think the results we have found so far are promising and motivate further research into energy-efficiency through parallelization. 4 Concluding Remarks and Future Work We have seen that low-power design, HW-techniques, multi-core architectures, parallelism, heterogeneity and novel software techniques both at the OS level and in user application all can contribute to improved energy efficiency. In Figure 7 we show the "5 Ps of parallel processing": performance, predictability, power-efficiency, programmability and portability. When designing for better energy-efficiency, it is important to have a holistic view and be aware of the possible conflicting goals. Optimizing for one of these Ps very often will reduce the possibility of achieving some of the others. As an example, the Cell processor has achieved outstanding results for power-efficiency at the cost of reduced programmability. Also, optimizing an application to use the underlying architecture in an energy-efficient way will very often introduce constructs and adaptations that reduce portability. Predictability denotes the wish of the user to know how long a computation will take. If it is of high importance, as in real time systems, the system must be designed to meet performance guarantees. But if predictability can be given low priority, the task can be postponed to a low-load period, running using less energy. We plan to extend and improve the research sketched in Section 3 in different ways. On the programming side, we will test OpenMP programs and other variants of task based programming. On the execution side, we will test and measure the energy consumption on the Intel Sandy Bridge architecture (Core i7) and eventually other multi-cores. We wish to extend our experiments towards a larger number of cores and at the same time try to model a more recent and energy-focused processor. The full-system simulations used are very time consuming and limit the current approach. Therefore, we work on using statistical sampling methods to be able to simulate the main effects with drastically reduced execution time. There are many challenges, but these are also opportunities for innovation in new products, solutions and exciting research. We see the increased focus on energy efficiency Figure 7: 5 Ps of Parallel Processing. Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 57

We see the increased focus on energy efficiency as a revitalizing force for large parts of the Computer Science field as a revitalizing force for large parts of the Computer Science field. References [1] B. Zhai et al. "Energy efficient near-threshold chip multi-processing". Proceedings of the 2007 Int l Symp. on Low Power Electronics and Design Series, ISLPED 2007. [2] V. Venkatachalam, M. Franz. "Power reduction techniques for microprocessor systems". ACM Comput. Surv., September, 2005. [3] M. Curtis-Maury et al. "Prediction models for multidimensional power-performance optimization on many cores". Proceedings of the 17th Int l Conf. on Parallel Architectures and Compilation Techniques, PACT 2008. [4] C. Isci et al. "An Analysis of Efficient Multi-Core Global Power Management Policies: Maximizing Performance for a Given Power Budget". Proceedings of the 39th annual IEEE/ACM Int l Symp. on Microarchitecture, MI- CRO 2006. [5] Intel Corp. "2nd Generation Intel Core Processor Family Desktop". <http://download.intel.com/design/processor/datashts/324641.pdf>. [6] J. Howard et al. "A 48-core IA-32 Message-Passing Processor with DVFS in 45nm CMOS". IEEE International Solid-State Circuits Conference, 2010. [7] Energy Micro website, <http://www.energymicro.com/>. [8] Advanced Configuration and Power Interface Specification, Revision 4.0, June 16 2009. <http://www. acpi.info/spec40.htm>. [9] M.A. Suleman et al. "Feedback-Driven Threading: Power-Efficient and High-Performance Execution of Multi-threaded Workloads on CMPs". Proceedings of the 13th int l conf. on architectural support for programming languages and operating systems, ASPLOS 2008. [10] C. Ruggiero, J. Sargeant. "Control of parallelism in the Manchester Dataflow Machine". In Lecture Notes in Computer Science - Functional Programming Languages and Computer Architecture, 1987. [11] M. D. Hill, M. R. Marty. "Amdahl s Law In the Multicore Era". IEEE Computer, July 2008. [12] A. Fedorova et al. "Maximizing Power Efficiency with Asymmetric Multicore Systems". Communications of the ACM, December 2009. [13] Zain-ul-Abdin et al. "Programming Real-time Autofocus on a Massively Parallel Reconfigurable Architecture using Occam-pi". Proceedings of IEEE 19th annual Int l Symp. on Field-programmable Custom Computing Machines, FCCM 2011. [14] K. Singh et al. "Real Time Power Estimation and Thread Scheduling via Performance Counters". SIGARCH Comput. Archit. News, July 2009. [15] R. Bertran et al. "Decomposable and Responsive Power Models for Multicore Processors using Performance Counters". Proceedings of the 24th ACM Int l Conf. on Supercomputing, ICS 2010. [16] S. Albers. "Energy-efficient algorithms". Communications of the ACM, May 2010. [17] S. N. Shavit. "Data structures in the multicore age". Communications of the ACM, March 2011. [18] N. L. Binkert et al. "The M5 simulator: Modeling networked systems". IEEE Micro, July-August 2006. [19] S. Li et al. "McPAT: An integrated power, area, and timing modeling framework for multicore and manycore architectures". Proceedings. of the 42nd annual IEEE/ACM Int l Symp. on Microarchitecture, MICRO 2009. [20] Suzanne Rivoire et al. "Models and metrics to enable energy-efficiency optimizations". Computer, December 2007. 58 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

Towards Sustainable Solutions for European Cloud Computing Kien Le, Thu D. Nguyen, Íñigo Goiri, Ricardo Bianchini, Jordi Guitart-Fernández, and Jordi Torres-Viñals Public Cloud Computing means that we are outsourcing our data to places where we cannot keep track of it. This creates a problem in terms of the privacy of our data and its availability. Unfortunately, the risk generated by unallocated computation and storage is not the only problem. In addition, the high energy consumption of the Cloud also contributes to climate change, since most of the electricity produced around the world comes from burning coal and natural gas, which are carbon-intensive approaches to energy production. This article reflects on these problems that arise with Cloud Computing and proposes sustainable solutions to mitigate them in countries like Spain. Keywords: Cloud Computing, Datacenters, Renewable Energy, Sustainability. 1 Datacenter Consumption The true heart of cloud computing is large datacenters, where energy-related costs are becoming the single largest contributor to the overall cost of operating them. Fortunately, new hardware and software technologies are creating exciting opportunities for improving the power management of these systems. Some examples of these technologies are the variety of low power modes into hardware components; the advances in virtualization that enable an efficient consolidation of applications to increase efficiency; or the introduction of new energy-aware algorithms at a resource management level [1][2]. Despite these improvements and the fact that the economic crisis has reduced the electricity used by the datacenters that are powering new Internet and Cloud com- Authors Kien Le is a graduate student in the Computer Science Dept. at Rutgers University, USA. He is a member of Panic Lab and Dark Lab under the supervision of Ricardo Bianchini and Thu D. Nguyen. His research focuses on building cost-aware load distribution framework to reduce energy consumption and promote renewable energy. <lekien@cs.rutgers.edu> Thu D. Nguyen received his PhD degree in Computer Science at the University of Washington in 1999, his MS at MIT in 1988 and his BS at UC Berkeley in 1986, all in the USA. He is currently an Associate Professor at Rutgers University, USA. His research interests include green computing, distributed and parallel systems, operating systems, and information retrieval. <tdnguyen@cs. rutgers.edu> Íñigo Goiri received his MS and PhD degrees in Computer Science at the Universitat Politècnica de Catalunya (UPC), Spain, in 2008 and 2011 respectively. Previously he received his bachelor degree in Computer Science from the Universitat de Lleida, Spain, in 2006. He also collaborated at Barcelona Supercomputing Center (BSC), Spain, as a researcher in the edragon Research Group. He currently holds a postdoctoral researcher position in the Dept. of Computer Science at Rutgers University, USA. <goiri@cs. rutgers.edu> Ricardo Bianchini received his PhD degree in Computer Science from the University of Rochester, USA, in 1995. From 1995 until 1999 he was a Research Associate and then Assistant Professor at the Federal University of Rio de Janeiro, Brazil. He is now a Professor at Rutgers University, USA. Prof. Bianchini s current research interests include the power, energy, and thermal management of datacenters. He has received many awards for his research, including the NSF CAREER award. He has chaired the program committee of many conferences and workshops. He is currently a member of the Editorial Board of the IEEE Computer Architecture Letters and of the Journal of Systems and Software. <ricardob@cs.rutgers.edu> Jordi Guitart-Fernández received his MS and PhD degrees in Computer Science at the Universitat Politècnica de Catalunya (UPC), Spain, in 1999 and 2005, respectively. He is currently an Associate Professor at the Computer Architecture Dept. of the UPC and an associate researcher at Barcelona Supercomputing Center (BSC), Spain. His research interests are oriented towards innovative resource management approaches for modern distributed computing systems. He is involved in a number of European projects. <jguitart@ac.upc.edu> Jordi Torres-Viñals is a Professor at Universitat Politècnica de Catalunya - UPC Barcelona Tech - and research manager at the Barcelona Supercomputing Center (BSC), Spain. He has been Vice-Dean of the Computer Science School and he is currently a member of the Board of Governors at UPC. He has more than twenty years of experience in the research and development of advanced distributed and parallel systems with more than a hundred research publications. He is currently doing RDI work to obtain more sustainable Cloud Computing environments. He acts as an expert on these topics for various organizations and collaborates with the mass media to disseminate ICT. He has just published the book "Empresas en la nube. Claves y retos del Cloud Computing", in Spanish. You can contact him through <http:// www.jorditorres.org>. <torres@ac.upc.edu> Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 59

This article reflects on the problems that arise with Cloud Computing and proposes sustainable solutions to mitigate them puting services, they still consume several megawatts of power [3], especially because citizens are gradually moving away from storing information on their personal computers and they will soon have a large part of their information stored in the Cloud. This Cloud is mainly made up of large data processing centers scattered around the world, some of them the size of several football fields. 2 Information Dependency As the Internet does not have borders as countries do, our information may end up being relocated to the other end of the globe to one of these datacenters, the information factories of the 21st century, where costs, such as energy costs, are much lower. Something similar happened in the textile industry in Spain where, at the end of the last century, most of the production was moved to countries in northern Africa or Asia. If we relocate the production and storage of our information abroad, are we ensuring the availability of this information? What if the same happens to us as has happened to the Spanish textile industry (e.g. in Maresme county), whose factories in Tunisia are not able to deliver goods because of the Arab Revolution which was taking place some months ago? Information is essential in our society. To guarantee the accessibility of information, we need to store our part of this Cloud inside our own European territory. However, information factories require many thousands of kilowatts of energy to operate, so if, for example, approximately 75% of the global energy system in Spain is imported, we can expect this dependency on energy from other countries to increase as we move such information centers here. The only viable option is to increase the production of renewable energy generated inside Europe to ensure power for the new information factories. 3 Macro-generation of Information One possibility is to move these information factories to renewable energy parks, so that these parks become not only suppliers of energy, but also information. This proposal is energy efficient in several aspects; for instance, transferring data via optical fiber from these parks has a minimal cost compared to the losses involved in the transportation of energy, which can be as high as 10%. Moreover, although this proposal should, for now, have power supply backup support (because of the discontinuity of renewable energy), technical solutions exist to ensure good availability of information using mainly renewable energy. For example, by relying on weather forecasting, it is possible to move work between different parks, depending on the energy that each park has, to complete an assignment [4]. 4 Micro-generation of Information However, an important part of the energy consumed by IT is actually due to countless small and medium-sized datacenters [5]. These facilities range from a few dozen servers housed in a machine room to several hundreds of servers housed in a larger enterprise facility. These datacenters are widely dispersed and closer to users. Nowadays, research is being carried out in this direction where the line of thought is to build small-sized containerized data-centers powered by photovoltaic panels or wind turbines. These datacenters would rely on cooling systems that can take advantage of the external temperature to reduce the overall power consumption. In addition, they are connected to the electrical grid as a backup because renewable energy source are not always available. 5 Research Challenges The information-generating landscape will be different in the future. As we mentioned, citizens are gradually moving away from storing information on their personal computers and soon they will have all this information stored in the Cloud. There seems to be two main ways: large information plants, generating a large amount of information (storage and computation) for a large number of people (macro), or households and businesses generating their own information (micro). However, in either case, the current research challenge regarding solar and wind energy is that it is not always available. For example, solar energy is only available during the day and the amount produced depends on the weather and the season. In this case, with micro-generation we can sometimes "bank" green energy in batteries or on the grid itself (called net metering) to mitigate this variability. However, both batteries and net metering have problems as we reported in [6]. Based on these observations, our vision at BSC/UPC [7] is that the best way to take full advantage of the available green energy is to match the energy demand to the energy supply as much as possible, moving work between green datacenters or delaying the execution of work according to service constraints and renewal power availability. However, these goals have made resource management a burning issue in today systems, adding energy efficiency to a list of critical operating parameters that already includes service availability, reliability, and performance. For BSC, self-management is considered the solution to this complex- 60 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

The true heart of cloud computing is large datacenters, where energy-related costs are becoming the single largest contributor to the overall cost of operating them ity and a way to increase the adaptability of the execution environments to the dynamic behavior of Cloud Computing [8]. To illustrate our current work, we selected an example of one of our current research works in micro-generation which is being carried out in collaboration with a research group at Rutgers University in the USA. In particular, we are working on intelligent mechanisms to automatically manage the applications under execution in small-sized containerized datacenter prototypes, in order to make them self-sufficient. 6 Example of Research in this Area The focus chosen as an example is scientific workloads running on commodity servers in small and medium datacenters (such as those operated by many enterprises or universities), rather than large facilities. In particular, we design a parallel batch job scheduler, called GreenSlot, for a datacenter powered by an array of photovoltaic solar panels and the electrical grid. Jobs submitted to GreenSlot come with user-specified numbers of nodes, expected running times, and deadlines by which they must be completed. The deadline information provides the flexibility that GreenSlot needs to manage energy consumption aggressively. GreenSlot seeks to maximize renewable or green energy consumption (or equivalently to minimize grid energy consumption) while meeting the jobs deadlines. If grid energy must be used to avoid deadline violations, GreenSlot schedules jobs for times when grid electricity is cheap. To be more precise, GreenSlot combines solar energy prediction, energy cost awareness, and least slack time first (LSTF) job ordering [9]. It first predicts the amount of solar energy that will likely be available in the future, using historical data and weather forecasts. Based on its predictions and the information provided by users, it schedules the workload by creating resource reservations into the future. When a job s scheduled start time arrives, GreenSlot dispatches it for execution. As should be clear by now, GreenSlot departs significantly from most job schedulers, which seek to minimize completion times or bounded slowdown. We implement GreenSlot as an extension of the widely used SLURM scheduler for Linux [10]. Our experiments use real scientific workloads (and their actual arrival times and deadlines) that are in current use at the Life Sciences Department of the Barcelona Supercomputing Center [11]. We model the datacenter s solar array as a properly scaleddown version of Rutgers solar farm in New Jersey. The grid electricity prices are from a power company in New Jersey. Our results demonstrate that GreenSlot accurately predicts the amount of solar energy to become available. The results also demonstrate that GreenSlot can increase green energy consumption and decrease energy costs by up to 117% and 39% respectively, compared to a conventional backfilling scheduler. Based on these positive results, we conclude that green datacenters and green energy aware scheduling can have a significant role in building a more sustainable Information Technology ecosystem. More details about GreenSlot can be found in [6]. 7 Energy Scenario We considered the scenario of solar energy for demonstrating the usefulness of our algorithm, although solar energy and wind are both promising clean energy technologies in Europe. Except for our (solar) energy predictions, the example selected is directly applicable to wind energy as well. Transforming solar energy into (direct-current or DC) electricity is most commonly done using photovoltaic panels. The panels are made of cells containing photovoltaic materials, such as monocrystalline and polycrystalline silicon. The photons of sunlight transfer energy to the electrons in the material. This extra energy causes the electrons to transfer between the two regions of the material, producing a current that is driven through the electrical load (e.g., a datacenter). Since current server and cooling equipment run on alternating current (AC), the DC electricity produced by the panels must be converted to AC electricity using an inverter. When the datacenter has to be operational even when solar energy is not available, it must also be connected to the electrical grid via a grid-tie device. The design we study in this paper does not include batteries or net metering, for the reasons mentioned in [6]. Regarding prices through the grid, datacenters often An important part of the energy consumed by IT is actually due to countless small and medium-sized datacenters Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 61

Figure 1: Scheduling 3 Jobs (J1-J3) with Backfilling (top) and Green-Slot (bottom). The Jobs Deadlines are the Vertical Lines. contract with their power companies to pay variable prices, i.e. different euros amounts per kwh of consumed energy. The most common arrangement is for the datacenter to pay less for electricity consumed during an off-peak period than during an on-peak period. Typically, off-peak prices are in effect during the night, whereas on-peak prices apply during the day. Thus, it would be profitable for the datacenter to schedule part of its workload (e.g., maintenance or analytics tasks, activities with loose deadlines) during the night. Our model for predicting the generation of solar energy is based on a simple premise: various weather conditions, e.g., partly cloudy, reduce the energy generated in a predictable manner from that generated on an ideal sunny day. We use weather forecasts widely available from sites such as The Weather Channel and Weather Underground. Unfortunately, weather forecasts can be wrong. Furthermore, weather is not the only factor that affects energy generation. For example, after a snow storm, little energy will be generated while the solar panels remain covered by snow even if the weather is sunny. To increase accuracy during the above "mispredictions" we also use an alternative method. Specifically, we assume that the recent past can predict the near future using the observed energy generated in the previous hour. Although we do not claim our prediction approach as a contribution of this paper, it does have three important characteristics: it is simple, relies on widely available data, and is accurate for medium time scales, e.g. a few hours to a few days. 8 Resource Management in Renewable Datacenters We propose GreenSlot, a parallel job scheduler for datacenters powered by photovoltaic solar panels and the electricity grid. GreenSlot relies on predictions of the availability of solar energy, as well as on a greedy job scheduling algorithm. Figure 1 illustrates the behavior of GreenSlot (bottom) in comparison to a conventional EASY backfilling scheduler (top) for three jobs. Each rectangle represents the number of nodes and time that each job is likely to require. The vertical lines represent the jobs deadlines. Note that backfilling uses less renewable energy (more grid energy) as it does not consider the energy supply in making decisions. Any scheduler (including a real-time one) that is unaware of renewable energy would behave similarly. In contrast, GreenSlot delays the execution of some jobs (as long as they do not violate their deadlines) to guarantee that they will use green energy. This delay is not a concern since users only need their jobs completed by the jobs deadlines. Similarly, GreenSlot may delay certain jobs to use cheaper grid electricity (not shown). GreenSlot is beneficial because datacenters are not fully utilized at all times. 9 The Inside Scheduling Algorithm GreenSlot seeks to minimize grid energy consumption by using solar energy instead, while avoiding excessive performance degradation. 62 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

We have designed a parallel batch job scheduler, called GreenSlot, for a datacenter powered by an array of photovoltaic solar panels and the electrical grid At submission time, users can specify the workflows to which their jobs belong. As in many other job schedulers, users must inform the number of nodes and the expected running time for each of their jobs. Deadlines can be specified per job or workflow. At the beginning of each slot, GreenSlot determines whether a new schedule needs to be prepared. If so, it goes through the list of queued jobs and schedules them (i.e., reserves resources for them) into the future. This scheduling window corresponds to the range of our hourly solar energy predictions, i.e. two days. The window is divided into smaller time slots (15 minutes in our experiments). The scheduling window moves with time; the first slot always represents the current time. GreenSlot is cost-aware in that it favors scheduling jobs in time slots when energy is cheapest. To prioritize green energy over grid energy, renewable (green) energy is assumed to have zero cost. In contrast, grid electricity prices often depend on time of use, as mentioned previously. When the price of grid electricity is not fixed and grid energy must be used, GreenSlot favors the cheaper time slots. To avoid selecting slots that may cause deadline violations, GreenSlot assigns a high cost penalty to those slots. GreenSlot is greedy in two ways: (1) it schedules jobs that are closer to violating their deadlines first; and (2) once it determines the best slots for a job, this reservation does not change (unless it decides to prepare a new schedule during a later scheduling round). The next job in the queue can only be scheduled on the remaining free slots. Moreover, GreenSlot constrains its Figure 2: GreenSlot Scheduling Window at Times T1 (top), T2 (middle), and T3 (bottom). Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 63

Figure 3: Conventional Scheduler and Average Week. scheduling decisions based on workflow information, i.e. a job belonging to phase i of a workflow cannot begin before all jobs of phases < i have completed. GreenSlot dispatches the jobs for execution on the cluster according to the schedule. Dispatched jobs run to completion on the same nodes where they start execution. GreenSlot deactivates any idle nodes to conserve energy. Figure 2 illustrates GreenSlot s operation, from time T 1 (top) to T 3 (bottom), with a very simple example. At T 1, job J1 is executing and job J2 is queued waiting for green energy to become available (according to GreenSlot s solar energy predictions). More than a day later than T 1, at T 2, J1 and J2 have completed, and J3 has just been dispatched. Because GreenSlot predicts two days of very little solar energy, J4 is scheduled for the following day during a period of cheap grid electricity. More than a day later than T 2, at T 3, we see that GreenSlot initially mispredicted the amount of solar energy at time T 2. It later adjusted its prediction and ran J4 earlier. Finally, we also see J5 queued waiting for solar energy to become available. 10 Evaluation For comparison purposes we also study a variant of EASY backfilling [12] which considers the deadlines in Figure 4: GreenOnly Scheduler and Average Week. 64 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

In the near future we should try to build new datacenter facilities which meet the Cloud computation and storage requirements of European companies and citizens sorting the job queue in LSTF order. The scheduler backfills jobs as long as the first job in the queue is not delayed. We refer to this baseline scheduler as "Conventional". Like GreenSlot, Conventional assigns a 20% tolerance to the user-estimated job run times. If a job s run time estimate and tolerance are exceeded, Conventional cancels the job. Conventional is oblivious to energy sources and costs. Figure 3 shows the behavior of Conventional for our real workloads, the Average week, and accurate job run time estimates. The X-axis represents time, whereas the Y-axis represents cluster-wide power consumption (left) and grid (we refer to it as a brown from now on) electricity prices (right). The figure depicts the green and brown energy consumptions using areas colored light gray and dark gray, respectively. The two line curves represent the green (renewable) energy available (labeled "Green actual") and the brown electricity price ("Brown price"). The figure shows that the cluster utilization is roughly 50%, which is comparable to (or even higher than) many real scientific-computing datacenters [13]. As Conventional schedules the workloads to complete as soon as possible, it uses the servers heavily early in the week and leaves them in a deep-sleep state late in the week. This approach is ideal in terms of conserving energy, since keeping modern servers powered on involves a high "static" energy [14]. However, Conventional wastes a large amount of green energy, which could be used instead of brown energy. In this experiment, only 26% of the energy consumed is green. Figure 4 depicts the behavior of GreenOnly, under the same conditions as in Figure 3. Note that, in this figure, we plot the amount of green energy that GreenSlot predicted to be available an hour earlier (labeled "Green predicted"). The green prediction line does not exactly demarcate the light gray area, because our predictions sometimes do not match the actual green energy available. A comparison between Figures 3 and 4 clearly illustrates how GreenOnly is capable of using substantially more green energy than Conventional, while meeting all job/workflow deadlines. Green-Only spreads out job execution across the week, always seeking to reduce the consumption of brown energy within resource and deadline constraints. Overall, GreenOnly consumes 47% more green energy than Conventional in this experiment. Although GreenOnly does not explicitly consider brown electricity prices in making decisions, its energy cost savings reach 20% compared to Conventional. More than 80% of these cost savings comes from replacing brown energy with green energy To demonstrate this in practice, we are building a prototype micro-datacenter powered by a solar array and the electrical grid. The micro-datacenter will use free cooling almost year-round and will be placed on the roof of our building at Rutgers University (New Jersey, USA). Our future work will further explore our approach for green energy prediction, improve GreenSlot s ability to gracefully handle very high datacenter utilizations, and extend GreenSlot for managing peak brown power consumption for those datacenters that are subject to peak brown power charges. With the description of this work we want to demonstrate that our vision is a realistic approach to solve real problem in this area. More details about GreenSlot can be found in [6]. 11 Looking ahead Recent world events remind us that we cannot be sure that our data is always available and safe outside our European borders. This means that in the near future we should try to build new datacenter facilities (which meet the Cloud computation and storage requirements of European companies and citizens) inside our borders. The kind of energy to be used for computation and storage in our European territory will have to be mostly renewable because of: (a) our current dependency on energy coming from foreign countries, (b) the environmental awareness and responsibility of European citizens and, (c) the limitations being imposed by the institutions themselves on companies regarding CO2 emissions. Because of all this, we are being forced to consider sustainable datacenters, that is, datacenters which are not responsible for CO2 generation during their operation and which help reduce our dependency on energy (fossil fuels) coming from foreign countries. There is no doubt that for future generations, information will form a crucial part of their society. There is therefore an urgent need to pull together to protect it. Citizens should be aware of this reliance on information and should support the building of infrastructure that make it possible. We should not forget that energy dependency constitutes a problem in itself. Without the energy to power computers, no data is available. But, above all, it takes foresight on the part of governments to stimulate the market itself in order to provide such services and actively promote Research+Innovation, not only in terms of power generation, but also with regard to ensuring the availability of information, which will soon be as basic a need as energy. Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 65

Acknowledgments This work is partially supported by the Ministry of Science and Technology of Spain (contract TIN2007-60625), the Generalitat de Catalunya (2009-SGR-980), the European Commission in the context of the COST (European Cooperation in Science and Technology) framework, under Action IC0804, NSF grant CSR-0916518, and the Rutgers Green Computing Initiative. References [1] E. Ayguadé, J. Torres. "Creating Power-Aware Middleware for Energy-Efficient Data Centres". October 2009. ISSN 0926-4981. [2] E. Ayguadé, J. Torres. "Holistic Management for a more Energy-Efficient Cloud Computing". ERCIM News, number 83, October 2010. ISSN 0926-4981. [3] J.G. Koomey. "Growth in Data Center Electricity Use 2005 to 2010". Analytics Press, 2011. [4] J. Torres. "Empresas en la nube, Ventajas y retos del Cloud Computing" (in Spanish). Libros de Cabecera, June 2011. ISBN: 978-84-939082-2-5. <http:// www.jorditorres.org/blog>. [5] US Environmental Protection Agency. "Report to Congress on Server and Data Center Energy Efficiency", August 2007. [6] Í. Goiri, K. Le, Md. E. Haque, R. Beauchea, T.D. Nguyen, J. Guitart, J. Torres, R. Bianchini. "GreenSlot: Scheduling Energy Consumption in Green Datacenters". To appear in Super Computing 2011 (SC 2011). [7] Autonomic Systems and ebusiness applications research line at BSC, <http:www.bsc.es/autonomic>. [8] J. Torres, E. Ayguadé, D. Carrera, J. Guitart, V. Beltran, Y. Becerra, R. M. Badia, J. Labarta, M. Valero. "BSC contributions in Energy-aware Resource Management for Large Scale Distributed Systems". Proceedings of the COST Action IC0804 on Large Scale Distributed Systems 1st Year. Jean-Marc Pierson, Helmut Hlavacs (Ed.) pp 76-79. ISBN: 978-2-917490-10-5. [9] R. Davis, A. Burns. "A Survey of Hard Real-Time Scheduling Algorithms and Schedulability Analysis Techniques for Multiprocessor Systems". Technical Report YCS-2009-443, Dept. of Computer Science, University of York, 2009. [10] A. Yoo, M. Jette, M. Grondona. "SLURM: Simple Linux Utility for Resource Management". In Proceedings of the International Workshop on Job Scheduling Strategies for Parallel Processing, June 2003. [11] Live Science department at BSC. <http://www.bsc.es/ plantillae.php?cat_id=3>. [12] D. Lifka. "The ANL/IBM SP Scheduling System". In Proceedings of the International Workshop on Job Scheduling Strategies for Parallel Processing, 1995. [13] P. Ranganathan, P. Leech, D. Irwin, J. Chase. "Ensemble-level Power Management for Dense Blade Servers". In Proceedings of the International Symposium on Computer Architecture, June 2006. [14] L. A. Barroso, U. Hölzle. "The Case for Energy-Proportional Computing". IEEE Computer, 40(12), December 2007. 66 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

A State-of-the-Art on Energy Efficiency in Today s Datacentres: Researcher s Contributions and Practical Approaches Marina Zapater-Sancho, Patricia Arroba-García, José-Manuel Moya-Fernández, and Zorana Bankovic Energy efficiency has become an issue of great importance in today s datacentres. Metrics like Top500, which measure speed and performance, are beginning to lose importance in favor of others such as Green500. In order to increase energy efficiency of datacentres and save energy costs, the research community proposes solutions from both the computing and the cooling point of view, while European and US Institutions publish best practice manuals on energy-efficiency for datacentre owners. However, even though best practices are beginning to be implemented, most of the solutions offered by researchers are not yet used in real production environments. This paper makes a survey of the solutions proposed by researchers as well as the practices that real datacentres apply in order to increase the energy-efficiency of their facilities, and to find the reasons that create this gap between research and innovation in datacentres. Keywords: Cooling, Datacentres, Energy Efficiency, Green Computing, Heterogeneous Systems, High-Performance Computing, Power Usage Efficiency (PUE), Reliability, Thermal-Aware Scheduling. 1 Introduction In today s world, immersed in the Information Society, the driving force of economy has changed from being fossil fuels and electricity to being information. But the usage of Information and Communications Technologies has not only been restricted to the economy field; it has extended to all areas of society. Computers and electronic devices are found in almost all areas of human activity. Within this framework, datacentres have turned into a key element in society. It is easy to find datacentres in all economic sectors, as they provide the computational infrastructure for a wide range of applications and services, covering from the specific needs of social networks to the requirements of high performance computing. Moreover, concerns over the reliability of these datacentres, understood as a concern about the availability and security of these infrastructures, from the point of view of the backup of information, is gaining greater importance. The most valuable possession of companies is information; this means that datacentres are supposed to be always available, with all their servers up and running and all information appropriately secured. A loss of information or the lack of availability could result in high economic costs. In this paper, we will study the state-of-the art on energy efficiency of today s datacentres. In Section 2 we will Authors Marina Zapater-Sancho received a Degree in Telecommunication Engineering and a Degree in Electronic Engineering from the Universitat Politècnica de Catalunya, Spain, in 2010. Currently she is a PhD Candidate in the Electronic Engineering Dept. at the Universidad Politécnica de Madrid, Spain. She has been awarded a Research Grant by the Program for Attracting Talent (PICATA) of the Campus of International Excellence of Moncloa. Her research focuses on thermal optimization of complex heterogeneous systems, proactive and reactive thermal-aware optimization of datacentres and security of embedded systems. <marina@die.upm.es> Patricia Arroba-García received a Degree in Telecommunication Engineering from the Universidad Politécnica de Madrid, Spain, in 2011. Currently she is doing her PhD in the same University in the Electronic Engineering Dept. Her research field focuses on thermal optimization of complex heterogeneous systems and mobile accessibility development. <parroba@die.upm.es> José-Manuel Moya-Fernández is currently an Associate Professor in the Dept. of Electronic Engineering, Universidad Politécnica de Madrid, Spain He received his MSc and PhD in Telecommunication Engineering from the same University, in 1999 and 2003, respectively. He has participated in a large number of national research projects and bilateral projects with industry, in the fields of embedded system design and optimization, security optimization of embedded systems and distributed embedded systems. In these fields he is co-author of a number of publications in prestigious journals and conferences. His current research interests focus on proactive and reactive thermal-aware optimization of datacentres, and design techniques and tools for energy-efficient computeintensive embedded applications. <josem@die.upm.es> Zorana Bankovic got the title of Electrical Engineer from the Faculty of Electrical Engineering, University of Belgrade, Serbia, in 2005, and the PhD title from the Universidad Politécnica de Madrid, Spain, in 2011. Currently she is a researcher in the Dept. Electronic Engineering at the Universidad Politécnica de Madrid. Her main research interests include reputation systems and energy efficient intrusion detection for wireless sensor networks based on artificial intelligence techniques. <zorana@die.upm.es> Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 67

This paper makes a survey of the solutions proposed by researchers as well as the practices that real datacentres apply in order to increase the energy-efficiency of their facilities present the main problems, then we will study the present solutions from a dual perspective. Section 3 reviews the actual trends for energy efficiency in high-performance computing datacentres, whereas Section 4 expands that study towards datacentres for cloud computing. Both these sections will make a survey from the research point of view. Section 5 will study how many of the technological advances offered by research in this area have a real market penetration, and which are the real solutions implemented in today s datacentres. Finally, some conclusions about the actual gap between research and industry in this area are drawn in Section 6. 2 Problem Statement 2.1 The Energy Problem For decades, datacentres have focused on increasing their performance, defined only in terms of speed. Examples include the Top500 list of the world s fastest supercomputers [1], which calculates the speed metric as floating-point operations per second (flops). However, this speed increase has not come for free, as the energy efficiency has not improved as quickly. In 2007, although there had been a 10,000- fold increase in speed since 1992, performance per watt had only improved 300-fold and performance per square foot only 65-fold [2]. The significant growth in processing capabilities has lead to a severe rise in the energy consumption by these infrastructures, not only to increase computational power but also for to run more powerful cooling equipment. The energy consumption of datacentres has a global impact and in 2009 used the 2% of the world electric energy production [3]. The huge performance improvement is mainly due to increases in three different dimensions: the number of transistors per processor, each processor s operating frequency, and the number of processors in the system. However, technological trends further increase the power density, which was expected to reach 60 kw/m 2 by the year 2010 [4]. Collectively, these factors yield an exponential increase in power needs of datacentres that is not sustainable. The focus on just speed has let other evaluation metrics go unchecked. As a result, the total energy cost of the cooling infrastructure has increased dramatically in recent years. This evolution can be seen on Figure 1. In latest datacentres, the site infrastructure accounts for about 30% of the total energy cost. Therefore, the cooling cost is one of the major contributors of the total electricity bill of large datacentres, and another 10-15% is due to power distribution and conversion losses [6]. We can consider that one megawatt (MW) of power consumed by a supercomputer today typically requires another 0.7 MW of cooling to offset the heat generated. In 2006, datacentres in the U.S. used 59 billion KWh of electricity, costing the US $4.1 billion and 864 million metric tons of carbon dioxide (CO2) emissions; this accounted for 2% of the total USA energy budget, while it has been projected that it reached 3% by the year 2010 [7]. Figure 1: Electricity Use by End-Use Component. Taken from [5]. 68 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

Figure 2: ASHRAE Working Environment Recommendation. In 2010, for every $1 spent on hardware, 70 cents have been spent on power and cooling; and by 2012, for every $1 spent on hardware, $1 will be spent on power and cooling. The advent of cloud computing has lead to an even faster growth in the number of datacentre facilities. Datacentres are beginning to appear everywhere, and they are becoming hungrier in terms of energy. This immediately translates into an urge for datacentre owners to save costs on infrastructure i.e. to make their datacentres more efficient in terms of energy and servers. As we see next, the first urge has lead to the proliferation of green datacentres. The second one puts stress on the importance of increasing the lifetime of servers, which necessarily needs increased reliability. 2.2 The Move towards Green Datacentres All the above mentioned factors contribute to the rise of a general concern about the high energy consumption of datacentres. Today, other metrics such as the datacentre being in the Green500 list [8] are beginning to be of importance. To be in the Green500 list [9], datacentres report the FLOPS/watt metric by measuring the average power consumption when executing the LINPACK (HPL) benchmark [10]. Also, some reference companies such as Google and IBM are already implementing measures to make their datacentres more efficient, and begin to measure the Processor Usage Efficiency (PUE) of their datacenters as a means to be aware of their efficiency. The PUE can be calculated as the total power entering by the facility divided by the power needed to run the IT equipment within it. Also, organisms all around the world are publishing best practice documents on how to improve the energy efficiency of datacentres, and promote the creation of networks for datacentre owners and operators complying with this best practice. This is the case of the Code of Conduct on Data Centres Energy Efficiency published by the European Commission on 2008 [11]. This Code of Conduct takes into account both the IT load and the facilities load in terms of energy, and defines a metric for infrastructure efficiency, the Data centre infrastructure efficiency (DciE), which is a percentage measure of the main IT equipment energy consumption divided by the total facility energy consumption. Also, the US Department of Energy, through the Energy Star program, published a report on datacentre energy efficiency on 2007 [5], which is a compendium of data on the state-of-the-art on datacentre facilities in the US. Concern about the availability and security of these infrastructures, from the point of view of the backup of information, is gaining greater importance Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 69

Figure 3: Energy-efficient Strategies. [Source: Impact Lab.] 2.3 The Importance of Reliability It is important to note that energy savings can never be obtained with prejudice to reliability. Solutions to the energy problem must not suppose a decrease in the lifetime of servers or, at least, a decrease in their desired or medium lifetime. If datacentre owners need to change their equipment before they had expected to, then the costs saved in energy can be reduced, but there will be an increase due to the hardware costs. That is the reason why some organizations such as the ASHRAE [12] publish metrics on the maximum inlet air temperature for a server, the redline temperature, as well as for the appropriate temperature and humidity conditions of the environment, so ensure that reliability is not affected. These appropriate environmental conditions can be seen on the ASHRAE psychometric chart of Figure 2. There is a lot of discussion in literature about this topic, as many authors believe that for every 10ºC increase over 21ºC in the inlet temperature will decrease the long-term reliability of electronics by 50% [13]. This belief is based on the fact that, for a constant fan speed, the CPU temperature follows the inlet temperature and so, the higher the inlet temperature, the higher the CPU temperature. ASHRAE, however, finds that these studies have not a strong basis and allows higher inlet temperatures, up to an inlet temperature of 27ºC in their 2008 recommendations. As we will see in next section, however, even though this is a key aspect, there are other issues related to reliability that can be of utmost importance, and that are sometimes overlooked. 3 Energy Efficiency in High-Performance Computing Energy efficiency for High-Performance Computing (HPC) has traditionally centered its research in two different fields: solutions that reduce computing energy or power, some of them centered also in the reliability of circuits, and solutions to reduce the cooling infrastructure costs. Even though there should be no impediment for these two different ways to be merged into just one, the literature has mostly treated them as separate realities, or different, sometimes even independent, variables of the same problem. Only in some cases, we can find contributions that take both into account at the same time [14]. Figure 3 summarizes all the strategies regarding energyefficiency in HPC that are explained in the next subsection. 3.1 Energy Efficient Computing and Reliability The first solutions that take into account both reliability and energy efficiency are those that descend to the lowest level, making an approach from the circuit and chip level in the design state. This is a very common approach for reliability, as most of the effects that reduce the mean time The cooling cost is one of the major contributors of the total electricity bill of large datacentres, and another 10-15% is due to power distribution and conversion losses 70 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

It is important to note that energy savings can never be obtained with prejudice to reliability to failure, can be faced from this perspective. The effects that have been observed to have a very strong impact on reliability of systems and processor architectures are the following [15]: Electromigration (EM): an effect that appears due to the charge exchange between the electrons and the aluminum ions in long metal lines. Time-dependent dielectric-breakdown (TDDB): an important failure mechanism that models how the dielectric fails when a conductive path forms in the dielectric, shorting the anode and cathode. Stress migration (SM): describes the movement of metal atoms under the in?uence of mechanical-stress gradients. Stress migration may cause electrical failures due to the resistance rise associated with the void formation. Thermal cycling (TC): this effect produces a permanent damage that accumulates each time the device undergoes a normal power-up and power-down cycle. Even though all the effects are present in datacentres, the most common are EM, caused when hot spots appear in processors, or TC, as a result of uneven load balancing in servers. The reduction of hot-spots, apart from increasing reliability, lowers the cooling effort because not so high temperatures are achieved in the chip. That is why many approaches in literature are focused on the reduction of hotspots. This is the case with all the literature on temperatureaware floor-planning of cores (especially in the world of MPSoCs) [16][17][18], which is devoted to getting the optimum floorplan that reduces the hot-spots. Also, hot-spots can be reduced by means of temperature-aware task allocation and scheduling algorithms [19]. Even though, at a first glance, the world of MPSoCs seems far apart from the world of datacentres, the usage of embedded systems in HPC to allocate tasks that do not have hard time constraints is becoming more important by the day. Some papers focus on trying to find the most energyefficient datacentre blocks between machines from the embedded, mobile, desktop and server spaces [20]. Going up in the level of abstraction we find some common approaches for thermal-aware task scheduling at the operative system level, such as heat balancing or deferred execution of hot jobs [21]. This is the case with temperature-aware Greedy algorithms that prevents hot-spots and imbalanced temperature distributions, while ensuring that all servers inlet temperatures are greater than a threshold, increasing the priority to retirement when the condition is not accomplished [4]. However, it is at the server level where the most popular measures to reduce power consumption without impact on performance are found. These measures are those based on Dynamic Voltage Frequency Scaling (DVFS) [22] which is a power-aware algorithm that automatically and transparently adapts the voltage and frequency settings of servers, Vary-On Vary-Off (VOVF). and all techniques based on switching off idle machines, in order to save the costs of keeping on under-saturated machines [23]. At the highest level, the last two techniques can be used alone or jointly with power-aware scheduling strategies [24], even taking into account system load considerations or task characterization [25]. Efficient job allocation [26] by means of adaptive power-aware scheduling algorithms help to reduce peak and average power consumption. Less common approaches try to reduce power by means of fans, networking or IO operations. The newest approaches are beginning to take into account some capabilities that are mostly used in Cloud Computing, as we will see next, such as virtualization solving multi-objective scheduling to minimize energy in virtualized datacenters [27]. Virtualization enables the consolidation of multiple workloads in a smaller number of machines. So, even though it incurs some additional overheads, because of virtual machine creation and migration dynamic job scheduling policies still mean considerable energy reductions. 3.2 The Cooling Perspective Energy efficiency from the cooling perspective can exploit two different facts. On the one hand, cooling efficiency can take into account that there are some physical locations in the datacentre which are more efficient to cool than others. If heavier workloads are placed in the easier-to-cool places, scheduling is more efficient in terms of energy [7]. A refined version would be the one that studies which are the steady state hot-spots and cold-spots of the datacentre, and uses them to generate temperature-aware workload placement [13]. On the other hand, some approaches try to maximize the temperature of the air supplied by the air conditioning units, or to minimize the heat recirculation of the datacentre so that cooling is more efficient e.g. in [28] less tasks are assigned to chassis that are known to have more recirculation. Some algorithms are known to work well for these conditions and generate an energy efficient job scheduling. Some of the most common algorithms are [29]: Uniform Outlet Profile: that tries to make uniform the outlet temperatures of all servers. Minimal Computing Energy: which generates schedules that spend the minimum energy. Coolest Inlet: which distributes the workload first to the servers with lowest inlet temperature [30]. Finally, some task allocation algorithms try to balance Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 71

load, migrating tasks from thermally saturated cores to cooler neighbors [31]. Both approaches are, after all, based on taking into account the thermodynamic effects of the server room. To prove the abovementioned hypotheses, it is very common to use Computational Fluid Dynamics software, such as Mentor Graphics Flovent [32]. This software tool computes the specific performance of data rooms, and allows researchers to prove the results provided by the algorithms. Typically the datacentre is considered a warehouse-sized facility with several rows of server racks. The room is usually considered to be of the hot-aisle/cold-aisle style, that is, each row is sandwiched between a hot aisle and a cold aisle. The cold air is supplied by the air conditioning units through perforated tiles in the floor underneath the cold aisles. The servers suck the cold air coming from the cold aisle into the rack using chassis fans, and push out the heated air to the hot aisle. The hot air is extracted from the air-conditioning units at the room ceiling. 4 The Contribution of Cloud Computing The move towards cloud computing has given a new vision to the energy efficiency problem. The key current technology for energy-efficient operation of servers in cloud datacentres is virtualization. The cloud datacentre is, by definition, heterogeneous. Heterogeneity might be overlooked in datacentres for HPC, even though the real datacentres are almost always heterogeneous, due to the different versions of their hardware, but it must be taken into account in Cloud Computing datacentres. Virtualization is useful in the sense that it can abstract the heterogeneity of the servers and thus make allocation and migration easier. This however, cannot be by itself the only practice applied. Some solutions for Virtual-Machine based datacentres propose to investigate the resource demands characterization of each workload and the live migration of virtual machines without QoS degradation [33], finding the minimum energy allocation of workloads to servers, using heuristics that maximize the sum of Euclidean distances for current allocations or avoiding bottlenecks so that machines are never idle [34]. Cloud computing provides opportunities for the usage of turn off policies and optimization, or even re-factory, of network protocols in a way that enhances the energy-efficient operation of the network elements. As cloud computing is becoming of significant importance, the amount of data that is transferred over the Internet is increasing exponentially. In order to create green cloud computing datacentres, content replication and dissemination algorithms will then need to consider energy as a key parameter of optimal operation [35]. The problem of Best-Fitting, that is, the problem of assigning each VM to the resource where it will best behave from the energy-efficiency point of view, as well as the policies for migrating that VM somewhere else, is comprehensive in Cloud Computing. The most common algorithms to approach this problem are [36][37]: Minimization of Migrations (MM): which consists of migrating the least number of VMs to minimize migration overhead as this overhead has an associated energy consumption. Highest Potential Growth (HPG): which consists of migrating VMs that have the lowest usage of CPU in order to minimize total potential increase of the utilization and SLA violation. Random Choice (RC): which consists of choosing the necessary number of VMs by picking them according to a uniformly distributed random variable. Modified Best Fit Decreasing (MBFD): this algorithm consists of sorting all VMs in decreasing order depending on their current CPU utilizations, and allocating each VM to a host that provides the least increase in power consumption due to this allocation (choosing the most power-efficient nodes first). As it can be seen, the need for cloud computing to become green has lead to a proliferation of algorithms that take into account virtualization, heterogeneity and high network demands. This opens a new path in research, which could feed back some of the cloud solutions to improve the energy efficiency of HPC datacentres. 5 Real Solutions in Today s Datacentres First of all, it must be taken into account that, from the datacentre owner s point of view, their facilities are something more than just air conditioning units, racks and servers. Datacentres have many security measures such as fire detection alarms, fire suppression, smoke dampers, fresh air supply and emergency off buttons. They are also real production and working environments that cannot assume to integrate any new measure that can put at a stake their availability or reliability. All research solutions that aspire to be brought to market must keep in mind the security restrictions of datacentres and the need for the machines to be always ready to absorb the changing loads of a datacentre, ensuring that the stability of the machines will never be at stake. There are of course some measures that have already been applied to datacentres. Most of them, however, are more related with best practices than with innovative techniques of workload scheduling. It is true that datacentres, such as the those with the BlueGene computers from IBM, An important gap exists between the contributions of researchers and the products offered by industry 72 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

which are at the top of the Green500 list, or Google, are beginning to constantly measure their PUE, and have realized the benefits of having good airflow management and better isolation between hot and cold aisles. The trend of keeping the inlet temperature "at the lowest possible", however, continues to be a common practice, even though this is not a recommendation anymore and does not serve the cause of energy efficiency nor the reliability of servers. Only some of the most innovative companies, such as Google, are applying measures to make their datacentres Green so that they can save energy costs. Google announced in 2008 that their Container Datacentre [38] had an effective PUE of 1.25. This high efficiency is reached on the one hand by the use of heat exchangers, optimized power distributions, that increase the efficiency of transformers and UPS, and free cooling. On the other hand, Google pays close attention to the airflow management: their datacentres try to eliminate hot/cold air mixing by isolating the aisles. Their machines work in cold aisles with temperatures of 27ºC, that is, at the limits recommended by the ASHRAE organization in 2008. 6 Conclusions The reality of today s datacentre seems, at a first glance, far apart from the research on the field. The solutions given by University departments are far away both from commercial datacentres and from the solutions sold by industry. Most contributions assume unreal datacentres; smaller rooms than the real ones with severe hot/cold air mixing and recirculation. Even though that is the case with many datacentres, there are commercial solutions based on better airflow isolation that solve the problem. However, the really important contributions, that is regulation over air conditioning units and energy-efficient scheduling, has not yet been integrated in real datacentres. Some trials have been carried out in experimental datacentres, but these solutions have not yet been put into industrial use. The most energy efficient datacentres those that are in the first positions of Green500 list do not use automatic energy-aware scheduling algorithms nor automatic air conditioning supply temperature control. They just follow the best practices and recommendations of the abovementioned institutions. An important gap exists between the contributions of researchers and the products offered by industry. Even though some solutions seem ready for the market, innovation in this field has lots of barriers when arriving to real production environments. Companies must be very positive about the strength and reliability of the proposed solutions before they allow testing in their datacentres, as unstable solutions could lead to severe economic losses for datacentre owners. Acknowledgements This work was partially funded by the Spanish Ministry of Science and Innovation through the Secretariat of State for Research, under Research Grant AMILCAR TEC2009-14595-C02-01, through the General Secretariat of Innovation under Research Grant P8/08 within the National Plan for Scientific Research, Development and Technological Innovation 2008-2011 and the Campus of International Excellence (CEI) of Moncloa, under Research Grant of the Program for Attracting Talent (PICATA). References [1] TOP500 Supercomputing Sites, 2009. [Retrieved October 25, 2011, from <http://top500.org/>.] [2] W. Feng, K. Cameron. "The Green500 List: Encouraging Sustainable Supercomputing". Computer, 40(12), pp. 50-55, 2007. [3] J.M. Kaplan, W. Forrest, N. Kindler. "Revolutionizing Data Center Energy Efficiency". McKinsey & Company, July 2008. [4] E. Pakbaznia, M. Pedram. "Minimizing data center cooling and server power costs". Proceedings of the 14th ACM/IEEE international symposium on Low power electronics and design - ISLPED 09 (p. 145). New York, New York, USA: ACM Press, 2009. [5] U. D. Of Energy. "Report to Congress on Server and Datacentre Energy Efficiency". Public Law pp. 109-431, 2007. [6] C. Bash, G. Forman. "Cool job allocation: Measuring the power savings of placing jobs at cooling-efficient locations in the datacentre". 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference (pp. 17-22). USENIX Association, 2007. [7] T. Mukherjee, A. Banerjee, G. Varsamopoulos, S.K.S. Gupta, S Rungta. "Spatio-temporal thermal-aware job scheduling to minimize energy consumption in virtualized heterogeneous datacentres". Computer Networks, 53(17), pp. 2888-2904, 209. [8] The Green500 List. Environmentally Responsible Supercomputing. [Retrieved October 25, 2011, from <http://www.green500.org/>.] [9] B. Subramaniam, W. Feng. "Understanding Power Measurement Implications in the Green500 List". 2010 IEEE/ACM Int l Conference on Green Computing and Communications & Int l Conference on Cyber, Physical and Social Computing (pp. 245-251). IEEE, 2010. [10] Linpack HPL Benchmark for HPC. [Retrieved October 10, 2011, from <http://www.netlib.org/benchmark/ hpl/>.] [11] European Comission, European Union s Joint Research Centre (JRC). "Code of Conduct on Data Centres Energy Efficiency Version 1.0". October, pp.1-20, 2008. [12] American Society of Heating, Refrigerating and Air- Conditioning Engineers. "ASHRAE Environmental Guidelines for Datacom Equipment - Expanding the Recommend Environmental Envelope", 2008. [13] J. Moore, J. Chase, P. Ranganathan, R. Sharma. "Making scheduling cool : Temperature-aware workload placement in datacentres". USENIX Association Proceedings of the General Track: 2005 UNENIX Annual Novática CEPIS UPGRADE Vol. XII, No. 4, October 2011 73

Technical Conference, pp. 61-74, 2005. [14] A. Banerjee, T. Mukherjee, G. Varsamopoulos, E.K.S Gupta. Integrating Cooling "Awareness with Thermal Aware Workload Placement for HPC Datacentres". Impact Lab. Arizona State University, 2011. [15] D. Atienza, G. De Micheli, L. Benini, J.L. Ayala, P.G.D. Valle, M. DeBole, V. Narayanan. "Reliability-aware design for nanometer-scale devices". Proceedings of the 2008 Asia and South Pacific Design Automation Conference (pp. 549 554). IEEE Computer Society Press, 2008. [16] Y. Han, I. Koren, C. Moritz. "Temperature aware floorplanning". Proc of the Second Workshop on TemperatureAware Computer Systems, 2005. [17] Y. Han, I. Koren. "Simulated Annealing Based Temperature Aware Floorplanning". Journal of Low Power Electronics, 3(2), pp. 141-155, 2007. [18] W. Hung, Y. Xie, N. Vijaykrishnan, C. Addo-Quaye. "Thermal-aware floorplanning using genetic algorithms". International Symposium on Quality Electronic Design, 2005. [19] Y. Xie, W.L. Hung. "Temperature-aware task allocation and scheduling for embedded multiprocessor systems-on-chip (MPSoC) design". The Journal of VLSI Signal Processing, 45(3), pp. 177 189. Springer, 2006. [20] L. Keys, S. Rivoire, J.D. Davis. "The Search for Energy-Efficient Building Blocks for the Datacentre". Workshop on Energy-Efficient Design, 2007. [21] J. Choi, C.Y. Cher, H. Franke, H. Hamann, A. Weger, P. Bose. "Thermal-aware task scheduling at the system software level". Proceedings of the 2007 international symposium on Low power electronics and design (pp. 213 218). ACM New York, NY, USA, 2007. [22] C-H.Hsu, W-C. Feng. "A Power-Aware Run-Time System for High-Performance Computing". ACMIEEE SC 2005 Conference SC05, (November), 1-1. IEEE, 2005. [23] X. Zheng, Y. Cai. "Markov Model Based Power Management in Server Clusters". International Conference on Green Computing and Communications (pp. 96-102). IEEE, 2010. [24] X. Zhu, C., He, Y. Bi, D. Qiu. "Towards Adaptive Power-Aware Scheduling for Real-Time Tasks on DVS- Enabled Heterogeneous Clusters". 2010 IEEE/ACM Int l Conference on Green Computing and Communications & Int l Conference on Cyber, Physical and Social Computing (pp. 117-124). IEEE, 2010. [25] W. Felter, K. Rajamani, T. Keller, C. Rusu. "A performance-conserving approach for reducing peak power consumption in server systems". Proceedings of the 19th annual international conference on Supercomputing ICS 05, pp. 293-302. ACM Press, 2005. [26] C. Bodenstein, G. Schryen, D. Neumann. "Reducing Datacentre Energy Usage Through Efficient Job Allocation". ECIS 2011 Proceedings. Brill, K. G. (2007). [27] J. Xu, J.A.B. Fortes. "Multi-Objective Virtual Machine Placement in Virtualized Datacentre Environments". 2010 IEEE/ACM Int l Conference on Green Computing and Communications & Int l Conference on Cyber, Physical and Social Computing (pp. 179-188), 2010. [28] Q. Tang, E.K.S. Gupta, G. Varsamopoulos. "Thermalaware task scheduling for datacentres through minimizing heat recirculation". In IEEE Cluster, 2007. [29] S.K.S. Gupta, G. Varsamopoulos. "Energy-Efficient Thermal-Aware Task Scheduling for Homogeneous High-Performance Computing Datacentres: A Cyber- Physical Approach". IEEE Transactions on Parallel and Distributed Systems, 19(11), pp. 1458-1472, 2008. [30] G. Varsamopoulos, A. Banerjee, S.K.S. Gupta. "Energy efficiency of thermal-aware job scheduling algorithms under various cooling models". Contemporary Computing, pp. 568 580. Springer, 2009. [31] G. Wu, Z. Xu, Q. Xia, J. Ren, F. Xia. "Task Allocation and Migration Algorithm for Temperature-Constrained Real-Time Multi-Core Systems". 2010 IEEE/ACM Int l Conference on Green Computing and Communications & Int l Conference on Cyber, Physical and Social Computing (pp. 189-196). IEEE, 2010. [32] Flomerics Ltd. "Flovent version 2.1," Hampton Court, Surrey, KT8 9HH, England, 1999. Available at <http:/ /www.flomerics.com/>. [33] K. Ye, D. Huang, X. Jiang, H. Chen, S. Wu. "Virtual Machine Based Energy-Efficient Datacentre Architecture for Cloud Computing: A Performance Perspective". 2010 IEEE/ACM Int l Conference on Green Computing and Communications & Int l Conference on Cyber, Physical and Social Computing, pp. 171-178. IEEE, 2010. [34] S. Srikantaiah, A. Kansal, F. Zhao. "Energy aware consolidation for cloud computing", 2008. [35] A. Berl, E. Gelenbe, M. Di Girolamo, G. Giuliani, H. De Meer, M.Q. Dang, K. Pentikousis. "Energy-Efficient Cloud Computing". The Computer Journal, 53(7), pp. 1045-1051, 2009. [36] A. Beloglazov, J. Abawajy, R. Buyya. "Energy-aware resource allocation heuristics for efficient management of datacentres for Cloud computing". Future Generation Computer Systems, 2011. [37] R. Buyya, A. Beloglazov, J.Abawajy (n.d.)." Energy- Efficient Management of Datacentre Resources for Cloud Computing: A Vision, Architectural Elements, and Open Challenges". [38] Official Google Blog: Designing lean, green, energysaving machines. (n.d.). [Retrieved October 7, 2011, from <http://googleblog.blogspot.com/2009/04/designing-lean-green-energy-saving.html>.] 74 CEPIS UPGRADE Vol. XII, No. 4, October 2011 Novática

UPENET IT for Education IT in Schools. A European Project for Teachers Training Pierfranco Ravotto and Giovanni Fulantelli Mondo Digitale, 2010 This paper was first published, in its original Italian version, under the title "Informatica nella Scuola: Un progetto europeo per formare i docenti", by Mondo Digitale (issue no. 4, December 2011, pp. 87-93, available at <http://www.mondodigitale.net/>). Mondo Digitale, a founding member of UPENET, is the digital journal of the CEPIS Italian society AICA (Associazione Italiana per l Informatica ed il Calcolo Automatico, <http://www.aicanet.it/>.) In a context in which both a competence-based educational approach and virtual environments are spreading and the digital competence has been set up up by the European Parliament and the Council as one of the 8 key-competences for lifelong learning, this article aims to teach teachers "what" and "how" to teach. The experience described is taking place in Italy, Romania and Slovenia under the European project Sloop2desc, financed by the European Union in the framework of the Lifelong Learning Programme. Keywords: Competence-based Educational approach, Digital Competences, ECDL, EU Lifelong Learning Programme, EUCIP, Sloop2desc, Teachers Training, Virtual Classroom. 1 Introduction The digital competence is one of the 8 key-competences for lifelong learning that the European Parliament and the Council states to be "necessary for personal fulfilment, active citizenship, social cohesion and employability in a knowledge society" [1]. Enabling students to acquire digital competence is one of the objectives that schools need pursue. That means both competences as digital users and professional competences in designing, producing and maintaining software and digital services. This theme entwines with the one of using digital environments and tools in teaching and learning activities. This article aims to explore "what" Authors Pierfranco Ravotto graduated in electronic engineering at the Politecnico in Milan, Italy, in 1974. He taught electro technology, electronics, automation and organizational systems in upper secondary schools and was involved in post-diploma courses, workbased learning projects, student exchanges abroad and elearning. He has coordinated various European projects among which the SLOOP Project - under Leonardo da Vinci programme and has participated in the design and setting up of teachers and educational managers training courses. He has been Project Manager of the Ensemble project for the Department of Science of Education at the University of Florence (2008-2010). Since 2008 he has been collaborating with the Italian CEPIS society AICA (Associazione Italiana per l Informatica ed il Calcolo Automatico). He is a member of the Milanese board of AICA and of the national executive board of the Sie-L, Società Italiana e-learning. He is a certified EUCIP IT Trainer. <p.ravotto@aicanet.it> Giovanni Fulantelli is a researcher at the Institute for Didactic Technologies (ITD) at the Italian National Research Council (CNR); he studies the use of new technologies to support learning. He is the coordinator of the European project "Sloop2desc - Sharing Learning Objects in an Open Perspective to Develop European Skills and Competences", and is in charge of the CNR research work as for "Multiculturalism and the school: methodologies and technologies to support learning and integration". He is also a member of the ITD committee. In 2005 he was awarded scientific recognition for innovative results of particular excellence and strategic relevance in the sector of didactic technologies. <giovanni.fulantelli@itd.cnr.it> and "how" to teach in a period in which both a competence-based educational approach and virtual environments are spreading. CEPIS CEPIS UPGRADE Vol. XII, No. 4, October 2011 75

UPENET The experience described in this article is taking place in Italy, Romania and Slovenia under the European project Sloop2desc The experience described is the one of the European project Sloop2desc, financed by the EU in the framework of the Lifelong Learning Programme. 2 Competence-based Education The old school view was, and to a large extent it is still like that, the one of a "syllabus" intended as a list of content to be transmitted to students. From this point of view technological changes often led to a mere substitution or addition of content to the list. Now there is a trend to substitute the old approach to syllabuses-lists of content with a competence-based one, as demanded by the labour market and supported by the European Union. According to the EU Commission, " 'competence' means the proven ability to use knowledge, skills and personal, social and/or methodological abilities, in work or study situations and in professional and personal development". Obviously, for those involved in education, it is necessary to define: "Knowledge": "the outcome of the assimilation of information through learning. Knowledge is the body of facts, principles, theories and practices that is related to a field of work or study". "Skills": "the ability to apply knowledge and use know-how to complete tasks and solve problems" [2]. But knowledge and skills cannot lead to a new list of content to be transmitted more or less in sequence. The teacher s task is to enable students to acquire them in a practical context, because it is in this way that competences develop. It is not a matter of adapting learning to technological changes by adding items to a list of content, but rather of designing practical contexts in which students acquire the relevant knowledge and skills. A competence-based approach requires schools and teachers to abandon selfreferencing and assume external points of reference which at IT level can be represented by CEPIS certification and its related syllabuses. As far as Upper Secondary Schools are concerned, ECDL with all its articulations represents the obvious reference framework: ECDL Core, Advanced ECDL - spreadsheet for mathematics, physics and business economics; database for business administration, ECDL4PS (for Problem Solving) for mathematics, physics and business administration; ECDL CAD for all subjects involving design; ECDL Health for biology, and microbiology, EUCIP 1 can instead be the reference framework for IT vocational courses, in particular EUCIP Core and IT Administrator certifications can be acquired. 1 EUCIP, the European Certification of Informatics Professionals, is a framework of competences with a related certification system developed by CEPIS and managed, in the case of Italy, by AICA. The EUCIP Syllabus is based around 21 professional profiles characterized by a common area of knowledge and abilities, the core, and a twenty second profile: the IT Administrator. The core Syllabus and the IT Administrator are the profiles which correspond most closely to the competences acquired by the end of IT courses at secondary school. 3 New Environments and Learning Styles Marc Prensky first made a distinction in 2001 [3] between digital natives and immigrants, or rather between students who are growing up in an environment rich in digital technologies and their teachers. It is unthinkable today to suggest that students should study at school without the communication tools which they are used to, or learn in an environment different from the one in which they "live, build and exchange meaningful knowledge" [4]. It is not simply a question of habits to be met, but also of learning styles: many people think that the early cognitive experiences of this generation have determined a change in their brain structures. As IT teachers are by definition the ones who should have a better knowledge of the new learning and communication environments originated by the use of ICT, they could play the role of promoting an ICT-based learning which takes place in the new digital environment where the young people are "always connected". However, this does not happen systematically, and, sometimes, teachers of other subjects are more inclined to use elearning, blogs, wikis, podcasts, mobile phones, etc. In most cases the teaching of ICT and the use of the computer takes place in a traditional manner: lessons, accompanied by a lot of laboratory work, but within the old approach according to which interaction occurs between teachers and classmates in the classrooms and informatics laboratory, while at home the students are "unconnected". Instead it is time now of mobile learning, of an interaction between teachers and students and within the peer groups Now there is a trend to substitute the old approach to syllabuses-lists of content with a competence-based one, as demanded by the labour market and supported by the European Union 76 CEPIS UPGRADE Vol. XII, No. 4, October 2011 CEPIS

UPENET Table 1: Sloop2desc Courses Organisation. which, thanks to the Internet and mobile devices, can take place outside school time and beyond the walls of the school. The way of teaching, or rather, of providing opportunities for learning, is destined to change. 4 The Sloop2desc Project Sloop2desc is a European co-financed project 2 which concerns with both "what" and "how" to teach and which particularly addresses teachers of IT and related subjects, those, in fact, who are to prepare IT professionals, or at least expert users, of the future. Sloop2desc focuses on teachers training as far as the following two goals are concerned: competence-based learning, the use of the Internet and Web 2.0 tools to integrate face-to-face and online learning. The Sloop2desc courses, "To design and develop online courses based on competence-oriented education" are elearning training projects which last 2 This is a TOI (Transfer of Innovation) project, financed in the LLP, Leonardo da Vinci programme (2009). It is promoted by CNR-ITD in Palermo. The partnership is made up of Italian institutions - CNR-ITD (promoter), AICA, Metid-Politecnico of Milano, ITSOS "Marie Curie" of Cernusco sul Naviglio, IIS "Danilo Dolci" of Palermo and the Consortium "Med Europe Export" - Irish partners, DEIS, Department of the Cork Institute of Technology, Romanians, University of Galati, and Slovenians, Ljubljana University and Informatika. The two-year project began in October 2009. The acronym recalls one of the previous project, SLOOP from Sharing Learning Objects in an Open Perspective, with the "2" which indicates a second phase of SLOOP but also sounds like to : 2desc means TO Develop European Skills and Competences. 16 weeks and are divided into 5 modules, as shown in Table 1. It is important to be clear about the term elearning. It often refers to courses, which are mainly based on a self-study approach: a set of learning materials which trainees study on their own. In the most "primitive" versions the learning resources were simple PDFs to download or web pages consisting of texts and images. In more up-to-date versions there are audio-video presentations, films, simulations, highly interactive texts. A trainees is basically on his own while learning, even though there are often tutors on hand to help if needed, and some forums where students can exchange opinions and information. Our model focuses instead on collaboration in a "virtual classroom", where the classroom represents an environment in which interactions be- CEPIS CEPIS UPGRADE Vol. XII, No. 4, October 2011 77

UPENET As far as Upper Secondary Schools are concerned, ECDL with all its articulations represents the obvious reference framework tween teachers and students and between the students themselves develop, as it would happen in a face-to-face dimension. Module 1 is dedicated to the MOODLE 3 learning environment which is used in the course and which teachers will use with their students. Here the trainees are required to learn its various functions, first as a trainee user of Moodle, and then as a teacher able to create courses and to fill them with learning resources and activities (exercises, forums, etc.). In this module, individual activities (getting familiar with Moodle tools) are central and cooperative activities consist "only" in discussion within forums (questions to teachers/tutors as well as to other trainees, answers and suggestions to "classmates"). In one of the two pilot courses in the Module 1 forum, about thirty trainees with two tutors started 18 discussions and posted 352 messages in two weeks! A similar phenomenon occurred in the other course. This clarifies what we mean by "courses based on interactions". Module 2 instead focuses on online tutoring and the use of Web 2.0 tools 3 MOODLE is the most popular Learning Management System or online learning environment. In the Moodle environment it is possible to create courses consisting of resources (written texts, web pages, links, etc) and activities (forums, interactive lessons, exercises, quizzes, wikis, SCORM objects, questionnaires, etc.). There are different levels of authorization (roles): administrator, course designer, teacher, not editor teacher, students, hosts. The teacher can divide students into groups and monitor their activities. The course can easily be duplicated for different classes or exported to another Moodle platform. for online teaching and learning. The trainees, divided into groups, are required to analyze some Web 2.0 tools and to prepare - using googledoc and a wiki - a description of them accompanied by some activities to be proposed to other trainees. This collaboration no longer involves discussions but "doing/acting together". In the experience of this module, the 10 discussions opened in the forum with 439 messages were only the tip of an iceberg of the communication process occurred among the trainees. The various groups carried out their collaborative activities using e- mail, Internet chatting, Skype and mobile phones! Module 3, like the first module, is based again on individual exercises, in this case concerning the acquisition of technical competences for producing learning resources using, for example, SlideShare to publish a presentation and also adding a voice comment or exelearning to produce SCORMs. Such activities are carried out individually but once again within a virtual classroom situation: trainees can ask questions, provide answers, request clarification and give suggestions: 13 discussions in Module 3 forum with 381 posts. Module 4 focuses on competences, in particular EQF 4, e-cf 5 and EUCIP. This module should have been based 4 EQF, European Qualification Framework, is the European document which aims to make different national qualifications more readable across Europe through a common definition of levels. 5 E-CF, European e-competence Framework, is a reference framework for ICT drawn up by CEN (www.ecompetences.eu). on discussion, but in fact there was less discussion than in other modules: 119 messages in 10 discussions. There are a number of possible reasons: several participants arriving late at this phase preferred directly entered the fifth Module where they took part the discussion on EUCIP syllabus, which was supposed to be used as a guide for as a guide for the development of resources. the topic was new for many participants and consequently they had little to say on the forums; not everyone taught IT professional courses and so they were not interested in e-cf or EUCIP. For these reasons, in the revised version for cascade courses, we introduced other parts regarding the "8 key competences" and the ECDL family. Module 5 is the most important and challenging: it is here that what has been studied previously is put into practice; it is not a coincidence that it lasts 6 weeks and that many participants have chosen to keep on working beyond the official end of the course. The participants are asked to form groups on the basis of their teaching subjects and classes, and to choose items from the EUCIP Syllabus (or, in the case of cascade courses, concerning 8 key competences or ECDL or EUCIP) and to produce related learning resources single resources (learning object) and whole courses - for their students. In this module there are no further didactic material to study as all the activities are based on the collaborative production of material and courses for the students. Also in this case, interaction between trainees have occurred only to some extent in the two module forums 34 discussions with 499 posts 6 since the trainees have interacted by Skype or other tools. The outcomes of the two pilot courses are: 6 To complete the overview of interactions it is necessary to add that there was also a general discussion forum: 42 discussions with 347 posts. 78 CEPIS UPGRADE Vol. XII, No. 4, October 2011 CEPIS

UPENET As far as Upper Secondary Schools are concerned, ECDL with all its articulations represents the obvious reference framework 15 SCORM packages related to Informatics content, 5 videos related to Informatics content, various learning resources for other subjects (in SCORM, doc, pdf and other formats); 4 courses on the Moodle platform: a course for IT Administrator, Module 1, Hardware (almost complete); a course on the topic "Network layer", item 4.5 of Module 4, Expert use of the networks, of IT Administrator; a course on databases, the Build area of EUCIP Core, item B2; a course on "Educational uses of the Web 2.0 tools". The two pilot courses in Italy were carried out from February to June (with an extension in July) and were attended by sixty teachers who were not all IT teachers. 20 teachers were selected to work in pairs as tutors in 10 cascade courses, which began in November 2010 and involved about 500 teachers. An eleventh course, for teachers of "business administration ", who are potentially interested in the Plan area of the EUCIP Core, started in December 2010. In November another two pilot courses began in Slovenia and Romania. The Slovenian course focused on the EUCIP Syllabus, while the Romanian one used a modified version of Module 4 referring to a competence system in the maritime transport sector. 5 Conclusions Sloop2desc has entered its second year with a result that we can consider outstanding. We needed to find 400 Italian teachers willing to undertake a challenging course, which does not provide any credits for career promotion. We received more than 1,700 applications! It is a sign of teachers interest in the course topics: the use of digital technologies in teaching and competence-based systems. It is a positive sign for the school in our country. At the moment we have provided courses only for some of those who have applied, but we will do our utmost to involve teachers of other subjects, whose interest shows that there is a potential for opening up Italian schools to the use of digital technologies and for innovating the educational system. The cascade courses, which have just started, will seek to achieve three objectives: To spread a model of elearning based on the idea of the Internet as a place for collaborative knowledge construction. We hope that the teachers involved in the courses will transfer this model of elearning to their class activities, integrating it with face-to-face lessons. The use of the Internet allows students to access learning resources which are varied, interactive, available at any time and suitable to their personal style of learning, but and particularly useful for extending the relationship with the teacher and the peer group beyond the bounds of the classroom and timetable. Moreover, when communication tools and digital environments are embedded in education, all the students can be actively involved and interact much more so than in traditional faceto-face activities when the number of students and the little time available 7 FreeLOms, Free Learning Objects management system, is the OER repository produced in a first version during the SLOOP project and now renovated in the Sloop2desc project. limits the number of questions which can be asked, the expression of different points of view, the connections, digressions and interactions, or rather all those processes on which knowledge building is based. To produce collections of Open Educational Resources OER, which any teacher can use to organize learning paths and environments for students. By "educational resources" we mean any educational material in digital form, from a whole course to single objects: lessons with texts and images, or in audio or audio-video form, simulations, tests, work proposals, etc. By the term "open" we mean three aspects: accessibility, modifiability and permissions. As far as permissions are concerned, the resources developed cannot be copyrighted with "all rights reserved" but must be left either in the "public domain" or under a licence like Creative Commons Attribution-Share alike, which guarantees freedom of use, distribution and modification. In order to make resources modifiable, access must be granted, where necessary, to their sources. To be accessible, the resources must be uploaded into a repository and must be easily traceable. In module 5 of the courses, the participants are invited to start using resources already developed in the pilot courses and stored in the FreeLOms 7, and then to improve and integrate them producing new ones. This is particularly relevant for IT Administrator and EUCIP core syllabuses. The setting up of a repository of open educational resources referring to these syllabuses, can really help teachers to design and set up learning environments for their students where they can combine competence-based education with new learning models. We have mentioned single learning objects and whole courses, but some CEPIS CEPIS UPGRADE Vol. XII, No. 4, October 2011 79

UPENET We needed to find 400 Italian teachers willing to undertake a challenging course, which does not provide any credits for career promotion. We received more than 1,700 applications! clarification is necessary. The syllabuses, which we have adopted as references, provide a detailed description of the knowledge and skills, which are at the basis of the required competences. An ideal repository should contain resources that correspond to each single item of knowledge and to each single skill (indeed, it would be better to have several resources for each item or skill, so that the teacher, or the student directly, has a choice). We therefore expect that in the Sloop2desc courses the teachers will produce resources with this extent of definition. But a competence is not simply the sum of knowledge and skills. It is up to the teacher to propose learning paths that offer collaborative activities to the students during which they acquire these skills and knowledge by means of a "learning by doing" approach. This is what we mean by "course". We expect the teachers of Sloop2desc to use the resources produced by themselves and by others to construct courses. In fact this collaborative activity, even more than the production of individual resources, is to be recommended. The courses can and must also be shared with the same characteristics of openness. To create a Community of Practice of IT teachers. The learning communities, which have been created in the pilot and in the cascade courses, and the discussion forums in the Sloop2desc site, already constitute the embryo of such a community of practice. The habit of collaborating in designing OERs, sharing them in repositories, modifying and using them will be another key element. When we concluded the previous SLOOP project we wrote that it fitted into an "idea of the web as a space in which people interact, collaborate, produce new knowledge together" [5]. To reach their students via the Internet and to use the Web in their teaching, teachers must acquire the habit of collaboration and sharing in a community, which is typical of Web 2.0. References [1] Recommendation of the European Parliament and of the Council of 18 December 2006 on key competences for lifelong learning (2006/962/EC). [2] EQF, European Qualification Framework for lifelong learning, <http://ec.europa.eu/education/- lifelong-learning-policy/ doc44_en.htm>. [3] Prensky M.: Digital natives, digital immigrants. MCB University Press, Vol. 9, n. 5, 2001. [4] Ardizzone P., Rivoltella P.C.: Media e tecnologie per la didattica (ISBN 978-88-343-1590-3). Vita e pensiero, 2008. [5] Ravotto P., Fulantelli G.: The SLOOP idea: sharing free/open learning objects in Sharing Learning Objects in an Open Perspective (ISBN 978-88-903115-0-5), 2007. Webgraphy SLOOP project site: <http://www. sloopproject.eu>. Sloop2desc project site: <http:// www.sloop2desc.eu>. FreeLOms: <http://freeloms2.pa. itd.cnr.it/xmlui>. EUCIP site: <http://www.eucip. org>. ECDL site: <http://www.ecdl. org>. e-cf site: <http://www. ecompetences. eu/>. 80 CEPIS UPGRADE Vol. XII, No. 4, October 2011 CEPIS

CEPIS News CEPIS Projects Selected CEPIS News Fiona Fanning New Research Shows that Future IT Professionals Need Career Paths and More IT-Focused Training to Give EU Competitive Edge A worrying picture of Europe s future emerges from new research showing that as the population ages and the demand for skilled IT professionals increases, the flow of fresh, young talent into the field of IT is sadly lacking. The lack of career paths damage the attractiveness of the IT profession for new entrants, while continuous professional development is needed for established professionals. Europe s chances at becoming a smart, sustainable and inclusive economy are severely threatened by the lack of Europeans with the right skills. As ICT pervades all industries, companies rely increasingly on technology for productivity and to compete globally. Yet the cost of IT project failures is estimated at 4.5 trillion worldwide, and with over half of IT projects running over budget - IT professionals are inextricably linked with Europe s ability to thrive and compete globally. Until now there has been little information on what e-competences are currently held by IT professionals in Europe. The Council of European Professional Informatics Societies (CEPIS) has carried out pioneering research into the actual e-competences of IT professionals across Europe and has just published the European report. The findings show that European IT professionals are ageing with a lack of young talent entering the IT profession; there is a clear need for further initiatives to promote the IT profession among young people. Further, only one out of every six IT professionals who responded is female, indicating that all countries need to urgently address this gender imbalance and increase the participation of women in IT careers. Overall only one fifth of IT professionals who participated in the research actually had the right e-competences for their IT career profile. The report also shows that a significant proportion of IT professionals do not have an IT-focused educational background and as a result continuous professional development for IT professionals is needed, through the completion of IT certifications for example. Developing and identifying the e- competences of IT professionals in Europe can help towards better matching the skills of our labour market to future jobs. More importantly knowing the e- competences of IT professionals can enable European employers, industry, policymakers, and educators to develop and implement a vision to manage the mismatches and shortfalls that hinder Europe s competitiveness and productivity. Almost 2000 IT professionals in 28 countries across greater Europe participated in the CEPIS Professional e-competence Survey. The European report by CEPIS compares all countries involved in the survey and creates a picture of the overall status of e-competences in Europe at the moment. The survey is a webbased self-assessment tool, based on the European e-competence Framework (e- CF) and uses 18 IT career profiles recognised by the labour market. The European report and the Executive Summary Brochure are available at: <https://www.cepis.org/professionale competence>. Greek CEPIS Member Society to Host Digital Trends 2011 The Hellenic Professionals Informatics Society (HePIS), and CEPIS are hosting Digital Trends 2011, the first forum in Greece to encourage dialogue on the contribution of the ICT sector to economic growth, increased productivity, the advancement of a creative digital culture, the adoption of Green and Cloud practices and the strengthening of the overall role of industry professionals within the business community. The event will take place on December 5 in Athens, Greece. All CEPIS members and CEPIS UPGRADE readers are invited to participate. You can find more about this conference at: <http://www.digitaltrends.gr/ default.asp?la=2>. Digital Agenda for Europe is Going Local Events in Every Country in Europe The European Commission has announced the second round of Digital Agenda Going Local. Following on from 2010, a series of meetings are being carried out in all Member States and other European countries. Some of the events announced by the European Commission have already taken place in several European countries, including Romania, Denmark, Poland and the Czech Republic. These events focused on IT security, and e-government among many other topics related to the Digital Agenda for Europe (DAE) such as innovation for digital inclusion. For all European citizens involved in DAE actions, this is an opportunity to influence your national environment and promote your digital agenda actions. The Digital Agenda Going Local initiative aims to: Report on progress of the DAE Identify challenges for the future Stimulate actions and commitments If you would like to get involved in the Digital Agenda Going Local, or if you wish to find out more about the events taking place in your country, please click here: <http://ec.europa.eu/information_ society/digital-agenda/local/index_ en.htm>. CEPIS CEPIS UPGRADE Vol. XII, No. 4, October 2011 81