SEVENTH FRAMEWORK PROGRAMME Integrated Infrastructure Initiative projects (I3)

Transcription

1 SEVENTH FRAMEWORK PROGRAMME Integrated Infrastructure Initiative projects (I3) INFRA : Simulation Software and Services ScalaLife Scalable Software Services for Life Science Grant Agreement Number: RI D5.4 Report on Knowledge Base Final Version: 1.0 Author(s): Rossen Apostolov (KTH) Date: August 18, 2012

2 Project and Deliverable Information Sheet ScalaLife Project Project Ref. : RI Project Web Site: Deliverable ID: D5.4 Deliverable Nature: DOC_TYPE: Report Deliverable Level: PU* EC Project Officer: K. Aniyan C. Varghese Contractual Date of Delivery: 31 / 08 / 2012 Actual Date of Delivery: 18 / 08 / 2012 * - The dissemination level are indicated as follows: PU Public, PP Restricted to other participants (including the Commission Services), RE Restricted to a group specified by the consortium (including the Commission Services). CO Confidential, only for members of the consortium (including the Commission Services). Document Control Sheet Document Authorship Title: Report on Knowledge Base ID: D5.4 Version: 1.0 Status: Final Available at: File(s): ScalaLife-D5.4-final.pdf Written by: Rossen Apostolov (KTH) Contributors: Mihai Duta (OeRC) Reviewed by: Ferdinand Jamitzky (LRZ) Berk Hess (KTH) Approved by: PMB Document Status Sheet Version Date Status Comments May 2012 Draft First draft June 2012 Draft Updated section 4 Future Jan 2012 Draft Implemented comments Name Surname Jan 2012 Draft Version for internal review Jul 2012 Draft Ready for PMB review Aug 2012 Final Submitted to EC ScalaLife - RI i

3 Document Keywords and Abstract Keywords: Abstract: Life Science, knowledge base, competence center In this document we summarize the status of the knowledge base, how it has been populated and used so far and derive recommendations for the final year of the project. Copyright notices 2012 ScalaLife. All rights reserved. This document is a document of the ScalaLife project. All contents are reserved by default and may not be disclosed to third parties without the written consent of the ScalaLife partners, except as mandated by the European Commission contract RI for reviewing and dissemination purposes. All trademarks and other rights on third party products mentioned in this document are acknowledged as own by the respective holders. ScalaLife - RI ii

4 Table of Contents Project and Deliverable Information Sheet Document Control Sheet Document Status Sheet Document Keywords and Abstract Table of Contents List of Figures List of Tables List of Acronyms and Abbreviations Executive Summary i i i ii iii iv iv v 1 1. Introduction 2 2. Knowledge base platform and structure 2 3. Population workflow Software development (WP3) Code analysis and profiling (WP6) Benchmarking and validation (WP4) File Formats and APIs (WP7) 5 4. Knowledge base contents and its usage (web traffic) 5 5. Future work and prospects 11 ScalaLife - RI iii

5 List of Figures Figure 1. Competence Center landing page Figure 2. Number of monthly visits to the web portal Figure 3. Number of unique monthly visitors to the web portal Figure 4. Number of daily visitors in April and May Figure 5. Visits per country List of Tables Table 1. Number of pages in main categories Table 2. Search engine keywords. Most of the traffic leading to the web portal comes from searches for GROMACS Table 3. Structure and access statistics of the knowledge base Table 4. Downloads of the pilot packages Table 5. Knowledge base population plan ScalaLife - RI iv

6 List of Acronyms and Abbreviations BSC CA CC DALTON DISCRETE DoW EAC EB EC EGI GA GROMACS HPC IRB KTH LRZ PD PM PMB PO PRACE ScalaLife SNIC SYN ToC UOXF.E9 WP Barcelona Supercomputing Centre, Spain Consortium Agreement Competence Center Molecular electronic structure; application Discrete Dynamics simulations; application Description of Work (Technical Annex) External Advisory Committee Executive Board European Commission European Grid Initiative Grant Agreement Versatile package to perform molecular dynamics; application High Performance Computing; Computing at a high performance level at any given time; often synonymous with supercomputing. Institute for Research in BioMedicine, Spain Royal Institute of Technology, Sweden Leibniz-Rechenzentrum, Germany Project Director Person Month Project Management Board Project Office Partnership for Advanced Computing in Europe; project acronym Scalable Software Services for Life Science; project acronym Swedish National Infrastructure for Computing, Sweden Synective Labs, Sweden Table of Contents Oxford e-research Centre, United Kingdom ScalaLife Work Package ScalaLife - RI v

7 Executive Summary A main component of the Competence Center pilot is the knowledge base, which contains the best practices and experiences of the developers as gathered during the lifetime of the ScalaLife project. The current deliverable presents a report on the structure, population status and usage statistics of the knowledgebase. The knowledgebase shares the same content management platform Drupal ( as the project portal. The information is organized in top-level sections about each of the codes and sections on the different types of work done in the project. These are code analysis and development (WP6 and WP3), performance benchmarking (WP4), and file format and APIs (WP7). Hardware resources and development tools are also presented. Furthermore the web portal includes sections with information about the organized training events, presentation of the user community and the help desk for expert support. The knowledge base contains roughly 150 pages of material: general information (nonapplication specific) (64) and information on ScalaLife software: GROMACS (35), DALTON (15), DISCRETE (13), ERGO (9), MUSIC (8) and XMIPP (7). Some pages contain links to external information sources as well. An outline of the structure can be found at The web portal is well linked from large external websites such as those of PRACE, EGI, GROMACS etc., which leads to its high Page rank, and comes at the top of many specialized term searches. This brings a lot of traffic from countries such as the USA where GROMACS (the most well known application from the pilot codes) is widely used. The portal receives on average 40 unique users per day, 80 unique users per month for an average of 3500 monthly visits since the launch in September Analysis of the web traffic also suggests that inclusion of famous packages such as GROMACS and DALTON stimulates interest in users to explore the lesser-known codes in the center. Our overall experience is that the knowledge base is of great value for the users. However, rapid developments in the whole IT sphere (both hardware and software) nowadays require major efforts for continuous maintenance of such structures. Researchers and engineers are also rather reluctant to invest time in maintenance of paper work when it does not bring an obvious positive effect on their work. Efforts based on good will may not be a sustainable power source. Based on that we would recommend that (1) fast transfer of knowledge can be achieved by successfully encouraging close interactions between software developers and its users which can be done through the establishment and promotion of long-term Centers Of Excellence, and (2) future EU funded projects on software development include a requirement for participation in those COEs for collection of the gathered experience and knowledge. The ScalaLife Competence Center is an excellent prototype that could potentially evolve into a long-term COE structure. ScalaLife - RI

8 1. Introduction One of the main goals of the ScalaLife project and WP5 is to prototype a Life Science Software Competence Centre, which will act as a one-stop-shop for groups seeking advice and help with life science software. A main component of the Competence Center is the knowledge base where we capture and make available to the wide community the experiences of the partners during the lifetime of the project. Apart from providing documentation and best practices, WP5 provides through the Competence Center expert advise by people who have worked intensely with the pilot codes of the project in different work packages. We have launched the Competence Centre pilot and currently have three external applications in addition to the pilot codes. In the current document we will refer to the ScalaLife project website as the web portal which includes both the project specific section The ScalaLife Project and the pilot Competence Center. Knowledge base refers to sub-sections of the Competence Center which include the gathered experiences and information through the project work. In this deliverable we present an overview of the available information in the knowledge base and its usage based on the monitored web traffic. We have monitored the usage of the portal and gathered statistics in order to understand Web Traffic - How many users are visiting the portal? (Figure 2, Figure 3 and Figure 4) Global visibility - What is the geographical spread of the visitors? (Figure 5) Why users visit the ScalaLife portal - How do visitors find about the portal, i.e. what keywords they use in the searches that lead to a link? (Table 2) Relevance and popularity - What information is most often requested, what are the most often visited sections? (Table 3) Software usage - How often the pilot codes are downloaded? (Table 4) The deliverable has the following sections: Section 1. is the introduction to the deliverable Section 2. introduces the platform and structure of the web portal and the knowledge base. Section 3. describes the population procedures of the knowledge base Section 4. gives a detailed look of the contents and analysis of the access statistics to the portal Section 5. discusses the future work and outlook. 2. Knowledge base platform and structure The knowledge base platform provides the means for efficient collection and publication of the accumulated knowledge during the project. As reported in the previous deliverable D5.1 Competence Center Service and Support Structure, the required functions of the platform were defined to be: Collect best practices from development work done during the ScalaLife project Publicize the collected knowledge in various formats ScalaLife - RI

9 Integrate existing information bases such as knowledge base and mailing lists Provide a help desk system and public forums for support Several platforms were considered MindTouch ( HubZero ( and Drupal ( After evaluation a decision was made to use Drupal, the same platform that is used for the main ScalaLife project website. Drupal is an open-source software Content Management System (CMS) which is very popular and powers millions of websites including many governmental and industrial ones. The platform is robust and presents a stable framework for extension through modules. Drupal has a very large and devoted community of developers and users. Development of the platform by WP5 and WP2 has been focused on the following main areas: Improving the layout, theme and structure of the website Including modules for additional functionality Integrating the existing GROMACS mailing list for seamless usage within the ScalaLife forum system Organizing the knowledge base structure to accommodate for different types of content The content in the knowledge base is organized in the following main categories (Figure 1): Applications (each of the codes is presented in a separate section) Development tools Performance (a large category that includes code analysis, performance profiling, benchmarking results and algorithm descriptions Hardware resources File Format Standards The rest of the Competence Center section includes also: Training (events) Community Expert support (help desk) ScalaLife - RI

10 Figure 1. Competence Center landing page. There is an extensive cross-linking between related sub-sections in different categories. E.g. each package has a sub-section for downloads, but those can be directly accessed from a toplevel Downloads page. The structure thus provides multiple entries for the different type of users who visit the portal. Users of the applications can start with the corresponding pages; developers would go directly in the Performance area where topics are not restricted to a particular application; getting the codes is easily done directly from the Downloads section etc. More details of the different sections are shown in Table 3 along with access statistics. ScalaLife - RI

11 3. Population workflow The Competence Center knowledge base aims to extract and present to the wide community results from the work done during the project. Information from the work of the different work packages is extracted as follows. 3.1 Software development (WP3) Improving the performance and scalability of the pilot codes requires the development and implementation of relevant algorithms and techniques. They are described in the Improving performance sections of each of the packages. 3.2 Code analysis and profiling (WP6) Code analysis of the software offers valuable information not only to the main developers but also to other researchers who might be facing similar issues with their codes. The results from WP6 s profiling and analysis work are extracted and presented in the Code analysis sections for each of the packages 3.3 Benchmarking and validation (WP4) End users are mainly concerned with the performance that they could expect from the codes on different HPC systems. Results from validation work of WP4 are shown in the Performance Validation and Benchmarking sections of each application. 3.4 File Formats and APIs (WP7) File format specifications and documentation of the available APIs is presented in the File Format Standards and APIs section of the knowledge base. The information is useful to other external programs that are interested in implementing support for them. 4. Knowledge base contents and its usage (web traffic) In this section we present an overview of the available information in the knowledge base and statistics of the usage, i.e. access to the different sections. As of the end of May 2012 the website of the project, which also hosts the knowledge base, has over 400 pages of content. This number includes pages used for presentation of the ScalaLife project and some for internal use. Currently there are 20 registered users on the website (the ScalaLife partners are not counted). Number of pages in the main categories is given in Table 1: Applications Development Tools Downloads Performance Hardware resources File Format Standards Training Community Expert Support (+20 sub pages) 3 1 (+11 external) Table 1. Number of pages in main categories. In order to quantify the success of the knowledge base, we have collected statistics on the web traffic on the Competence Center, which go back to the launch of the portal. The overall numbers show the portal receives on average in excess of 3500 visits per month (Figure 2. Number of monthly visits to the web portal.), from roughly 80 unique visitors (Figure 3. Number of unique monthly visitors to the web portal.). ScalaLife - RI

12 Figure 2. Number of monthly visits to the web portal. The portal receives ~3500 visits per month. Figure 3. Number of unique monthly visitors to the web portal. There are ~80 unique monthly visitors per month. The peaks at the end of February are due to the increased traffic during the ScalaLife Winter School at LRZ. Also there are on average more than 40 daily unique visitors (Figure 4. Number of daily visitors in April and May 2012.) to the site. Figure 4. Number of daily visitors in April and May The average number of daily unique visits is ~40. Only the last two months are shown for better understanding and displaying the latest statistics. ScalaLife - RI

13 Overall, The amount of traffic is healthy and it is very likely (based on current trend) that traffic will increase with the expansion of the center and its activities. Another observation derived from the statistics is that the numbers are representative as shown, for example, by the number of visitors, which shows a relatively even variation across a month (Figure 4), with predictable troughs during the weekends and roughly constant usage during the week. Another overall characterisation of the statistics shows the geographical spread (Figure 5), indicating that the greatest number of the visitors are from the USA, which is followed by the number of visitor from the countries of the project partners. Figure 5. Visits per country. The data covers the period from 1 Oct 2011 to 29 May Most of the visitors are from USA where GROMACS is a widely used application. At the same time, Table 2, that shows the traffic statistics broken down on search engine keywords, helps to explain the large discrepancy between the number of visits from the USA and other countries. Keyword not defined Keyword not defined - Google scalalife scalalife - Google scalalife - Google Custom Search scalalife - Baidu scalalife - Babylon scala cuda scala cuda - Google scala cuda - Google Custom Search gromacs 4.6 Label Visits ScalaLife - RI

14 gromacs Google gromacs analysis gromacs analysis - Google nwchem tutorial nwchem tutorial - Google nwchem tutorial - Google Custom Search gromacs 4.6 download gromacs 4.6 download - Google scala life scala life - Google scalalife project scalalife project - Google scalalife project - Google Custom Search porphyrinringsystem porphyrinringsystem - Google gromacs download gromacs download - Google gromacs download - Google Custom Search scala mpi scala mpi - Google gromacs analysis tools gromacs analysis tools - Google cuda scala cuda scala - Google gromacs openmp gromacs openmp - Google gromacs workshop 2012 gromacs workshop Google alinea ddt Table 2. Search engine keywords. Most of the traffic leading to the web portal comes from searches for GROMACS The explanation is that a significant part of the traffic correlates with GROMACS related searches and the number of GROMACS users in the USA is large. Overall, the large number of visits is a direct proof of the web portal, being very well linked from other large web portals (PRACE, EGI etc.) and, indeed, its Page rank index, currently at 5/10, is reasonably high. The Page rank index shows the relative importance of a given website as linked from other websites. The index is used to sort the results after a search query through Google.com. Very few websites have the maximum 10/10 (e.g. Google.com, Facebook.com), major portals (europa.eu, cnn.com, bbc.co.uk) have 9, etc. Another lesson to learn from Table 2 is that for many specific searches about accelerators and GROMACS (e.g. FPGA, CUDA etc.), the portal comes in the top search results, thus bringing a lot of traffic. This is an aspect to be considered for the future development of the Competence Center - even though it is important to work closely with new codes where big ScalaLife - RI

15 performance/scalability gains can be achieved, it is important to include famous packages for better promotion of the center. Table 3, which shows the total number of visits per Competence Center page over a period of 8 months (Oct. 2011~May 2012), is confirming these findings. Menu Item Description # unique accesses * Applications Overview of the pilot codes - GROMACS GROMACS overview Quick Start Basic usage of the application HPC Usage Using GROMACS on HPC systems. Links to 190 installation and usage guides to other centers Grid Usage Running GROMACS on Grid resources. Links to 51 WeNMR documentation Tutorials Tutorials on using GROMACS Best Practices Best practices for using GROMACS, emphasis on 108 HPC Improving Performance Includes results from the code analysis work, 124 benchmarking results, description of parallel algorithms and FPGA/GPU implementation results Downloads Official releases DALTON DALTON overview Quick Start Basic usage of the application HPC Usage Using DALTON on HPC systems. Links to 129 installation and usage guides to other centers Tutorials Tutorials on using DALTON Best Practices Best practices for using DALTON, emphasis on 86 HPC Improving Performance Includes results from the code analysis work, 110 benchmarking results, description of parallel algorithms and GPU implementation results Downloads Official releases DISCRETE DISCRETE overview Quick Start Basic usage of the application HPC Usage Using DISCRETE on HPC systems. Links to 62 installation and usage guides to other centers Tutorials Tutorials on using DALTON < Best Practices Best practices for using DISCRETE, emphasis on 53 HPC Improving Performance Includes results from the code analysis work, 50 benchmarking results, description of parallel algorithms Downloads Official releases ERGO ERGO overview Quick Start Basic usage of the application HPC Usage Using ERGO on HPC systems. Links to < 30 installation and usage guides to other centers Best Practices Best practices for using ERGO, emphasis on HPC Improving Performance Includes results from the code analysis work, benchmarking results, description of parallel algorithms and GPU implementation results < 30 ScalaLife - RI

16 ----- Downloads Official releases 32 - MUSIC MUSIC overview Quick Start Basic usage of the application < HPC Usage Using MUSIC on HPC systems. Links to < 30 installation and usage guides to other centers Best Practices Best practices for using MUSIC, emphasis on < 30 HPC Improving Performance Includes results from the code analysis work, < 30 benchmarking results Downloads Official releases < 30 - XMIPP XMIPP overview < Getting Started Basic usage of the application < Tutorials Tutorials on using XMIPP < How to Install Installation guides < User Documentation Documentation for users of XMIPP < Developing XMIPP Documentation for developers < Downloads Official releases < 30 Development Tools Description of the available tools, links to 334 documentation and their usage at different centers Downloads Official releases of all codes 339 Performance Documentation of using HPC and accelerators, 388 ensemble computing techniques, results from code analysis and benchmarking results Hardware resources Description of the HPC systems that are provided 308 by the partners File Format Standards Description of the developed file format standards 190 and APIs for data management and exchange Training Announcements of ScalaLife specific and related 223 events such as workshops and seminars Community Overview of the Competence Center community, 136 the current users and procedures for joining Expert Support Competence Center support forums 154 Integrated external mailing lists Links to other support structures Table 3. Structure and access statistics of the knowledge base. * # unique accesses (page views) are for the period since 1 Oct 2011 (1 month after launch) until 31 May Thus, there is a robust number of visits to GROMACS and DALTON information sections, and there is also a good number related to those pages that report on the use of accelerators. DISCRETE and ERGO get about 50% of the traffic of GROMACS and DALTON. This traffic is in fact quite big when considering the very small user base of these codes. Thus famous codes attract visitors who would later explore other less known software packages. Also apparent is a large difference between the number of visits on the landing pages (e.g. the GROMACS landing page) and those on the associated subpages. This can be explained by the fact that any visit to the subpages invariably goes through the landing page. An encouraging interpretation of numbers in Table 3 is the good number of visits on the Performance and Data Formats pages in the Competence Centre, which are very specialised and likely to interest the Life Science software developers only. The knowledge base is the main part of the Competence Center and has been structured to cohesively organize the collected experiences. Table 3 shows the main structure of the ScalaLife - RI

17 knowledge base along with statistics of the page views since a month after launching the Competence Center. In Table 4 we show the number of downloads of the official releases of the pilot packages. ERGO and MUSIC are distributed through their corresponding external pages and thus we don t have statistics. Package GROMACS 53 DALTON 15 DISCRETE 11 Table 4. Downloads of the pilot packages. # downloads 5. Future work and prospects Creating and maintaining the Competence Center knowledge base has shown to be a lengthy process. It requires a lot of efforts in many different aspects of: Choosing an appropriate organizational structure of the material Extracting information from the software development processes for inclusion Coordinating the population of the knowledge base Keeping up-to-date the information Avoiding unnecessary presentation of information that is available elsewhere Population of the knowledge base will continue in parallel with the results obtained from the work of different work packages (Table 5). Milestone/Deliverable D4.4 - Report usage/issues/results of tests D7.2 - WP7 Progress report and updated plan MS18 - High-parallelization scaling version of DISCRETE MS19 - Dispatcher-queue based parallel GMX analysis tools MS35/36 - Stream computing (GPU) algorithms running MS24 - Final VS running, collection and analysis of results MS37 - Advanced ensemble computing algorithms MS45 - Release version of new API available MS46 - Release version of XML file formats MS47 - Structure of the Database of MD with IO interfaces WP# Description of material to be added 4 Results from the benchmarking runs of the pilot codes on different architectures 7 File format use and specification; Usage of quality check and file transfer; Usage of the automatic trajectory analyzer 3 Usage guide and best practices 3 Usage guide and best practices 6 Usage guide and best practices 4 Usage guide updated 6 Usage guides and best practices 7 Usage guide 7 Usage guide 7 Usage guide ScalaLife - RI

18 D3.3 - Project software releases with 3 Usage guides and best practices documentation D4.5 - Report usage/issues/results of tests 4 Results from the benchmarking runs of the pilot codes on different architectures D5.5 Competence Center Pilot and 5 Newly added code bases D5.6 - Competence Centre business plan Table 5. Knowledge base population plan. The knowledge base will be further expanded with material and links to external resources from the collaboration work with partner projects as well. Undoubtedly, the knowledge base is of great value for the users. However, rapid developments in the whole IT sphere (both hardware and software) nowadays require major efforts for continuous maintenance of such structures. Researchers and engineers are also rather reluctant to invest time in maintenance of the paper work when it does not bring an obvious positive effect on their work. Efforts based on good will may not be a sustainable power source. It is known from analysis of online social networks that only ~1% of a group would contribute to discussions in the community. We can expect a similar percentage to apply to knowledge base. E.g. GROMACS has thousands of users worldwide but only a handful of people (outside of the core developers group) participate in maintenance of the project s wiki website. Based on the experience we would recommend that (1) fast transfer of knowledge can be achieved by successfully encouraging close interactions between software developers and its users which can be done through the establishment and promotion of long-term Center Of Excellence, and (2) future EU funded projects on software development include a requirement for participation in those COEs for collection of the gathered experience and knowledge. ScalaLife - RI