open source software in the Valencian Regional Ministry of Infrastructure and Transport

Transcription

1 The project EUROPEAN UNION EUROPEAN REGIONAL DEVELOPMENT FUND A way to build Europe Complete migration to open source software in the Valencian Regional Ministry of Infrastructure and Transport

2 Complete migration to open source software in the Valencian Regional Ministry of Infrastructure and Transport

3 IndexPart 1 Corporate and 7 web developments Chapter 1 Overview 8 Coordinated and published by Legal Deposit M Translated by Crown Communication, S.L. C/. Archiduque Carlos, nº65, Valencia Design and layout Edit Lin Editorial, S,L Avda. de Portugal, 85-local Madrid Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 gvdades: experiences with database management systems 11 gvmétrica and MOSKitt: definition of a development and support methodology 17 gvhidra: development of a framework for PHP 22 Implementation of version control systems: CVS and Subversion 24 Implementation of a report generation tool 25 Licence This publication is distributed under the Creative Commons Attribution - Share Alike 2.5 Spain licence. To see a copy of this licence, go to licenses/by-sa/2.5/es/ Chapter 7 Migration of the web portal and intranet 28 Chapter 8 Workflow for business process management 31 Chapter 9 gvadoc: document management system 33

4 Part 2 Operating systems 37 and communication Chapter 10 End user PC environment 38 Chapter 11 Local network server environment 40 Chapter 12 Networking and communication environment 43 Chapter 13 Corporate server environment 44 Part 3 GIS and CAD: gvsig 47 Chapter 14 gvsig: introduction 48 Chapter 15 Chapter 16 gvsig: description and justification of our initial situation 49 gvsig: how it has evolved into the current solution 52 Chapter 17 gvsig: conclusions 63 Chapter 18 gvsig: future lines of work 64 References 64 Glossary 65

5 INTRODUCTION gvpontis, a success story gvpontis is the name we have given the project to migrate all the Valencian Regional Ministry of Infraestructure and Transport s (CIT) Information Systems from commercial to open source programmes. The idea behind the project goes back to 2003, a time of major upheaval in programme licensing costs. This change in commercial software sales strategy, together with the CIT s policy of having legal licences for all our users, meant a considerable increase in licence costs which became unsustainable as the majority of our budget went on acquiring these licences. In addition, many attempts had been made to obtain accurate, centralised information through corporate information systems but this had to take into account the fact that our staff used a huge range of computer tools in their daily tasks and these all needed to be brought under the same umbrella. To give you some idea of the scale of the project, around a thousand members of staff use the different information systems. Six hundred of these staff work in general administration whilst the other four hundred work in more specialist positions, mainly as engineers and architects. These staff are divided between the CIT s headquarters, three regional offices and some area offices. In the light of this situation, we made a proposal to the Valencian Directorate General for Modernisation to migrate all our tools and systems to open source software. The proposal was accepted and the migration plan was authorised in September The last quarter of 2003 was spent studying the technical feasibility of the project. The University of Valencia and the Universidad Politécnica de Valencia were entrusted with several studies about the open source software alternatives on the market at the time. Several SMEs which used these new technologies were also given the task of developing small-scale computer applications using the new concepts. All the reports and applications indicated that there were no dangers or reasons not to start developing the project. All these reports are available on the project s web site: The only real risk was the technological change which needed to be implemented. This was our only fear but this will be better illustrated in the report which follows. Finally, in January 2004, gvpontis was begun. The project was to be carried out over a period of four years and was to include the migration of all the CIT s information systems. It was to be carried out in an orderly, gradual manner based on two principles which were to govern the whole process: dd The project was not to be allocated any additional resources. Only resources already allocated to the Organisation and Computing Department were to be used. 4

6 dd The negative effects of the project on the CIT staff s daily work were to be kept to a minimum. The work itself is more important than the means. Once the project was started, it was found that even the most rudimentary tools evolved to a more than acceptable standard. The progression has been almost geometric and the implementation of the interoperable systems, open standards and open source tools and software are already on an even footing in our information systems. As of 2008, commercial software is only used more than open source programmes for multimedia and other similar tasks. As explained below, we have encountered problems, we have managed to solve most of them and others are still pending. We have spent time on training, which is so essential and needs to be a constant in the computing business, and have trained the users and above all the departmental staff. We must admit with hindsight that the major problem over the years has been the fear of change. This apprehension over unbuffered technological innovation has proved to be this project s worst enemy development-wise. Tackling fear is also a challenge yet we have learned from our own experience that a challenge can always be taken on board. In our case, we have faced up to the challenge with well-laid plans, training and an alternative plan of action just in case. We will have finished the migration process by December There will still be a few remaining areas and some obsolete applications which have not been included in the migration process but only because this is not an economically attractive proposition. Time will do away with these leftover applications which do not affect our day-to-day work because we have managed to include them in the new working environment. the past four years. This also includes our technical experts and our clerical staff. They have all helped to overcome the problems that have arisen. There is no room to list them all here, but my sincere thanks go out to all the Department s own staff and to our external partners. To the successive Administrative General Secretaries who have worked at the CIT over the years and have supported and on some occasions protected the project. To Isabel Villalonga whose characteristic energy gave the project a considerable boost at the start. To José Manuel Palau who also saw how important the project was. And to Ana Climent, who has suffered the project the longest but has always championed it. During these four years, the two Regional Ministers have also influenced the project: José Ramón García Antón, who passed the project at the start and who had to put up with it on several occasions, and Mario Flores, who joined the project when its teething problems were over but who has been involved in the final definitive part of the project. Only one person has been at the helm of the project during these four years. Gaspar Peral, the CIT s undersecretary, has been the continuous driving force behind gvpontis. Without his support and help, this report would not be telling a success story, it would be recounting a failed or unfinished project. Martín García Hernández Head of the Organisation and Computing Department We have been told that this is what is known as a success story. Yet we would prefer to call it a unique professional experience. In conclusion, there are a lot of people I am extremely grateful to. First and foremost, I would like to thank all the staff in the Organisation and Computing Department. They have worked extremely hard over 5

7 Part 1

8 Corporate and web developments gvpontis includes a wide variety of application development projects and tasks, a brief summary of which will be given below as an overview of the main lines of work undertaken. We shall then move on to give a more detailed explanation of each one in the subsequent sections. 7

9 CHAPTER 1 Overview In line with the framework of the gvpontis project and based on the proposals put forward after the preliminary studies in corporate and web developments, plans were started to implement a project to migrate to open source software so that the development of new applications would be based on this type of software. The migration of applications was influenced by the simultaneous decision to modify the client/server development architecture which was used at the time the project begun to a three-layer and later multi-layer architecture. Initially this change of paradigm introduced another element of uncertainty in the project. However, over time this has been beneficial as a result of the abstraction it has produced when programming each of the layers the information system is divided into - presentation, business and data. It has made our developers more specialised and has defined our information systems functions more clearly. The two working tools used to develop applications in the client/ server environment at the start of the migration project were Oracle Developer and PowerBuilder. However, in order to be able to use three-layer architecture, we incorporated PHP and Java for the new application developments. It was decided that the applications which were already being developed in the client/server environment would be implemented in three layers and that Oracle Developer, the language they were initially programmed in, would be maintained. These applications would gradually be converted to PHP or Java languages according to the features of each one. Data base management systems (DBMS) are one of the cornerstones of the gvpontis project. Thus, the gvdades project was developed to integrate the research and implementation efforts in this field. The Valencian Regional Ministry of Infrastructure and Transport s (CIT) departmental applications were developed in Access whilst its corporate applications used Oracle. As there were no sufficiently well developed alternatives to migrate to in the case of the Access applications, a decision was taken to develop new applications in PHP, PosgreSQL and MySQL. It was decided that Oracle should be maintained in the corporate developments already underway and that new applications should be developed independently from the data base and that they should at least be able to run on PostgreSQL. MySQL was also to be used to support the CIT s web portals because of its user-friendliness. The existence of numerous other developments, as well as the need to deal with other Valencian Regional Government ministries which used Oracle, meant that co-existence with Oracle on Linux servers was essential. Thus, the decision was taken to migrate from Oracle 8i to Oracle 10gR2. Another important area which needed to be sorted out was how the applications using two different DBMS, one of which was Oracle, could obtain data. We found ourselves having to develop a tool that carried out replications at table level. This solution is currently being replaced by using WebServices. 8

10 Overview 1 We would also like to mention the problems we have still not found an optimum solution for. Our main lines of work centre on these problems which include the search for better PostgreSQL management and monitoring tools, and security management and access control improvement. Previous paragraphs have mentioned the technological changes implemented in the migration project. However, cultural change was also necessary to streamline the interaction of different profiles involved in the different development phases both as a result of the migration of a large part of the application inventory because of the reengineering they underwent as they had been developed a good few years before and to ensure that the new developments the CIT might need would stand the test of time. This created an imperative need to adopt a development methodology that, amongst other objectives, would allow us to: dd Generate a working method whose tasks were clearly de- fined and assigned to different collaboration profiles. This would allow us to integrate the work carried out by the Organisation and Computing staff who took part in each of the development projects. dd Homogenise the output product documentation that each of the tasks transmitted to the next in order to make the projects more efficient. Based on the recommendations of the document of conclusions, the methodology chosen was MÉTRICA as it supports the software s whole life cycle and covers object-oriented development. Thus, the gvmétrica project was begun with the aim of adapting MÉTRICA Version 3 to the needs of the CIT. As the project advanced and was implemented in the analysis and design phases of new applications, the need to find a CASE 1 tool which supported its implementation and reduced the use of our own templates as much as possible became evident. This was the seed for gvcase, a subproject of the gvpontis project which aimed to develop MOSKitt, an open source CASE tool developed on the Eclipse platform. 1 CASE (Computer Aided Software Engineering) tools are computer applications aimed at increasing productivity in software development in all aspects of its life cycle. At the programming stage, it is important to have a framework 2 when numerous applications are going to be developed. This led to the creation of gvhidra, another gvpontis subproject whose objective was to become the working framework and basis for the development of PHP management applications in web environments, as per the CIT s Style Guide 3. The decision as to which framework to use for the development of Java applications is still under study, although initial trials were carried out using Struts and Turbine. We are currently working with a modular combination of JSF in the presentation layer, Spring in the business layer and Hibernate in the data persistence layer. gvcase, a subproject of the gvpontis project which aimed to develop MOSKitt, an open source CASE tool developed on the Eclipse platform One of the most important functions required when several developers coincide in the development of an application is having a version control system which uses a common repository to store the programmed files and compare changes that have been made in the different versions. Initially, CVS was adopted. However, Subversion is currently being implemented because of the advantages it has over CVS as it includes permission control to access projects and improve file versioning. An Integrated Development Environment was required to make application programming easier. Eclipse was the most mature open source IDE 4 at the time the decision was taken. Its advantages 2 Working environment which groups together a predefined set of programmes, libraries and interpreted language which help to develop and standardise the different parts of a project. 3 A guide to streamline appearance and usability criteria in the application development process. 4 Programme composed of a set of tools which has been packaged as an application programme, i.e. it has a source code editor, a compiler, a debugger and a graphic user interface builder. 9

11 Complete migration to open source software include its large community of users and the use of plug-ins 5 which allow programming in different languages and the extension of its functions. Up until now, we have simply enjoyed the advantages of the application and of its evolution which has brought with it ever-improving features. Time has confirmed that we made the correct decision in choosing this tool. The MOSKitt project is being developed as an Eclipse plug-in. Another aspect to be taken into account was the generation of reports in open source code. The CIT selected the ireport tool which includes the JasperReports engine. This has shown itself to be a tried and tested, highly reliable ally. ireport is a Java-based multiplatform tool whose major advantages include the ease with which highly detailed reports (graphs, pivot tables, etc.) with dynamic content can be generated and the multiple formats it can export the information to (XML, PDF, etc.). Data can be extracted from a wide variety of sources, including all data bases. It also includes a viewer which allows users to preview their reports on screen. Finally, the web portal and the intranet were also part of this migration process. The information was redesigned and restructured using Typo3 as the content manager. Solutions are still to be adopted for the development of fill-in forms in PDF and for some aspects of advanced image editing not covered by GIMP. 5 Computer application which interacts with another application to provide it with a specific function or utility. It is a way of expanding programmes by modules so that new functions can be added without affecting existing ones or complicating the development of the main programme. In short, the main lines of work undertaken up until now in the field of corporate and web developments, in the framework of the gvpontis project, are listed below: >> General: dd gvdades: work with data base management systems. dd gvmétrica and MOSKitt: definition of a development methodology and support for it. dd gvhidra: development of a framework for PHP. dd Implementation of version control systems: CVS and Subversion. dd dd Implementation of a report generation tool. Migration of the web portal and intranet. >> Specific lines of work have included (main examples): dd MASTIN migration. Workflow for business process management. dd gvadoc Document management system. Further details on each of these lines of work are given below. 10

12 gvdades: experiences with data base management systems CHAPTER 2 When the gvpontis project to migrate to open source software was started in the CIT it was divided into various parts. One of these parts gave rise to the gvdades project which aimed to tackle the migration of the different data base managers to open source software. Two database managers were mainly used: d d Oracle as a data container for corporate applications. The applications that used this data were developed using tools such as PowerBuilder and Developer. d d Microsoft Access for departmental and user applications. This was used both as a data base and to develop queries, reports and applications via the actual tool. We should stress here that the actual users requested the use of Access as some of them were well versed in the programme. ODBC was used to connect to data bases from different systems to capture fields which were then included in the generation of documents, spreadsheets, etc. The main problems we needed to solve here were: 1. Finding alternatives to the use of Microsoft Access as a departmental data base. At the time, there were no alternative office management programmes. However, it was found that as a data base manager it could be connected to MySQL or PostgreSQL via ODBC. This meant that these programmes could be installed in the users computers or in servers if they needed to share data. 2. Finding alternatives to the use of Microsoft Access as a development tool for applications, queries, reports, etc. MS Access had two advantages:»»»» end users found it easy to use. Thus they could define their own tables, queries and reports. it was easy to develop and implement small departmental applications. At the time, no other alternative which could meet these requirements was found. Applications had to be developed using other tools, such as PHP and data bases had to use PostgreSQL or MySQL. 3. Finding alternatives to the use of ODBC/OLE for access from applications such as Word, Excel and in-house developments to the data bases. 11

13 Complete migration to open source software In this case, there was no problem deciding because there was a mature alternative called UnixODBC and it was easy to make ODBC connections from applications that supported it, such as OpenOffice. Thus it was installed in the computers that needed it using the ODBC drivers provided by the manufacturers. Standard JDBC drivers were used for the Java applications that supported them code in the data base, we had no choice if we were to maintain the existing applications it had to support large objects (raw, long, lob, etc.) it had to have software to connect and manage data bases. 4. Finding alternatives to the corporate data base system (Oracle), and its use on Linux. Two major objectives were set out: dd Oracle had to coexist with Linux. We had to make sure that the operating system Oracle ran on was open source. We were aware that we could not totally eliminate the use of the Oracle data base for two reasons: the existence of old developments the relationship with other Valencian Regional Ministries that use Oracle. dd The choice of an alternative DBMS with the following features: -- it had to comply with corporate DBMS requirements (transaction manager, queries, user management, storage management, integrity, source code in data base, scaleability, portability, performance, availability, etc.) the existence of ODBC/JDBC clients the existence of software to connect to from Windows clients when migration was not possible it had to allow data migration 2.1 Analysing the alternatives and making a choice A study 1 was carried out to compare the three main data bases existing at the time: PostgreSQL, MySQL and Interbase. A technical comparison was made and an analysis of how Oracle data could be migrated to these other managers was also carried out. dd Interbase 6.0. Was stable, fast, scaleable and sufficiently functional. However, documentation was scarce and we felt that Borland should have provided more support for this project so it could be successful in the open source software community. d d MySQL 4. Was user-friendly, fast and the most widely used. It was well documented and supported yet it did not support many essential functions (ACID transactions, referential integrity, stored code, triggers, etc.). d d PostgreSQL 7. Seemed to be the most comprehensive solution. It was supported by the open source software community, supported transactions and concurrence that are ACID compliant, referential integrity and sequences and it included procedural languages, backup system and fault tolerance. In addition, the added value of this data base was that it had a reasonably mature module, PostGIS, which maintained geospatial data which are highly important to our ministry. -- it had to allow source code migration. Although we wanted to carry out the developments without a source 1 V. Comparative study of different DBMS under Linux. 12

14 gvdades: experiences with data basa management systems 2 This study led to the decision to implement PostgreSQL version 7.3 as an alternative DBMS to Oracle. However, we continued to monitor MySQL as we imagined that more functions would be added to it as it is the most widely used tool. 2.2 Implementing the chosen solution Once we had taken the decision to choose PostgreSQL, we installed this data base on two servers, one for development and another for production. We tried out some initial developments so we could become familiar with the features of this data base and test how well it worked. We took the decision to use PostgreSQL for: dd all new developments, using it as our new OLTP data base server dd the new document file data base we were going to develop (V. gvadoc) dd the GeoSpatial data base using PostGIS (essential for gvsig) dd we later used PostgreSQL as a repository for the gforge tool. We then began a phase of adaptation in which: dd we had to take the incompatibilities/compatibilities between Oracle and PostgreSQL into account. We discovered many of them whilst we were actually developing the programme, such as different names for data types or commands which worked differently from one system to another dd we established the way the applications had to work with the data base (definition of data bases, schemas, data base objects, users, groups, tablespaces, security management, etc.) dd we developed procedures to maintain the data base (startup, shutdown, backups, support, session management, etc.). We have continued to make progress with PostgreSQL in its successive versions, from version 7.3 to version 8.2 (including versions 7.4 and 8.1). We have had no problem with updates when migrating from one version to the next Data replication problems One of the main problems we encountered was that on one hand our new developments needed to access their own data, which was created in the new PostgreSQL data base but on the other, they also needed to access other data which was in other Oracle data bases. What could we do if we needed data which were in another data base or we needed to combine tables which were in other data bases? This problem would not have occurred if we had had an intermediate data access layer, such as WebServices or Hibernate, but at the time this layer did not exist. In the face of this problem, we considered two solutions: 1. That the new applications should maintain several connections, one to the new data base and one to the old data base to recover the data they needed. 2. To replicate the Oracle data base tables in the PostgreSQL data base when large sets of data needed to be worked with in both data bases. We had to take the incompatibilities/ compatibilities between Oracle and PostgreSQL into account Both Oracle and PostgreSQL feature the dblink concept, i.e. data base links which allow data to be related between different data bases, as long as they belong to the same manufacturer, but not between different ones. We thus began to replicate data by dumping complete data sets from one data base to another. This was not a satisfactory solution for the following reasons: 13

15 Complete migration to open source software dd They were complete dumps, i.e. the data in the target table was deleted and they were dumped again. This was fine for small tables but if there were many large tables, the system could become inefficient. dd As the data was deleted from the tables, foreign keys could not be defined for the tables as this prevented the data from being deleted. dd As they were complete dumps, they ran once a day. This meant that we did not have the instant, necessary or advisable replication in some cases. We have recently developed a tool to maintain replication at table level, both for complete and fast refreshes. This basically consists of defining a repository of master tables and slave tables, a trigger system which is used for fast refreshes and programmes in PHP which carry out the refreshes and control the replications. This system is already up and running and it is used for the cases we are not able to resolve with WebServices Oracle migration to Linux servers In parallel, we also started a study to be able to use Oracle on Linux servers. The version we had at the time was version which was not certified for the Linux versions we were using (Suse 8). We thus began migration from Oracle 8i to Oracle 10gR2, which is specifically intended for Linux servers. Our development servers currently use Oracle 10g on RedHat ES4 and we expect to move on to our production servers in the near future Internationalisation MySQL We also implemented MySQL4 on a small-scale thanks to the acquisition of SUN s Typo3 and Identity Manager which use this data base as a repository. This has allowed us to see the functions of this data base engine Client access to the data bases Another problem that occurs when an environment is changed is that the tools that ran perfectly on a Windows environment with Oracle, such as SQL*Plus, TOAD and Enterprise Manager, do not now run on a Linux environment with other data bases. The new tools must comply with some of the following requirements: dd Run on Linux. dd Connect to Oracle, PostgreSQL or MySQL (depending on the tool). dd Have functions to create tables, views, source code. dd Have administration and monitoring functions. dd Be open source or free software. Our development servers currently use Oracle 10g on RedHat ES4 and we expect to move on to our production servers in the near future We have reached PostgreSQL version in successive data base version migrations. We used these migrations to tackle the subject of character set internationalisation. We can currently support UTF8 when we previously only supported ISO (Latin1). In our search the experience has been good. Although we have not found any products as comprehensive as TOAD (or other commercial tools), we have managed to find something similar. The following Table 1 shows the clients we have worked with: 14

16 gvdades: experiences with data basa management systems 2 Table 1. Clients we have worked with. Client SGBD DESCRIPTION pgadmin3 PostgreSQL»» Useful for programmers and poor for administrators.»» Has a web version in PHP (phppgadmin3).»» Runs on Linux and Windows Training When the migration project began, a course was organised at the Universidad Politécnica de Valencia on PostgreSQL administration. Since then, we have made progress on our own by consulting the documentation and practicing. The CIT is currently ready to prepare a course on data base administration using PostgreSQL. The course manual is available on the gvpontis web site Difficulties and problems SQLDeveloper Oracle MySQL»» Oracle s own, certified to connect to versions 9.x and 10.x.»» Requires JDK.»» Runs on Linux and Windows.»» Useful for programmers. We have come across some problems during this process, some of which have not yet been solved: dd A lack of GPL tools to administer and monitor PostgreSQL. Squirrel Oracle MySQL PostgreSQL»» GPL, plugins can be made to add functions.»» Created in Java, runs on Linux and Windows.»» Connects via JDBC to any DBMS. dd A lack of companies which provide qualified support for PostgreSQL. A solution is on the way for this problem. dd Security management and access control. As PostgreSQL does not have fine control mechanisms like Oracle Label, Oracle Vault, etc., security implementation has to be done using our own programmes. We have not managed to develop these programmes yet. isqlplus Oracle»» SQL*Plus web client to connect to Oracle. dd Loss of product integration. Compared to Oracle which has a huge number of complementary products, PostgreSQL only has the data base. The fact is that we have worked satisfactorily with the first three and although they do not have as many functions as TOAD, it is a question of personal preference over which one to choose. Another aspect to bear in mind was the data base administrator s point of view. In the case of Oracle 10g there were no problems as Oracle provides a very powerful web interface for Enterprise Manager 10g. However, in the case of PostgreSQL we did not find any free GPL tools that convinced us, especially in the field of data base monitoring. We obviously imagined these difficulties when we started the project and we accept them Impressions of the migration The migration has left us with a positive impression. We have managed to work with PostgreSQL and MySQL and have managed to solve the few incidents we had by turning to the knowledge and services of the users community to a great extent. 15

17 We have not taken the change lightly but have not been afraid to go through with it. It should be remembered that the change to open source software requires a change of mindset. We cannot expect to obtain the same features in PostgreSQL as we have in Oracle. However, we can see that the migration is proving useful to us. We accept certain losses which are balanced out by certain benefits both of a financial nature and in collaboration in projects which are of interest to society and to the CIT. 2.3 Future lines of work dd Improvement and possible migration of the PHP replication system to Java. dd Creation of a data warehouse system using PostgreSQL with support from Mondrian and JasperETL. dd Integration of alphanumeric data bases with gvsig s ge- ospatial data bases. dd Analyse whether it is feasible to use MySQL5. This data base is improving in leaps and bounds especially now that is being backed by SUN. dd Implementing a system which allows us to monitor data bas- es using open source products, such as Cacti or Munim. dd Developing a security module for the applications which manages user access and privileges, audits access and runs independently from the data base. dd OpenBase2 is going to be used as the open source alterna- tive to Microsoft Access. dd Selecting companies that are willing and prepared to work with us and provide PostgreSQL support to improve features, i.e. implementing data base clusters, high availability, etc. 2.4 Recommendations We would like to make some recommendations based on our experience: dd If Access or similar programmes are used as data base en- gines in small data bases, they can be replaced by MySQL or PostgreSQL as these two programmes are easy to install and cost nothing in licence fees. dd If there is a need to create many departmental data bases, for example, an organisation which has many work centres and requires a data base for each one, PostgreSQL can be installed. This is easy to administer remotely from a central node and also saves on costs. dd Implementing these systems in a large organisation, which is our case, implies the co-existence of different data base systems which must be able to work with each other. In this case, it is very important to remember that this change does not only affect data bases. Software developments must also be modernised to make up for the disadvantages of not using a sole software service provider. Developments must be separated into layers to prevent coupling. dd We should not be afraid of innovation. These data base managers have the majority of the functions required in many environments. We would like to encourage universities to use these data base managers for teaching and for their work, and also encourage technology-based firms to use them so that they can see that this software offers business opportunities by developing improvements or providing support. dd Asking ourselves what we want our data bases to do for us can avoid us having to pay out for high licence fees and support for these applications, if we opt for the existing open source alternatives that offer the features we need. 16

18 gvmétrica and MOSKitt: definition of a development and support methodology CHAPTER 3 One of the main objectives of the gvpontis project set out in the document of conclusions was the definition of a comprehensive working environment which was suitable for the type of developments required by the CIT 1. Basically, the two main lines of work to be followed in this area were: dd The definition of a software development methodology adapted to the CIT. dd The definition and implementation of the supporting architec- ture for the proposed developments. These lines of work covered a huge area. Thus several projects were created within the gvpontis project to tackle them. These projects had to be well coordinated since the definition of the development methodology had to adopt the conclusions reached with regard to their technical constraints and operational requirements as imposed by the CIT Support Architecture. The project to be developed under the first line of work was called gvmétrica. The following criteria 2 were agreed before work was started: dd The methodology chosen had to be an adapted version of MÉTRICA Version 3 3. dd Although the proposed methodology had to cover the whole development process, the adaptation of the analysis and design phases were given priority. dd Object orientation techniques were encouraged in these two phases. Thus, the objective we wish to meet is to define an approach for the CIT and the companies that work with it, which is particularly suitable for building various similar information systems which automate administrative procedures and other processes designed by the CIT s process engineers Developing the gvmétrica Project This project was tackled in stages, which have gradually extended the range of the project. 1 See section C.2. Alternatives for new Corporate Developments and Web Developments using Linux in the concluding document. 2 The justification for each of these decisions can be found in the concluding document put together by the Linux group. 3 MÉTRICA Version 3 is a method developed by the MAP (Public Administration Ministry) which must be followed in the civil service. It divides the development process in the following phases which are also called processes: Planning, Feasibility Study, Analysis, Design, Building, Implementation and Maintenance. For further information, go to 17

19 Complete migration to open source software The first phase consisted of creating a working team made up of members of the CIT and the Universidad Politécnica de Valencia. This team adapted MÉTRICA s analysis and design processes. This gave rise to the first version of gvmétrica. The work covered in this phase included: dd Selecting the MÉTRICA Version 3 activities and tasks which were deemed to be necessary in the CIT in its analysis and design processes. dd An interface was designed between the Organisation and Computer staff which included all the department s members in the development process. The initial tasks in the analysis phase belonged to the Organisation department which transferred knowledge about the process to be developed to the Computer Department. dd The techniques and standards to be applied were defined. It was decided that event analysis would be the main analysis technique, using UML as the specification language. A Style Guide was defined which was to be used for all the CIT s applications. dd The gvmétrica process was documented. The input and output products, the participants profile in the task and the techniques to be used in the design of each product were specified for each task. dd The set of handouts or templates which support the contents of the products that flow in the process were created. dd The method was integrated with the collaborative working envi- ronment defined for the CIT by the gvpontis project. In general terms, this consisted of using gforge to manage projects and Plone as a content manager to publish project documentation. A test phase was begun to validate the methodology. In this phase, we realised that it was impossible to tackle the developments only using the templates defined by gvmétrica because managing the documentation (publishing and updating it) was often more costly than the analysis itself. A problem with the templates also became apparent. There was too much information to fill in which was not always necessary. Thus the decision was taken to review its publication to make it easier to learn. This gave way to the need to start a new phase in the project. The main objective of this new phase was to make it easy to apply the methodology. To do this we had to (1) provide the CIT with tools to be able to apply the methodology more efficiently, (2) define the minimum requirements that analyses in the CIT had to comply with in order to balance efforts across the different phases and (3) improve the process documentation to make it more user-friendly and thus make it easier to apply the methodology. The work carried out during this phase included: dd Establishing UML as the standard to use when specifying the information systems. dd Giving greater weight to UML models to the detriment of templates. dd Defining which UML language elements were to be used to model the CIT s information systems. Specifying the UML profiles needed to adapt the UML models to the CIT s information systems and define the gvmétrica template subset which was to be replaced by these models. dd 4 Selecting PowerDesigner v.11 as the analysis tool mainly because it was well known in the CIT. It had been previously been widely used by analysts in defining and implementing relational data bases from conceptual models based on entity-relationship models. Although this was a commercial programme, it was valid as a temporary solution because it avoided having to include another new element and allowed us to tackle this issue at a later date. 4 It supports UML

20 gvmétrica and MOSKitt: definition of a development and support methodology 3 dd The full structure was defined applying the criteria set out in gvmétrica. Any project modelled in PowerDesigner must have this structure. dd The aforementioned UML profiles were implemented in the tool and the mechanisms were provided to complete the analyses linking the models to the templates which could not be replaced. dd Finally, user and programmer-oriented reports were defined. When we reached this point, we decided that the methodology was mature enough to be applied in the CIT and that we had tools to do it, even though we knew there were areas which needed to be completed and others which could be improved. Thus a monitoring phase was initiated to detect errors and above all to prioritise improvements. This task continued throughout the lifetime of the project. Regular meetings were held and all those involved in the project were directly monitored and supported. In parallel, many of the projects started in gvpontis also reached their first version, as was the case of gvhidra (implementation of the CIT s Style Guide in PHP) which meant it was possible to tackle the integration phase of the methodology with the support architectures the CIT s developments were to be built on. Thus: dd The UML model catalogue was extended to include a UML profile which facilitated the definition of the user interfaces, based on the patterns defined by the Style Guide and implemented by gvhidra. dd A new UML profile was defined to be applied in the develop- ments which aimed to automate administrative procedures using a Workflow engine. Following the approach defended in MDD 5 (Model-driven Development), and more specifically in MDA 6 (Model-driven 5 MDD is a software development approach based on software system modelling and their creation using these models. 6 MDA is an OMG (Object Management Group) standard which promotes MDD Architecture), we tried to maintain the models at an abstraction level which allowed information systems to be described as independently as possible from the platform they were to be built on. Unfortunately, this is not always possible because the guides which include all the rules the programmers must apply to transform all the information that they receive in the models into source code are not always available. This has forced analysts to include this implementation information in the diagrams (the models) to be able to get it to the programmers. The project is currently at this point (the generation of implementation guides). In parallel, the tool chosen to apply the methodology was reviewed for several reasons: dd The use of PowerDesigner, a commercial tool which only works on Windows was not coherent with gvpontis s general strategy. dd The difficulty of tackling the whole process proposed by gvmétrica only using UML models. UML and the defined profiles were insufficient to specify the systems user interfaces. Moreover, some of the templates still have to be defined for OpenOffice documents. dd A theoretical base was built to document the different UML models deemed necessary to complete the analysis of an information system using PowerDesigner. These models were interrelated so that as a whole they completed the information which corresponded to the different concepts set out by the methodology. There was a model to specify the system s functions, a model to define the entities required to provide these functions, a model to determine the graphical interface the user employs to access these functions and how this affects the system s different entities, etc. However, these relationships between the different models could not be defined in PowerDesigner and thus these models could not be built based on others. They obviously could not be automatically and groups together several languages which can be used to follow this approach (UML is one of these languages). MDA explicitly aims to increase the level of abstraction when describing information systems. 19