ASSESSMENT OF INSTITUTES OF ITALIAN NATIONAL RESEARCH COUNCIL PANEL L2 INFORMATION TECHNOLOGY. January 2010

Transcription

1 ASSESSMENT OF INSTITUTES OF ITALIAN NATIONAL RESEARCH COUNCIL PANEL L2 INFORMATION TECHNOLOGY January 2010

2 PART I - Introduction The Panel L2 is composed of Proff. Giorgio AUSIELLO (Coordinator), Michel COSNARD, José Luis ENCARNAÇÃO, Dieter W. FELLNER, Ferrante PIERANTONI, Aldo ROVERI. The activities of the Panel consisted in a series of visits to the Institutes and meetings with Directors and selected groups of researchers in Rome at CNR Headquarters in the case of institutes for which only part of the activity was in the scope of Panel L2 and for which turned out to be impossible to organize specific visits. The activities of the Panel took place from September to December 2009 with the following schedule: September 8: ICAR Rende (Ausiello, Roveri) October 1: IEIIT Torino (Ausiello, Roveri) October 19: IIT Pisa (Ausiello, Cosnard, Roveri) October 20: ISTI Pisa (Ausiello, Cosnard, Roveri) October 19: ILC Pisa (Ausiello, Cosnard) October 30: IASI Roma (Ausiello, Cosnard, Roveri) November 18: IREA Napoli (Ausiello, Roveri) November 19: Meeting in Rome with ISIB, ISTC, ITD (Ausiello, Encarnação, Roveri) December 9: Meeting in Rome with ISSIA and ITIA (Ausiello, Roveri) December 10: Panel Meeting in Rome (Ausiello, Encarnação, Roveri). As a general comment the Panel wants to express its disappointment for the inadequate organization of the assessment. This has resulted in various difficulties in performing the assessment activities. Moreover the goals of the evaluation have appeared unclear (and they are still unclear) especially to the foreign Panel members who are less familiar with the role and mission of CNR. As a consequence the Panel feels uncertain in the balance of the advice and of the grading scale. In Part II we have provided the evaluation table and a synthetic report outlining a few major observations for each institute. The evaluation has been based on the information provided by CNR, on the material presented by directors and researchers of the institutes during the visits and the material available on the Website. It is important to underline that the scores assigned to the institutes have the explicit approval of five out of the six Panel members. Professor Fellner has not provided any comment. Finally Part III contains a series of comments that derive from Panel s experience and that point out some critical issues that have to be addressed for a successful management of the ICT sector of Italian National Research Council.

3 PART III- General recommendations During its work, the Panel has realized that, generally speaking, all institutes in the ICT domain have a great research potential: first class researchers good scientific management and motivated personnel. On the other side the Panel perceived various problems that affect the life of several institutes: - lack of collaboration among institutes addressing similar research domains; in particular, taking into account the remarkable overlap existing among the research topics addressed in the three institutes in Pisa, and in view of the foreseen creation of a separate suitable structure for the management of the.it registry, the Panel suggests to revise the organization of the Pisa ICT institutes; - lack of coordination between the various sites of the same institute; - lack of focus; dispersion in too many research directions; - lack of regular deadlines for recruitment; researchers under temporary contract live in a continuous state of uncertainty due to the fact that recruitment does not take place at regular time intervals; - low number of research exchanges (in particular outbound); CNR researchers should be encouraged to spend frequent research periods abroad; - low number of PhD students (at least in comparison with similar foreign institutions such as INRIA). In particular it can be observed that, in many cases, researchers (due to their high qualification) are able to participate in EU projects and to obtain external funding even without a clear relationship with the mission of their institutes; this leads to a great dispersion of efforts. It is the Panel opinion that, in order to avoid such dispersion, CNR should invest some money to encourage institutes and research groups to focalize around few relevant objectives and to promote cooperation among institutes operating in the field of computer engineering. Finally the Panel has observed that in most cases the quality of websites of the institutes is rather low. Websites are the principal instrument for promotion and dissemination of research activity and the quality of the presentation of the activities of the institutes should definitely be made more effective. We also suggest that institutes should have a regular Newsletter containing short summaries of the internal and worldwide news in the field, with pointers to the relevant URLs.

4 PART II - Assessment of the Institutes INSTITUTE: IASI DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i = b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 15 TOTAL SCORE

5 INSTITUTE: IASI Introduction The research activity of this institute is characterized by a very good synergy between various areas of ICT and system theory: data and knowledge base systems, discrete mathematics and operations research, discrete and continuous optimization, system and modelling applications in biology. The recent aggregation of IASI with a research group in biomedicine has given excellent fruits. 1. Scientific publications The average quality of research work at IASI is very high. IASI has a good set of active researchers with several first class researchers. In particular the research work in all areas related to applications of information technology and system theory to biology and medicine (bioinformatics, mathematical models in biology and medicine, neural networks and the activity of the biomathlab ) is of very good quality both from the theoretical and from the experimental points of view. The research work in the area of discrete optimization appears again of very high quality and founds relevant applications but in our opinion is fragmented in too many directions and seems to be guided only by mathematical interest without a precise direction and focus. Interesting is also the work in the domain of data and knowledge base systems, recently focused in the direction of semantic web. Although part of the activity is successfully devoted to applications, the overall applicability of the research work carried on in the Institute is moderate and the research efforts seem to be mostly driven by mathematical curiosity. 2. Promotion of the scientific and technological dissemination The promotion and dissemination activity is very good. The editorial activity consists of the editing of two journal special issues and of the proceedings of some conferences and it is limited to some of the scientific areas in the institute. 3b. Intellectual property 2 (patents) Although part of the activity in the area of applications in biology and medicine might lead to patents in perspective no patents have been achieved until now. The partnership with external bodies (companies, associations etc.) could be improved.

6 The education activity and collaboration to University courses is good. Scientific exchanges (inbound and outbound) could be improved. The participation to international projects and to contracts is good. 4c. Realization and management of facilities and infrastructures The only relevant infrastructure managed by IASI is the library that is a useful support for the ICT research community in Rome. 5. General comments IASI is a good institute with a clear management, a group of first class researchers with international reputation, and excellent implication in projects. However the impact appears low. With the exception of the well focused area of applications in biology and medicine the choice of applications seem to be occasional and mostly driven by mathematical curiosity. The support from CNR should be improved.

7 INSTITUTE: ICAR DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i = b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 15 TOTAL SCORE

8 INSTITUTE: ICAR Introduction ICAR originated from the aggregation of the former Institute for Systems and Informatics with three smaller research units, the research Center for Parallel Processing located in Naples and two groups located in Palermo working in the area of networks and of artificial intelligence. The research activity of ICAR is based on two fundamental domains: high performance computing and intelligent systems. The application areas are very broad and extend from logistics to robotics, from bioinformatics to e-health, from geoprocessing to security. Recently new developments in the direction of bio-inspired computation paradigms and more generally of natural computing have been undertaken. 1. Scientific publications The average quality of the research work carried on at ICAR is good. In particular very good results have been achieved in Rende in the field of distributed knowledge management. Very good results have also been achieved in Naples in the area of computer architectures. Most of the activities are devoted to applications, still the overall impact of the research work developed in the Institute is low. 2. Promotion of the scientific and technological dissemination The promotion and dissemination activity is very good. Very limited editorial activity (most of the indicated books are indeed papers published in books). 3b. Intellectual property 2 (patents) Although part of the activity developped at ICAR might lead to patents in perspective, no patents have been achieved until now. Partnership with external bodies (companies, associations, etc.) could be improved. The education activity and collaboration to University courses is good. The researchers have taken part in very few scientific excahnges.

9 The participation to international projects and to contracts is good. 4c. Realization and management of facilities and infrastructures Relevant facilities managed by ICAR are computer facilities for High Performance Computing available in Naples and in Rende. 5. General comments on the Institute Although there is some overlapping between the activities carried on in the three units of ICAR (Naples, Rende and Palermo), the integration between the three groups is still far from complete but efforts in this direction are being carried on. The aggregation of the four groups that originated ICAR can be very fruitful in the future provided that the complementary competence are better exploited and that a better integration of research activities is achieved. Also a more precise focus of the mission of ICAR has to be identified even if this might lead to the reduction of some of the less meaningful research activities.

10 INSTITUTE: IEIIT DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i = b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 20 TOTAL SCORE

11 INSTITUTE: IEIIT Introduction IEIIT activity covers all aspects of ICT, from computer science to electronics and from telecommunications to control theory. IEIIT laboratories are located in Torino, where are IEIIT headquarters, Bologna, Genova, Milano and Pisa. The personnel in Pisa is limited to one researcher. 1. Scientific publications The quality of the research work is very high and reaches excellent levels in the area of telecommunications and electrical engineering. Most of the activities have a strong applicative impact that is achieved through strong industrial collaborations, the registration of several patents, the creation of spin-offs. The scientific level of publications is very high and so is their applicative validity. Excellent publications have been produced in several of the research areas: wireless and mobile communication (Bologna), telecommunications (Milano), bioengineering (Genova), electronics (Torino). The high quality of the research is also testified by various awards achieved by the researchers of IEIIT for their theoretical and applicative activity. 2. Promotion of the scientific and technological dissemination Promotion and dissemination activities are very good. Several publications in books but limited editorial activity. 3b. Intellectual property 2 (patents) The production of patents is excellent. The research work at IEIIT has produced 6 (5 European and 1 Italian) patents in the fields of electronics, telecommunications and computer architectures.. The collaboration with the University is very strong in all sites of IEIIT. Very good inflow of visiting researchers. The visits of IEIIT researchers in foreign institutions are limited.

12 IEIIT is able to attract a very large number of research projects and contracts (most of which are research contracts). It is also important to underline that two spin-off companies have been created by IEIIT researchers in the areas of security and wireless communication. 4c. Realization and management of facilities and infrastructures The most important facilities managed by IEIIT are the network infrastructure for the Genova Research Area and the meteorological radar facilities in Spino d Adda (Milano). 5. General comments on the Institute The activity of IEIIT is very good from all points of view that have been taken into consideration. Although the publication rate of IEIIT is already excellent it has to be noted that, due to privacy issues, a great part of the research work done under contracts cannot lead to publication or only allows the publication of the less critical aspects. The activities of the five sites still appear to have very little correlation. The Institute addresses a large number of topics but the research directions that appear more promising are those located in Torino, Bologna and Genova. In particular the activity in Pisa suffers for being isolated. We recommend to concentrate the research efforts in a smaller number of directions. We also remark that, since the research activity done under contract aims at very short term results (at most one year) it is important that the Institutes finds other funding sources (CNR funding in particular) for providing support to more long term activities.

13 INSTITUTE: IIT DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i = b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 20 TOTAL SCORE

14 INSTITUTE: IIT Introduction IIT has been created by assembling various ICT research groups from the Pisa area around three main research directions (networks, with special attention to the problems of future Internet, security and algorithmic methods for complex problems) and various technological services (among which the management of the registry of the domain.it is by far the most relevant). All aspects of IIT life, like scientific productivity, acquisition of funding, distribution of personnel with various skills, etc. is strongly influenced by the twofold mission of the Institute. 1. Scientific publications The number of active researchers is low (55%) however some permanent researchers are first class. Very good work is carried on in the area of networks and security. Also in the area of algorithmic methods for complex problems very good activity is carried on although objectives and applications of the research work seem to be dispersed in too many directions. The quality of the research work is testified by numerous awards and by the involvement of IIT researchers in top positions in international research organizations. 2. Promotion of the scientific and technological dissemination Promotion and dissemination activities are very good although only a small number of researchers are very active in international conferences and program committees. The editorial activity consists of editorial work for international journals, editing of proceedings of conferences and production of a large number of special issues of top quality journals. Also several chapters of books have been produced. 3b. Intellectual property 2 (patents) Although large part of the research activity might have a strong applicative impact no patents have been produced and there is almost no direct partnership with companies. The collaboration with University is very good. Good inflow of visiting researchers. Taking into account size and role of the Institute, the number of

15 scientific exchanges is very low. In the evaluation of this aspect it should be taken into account that by far the largest flow of funds to IIT derives from the registry activity. Nevertheless, even by not considering such flow, it has to be appreciated that IIT is able to attract a large number of research projects. 4c. Realization and management of facilities and infrastructures Of course the most important infrastructure managed by IIT is the registry of the domain.it (clearly an infrastructure of national value). Beside this activity IIT also manages the network infrastructure of the Pisa research area. 5. General comments on the Institute There are worldwide first class researchers in the Institute, but the overall quality and impact of the research remains limited. The top researchers should be given more visibility within CNR. Although the heavy service activity devoted to the management of the registry has not inhibited the development of a strong research activity in some of the areas of the Institute we recommend to separate the management of the.it registry from the research groups of the Institute. Moreover the institute should develop its own scientific strategy for the next 5 years and not only rely exclusively on external funding. There is no intellectual property policy. Hence there is almost no IP production. The relation with industry is very low. Hence the impact on the economy and the society of the research is negligible. High level researchers have there own international networks but the overall number of foreign visitors remains very low. Moreover the number of IIT researchers who made a research stay outside Italy is negligible.

16 INSTITUTE: ILC DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i =10 7 3b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 15 TOTAL SCORE

17 INSTITUTE: ILC Introduction ILC has been created in the late Seventies as a spin-off of the Institute CNUCE in order to organize and promote research in computational linguistics. Since then the Institute has played a major role in the promotion of the field in Italy and in Europe and has accumulated a huge series of applications, tools, systems databases etc. 1. Scientific publications The average quality of research work at ILC is good in relation to the mission of the Institute. In particular good results have been achieved both in the area of linguistic resources and databases and in the area of computational linguistic tools. Some of the ILC researchers are first class and have achieved very good international visibility. The scientific quality of publications seems to be good but difficult to assess in quantitative terms (impact factor, citations etc.). In fact, except very few cases, all publications of ILC researchers appear in specialized journals and conferences totally unrelated to the classical IT journals and conferences. In many cases the publications are in Italian journals and often in the same journal published by ILC. The applicative validity is also good. 2. Promotion of the scientific and technological dissemination The promotion and dissemination activity are moderately relevant on the average. Most of the invited lectures and invited papers are produced by only three researchers. Moderate editorial activity. Only three researchers appear to be involved in editorial activity. 3b. Intellectual property 2 (patents) Although various application of the research activity might lead to patents no effort in this direction has yet been made. Partnership with external bodies (companies, associations, etc.) could be improved.

18 The collaboration with University is good. Good inflow of visitors but only one ILC researcher has spent research time abroad. The participation to international projects and to contracts is very good. 4c. Realization and management of facilities and infrastructures The most important infrastructure managed by ILC is the library that collects 5000 volumes on computational linguistics since the birth of the discipline and that is probably unique in Italy. Besides, the Institute also manages a large series of databases of great value for the discipline. 5. General comments on the Institute ILC produces a good research activity and relevant applications in the field of computational linguistics. The Institute has a good relationship with users and has strong visibility at international level. ILC suffers of various problems. First of all the mission of the Institute is well defined but the relationship between software development and computer applications (on one side) and mainstream linguistics research is still unresolved. As a consequence there is a big concern with cohesion of this Institute. The question of its evolution should be addressed rapidly. Some activities that are much more technological may leave ILC and join ISTI, which will result in a more compact ILC and a more complete ISTI. In second place, the separation of the computational linguistic community w.r.t. the current research trends in computer science and engineering does not allow a flow of new updated computer science concepts in linguistic applications neither the application to software systems of the techniques developed by the computational linguistics community. It is strongly recommended that a better (cultural) integration between the Pisa computer science community and the computational linguistics community is pursued. With the same aim in mind it is necessary to encourage ILC personnel to spend sabbatical periods in institutions devoted to mainstream research in areas of computer science related to computational linguistics (web searching, advanced information retrieval techniques, ontologies, semantic web etc.).

19 INSTITUTE: IREA DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i = b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 18 TOTAL SCORE

20 INSTITUTE: IREA Introduction IREA has been created by assembling two of the major CNR research structures working in the field of electromagnetism in Napoli and Milano. Currently the two sections of the institute located in the two cities carry on a synergic research work on various aspects of remote and in situ sensing, GPS and georeferencing, geographical information processing. Besides the section in Napoli also carries on research activity on systems and components for electromagnetic diagnostic and a more traditional study on biological effects of electromagnetic fields. 1. Scientific publications The production of publications is very good both from the scientific point of view (in qualitative and quantitative terms) and from the applicative point of view. Excellent applied and experimental work is carried on in the field of electromagnetic sensors, in the field of 3D and 4D tomographic processing of SAR data, in the field of advanced data processing techniques for earth resources monitoring. Very important applications of this research work concern the monitoring of water resources, of burnt surfaces, of the uplift of the Campi Flegrei region etc. The applicative validity is confirmed by research and technology transfer contracts established by IREA with some of the major Italian companies. The quality of the research results is also testified by a long series of awards received by IREA researchers. 2. Promotion of the scientific and technological dissemination Promotion and dissemination activities are very good. The editorial activity consists of editing of proceedings of conferences and production of some special issues of journals. Also some chapters of books have been produced. 3b. Intellectual property 2 (patents) Two patents have been registered in the period of interest, concerning components for optical fiber sensing and 4D processing of SAR data. Several activities of the Institute might lead to other patents.

21 The collaboration with University is good. The exchange of visitors is rather poor both inward and outward. IREA is capable to attract a large number of research projects and research contracts. Particularly important are the research projects carried on with national agencies (such as the Space Agency). Most of the research contracts are with major national industries. 4c. Realization and management of facilities and infrastructures IREA manages various important infrastructures: a Computer Cluster for interferometric SAR data processing, the laboratory for the study of biological effects of electromagnetic fields, the laboratory for the development of component and system for optic fiber sensing, the Garda station for the monitoring of the lake. 5. General comments on the Institute The activity of IREA is very good. The mission is clear and the activities are well focused. Research projects and contracts are strongly related to the mission and this avoids dispersion of efforts and of funds. IREA is definitely a healthy research structure. Two observations are necessary. First, integration of the activities in the two sites has to be improved. Second the exchange of visitors has to be strongly developed in both directions. The flow of foreign visitors is important but also important is that IREA researchers regularly spend part of their time abroad.

22 INSTITUTE: ISIB DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i = b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 15 TOTAL SCORE

23 INSTITUTE: ISIB Introduction ISIB research work is essentially devoted to modeling in biology and sees the cooperation of mathematicians, system theorists and biologists. Only a fraction of approximately 25% of the activity is relevant for the field of ICT. The Headquarters of the Institute are in Padova. Further activities are carried on in Milano and Roma. 1. Scientific publications ICT experts are 25 % of the research staff of the institute, they are experts in Computational Engineering, Simulation and Modelling. Their expertise and reputation in these area is quite high. Altogether the number of active researchers is low (63%). Relevant publications are very good and appear in top journals in the fieldss of modeling, simulation and system theory. Still the impact on mainstream ICT topics is modest. 2. Promotion of the scientific and technological dissemination Promotion and dissemination activities are very good. The editorial activity is very good. 3b. Intellectual property 2 (patents) The Institute has registered 3 patents but they are not relevant with respect to ICT. The collaboration with University is good. The flow of visiting researchers from abroad is very good but very few researchers of the Institute have done research activity in foreign institutions. ISIB has been able to attract a large number of research projects and research contracts many of which relevant to ICT topics.

24 4c. Realization and management of facilities and infrastructures The only relevant facilities managed by ISIB are the library in Padova and the Computing Center that provides computing support to research centers and university in North-Eastern Italy. 5. General comments on the Institute The activity of ISIB is very good from the point of view of both theoretical and applicative research. Only 25% of the researchers belong to the field of ICT. They are experts in computational engineering, simulation and modeling. Their expertise and reputation in these area is quite high but the relevance of the research work in terms of innovation in the ICT domain is limited.

25 INSTITUTE: ISSIA DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i = b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 18 TOTAL SCORE

26 INSTITUTE: ISSIA Introduction ISSIA research work falls in various fields: ICT, Industrial Engineering, Environment. The Institute is based on three sites: Bari (Headquarters), Genova and Palermo. 1. Scientific publications The vocation of ISSIA is mainly directed toward ICT applications. Globally the research activity of the Institute is good. Some of the researchers are first class and publish their results in top level journals. Very good work is done in the field of bioinformatics, image processing and understanding, remote sensing. Unfortunately the number of non active researchers is high (27%). Apparently the fact that most of the activities are done under external contracts reduces the possibility to publish the results of the research work. 2. Promotion of the scientific and technological dissemination Promotion and dissemination activities are very good. The editorial activity is rather limited and consists mostly of chapters of books. 3b. Intellectual property 2 (patents) The Institute has registered 6 patents. The collaboration with University is very poor. The exchange of researcher visitors is practically zero both inward and outward. ISSIA is able to attract a very large number of research projects and research contracts.

27 4c. Realization and management of facilities and infrastructures ISSIA manages geographical networks and telematic services for various research institutions in Southern Italy. 5. General comments on the Institute Globally the research activity performed at ISSIA is good but one can notice that too many research objectives are pursued with respect to the number of active researchers, with very little coordination between the three sites. Further efforts should be made to focalize the mission of the Institute and to integrate the research work in the various sites. From this point of view, while it has to be appreciated that the Institute is capable to obtain a great amount of external funding (18 times the CNR funding, an absolute record among all institutes the Panel has visited), we think that the Institute should select the external projects and contracts in order to concentrate on few main research directions with major applicative relevance.

28 INSTITUTE: ISTC DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i =10 8 3b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 15 TOTAL SCORE

29 INSTITUTE: ISTC Introduction ISTC is located in three sites, Roma (Headquarters), Trento and Padova and has a strong multidisciplinary character deriving from the cooperation of experts in neurophysiology, psychology, artificial intelligence, computer engineering, information science. 1. Scientific publications The publication record of ISTC is very strong. First class researchers can be found practically in all groups related to ICT, especially in the areas of ontology studies, planning and scheduling and cognitive and social robotics. The quality of research is testified by numerous awards and by the number of citations. However the number of publications that are relevant from the point of view of computer engineering is limited. Although the potential impact of the research carried on at ISTC is very strong the actual impact on applications is also limited. 2. Promotion of the scientific and technological dissemination Promotion and dissemination activities are good. The ICT relevant editorial activity is rather limited. 3b. Intellectual property 2 (patents) The Institute has not produced any patent The collaboration with University is good. The flow of visiting researchers from abroad is good but few researchers of the Institute in the ICT area have spent meaningful periods of research activity in foreign institutions. ISTC has been able to attract a large number of research projects and contracts mostly funded by public institutions and organizations. Remarkable

30 are the contracts with the European Space Agency in the field of planning and scheduling. Still, globally, the applicative impact of the research work carried on at ISTC is limited and the cooperation with companies should be improved. 4c. Realization and management of facilities and infrastructures The only ICT relevant facility managed by ISTC is the library, the most important cognitive science library in Italy. 5. General comments on the Institute Overall the ICT relevant activity of ISTC is very good and the Institute has several researchers with good international reputation. Unfortunately the mission of the Institute concerning ICT is not at all clear. The various groups seem to have divergent research interests and do not fit into a coherent and focused mission. The common scientific basis of some of the research activities seems to be limited to a weak notion of artificial intelligence. The fall out of the research activity on innovation in mainstream ICT is also modest. Also the applicative impact is still limited and the cooperation with companies should be improved.

31 INSTITUTE: ISTI DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i = b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 20 TOTAL SCORE

32 INSTITUTE: ISTI Introduction ISTI is the major CNR institute operating in the field of computer science and computer engineering. It derives from the aggregation of activities formerly carried on in the two large institutes CNUCE and IEI (two institutes that have deeply contributed to the history of computer science in Italy). The research activity at ISTI is organized in four major IT thematic areas devoted to networks, knowledge based systems, software, visual and high performance computing. A fifth research area is devoted to mechanics and is related to IT only from the point of view of computational aspects. 1. Scientific publications The scientific production of ISTI is on the average very good with several first class researchers. Excellent work is carried on in the areas of networks, knowledge based systems, software, visual and high performance computing (although in the last case the activities seem to be rather dispersed and less focused). Also scientifically valid are the activities in the area of flight and structural mechanics but their relation with the mission of the Institute is less clear. The quality of the research work is testified by numerous awards (best paper awards, awards for prototypes and personal and group awards) and by the leading roles played by several ISTI researchers in international research organizations. Most of the research work is also very good from the applicative point of view. ISTI presents a great number of papers that received more than 20 citations (also taking into account the overall number of researchers) but it has to be observed that a great part of the most cited papers are indeed produced by a small number of excellent researchers active in the four major areas. 2. Promotion of the scientific and technological dissemination Promotion and dissemination activities are very good. Several researchers are very active in international conference program committees. The editorial activity that has been presented mainly consists in the production of a very large number of chapters of books and in a relevant number of proceedings of conferences edited. 3b. Intellectual property 2 (patents) Although part of the research activity might have an applicative impact no patents have been produced until now. Still, since several research activities have been carried on in the framework of industrial contracts in collaboration with Italian and foreign companies, it can be expected that

33 some patents will be produced in the future. Besides there is a good number of software which are transferred mainly as open source. In any case the way the IP is handled could certainly be improved. The collaboration with University is very good. Good number of exchanges of research visits, both inward and outward. ISTI attracts a large number of research projects and research contracts. In all areas the Institute has been involved in a large number of international research projects (altogether about 60) and a great variety of software systems and environments have been developed. The involvement in international projects is good but in some cases may lead to non focused and divergent research efforts. The number of contracts with national and international companies and organizations, although satisfactory, might still be increased and might be extended also to those areas of ISTI that currently have no industrial contract. 4c. Realization and management of facilities and infrastructures The most important infrastructure managed by ISTI is the computer science library that is one of the largest computer science libraries in Italy. Also relevant are the infrastructures supporting research, virtual research and computational activities, and the network and wireless network infrastructures. Important is also the software quality evaluation and certification service infrastructure offered to third parties. Finally various software systems developed in the framework of research projects and contracts are managed by ISTI and used by ISTI and external personnel for research purposes. 5. General comments on the Institute ISTI is a very good Institute with a clear management, a group of first class researchers with international reputation, and excellent implication in projects. The support of the CNR should be improved. The dimension of the Institute provides the critical mass to develop not only research activities but also a number of support infrastructures and a good environment for the education and training of young researchers. It has to be positively remarked that the Institute has decided to activate internal grants for doctoral students and young researchers that (although consisting of small amounts of money) may help to promote new advanced research efforts. The involvement in research projects and contracts is very good but sometimes the focus of the mission of the Institute seems to be lost because too many projects lead to the dispersion of research efforts. The large dimension of the Institute also leads to an overlapping between various areas (as it happens for the work in data mining) that should be resolved. Although the historical reasons that led to the inclusion of the area of structural mechanics in ISTI may be understood, in the future this research work should find a more suitable location inside CNR.

34 INSTITUTE: ITD DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i = b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 10 TOTAL SCORE

35 INSTITUTE: ITD Introduction ITD is the institute of CNR devoted to educational technologies and methodologies. It is characterized by a strong attitude toward applications and by a wide service activity toward the education organizations both at local and at national level. The Institute is based in two locations: Genova (Headquarters) and Palermo. 2. Scientific publications The production of scientific publications is good, although limited to specialized conferences and journals and only rarely published on main computer engineering journals. Most of the research activity has a strong orientation toward applications and toward the production of educational software (mainly tools and environments). 2. Promotion of the scientific and technological dissemination The promotion and dissemination activity is very good. The editorial activity is rather limited. 3b. Intellectual property 2 (patents) Three patents have been produced, two devoted to environments for teaching mathematical concepts and one for general purpose collaborative learning. It is worth noting that from the activity of the Institute spin offs have been created in the past. Good cooperation with University. ITD has a good ability to attract projects and contracts. It is worth noting that the Institute has very good cooperation with software companies and

36 with users (educational institutions). 4c. Realization and management of facilities and infrastructures The most relevant facilities managed by the Institute consist in laboratories and classrooms equipped for distance learning. Besides the Institute manages various libraries and databases regarding educational software and education curricula for computer assisted learning. 5. General comments on the Institute ITD has a clear mission and the activities are well focused. Although the relevance of the activities toward mainstream computer engineering is limited, the Institute has an important role with respect to innovation in computer assisted learning systems and has a good cooperation with educational institutions for the experimentation of new learning methodologies and technologies. The synergy between the two sites is good. The ability to create spin offs may lead to interesting developments in the future.

37 INSTITUTE: ITIA DEPARTMENT: ICT AREA PANEL: L2 Information Engineering Maximum assessment 1. Scientific publications X i =40 Real assessment Promotion of the scientific and technological dissemination Y i = b. Intellectual property 2 (patents) Z 1,2 = c. Realization and management of facilities and infrastructures Σ i W i Ω i =20 20 TOTAL SCORE

38 INSTITUTE: ITIA Introduction ITIA is the institute of CNR devoted to innovation in manufacturing systems. Research is developed with the aim to create new products, new production systems, advanced support to new business models. Beside the Headquarters located in Milano, the Institute has other two locations in Roma and Bari and an experimental industrial plant in Vigevano. The Institute has a strong experimental and applicative orientation. 3. Scientific publications The production of publications is rather limited but of good quality. One of the reason for such limited production and for the fact that only 24% of the researchers appear to be active is due to the strong applicative orientation and to the fact that most activities are developed in the framework of contracts with industries and are covered by non disclosure agreements. Notwithstanding such limitations some of the publications are of very good quality and some of the researchers are definitely first class. 2. Promotion of the scientific and technological dissemination The promotion and dissemination activity are very good. The Institute is not only on the forefront of research in the field of advanced manufacturing but also takes active part at EU and international level to the definition of future directions of the domain (see for example the participation to the definition of the European Manufuture Technology Platform). The editorial activity mostly consists of chapters of books and some edited volumes. 3b. Intellectual property 2 (patents) ITIA has produced seven patents all of which, except one, highly relevant to ICT (beside of course their relevance for industrial engineering). The collaboration with University is very good. Unfortunately very few research exchanges have taken place nevertheless the international cooperation within EU projects is very intense.

39 ITIA is able to attract a great number of research projects and contracts mostly well harmonized with the mission of the Institute. The cooperation with industry is excellent. 4c. Realization and management of facilities and infrastructures The most relevant facility managed by ITIA is the pilot industrial plant in Vigevano. Such facility plays an important role in the collaboration of ITIA with industry. Other facilities, more relevant for ICT aspects are the various laboratories and in particular robotic, control and virtual reality laboratories. 5. General comments on the Institute ITIA has a clear mission and a well focused research activity. Although the publication activity is limited the Institute has various first class researchers who publish on top quality journals. The applicative work in the domain of computer integrated manufacturing is impressive and the Institute has developed an intense cooperation activity with several industries. It has also achieved a strong position in the field at European and, more broadly, at international level and plays an important role in the definition of future advancements of the field.