International Laboratory for the Study of Judicial Systems Parthenope University Naples

Transcription

1 International Laboratory for the Study of Judicial Systems Parthenope University Naples XXVI Convegno SISP Università Roma Tre - Facoltà di Scienze Politiche, Dipartimento di Studi Internazionali e Dipartimento di Istituzioni pubbliche, Economia e Società settembre 2012 Court Management in the Justice System: A Performance Evaluation Model Luigi Lepore Assistant professor and Researcher of Business administration and Public management Department of Business Administration and Public management Parthenope University of Naples - Italy lepore@uniparthenope.it Concetta Metallo Associate professor of Organization Department of Business Administration and Public management Parthenope University of Naples - Italy metallo@uniparthenope.it Rocco Agrifoglio Assistant professor and Researcher of Organization Department of Business Administration and Public management Parthenope University of Naples - Italy agrifoglio@uniparthenope.it

2 Court Management in the Justice System: A Performance Evaluation Model Abstract The aim of this research is to develop and evaluate the usefulness of a Performance Measurement System (PMS) for Italian courts in planning and control activities. The tool developed, named Court Performance Measurement System (CPMS), is based on Balanced Scorecard framework and it consists of five perspectives. We decided to add another perspectives to the traditional four of the BSC (financial, customer, internal operating, innovation and learning), that is the perspective of Information System success (IS success). We tested CPMS in the Bari and Naples courts. Data were collected by using a qualitative and quantitative methodology. We analyzed data collected by calculating CPMS indicators for each court and comparing them by an ad-hoc statistical technique, t-test, in order to verify the similarities and the differences. Our results have shown that these tools could be useful to support managers in decision making processes in order to improve the effectiveness and efficiency, and that there is not a clear relationship between the perceived improvement of individual performance deriving from the ICTs implementation and organizational performance. Our findings have important implications for theory and practice for court administrators and presiding judges. Keywords: Judicial System, Court Management, Performance Measurement System, IS Success 1. Introduction This study is part of wider research project of the International Laboratory for the Study of Judicial Systems of Parthenope University of Naples aiming at developing and testing a Performance Measurement System (PMS) for courts that could be useful in planning and control activities. The work was also conducted with the cooperation and support of network involved in the European project Menu for Justice - Toward a European Curriculum Studiorum on Judicial Studies that is a project aiming to assess the key gaps in legal and judicial education in all European countries at all stages of education: from undergraduate to graduate and PhD programs to vocational training of lawyers and judges. In particular, the aim of our research is to test and to evaluate the usefulness of this tool in supporting manager/court administrator and presiding judges in decisionmaking, in order to improve the resources allocation, the timeliness of cases resolution, the quality of judicial services, and the accountability of Italian Judicial System. Judicial System (JS) plays an important role in socio -economic progress in every country. Like other public services, efficient and effective judicial services may be a critical factor in creating development opportunities for businesses, credit availability, and in attracting financial resources from other countries. However, over the last twenty years Italian JS has been facing a dramatic crisis of performance, such as the unacceptable length of its proceedings, a very large amount of both pending civil and penal proceedings and a significant amount of money invested.

3 As a consequence, Italian Legislator and several national institutions are making efforts to realize a modernization process of the JS in order to recover functionality and efficiency. This modernization process seems to place ever greater emphasis on management approaches and performance measurement in order to improve JS productivity. Italian courts have experimented innovative management and organizational solutions, such as greater autonomy for court administrators and new ways to work supported by Information and Communication Technology (ICT). Italy has been one of the European Countries that has spent the most in ICTs in order to improve the efficiency and effectiveness of its JS. The Italian Legislator, through considerable investment in ICT, is trying to develop an e-government approach for the JS (e-justice). For instance, the project Processo Civile Telematico (PCT), through use of ICTs in the courts activities, aims to allow full on-line transaction between ministry of justice, courts, lawyers, and other actors of the JS. Despite the modernization process and the considerable investments in ICT, to date the results achieved have been very few. In fact, the Italian JS is still characterized by bad performances. In this context, a managerial approach for the courts, and the use of PMSs in particular, could be useful for court administrators and presiding judges in order to monitor the court activities, the achievement of goals and thus to improve courts efficiency and effectiveness. The importance of these tools is shown by the efforts made over the last decades by scholars of Court Management and Judicial Administration, which aim was to produce systems for measuring the performance of the US courts. As suggested by literature, performance measurement is crucial to a court s ability to provide high quality yet cost-effective and efficient services to its customers. Court managers and presiding judges increasingly embrace the idea of systematically integrating performance measurement into the daily operations of the courts (Ostrom et al., 2008: i). In particular, the decision support tool that we used to evaluate the Italian courts performance is based on the Balanced Scorecard (BSC) framework. We have chosen the BSC to measure court performance because it, more than any other PMSs, makes extensive use of qualitative and non-financial indicators to demonstrate the overall ability of the organization to adequately satisfy the stakeholders. The BSC, thus, appears to be particularly suited to measuring the results in complex institutions such as courts, whose mission includes the achievement of several goals, in combination with other institutions of the State. Considering the substantial efforts the Italian Legislator is doing in order to develop an e-government approach for the JS, we decided to add another dimension to the traditional four of the BSC (financial, customer, internal operating, innovation and lear n- ing), that is the dimension of Information System success (IS success). The inclusion of this dimension is useful to evaluate the contribution of the ICTs to the performance improvement of JS. Thus, our PMS, named Court Performance Measurement System (CPMS), consists of five dimensions. For each dimension of the CPMS, we have identified different indicators. Our CPMS was tested into Naples and Bari courts, in order to verify its usefulness in planning and control activities. Data were collected by using a qualitative and quantitative methodology. Based on case-study method, we conducted a qualitative analysis in order to collect data on organization and performance of courts using ethnographic interviews and document analysis techniques. Moreover, a quantitative analysis was car-

4 ried out administering a questionnaire to administrative staff of Bari and Naples Courts. The questionnaire was aimed to measure some CPMS variables, such as IS success and organizational culture. Subsequently, we analyzed data collected by calculating CPMS indicators for each court and comparing them by an ad-hoc statistical technique, t-test, in order to verify the similarities and the differences. Our findings could be useful to support the court management, the resources allocation, the organizational change, the e-justice success and more generally the process of performance improvement in the two courts investigated. These results offer important theoretical and practical implications for court administrators and presiding judges. The structure of this paper is as follows. First, we introduce the theoretical background, we begin by explaining PMSs and BSC in particular within non-profit organizations; then we deepen the dimension of IS success in the Italian courts. Second, we present the PMS proposed, the CPMS. In the following section, we describe the research methodology and the results of the analysis. Finally, we discuss the findings and their implications for research and practice. 2. Measuring performance in non-profit organizations and the Balanced Scorecard Measuring the performance of non-profit organizations is a well-consolidated topic in the literature. Over the years, various systems have been proposed to assess the effectiveness and efficiency of organizations which, not operating in a market system, cannot refer to profit and to the traditional indicators of performance. In non-profit organizations and in public administration institutions in particular, the importance of non-financial indicators is demonstrated by the relevance assumed during last decades by those models, like the BSC, that through these indicators can be used to best represent the complex and articulate goals of public organizations (Kaplan, Norton, 1992, 1996a, 1996b; Kaplan, 1999). Several PMSs have been developed to evaluate the business performance of both profit and non-profit oriented organizations such as: BSC (Kaplan, Norton, 1992, 1996a); Integrated Performance Measurement System (Nanni et al., 1992); Intangible Asset Monitor (Sveiby, 1997); Piramide SMART (Lynch, Cross, 1991); Macro Process Model (Brown, 1996); the Performance Prism (Neely, Adams, 2001); Skandia Navigator (Edvinsson, Malone, 1997). These systems assume a very important role in organizations, supporting their strategic and operational decision making. Scholars and practitioners have used the BSC framework to investigate performance in non-profit organization (Niven, 2003; Busco et al., 2008) and, especially, in court (Ostrom et al., 2008). These PMSs have begun to play an important role also in public management approaches due to the spread of inefficiency in PA activities, the wasteful use of resources and the consequent crisis of public confidence. In fact, these PMS are considered useful for motivating and sanctioning, as well as simply to monitor the processes and the results (Fattore, 2009). The growing number of non-profit organizations which have implemented similar systems is due to the limits of the traditional PMSs, for instance the inability to provide precise information about the long-term aims, the inadequate ability to incorporate and show the value produced by intangible assets, the inability to communicate organizations strategies and priorities, and the purely economic and financial nature of the performance indicators used. In order to overcome these limitations, BSC uses indicators to monitor targets achievement from four points of view: financial, customer, internal process, and innova-

5 tion and learning. More specifically, BSC is a management tool that provides a periodic and concise assessment of how well the organization is progressing towards achieving its strategic goals. The model was created by Kaplan and Norton in the early 1990 s and has grown in popularity ever since. To each strategic objective of the organization are associated some measurements on the four perspectives of the BSC. Furthermore, this tool has been used to evaluate IT investments and IS performance in different kinds of organizations, both profit and non-profit oriented (Martinsons, 1992; Martinsons et al., 1999; Wright et al., 1999; Rosemann, Wiese, 1999). Implementing more balanced systems for measuring performance of public institutions, while being useful or perhaps necessary, leads to a clash with the complexity arising from the multidimensionality of the result achieved by public organizations. The use of these systems, in fact, must address various constraints, both theoretical and practical, relating to the assessment of efficiency and, in particular, effectiveness of the public non-profit organizations. The goals pursued by public organization are normally composite and often intangible, and so the overall output is difficult to measure. Due to its social function, the activity of public organization is characterized by the simultaneous attempt to satisfy a wideranging number of stakeholders. This leads to the multidimensional nature of the overall performance. Assessment, in all cases, regardless of any difficulty, represents an indispensable element in safeguarding the interests of the different stakeholders. In fact, the absence of mechanisms able to adequately measure the results, as well as the inadequacy of instruments for communication of results to public services users, would impede the operation of accountability mechanisms that protect the stakeholders. Over the last two decades, fast technological development has optimized the way information is managed, making it possible to overcome some of the limitations alluded to above. The ability to gather and manage a great amount of data, typical of the new ICT, makes it possible to improve the efficacy of two principle functions of information in public organizations, external accountability on the levels of performance and support the internal decision-making processes. In other words, IS make it possible to satisfy different knowledge requirements about performances of organizations and. On the other hand, facilitate decision-making processes at all levels of organization. 3. Information systems and performance of the Italian courts During the last two decades, there was a considerable growth of e-justice projects aimed at using ICT to improve efficiency and effectiveness of JS. Several computer applications are being introduced in judicial organizations to support administrative activities, evaluation of court performances and case-flows management. ICTs are considered to be important tools to implement the managerial philosophy in the JSs (Fabri, Langbroek, 2000), planning and control instruments, new budgeting methods, accounting systems, performance indicators, measurement of workload, etc. With the rise of e-justice, scholars and practitioners have begun to research this topic. The traditional research in this area concerned mainly descriptive analysis of the phenomenon, investigating the evolution of software applications for JS, the amounts of money invested by public sector, experience of use of ICT, and different e-justice strategies by European Union members (e.g., Fabri, 2008, 2009; Oskamp et al., 2004; Carnevali et al., 2006; Casanovas et al., 2008; Cerrillo, Fabra, 2009; Velicogna, 2008;

6 Velicogna et al., 2011). Fabri (2009 ) considers the Italian case in a comparative prospective with some other European countries. His study have shown e-justice strategies adopted in some European states to implement software applications for JS, and the main ICT projects and tools adopted in Italian courts. Cerrillo and Fabra s (2009) book presents experiences and best practices concerning the use and impact of ICTs in the JS. Based on the contributions of many authors, this research to provide the most comprehensive overview of present developments in e-justice. These recent studies have shown that have been done considerable investments to project, develop and implement new ICTs in the JS, with a whole body of practical experience in the courts. However, functionality and performance of technologies to support judicial administrations vary significantly from different countries (Carnevali et al., 2006; Fabri 2007; Velicogna 2008; CEPEJ, 2010). In many experiences, these investments in e-justice solutions have not given the expected results. In particular, in Italy, the results achieved have been very few, despite the considerable investments in ICT (Carnevali et al., 2006; Falletti, 2008). This situation has attracted the attention of numerous researchers (Fabri, 2004, 2006; Masciandaro, 2000; Marchesi, 2003; Zan, 2004; Bruti Liberati et al., 2005; Contini, Cordella, 2007) who, as well as highlighting the state of crisis in the Italian JS, have tried to propose solutions mainly in terms of management innovations. For example, the study carried out by Contini and Cordella (2007) has emphasized the role of the new technology in the Italian JS and has provided a first assessment of its impact on performance, showing the discrepancies between expectations and the results obtained. In this context, the evaluation of the success or effectiveness of the ICT investments could be useful for court administrators and presiding judges, in order to monitor the achievement of goals and, thus, to improve court efficiency and effectiveness. IS success or effectiveness is critical to our understanding of the value and efficacy of IS management actions and IS investments ( DeLone, McLean, 2003: 10). Many scholars recognize the need to evaluate IS success (or effectiveness) and a large number of systems success measures exist (e.g., Bailey, Pearson, 1983; DeLone, McLean, 1992). The most widespread and recognized indicator is user satisfaction (Bailey, Pearson, 1983). An information system is a successful information system if it is capable of satisfying the information requirements of its users. Bailey and Pearson (1983) defined user satisfaction as the sum of an individual s reactions to a set of factors affecting IS success. They identified thirty-nine factors that influence IS user satisfaction and developed an instrument to measure them. Bailey and Pearson s instrument has been adopted in numerous researches as basis to measure IS satisfaction and success (e.g., Ives et al., 1983; Saariner, 1996). According to DeLone and McLean (1992), however, a single indicato r is not sufficient to measure such a complex construct as the success of an IS. The authors therefore developed a model, known as the IS success model, based on six dimensions, i.e., the quality of the system, the quality of the information, the use of the system, user satisfaction, individual impact, and organizational impact. DeLone and McLean s model is based on three levels: technical, the accuracy and efficiency of the communication system that produces information; semantic, the success of the information in conveying the intended meaning; effectiveness, the effect of the information on the receiver. Therefore, the systems quality dimension measures technical success; information quality dimension measures semantic success; and effectiveness success is measured

7 from several dimensions such as use, user satisfaction, individual and organizational impacts. Moreover, DeLone and McLean (1992, 2003) argued that this model is based on both process and causal considerations and, therefore, these dimensions of success are interrelated rather than independent. According to the authors, the aim of their model was to investigate a process understanding of IS and their impacts. Therefore, the process model analyzed three components: creation, use, and consequences of system use. They argued that each of these steps is a necessary, but not sufficient, condition for the resultant outcome(s) (DeLone, McLean, 2003:16). Seddon (1997) proposed two dimensions for classifying IS effectiveness measures: type of system and stakeholder. Compared with DeLone and McLean s Model, Seddon (1997) argued that different IS success measurements are needed to assess not only the individual and organizational impact, but also the effectiveness of a system for different groups of stakeholders. Seddon (1997) proposes an alternative model for IS success, substituting DeLone and McLean s (1992) IS use with perceived usefulness because he believed that system use dimension represents a behavior, not a success measure, appropriate for inclusion in a process model, but not in a casual model. DeLone and McLean (1992, 2003), on the other hand, argued that system usage is an appropriate measure of success in most cases. In this way, Saariner (1996) also highlighted that IS use represents a necessary condition for success. Rai and colleagues (2002) empirically and theoretically compared DeLone and McLean s (1992) and Seddon s (1997) models of IS success and they found that both models exhibit a reasonable fit with the collected data. Since 1992, DeLone and McLean s IS success model has served as a basis for numerous studies and empirical investigations (for a review Petter and McLean, 2009). Other scholars (Martinsons, 1992; Martinsons et al., 1999; Wright et al., 1999; Rosemann, Wiese, 1999) have tried to evaluate IS success using the BSC model. Particularly, Wright and colleagues (1999) argued that it could be reasonable to apply BSC also for the evaluation of software performance. In fact, in their BSC, software evaluation is included in the innovation and learning perspective. Rosemann and Wiese (1999: 774), however, used the BSC for the evaluation of IS tasks such as the process of implementation and operational use. As for the four dimensions of the BSC, they add a new project perspective, increasing the completeness and the quality of IS reports including measures such as processing time along the critical path or remaining time to the next milestone. Finally, Martinsons and colleagues (1999) developed a BSC for IS. They used Kaplan and Norton s BSC framework, adapting it to IT application projects and/or an IS department or functional area. Particularly, they suggested four perspectives to measure and evaluate IS: user orientation, business value, internal processes, and future readiness. Despite the increased research interest on software applications for JS, the field currently is poor of theoretical frameworks that can be useful for addressing important issues concerning the implementation success of IS, such as the factors influencing the usage behavior of these applications. It would seem very important to understand what factors influence the usage behavior of e-justice solutions, so that policy makers may improve the likelihood of success when introducing or refining the use of these tools in JS. 4. The model proposed for measuring the performance of Italian courts Courts are complex non-profit organizations that provide users with various types of services. According to Ostrom and colleagues (2008) their mission is to provide a ju s-

8 tice service in order to protect guaranteed rights and individual freedom. Therefore, the courts must accessibly, fairly, efficiently and efficaciously solve disputes arising from the application of the law, and apply the law constantly, impartially and independently, thus guaranteeing the certainty of law, protection under the law, safety and public order. Figure 1 illustrates the above in summarized form. Fig. 1 Output and outcome of Judicial System Source: Kettinger (2005). As happens for different categories of public services, provision of judicial services consists of a combination of various production processes that lead to the provision of two different types of service: core and accessory. The core services are directly connected to the satisfaction of the need for justice and consist mainly in the exercise of judges adjudicatory and investigative functions. The accessory services are administrative services that support provision of primary services. Examples are planning and control activities and, generally speaking, activities that have organizational nature (Lepore, 2009; Alvino, Lepore, 2009, 2010; Lepore et al., 2009, 2010; Lepore, 2011). As for the accessory services, it should be underlined that courts do not have a dedicated structure for planning and control. Planning activity should be carried out by the cooperation between presiding judge and court administrator, who are legally bound to work out a yearly program of the activities. The internal control activity instead is generally absent. Italian courts in fact have never had a system to assess efficiency in the use of resources, effectiveness of their services, and in general their performances. In other words, they ever had any internal or external accountability systems. The main type of control is by inspection and is carried out by the Ministry of Justice and Superior council of magistracy. As happens in all public administrations, assessment of courts performances comes into conflict with the problem of quantifying the utility or the value created, comparing it with the utility sacrificed. Therefore, the need to create a specific PMS emerges. This tool should be able to evaluate and express the degree to which the different aims of the

9 court have been reached, using quali-quantitative indicators. The importance of these kinds of tool is shown for example by the efforts made over the last decades by scholars of Court Management and Judicial Administration to produce systems for measuring the results of the US courts. Following the previous studies (Ostrom et al., 2008), we have chosen the BSC to measure court performance because it, more than any other PMS, provides a range of information that goes beyond mere economic and financial performance. The BSC, in fact, makes extensive use of qualitative and non-financial indicators to demonstrate the overall ability of the organization to adequately satisfy the stakeholders. The BSC, thus, appears to be particularly suited to measuring the results in complex institutions such as courts, whose mission includes the achievement of several goals, not just financial ones. Some authors (Martinsons, 1992; Rosemann, Wiese, 1999) have argued that although the innovation and learning dimension of BSC could also provide the measurement and evaluation of IS performance, in some cases it is not suitable for this purpose. We believe, in other words, that business performance should also include an IS perspective. Particularly, we assume that the profit and non-profit oriented organizations business performance could be measured and evaluated using a decision support model based on both the classic four dimensions of the BSC and another that measures IS performance: IS success. Therefore, we developed a decision support tool that evaluates court performance based on different kinds of measures included in the following five perspectives: customer, internal operating, financial, innovation and learning (based on the BSC framework), and IS success. The indicators included in the four classic perspectives provide the courts management with basic information on the ability of the organization to be: - effective, adequately satisfying the requirements of the stakeholders (customer perspective); - efficient, saving resources (financial perspective); - expeditious in disputes resolutions (internal operating perspective); - capable of improving overall performance, promoting organizational processes able to facilitate creation and sharing of knowledge (innovation and learning perspe c- tive). The indicators included in the customer, internal operating, and financial dimensions have been developed and validated using some indicators included in the CourTools and some indicators developed by the European Commission for the Efficiency of Justice (CEPEJ) that according to us are adapt to Italian courts. CourTools is a PMS used in the USA to evaluate first instance court performance. Based on the literature, we have selected and adopted other indicators. Figure 2 shows our decision support tool.

10 Fig. 2 - Court Performance Measurement System (CPMS) The five dimensions comprising our decision support tool are the following: 1) customer perspective: the customer of a court is a person or an organization that receives the services provided by the court. Indicators that we include in this dimension are measures of a court s accessibility and treatment of customers in terms of fairness, equality, and respect. To define these measures, customers are asked to answer different questions about how they are treated in court and whether the court s decision making process seems fair. These questions have been defined on the basis of the questions used by the National Center for State Courts (NCSC); 2) internal operating perspective: the indicators used in this perspective measure the court s ability to be efficient, controlling its internal procedures and environments. The efficiency measurements used inform presiding judges and court administrators about how well resources are used to achieve intended goals. For this perspective we use the clearance rate indicator, which is the number of cases resolved as a percentage of the number of incoming cases. Another indicator is the case turnover ratio measured as the resolved cases/unresolved cases ratio. Finally, we use the disposition time calculated as the 365/case turnover ratio. These measurements are a fundamental management tool that evaluate the length of time it takes a court to process cases; 3) financial perspective: the indicator used is a cost indicator such as cost per case. It is a measurement of the average cost of processing a single case by case type (e.g., civil and/or penal cases). Cost per case will be measured using an indicator developed by the National Center for State Courts. This measurement aids managers in decision-making in order to improve efficiency and effectiveness of court; 4) innovation and learning perspective: in this dimension we include some indicators that measure the contribution of human resources, the information capital, and court culture to support innovation and learning. Particularly, for human resources, we use the following indicators: the number of administrative staff, the number of professional judges, and the number of IS end-users. Information capital, on the other hand, will be measured using the following indicators: ICT software and hardware investments. Finally, court culture will be measured using a specific scale ( Ostrom et al., 2007), the Court Culture Assessment Instrument (CCAI), which has been adapted by researcher of NCSC from the Organizational Culture Assessment Instrument (OCAI) scale (Cameron,

11 Quinn, 2006). Culture is an important element enabling court performance because it encompasses and makes coherent [ ] values, expectations, and assumptions about how work gets done in a particular court (Ostrom, 2010: 22). The inclusion of indicators to measure the cultural model in the perspective of innovation and learning is important, because obtaining measure of its culture, a court obtains relevant information about itself, and about the opportunity to change the ways to do business. In other words, this is necessary to find new and better ways to organize and conduct business. According to Ostrom (2010: 26), gauging a court s culture is a very useful step in moving toward high performance, because it focuses on how judges and administrators do their jobs. The CCAI is based on two specific dimensions such as solidarity and sociability. The first dimension refers to how unified is the court and shows the degree to which judges and court personnel working by shared goals, mutual interests, and common tasks in order to get common ends. On the contrary, sociability refers to how closely knit are members of the court, highlighting the degree to which judges and court personnel to work cooperatively with one in a cordial fashion. Combining solidarity and sociability dimensions, the CCAI provides a classification scheme that systematically produces four distinguishable types of cultures: communal, networked, autonomous, and hierarchical. Communal culture (High Sociability - Low Solidarity) emphasizes the importance of getting along and acting collectively. Judges and court personnel recognize the importance of group involvement and mutually agreed upon norms rather than established rules and authority. Networked culture (High Sociability - High Solidarity) emphasizes collaborative work environment and effective court-wide communication. Judges try to build court policies and practices basing on the deliberate involvement and consensus extend to involving other court system partners. Autonomous culture (Low Sociability - Low Solidarity) is characterized from judge wide discretion to conduct court business. Despite there is an agreement on court performance and aims, presiding judge leadership is inhibited as individual judges exercise latitude on key procedures and policies. Finally, hierarchical culture (Low Sociability - High Solidarity) defines rules and procedures to meet clearly stated court-wide objectives. Judges leader seek to achieve the efficiency advantages resulting as good managers and coordinators. In order to understand prevailing court cultural models, four types of cultures are measured by five content dimensions such as case management style, judge-staff relations, change management, courthouse leadership, and internal organization. 5) IS success perspective: Based on original DeLone and McLean s IS success model, we consider six dimensions: 1) system quality, the measurements of IS itself; 2) information quality, the measures of the IS output; 3) user satisfaction, recipient response to the use of IS output; 4) information use, recipient consumption of IS output; 5) individual impact, the effect of information on the behavior of the recipient; and 6) organizational impact, the effect of information on organizational performance. Particularly, system quality concerns the consistency of the user interface, the quality of documentation, and whether there are bugs in the system. Information quality concerns the relevance, timeliness and accuracy of information generated through the system. User satisfaction is the sum of an individual s reactions to a set of factors affecting information system success (Bailey, Pearson, 1983). IS use, instead, concerns the use of the system by the user. Finally, individual and organizational impacts are the outcomes attributed to user satisfaction (DeLone, McLean, 1992) and IS use (Seddon, 1997).

12 5. Research methodology Our CPMS was tested into Naples and Bari courts, in order to verify its usefulness in planning and control activities. Data were collected by using a qualitative and quantitative methodology. Based on case-study method, we conducted a qualitative analysis in order to collect data on organization and performance of courts using ethnographic interviews and document analysis techniques. In particular, firstly we conducted two preliminary interviews with the court administrators in order to obtain general information about the two courts. Later, eight semi-structured interviews for each court were carried out with court administrators, IS managers, chancellery officers and other administrative officers. These interviews have allowed us to obtain information about organization as well as the management of the courts investigated. Furthermore, through the analysis of court documents (Judiciary Administration Report; Directorate General for Automated Information Systems; Cepej reports) and data collecting we obtained some important information about the performance of the courts and calculated the PMS indicators included into the following dimensions: internal processes and innovation and learning perspectives. Subsequently, a quantitative analysis was carried out administering a questionnaire to administrative staff of Bari and Naples courts, during the period from the 20th of October 2009 to the 15th of January The questionnaire was aimed to measure some CPMS variables, such as IS success and organizational culture. Data were gathered from court users of various applications such as sistema informativo della cognizione penale (SICP), sistema informativo della cognizione civile (SICC), and sistema informativo dell area amministrativa (SIAMM). These applications, known as Case Management Systems (CMS) and Case-tracking System (CtS), are technologies for automation of administrative tasks. In particular, one of the first applications developed for the courts activities is CtS that automate court dockets and other court registers. To better support the activities of the administrative personnel, different CMSs were developed in European Countries in the last decades. These applications are principally devoted to support and automate the back-office and court staff s administrative work, the case tracking, the case planning, document management, scheduling of hearings and in this way they also support judicial activities. With the objective to improve these applications, more recently different European JSs are trying to open the CMS database to external users, such as lawyers and parties. All questionnaires were administered in Italian language, existing scales were translated from English and then back-translated to check the reliability of the translation using the procedure suggested by Brislin (1970). We conducted a pre and pilot testing to verify and validate the measures used and obtained feedback by some representatives of the court administrative staff in the first instance. Findings of the pre-test highlighted the reliability and consistency of the scales used. Of 620 court administrative staff, of which 212 from court of Bari and 408 from court of Naples, we received a total of 321 complete questionnaires: 59 from court of Bari (response rate 27.83%) and 262 from court of Naples (response rate 64.21%). To minimize data entry errors, all the collected data were checked for consistency. As a result, 314 valid responses were collected. Concerning IS success perspective, information quality was measured using Rai and colleagues (2002) seven-item scale. Two items were used to measure system quality by

13 adapting Rai and colleagues (2002) scale. IS use was measured with a single item based on Rai and colleagues (2002) scale. Twelve items were used to measure user satisfaction based on Doll and Torkzadeh s (1988) end-user computing satisfaction scale. Finally, individual impact was measured using Etezadi-Amoli and Farhoomand s (1996) user performance four-item scale. We have not measured the organizational impact because ICT impact on court organizational performance may be inferred from indicators related to the other CPMS perspectives. All the IS success dimensions were measured by five-point scale ranging from Strongly disagree (1) to Strongly agree (5), except for IS use and individual impact were measured by seven-point scale ranging from Strongly disagree (1) to Strongly agree (7). Concerning innovation and learning perspective, court culture was measured using a specific scale developed by Ostrom and colleagues (2007) called CCAI, previously adapted from the Organizational Culture Assessment Instrument (OCAI) developed by Cameron and Quinn ( 2006). This scale allows to measure both current and preferred cultural models by assessing five key dimensions of court culture such as case management style, judge and court-staff relations, change management, courthouse leadership and internal organization. Each court dimension is composed by four sets of statements; responders should divide 100 points among these four statements giving a higher number of points to alternative that is most often emphasized. 6. Results With reference to the customers perspective, the indicator access and fairness was not measured because the results of our survey about how customers are treated in court and whether the court s decision making process seems fair are not yet available. With reference to the internal operating perspective, clearance rate for Bari court for the year 2010 is equal to %, while clearance rate for Naples court is %. These values highlight that both Bari and Naples courts have been able not only to satisfy demand of justice occurred in the year, but also to reduce the backlog of cases. The longitudinal analysis has highlighted a general increase in productivity of the two courts, there is only a slight decrease in production capacity on Naples court from 2008 to 2009 (Fig. 3). Fig. 3 Clearance Rate

14 The case turnover ratio for Bari court in 2010 is equal to 0.43, while for Naples court it is equal to A case turnover ratio compare the number of resolved cases during the observed period and the number of unresolved cases at the end of observed period. So, these ratios measure how quickly courts analyzed turns over received cases, that is, how long it takes for a type of cases to be resolved. The longitudinal analysis has shown that the value of case turnover ratio for Bari court increases from 2007 to 2008 and then decreases (Fig. 4). Instead, for Naples court, it is almost constant during the observed period. Fig. 4 Case Turnover Ratio Information provided by previous indicators are integrated and confirmed by the value of the disposition time. This indicator provides further insight into how a court manages its flow of cases. Figure 5 highlights that timeliness of resolving cases for Bari court at first decreases and then increases. The number of days required to resolve civil cases decreases from 1077 days in 2007 to 796 days in 2008, subsequently it increases in 2009, it is equal to 844 days, and remains constant in For Naples court, timeliness of case resolution at firs increases from 781 day in 2007 to 805 days in 2008 and 824 in 2009, and then it decreases to 783 days in Fig. 5 Disposition Time

15 Figure 6 provides a summary of civil cases management in the two courts, showing the number of incoming, resolved, and pending cases Fig. 6 Civil Cases of Naples and Bari Courts Civil cases Naples incoming resolved pending Civil cases Bari incoming resolved pending With reference to the financial perspective, for now it was not possible to define the value of cost per case due to different difficulties we found to obtain necessary information. In fact, the determination of cost per case for Italian courts suffers several criticality as, for example, the difficulty to obtain different cost data (cost for salaries, ju s- tice cost, etc.), which are available from different administrations (Municipalities, Ministry of Justice, Appellate courts) that contribute in various way to finance the activities of a court. A second limitation relates to the fact that some data costs, sometimes available for example in the Ministry database, are not disaggregated by type of cases.

16 These limits, more in general, reflect a limit broader represented by the lack of a mechanism to monitor costs and revenues in Italian court, there is, in other words, no accounting systems. With reference to innovation and learning perspective, the results have shown that the Bari court is composed of 106 judges and 252 administrative staff. Investment in ICT hardware amounted to 40,000, of which 35,000 for the purchase of 50 computers and 5,000 for other computer equipment. Investment in ICT software, by contrast, are not quantifiable because these investments are often provided by the central government without any indication of costs. The court of Naples is composed of 438 judges and 674 administrative staff. Data relating to ICT hardware and software investments are not available. Finally, we also assessed the organizational culture for both courts by using CCAI. Firstly, we compared for each court the current and preferred court cultural types such as communal, networked, autonomous, and hierarchical, in order to understanding the gaps among current and preferred organizational models. After, we compared the court culture models of Bari and Naples courts in order to understanding the gaps among two courts about current and preferred organizational cultures. The comparative analysis between two courts was conducted by using the t-test statistical test. It assesses whether the means of two groups are statistically different from each other. Table 1 shows the results of comparative analysis among current and preferred court culture for each court, while Table 2 shows the results of comparative analysis among court culture types from two courts. Content dimension Case Management Style Judge-Staff Relations Change Management Courthouse Leadership Internal Organization Tab. 1 Current and preferred culture comparative analysis for each court Bari Naples Culture type Current Preferred t-test Current Preferred t-test Mean Std. Dev. Mean Std. Dev. "t" value Sig. Mean Std. Dev. Mean Std. Dev. "t" value Sig. COMMUNAL NETWORKED AUTONOMOUS HIERARCHICAL COMMUNAL NETWORKED AUTONOMOUS HIERARCHICAL COMMUNAL NETWORKED AUTONOMOUS HIERARCHICAL COMMUNAL NETWORKED AUTONOMOUS HIERARCHICAL COMMUNAL NETWORKED AUTONOMOUS HIERARCHICAL

17 Tab. 2 Bari and Naples Courts comparative analysis for current and preferred court culture Current Court Culture Preferred Court Culture Content dimension Culture type Bari Naples t-test Bari Naples t-test Mean Std. Dev. Mean Std. Dev. "t" value Sig. Mean Std. Dev. Mean Std. Dev. "t" value Sig. COMMUNAL Case Management Style NETWORKED AUTONOMOUS HIERARCHICAL COMMUNAL Judge-Staff Relations NETWORKED AUTONOMOUS HIERARCHICAL COMMUNAL Change Management NETWORKED AUTONOMOUS HIERARCHICAL COMMUNAL Courthouse Leadership NETWORKED AUTONOMOUS HIERARCHICAL COMMUNAL Internal Organization NETWORKED AUTONOMOUS HIERARCHICAL The results of comparative analysis within two courts have shown that current and preferred court culture types are often statistically different from each other (Sig ). On the contrary, the results of comparative analysis between two courts have shown that cultural types, both current and preferred, are not statistically different form each other (Sig ), apart some exceptions. In this regard, the current and preferred culture models differ from each other for both courts, pointing out that court personnel prefers other cultural models rather than those currents. Furthermore, the CCAI results have also shown that two courts are characterized from similar current and preferred cultural models, highlighting a similar current state and different needs to achieve. In particular, prevailing current cultural models are hierarchical and autonomous for each of five dimensions collected in the Bari court, apart for judge-staff relations in which prevails the communal culture. On the contrary, apart some exceptions, cultural models preferred by the Bari court personnel are communal and networked. Moreover, prevailing current cultural model is autonomous for each content dimensions of the Naples court, followed often by the hierarchical model. On the contrary, the Naples court personnel mainly prefers communal and networked cultural models as well as with some exceptions the hierarchical model too. Towards a better understanding of CCAI results, we build the following graphs work out again from two previous tables in order to better show shows the gaps among current and preferred court culture types for each of five dimensions and for each court. Figure 8 shows the results of CCAI for the Case Management Style dimension.

18 HIERARCHICAL Case Fig. Management 8 Case Management Style Style COMMUNAL NETWORKED Bari (current) Bari (preferred) Naples (current) Naples (preferred) AUTONOMOUS With reference to Case Management Style, the court of Bari is current characterized from autonomous and hierarchical cultural models, while communal and networked are preferred. Moreover, autonomous and communal current cultural models prevail in the court of Naples, while communal and networked are preferred. In both courts, despite there is a general agreement on court s goals, judges are relatively free make their own determinations on case flow management. Moreover, the court of Bari is also characterized from more rules and procedures to meet clearly stated court-wide objectives, while the other court emphasizes the importance of group involvement and mutually agreed upon norms rather than established rules and procedures. Finally, both court personnel prefer a collaborative work environment and effective court-wide communication aimed to involve people and to decide on policy guidelines built. Figure 9 shows the results of CCAI for the judge-staff relations. HIERARCHICAL Judge-Staff Fig. 9 Relations Judge-Staff Relations COMMUNAL NETWORKED Bari (current) Bari (preferred) Naples (current) Naples (preferred) AUTONOMOUS With redard to judge-staff relations, the CCAI results have shown that the court of Bari is currently characterized from communal and autonomous cultural models, while it prefers communal and hierarchical types. On the contrary, the court of Naples is characterized from communal and autonomous cultural types, but it prefers communal and networked cultural archetypes. In this regard, in both courts, judges seek to involve and to collaborate between them and administrative staff using flexible ways, such as norms, rather than established rules and firm lines of authority. On the contrary, the court of Bari prefers the current model, but its court personnel would like also use evaluation systems and performance appraisals in order to get rewards, promotions, and merit recruitment. The court of Naples s personnel, instead, also prefers more coordina-

19 tion and collaborative work environment to manage the court activities respect than wide discretion of judges. Figure 10 shows the results of CCAI for the change management. HIERARCHICAL Fig. Change 10 Change Management Management COMMUNAL NETWORKED Bari (current) Bari (preferred) Naples (current) Naples (preferred) AUTONOMOUS With reference to change management, the CCAI results have shown that both courts are mainly characterized from autonomous and hierarchical cultural models, while the hierarchical type is even more preferred by administrative staff of courts. Thus, judges seek individual ways to change management resisting a rule and process bound organizational setting rather than centralized change initiatives. On the other hand, judges are inclined to use technology and new ways of working and interacting inspired by principles of management in order to improve the timeliness of case processing and accuracy of record keeping. In both courts, judges perceived as good coordinators and organizers and who seek to achieve the advantages of order and efficiency are even more preferred by administrative staff. Using new technologies and principles of management court personnel could improve their individual performance promoting widespread benefits for all organization. Figure 11 shows the results of CCAI for the courthouse leadership. HIERARCHICAL Fig. Courthouse 11 Courthouse Leadership leadership COMMUNAL NETWORKED Bari (current) Bari (preferred) Naples (current) Naples (preferred) AUTONOMOUS With regard to courthouse leadership style, the CCAI results have highlighted that both courts are characterized from hierarchical and autonomous cultural models, while

20 they would like to go to other opposite cultural models such as communal and networked. In both courts, presiding judge leadership is inhibited because each judge prefers to work with few corresponding staff members of own choice. Furthermore, each judge establishes rules and directives to guide court operations and uses own channels to get things done. On the contrary, the personnel of two courts would like to emphasize human relationships in order to mutually agree upon the court performance goals, to obtain more satisfaction from work, and to build an integrated court system community. Finally, figure 12 shows the results of CCAI for the internal organization. HIERARCHICAL Fig. Internal 12 Internal Organization COMMUNAL NETWORKED Bari (current) Bari (preferred) Naples (current) Naples (preferred) AUTONOMOUS With refer to internal organization, both courts are currently characterized from the prevalence of autonomous cultural model. However, the court of Bari prefers communal and networked cultural types, while the court of Naples would like to go to networked and hierarchical models. Thus, in both courts the internal organization is autonomous so that each judge decides how to organize own work and has wide discretion to get things done. Two courts emphasize stability and slow change of practices, while the confrontation is minimized. On the contrary, the court of Bari prefers the collegiality and teamwork pointing out the role of personal relations in workplace. Court personnel prefers informal channels to communicate and to share information among judges and administrative staff, and to work collaboratively to perform case processing. The court of Naples, like the court of Bari, prefers internal organization based on collaborative work, but also would like to have a clear division of labor and formalized roles. Regarding IS success perspective, findings are presented in Table 3. Tab. 3 The IS Success s results Court IS Success Bari Naples t-test Mean Std. Dev. Mean Std. Dev. "t" value Sig. System quality Information quality User satisfaction IS use