Methods of Evaluation on Manufacturing Information Quality Xuzhi Lu 1, a, Changping Zou 1, b

Transcription

1 Advanced Materials Research Online: ISSN: , Vol. 650, pp doi: / Trans Tech Publications, Switzerland Methods of Evaluation on Manufacturing Information Quality Xuzhi Lu 1, a, Changping Zou 1, b 1 Chongqing Radio and TV University,Chongqing, P.R. China, a lxz6411@hotmail.com, b zcpctbu@yahoo.cn Keywords: Information quality; Manufacturing information quality; Methods of evaluation; Information analysis; Information quality evaluation Abstract.Information quality (IQ) is one of the basic factors limiting the development of enterprises. And now it becomes a determinable factor. It s urgent to improve IQ, especially methods of IQ evaluation. After comparing the views of the scholars at home and abroad, a method of IQ evaluation was developed. The evaluation of information quality includes two parts, data quality and information quality evaluation.subjective and objective evaluation of data quality were listed, including three basic forms,simple ratio, min or max operation and weighed average. It also introduces developing data quality measurement in engineering practice.the method of IQ evaluation was studied,and the basis of IQ evaluation was established. It reflected the status of organizational IQ using the data collected by the investigation questionnaires. These analyses are applied to analyze the gap between an organization and the best practice. The results of these techniques can be applied to assurance of IQ improvement and improvement of data quality. It s also an effective way to lower information costs. Introduction Information is the basis of operation of digital manufacturing system. The situation of a company s data quality is a direct factor, which influences its information quality. In a digital manufacturing system, if there are no good data measures, correct information data standards and effective control method, it will be more possible to cause bad results, such as determination error, increasing hidden cost and lowering profits. At present, most data quality matrix are developed on an ad hoc basis to solve specific problems, but it lacks of fundamental principles necessary for developing usable matrix in practice. Data quality evaluation Studies have confirmed data quality is a multi-dimensional concept. Companies must deal with both the subjective perceptions of the individuals involved with the data and the objective measurements based on the data set in question. Subjective data quality evaluation reflect the needs and experiences of stockholders. One can use a questionnaire to measure stakeholder perceptions of data quality dimensions. Many healthcare, finance and consumer product companies have used such a questionnaire (table 1) to assess data quality dimensions. Objective evaluation can be task-independent or task-dependent. Task-independent matrix reflect states of the data without the contextual knowledge of the application, and can be applied to any data set, regard. Task-dependent matrix are developed in specific application contexts, including the organization s business rules, company and government regulations and constraints provided by the database administrator. Table 1 Data quality dimension dimensions definitions accessibility The extent to which data is available,easily and quickly retrievable Appropriate amount of data The extent to which the volume of data is appropriate for the task at hand Believability The extent to which data is regarded as true and credible Completeness The extent to which data is not missing and is of sufficient breadth and depth for the task at hand All rights reserved. No part of contents of this paper may be reproduced or transmitted in any form or by any means without the written permission of Trans Tech Publications, (# , Pennsylvania State University, University Park, USA-19/09/16,16:44:54)

2 712 Advances in Materials Science and Engineering Concise representation Consistent representation Ease of manipulation Interpretability Objectivity Relevancy Reputation Security Timeliness Understandability Value-added The extent to which data is compactly represented The extent to which data is presented in the same format The extent to which data is correct and reliable The extent to which data is in appropriate languages, symbols and units, and the definitions are clear The extent to which data is unbiased, unprejudiced and impartial The extent to which data is applicable and helpful for the task at hand The extent to which data is highly regarded in terms of its source or content The extent to which access to data is restricted appropriately to maintain its security The extent to which data is sufficiently up-to-date for task at hand The extent to which data is easily comprehended The extent to which data is beneficial and provides advantages from its use When performing objective evaluation, companies should follow a set of principles to develop matrix specific to their needs. Often, the most difficult task is precisely defining a dimension, or the aspect of a dimension relate to the company specific application. Formulating the metric is straightforward once this task is complete. In this article, we described three functional forms for developing objective data quality matrix. We present an approach that combines the subjective and objective evaluation of data quality, and illustrate how it has been used in practice. Unless specified otherwise, this paper will use data interchangeable with information. Simple ratio. The simple ratio measures the ratio of desired outcomes to total outcomes. However, a proffered form is the number of undesirable outcomes divided by total outcomes subtracted from 1. This simple ratio adhere to the convention that 1 represents the most desirable and 0 the least desirable score. From our experience, managers prefer the ratio showing positive outcomes, since this form is useful for longitudinal comparisons illustrating trends of continuous improvement. Many traditional data quality matrix, such as free-of-error, completeness and consistency take this form. Other dimensions can also use, including concise representation, relevancy and ease of manipulation. Min or Max operation. Applying Min. or Max, operates to handle dimensions that require the aggregation of multiple data quality indicator (variables). One computes the minimum (or maximum) value from among the normalized values of the individual data quality indicators. The min operator is conservative in that it assigns to the dimension an aggregate value no higher than the value of its weakest data quality indicator (evaluated and normalized to between 0 and 1). The maximum operation is used if a liberal interpretation is warranted. The individual variables may be measured using a simple ratio. Believability and appropriate a lot of data can make use of the min operator. The max operator proves useful in more complex matrix applicable to the dimensions of timeliness and accessibility. Weighted average. For the multivariate case, an alternative to the min operator is a weighted average of variables. If a company has a good understanding of the importance of each variable to the overall evaluation of a dimension, for example, then a weighted average of the variables is appropriate. To insure the rating is normalized, each weighting factor should be between zero and one, and the weighting factors should add to one. Regarding the believability example mentioned, if the company can specify the degree of importance of each of the variables to the overall believability measure; the weighted average may be an appropriate form to use. To use the subjective and objective matrix to improve organizational data quality requires three steps: Performing subjective and objective data quality evaluation; Comparing the results of the evaluation, identifying discrepancies, and root causes of discrepancies; Taking necessary actions for improvement.

3 Advanced Materials Research Vol Experience indicates that there is not a good measure for all situations. Thus, it need continuous endeavor. It s necessary to understand the principal of subjective and objective measures. In the paper, we listed subjective and objective evaluation of data quality and three basic objective evaluation: simple ratio, min or max operation and weighed average. It helps to develop data quality measures in practical life. Information quality evaluation Research on information quality matrix originated from scholars in and abroad. Liu Baohua pointed out the main reasons for information quality (1)The accuracy of partial information is inferior;(2)the information feedback is not timely;(3)much information misses;(4)information collection isn t complete etc. While the reasons which make information quality inferior include:(1)leaders do not pay enough attention and organization is morbid; (2)Fluidity of troop staff is considerable; (3)Information forms are not perfect; (4)Product factories don t carefully fill out the biographical sketch;(5) Died information can t be disposed timely, which lead to information misses;analyze the quality character relations among data, information and information system and put forward to information quality matrix system. Also design the quantitative analysis method of information quality. Data quality evaluation matrix include:(1)accuracy:whether the recorded data accord with genuine data.(2)time limit: whether the recorded data are out of date ;(3) Completeness:whether the recorded data are complete; (4)Consistency: whether data expression is consistent all the time. Evaluation matrix of information quality include:(1) Time limit: there is apparent time limit. It has no value over time;(2) Usefulness: it is useful when the users think it is;(3) Completeness: not only information is useful to users, but also its content need to have appropriate completeness;(4) Credibility: users think the acquired information credible;(5) Usability:information is liable to use for users Moreover, the relations among data, information and information system and the relations among data quality(dq), information quality and information system quality (ISQ) are depicted in detail(listed in the following graph) Fig.1 The relationship among DQ, IQ and ISQ Information quality evaluation. Information quality has become a critical concern of organizations and an active area of Management Information System (MIS) research. The growth of data warehouses and the direct access of information from various sources by managers and information users have increased the need for, and awareness of, high-quality information in organizations. MIS researchers have always considered the quality of information to be important. A survey of the variables used to measure IS success reported IQ as one of the six categories commonly employed in MIS research. The AMIQ (AIM quality) of MIT provides a basic IQ evaluation benchmark. Research of information quality dimensions. MIT has no clear definition to information.first,mit thinks data and information are interchangeable. But most scholars have the converse views, considering data are used to describe things and facts. Information is the final product based on data. Second, data and information both are processed as a product with the same method.

4 714 Advances in Materials Science and Engineering MIT gives concerns to general information quality dimensions. Information quality dimensions of different conceptions are listed in table 2. Table 2 Information quality dimensions of different conceptions IQ conceptions IQ matrix Intrinsic IQ Objectivity Contextual IQ Relevance, Value-added, Timeliness, completeness Representational IQ Interpretability, Concise representation, Consistency Accessibility IQ Reliability, Security AMIQ information quality evaluation. AMIQ information evaluation is based on afterwards evaluation of the AMIQ information evaluation of MIT. The AMIT provides a basic IQ evaluation and benchmark. The PSP/IQ model consolidates the dimensions into four quadrants: reliable, dependable, useful, and usable information. (Table 3).These four quadrants represent IQ aspects that are relevant to IQ improvement decisions. The IQA instrument measures IQ for each of the IQ dimensions. These measures are averaged to form measures for the four quadrants. The IQ Gap Analysis techniques assess the quality of an organization s information for each of the four quadrants. These gap evaluation are the basis for focusing IQ improvement efforts. The AIMQ Methodology consists of the following three components: The PSP/IQ Model The IQA Instrument The IQ Gap Analysis Techniques The details are as follows: (1)PSP/IQ model Table 3 PSP/ IQ Model Comfort to specifications Product quality Importance:, accuracy, concise, expression, completeness, consistence Service quality Reliability,: Timeliness, security meet the customer s expectations Usefulness: relevancy, Understandability, objectivity Usability: reliability, Value-added, IQ model contains four parts: importance, reliability, usefulness and usability. According to quality, it includes product quality and service quality. With content, it can be classified by conforming to specifications and meeting the customer s expectations. (2) IQA instrument There are three methods. The first step is the development of items for each of the IQ dimensions; The purpose of the pilot study was to provide an initial evaluation of the reliability of the items for each of the dimensions and to use this to reduce the number of items per dimension; The full study used the final questionnaire to assess IQ organizations. Statistical analyses were performed using SPSS for Windows. (3) IQ gap analysis techniques There are two analysis techniques: IQ benchmark Gaps and IQ role gaps. The first technique, IQ Benchmark Gaps, assesses an organization s information quality against a benchmark. This is the IQ evaluation of a best-practice organization. Figure 2 shows an example IQ Benchmark Gap diagram for the Usability Quadrant using the data from the full study. The y-axis is the level of quality, which can range from zero to ten. The x-axis is the percentage of respondents, e.g., the level of information quality reported by the bottom 10% of the respondents.

5 Advanced Materials Research Vol When analyzing IQ Benchmark Gaps, three indicators should be considered: Size of the gap area Location of the gap Different size gaps over the x-axis Fig.2 An example of the IQ benchmark gap When analyzing IQ benchmark differences, there are three indexes, role scale, role location and the amout of roles reflected in x-axis. In the above graph, it reflects the true differences between a good organization and other four organizations. We can conclude there is space to improvement toward the best organization. The second technique is IQ role gaps. It compares the IQ evaluation from respondents in different organizational roles. When analyzing IQ role gaps, three indicators should be considered: Size of the gap area Location of the gap Direction of the gap(positive vs. negative) Fig.3 An example of the IQ role gap The size of the IQ Role Gap is much greater in organizations 2 and 5, which means that information consumers and IS professional do not agree about the level of IQ for usability. The location of the gap for the best practices organization(number one) is around an IQ level of seven, which is quite good; whereas the location of the gap for organization three, which is also small, is around 4.5.Thus, although their size was similar, organization one had much better IQ than organization three. The direction of the gap is defined to be positive when IS professionally assess the level of IQ to be higher than information consumers. Thus, organization five had a large positive gap. The best practices organization had a small negative gap. The organizations find their fault by using Role Analysis Techniques. Compare to the IQ differences among different sections and managers in an organization to find solutions for information quality improvement

6 716 Advances in Materials Science and Engineering Each component of the AIMQ has merit in itself and therefore makes a contribution on its own. The PSP/IQ model assesses IQ in terms of conformance to specifications and as exceeding consumer expectations on the one axis and IQ as a product and as a service on the other. It is a management tool for conceptualizing and assessing IQ in business terms. Furthermore, it serves as a theoretical foundation for performing gap analyses. Summary The IQ evaluation (IQA) instrument provides the measurements underpinning the PSP/IQ model and the gap analyses. It collects data to assess IQ status along the key IQ dimensions. Valid measures of IQ are critical for further research progress in IQ. The data collected from the IQA are the prerequisite for PSP/IQ modeling and gap analyses. The gap analysis techniques provide the tools by which organizations can understand their IQ deficiencies as compared to other organizations and to different stakeholders within one organization. Using these analysis techniques, organizations can benchmark their IQ and determine appropriate areas to focus improvement efforts. The key contribution of the overall research, however, stems from the integration and synthesis of these components. The AIMQ methodology as a whole provides a practical IQ tool to organizations. It has been applied in various organizational settings, such as the financial, healthcare, and manufacturing industries. The methodology is useful in identifying IQ problems, prioritizing areas for IQ improvement, and monitoring IQ improvements over time. References [1] Lee, Y. Strong, D., Kahn, B., and Wang, R., AIMQ: A Methodology for Information Quality Management, Information & Management, 2002, V40(2) [2] Wang,R.,Ziad,M,Lee,Y..Data Quality.Norwell,Massachusetts:Kluwer Academic, 2001 [3] Larry P. English. Improving Data Warehouse and Business Information Quality. New York: John Wiley & Sons, 1999 [4] Bopeng Zhang. Manufacturing Informatics. Peking:Tsinghua University Press, 2003 [5] Yin Su,Ming Yu,Bopeng Zhang. Theorism research based on quality function configuration of information. Computer Integration Manufacturing System,2002(10) [6] Baohua Liu. Research of reliability information quality F-8 aircraft outfield. Aviation standardization and quality,1996(6) [7] Li Jing, Tong Shurong, Ma Chunhui.Structural model of product manufacturing information supporting design for quality. Manufacturing Automation, 2011(03) [8] Gong Zhuping. Multiple order parameters evaluation of order-degree in manufacturing information systems with dissipative structure. Computer Engineering and Design, 2009(21) [9] Song Lirong. Establish and Strengthen the Self-constraint Mechanism of Information Quality Management in Science and Technology Information Sharing Construction Agencies. Science & Technology Progress and Policy,2010(11) [10] Liu Jianyong. Prensent Situation Analysis of Data and Information Quality in China. Information Science,2010(2)