Defect Analytics in a High-End Server Manufacturing Environment

Proceedings of the 2015 Industrial and Systems Engineering Research Conference S. Cetinkaya and J. Ryan, eds. Defect Analytics in a High-End Server Manufacturing Environment Faisal Aqlan Industrial Engineering Department The Pennsylvania State University, The Behrend College Erie, PA 16509 Chanchal Saha Department of Systems Science and Industrial Engineering State University of New York at Binghamton Binghamton, NY 13902 Sreekanth Ramakrishnan IBM Corporation 1 Rogers St, Cambridge, MA 02142 Abstract Server manufacturing is characterized by extensive test processes to ensure high quality and reliability of the servers. Server components are obtained from different suppliers who may have different specifications. Although outsourcing of components provides many potential benefits to the company, it can also cause quality issues. If quality issues are not addressed effectively at the initial stages, defects can transit through the supply chain. Thus, quality control is one of the major challenges for the high-end server manufacturing industries. Defective parts are either disposed, repaired, or returned to the supplier depending on the type of defects. Product quality is ensured through multiple test processes at the manufacturing and design stages are substantially expensive. The defect-related quality test results are stored in different databases in both structured and unstructured data format. In this study, defect analytics models are used for defect assessment of more than 5,000 different defect instances collected from different databases sources of a highend server manufacturing environment. Analytics models including cluster analysis, neural networks, and text mining to characterize and predict the defect root causes and solutions. The proposed defect analytics framework replaced the current manual defect analysis method which is based on trial and error. Keywords Defect analytics, defect characterization, cluster analysis, artificial neural network, text mining, server manufacturing 1. Introduction Data analytics has emerged as one of the main research areas in the last few years. Companies also found data analytics as a big opportunity to utilize for improving their performance. In integrated manufacturing environments such as high-end manufacturing, parts, and components are supplied by different suppliers who may have different specifications. Extensive test processes are required to ensure high quality and treat any defect at the earlier stages. Manual and automated systems have been developed to detect and resolve the defects in such environments. However, even with the automated defect detection systems, defect can still arise in which root causes and solutions are not known. In many manufacturing environments, defect resolutions tend to be based on trial and error. This process consumes time and effort to troubleshoot the defect root causes and to identify proper solutions. The typical characteristics of server manufacturing include aggressive new product introduction cycles, continuous quality improvements, extremely skewed demand patterns, high penalty costs from end-product order fulfillment, lower forecasting accuracies due to nature of production process and long lead times, thin profit margins, and a continuously increasing number of parts and features [1-3]. As a result of these characteristics, extensive test processes to ensure high quality and reliability of the servers are extremely critical in this environment. Quality issues are major disruptions of the operations in the high-end server manufacturing. Defective parts may be disposed, repaired, or

returned to the supplier depending on the issue. Removal of defective parts is necessary to protect the company s product, image and reputation, and customer satisfaction. Product quality is ensured through test processes, manufacturing, and design. Since server components are very expensive and they should have high quality, they are tested multiple times in both suppliers sites and manufacturers sites. Figure 1 shows the material flow and test processes for the server manufacturing environment. Major quality risk events can disrupt the smooth flow of products and operations in the supply chain. Quality management is responsible for stopping the flow of defective materials to the customers. Figure 1: Material flow and test processes for server manufacturing environment Defect management in manufacturing environments requires effective identification of the defects, finding the proper solutions for these defects, and providing the required resources and tools to repair the defects. Predicting and preventing the defects or quality issues before they can occur is the focus of quality risk management. Several tools are used for analyzing the defects such as Risk Ranking and Filtering (RRF), Failure Mode and Effect Analysis (FMEA), Hazard and Operability Analysis (HAZOP), and Fault Tree Analysis (FTA). Furthermore, automated systems have been proposed to identify defects and retrieve related solutions from the database. However, these systems do not consider the required skills and resources to solve the problem. The remainder of this paper is organized as follows: Section 2 discusses the literature related to data analytics methods for defect management. Section 3 presents the proposed framework for defect analytics. Section 4 discusses the case study for defect analytics in a high-end server manufacturing environment. Finally, conclusions and recommendations are discussed in Section 5. 2. Literature Review The challenges that are faced by the companies in quality risk management mainly arise from the lack of early defect detection mechasims. Furthermore, the problem is exacerbated by the time consuming yet erronous solutions retrival mechanisms that are currently available. Thus, accurate defect prediction and prompt retrival of defect resolution mechanisms are important for the quality risk management sysem of a company. Many predictive models can be found in literature including discriminant analysis, statistical methods, logistic regression, factor analysis, fuzzy classification, classification trees, Bayesian network, Artificial Neural Networks, support vector machines for defect prediction. It was claimed that NN has been proven to be more effective in prediction compared to statistical tools and expert systems [4]. However, a predective model should be chosen based on the complexity of problem, i.e., types of inputs and outputs (data structure), their relationships, data availability, nature of problems, and expected outcomes. This section presents a thorough review of literature related to the defect detection predective models using structured data, their application areas more specially, application of ANN based models, and scope of defect predection and resolution through through text mining of unstructured data. An intelligent defect analysis framework was proposed that automatically gathers manufacturing process data from all the related databases to determine the root-cause of a process excursion. The proposed model combined both special and temporal data, and analyzed them using artificial intelligence methods. The real-time output was presented through a multi-dimensional cubic structure. Although, the framework outlined an intelligent defect analysis method, however the author did not measure its effectiveness by implementing the model into any real environment [5]. Thus, this study can be extended by conducting performance measures, i.e., survey among the users, accuracy and reliability analysis, time-saving experiments for the proposed framework. An ANN based classification method was proposed to classify software into defect prone and non-defect prone classes. This early defect detection approach compared three algorithms to capture the misclassifications of non-defect prone software considering time and cost metrics. The 2

threshold-moving algorithm was claimed to be the most cost-sensitive software for the defect prediction [6]. A data mining approach was proposed to identify the attributes responsible for the defective software modules. This extracted knowledge was applied in defect prediction using a data mining model that is a weighted voting rule of four data mining clustering algorithms, namely Naïve Bayes, ANN, Association Rules, and Decision Tree algorithms [7]. Generalization of the proposed model can be a potential future direction to detect defects in manufacturing processes. A case-based reasoning system was proposed to predict the defects in the Printed Circuit Board (PCB) design. In casebased method, a case database stores all the past defect cases along with their design specifications, defect items, and corresponding costs. The past cases were clustered and ranked using vantage based case indexing mechanism to accelerate the case retrieval efficiency for a new case similar to past cases. Finally, a reasoning algorithm proposed the defect costs for the defective items [8]. Thus, in future, a factorial analysis of the design parameters can be conducted to determine the value of threshold parameters of the reasoning algorithm. Another study proposed a Naïve Bayes classifier based statistical method for defect prediction. The authors recommended to pay more attention to calibrating defect prediction model for that particularproblem rather searching for complex algorithms [9]. ANN is an effective tool for prediction because it can analyze the behavior of a system with certain amount of data to train the system and correlate it with other system parameters. Accurate predictions are important for a Supply Chain Network (SCN), as incorrect prediction not only affects a single stage of a company s Supply Chain (SC) but also the entire SC of that company as well as other stakeholders compancies. As stated earlier, ANN has been proven to be more effective in prediction compared to statistical tools and expert systems [4]. From the users perspective, prediction is probably the most discussed application in ANN domain. ANNs are increasingly used for short and long term demand forecasting and automatic defect predictions for electric loads, energy consumption, pattern recognitions, and stock markets [4, 10]. Reducing total cost in SC has become a crucial issue. Thus, ANNs can help in reducing or eliminating defects that are affecting the production or supply network of an SC by developing better forecasting models. ANNs are used in SC for optimization (logistics management, resource allocation, and scheduling), modeling and simulation (discrete event simulation, dynamic systems theory), defect prediction, globalization (interactions among different activities at different locations), decision support (data query, analysis, and management), and forecasting (any state from one echelon propagate to others in a SC) [10, 11]. However, ANN-based Artificial Intelligence (AI) models are very effective in analyzing only the structured data. Thus, for analyzing unstructured data, attention can be extended to Natural Language Processing (NLP). In current times, many sources including social media, mobile transactions, business networks, scientific experiments as well as operational domains such as healthcare, bioinformatics, finance, manufacturing industries are generating a remarkable amount of data and the amount is increasing rapidly. In response to that, studies on collecting, storing, cleaning, analyzing, and presenting new meaningful and real-time insights of these data have gained tremendous growth. The analytics associated with the big data analysis not only complements traditional statistics, surveys, archival data sources, hypothesis testing but also aim to explore novel patterns or predict future trends from the big data [12, 13]. In the research paradigm of big data analytics, one of the application areas of growing interest is text analytics which can be used for opinion mining and sentiment analysis [14]. In general, sentiment analysis and opinion mining refer to the same techniques that are derived from and based upon NLP, Information Retrieval (IR), Information Extraction (IE), and AI. Typical tasks of sentiment analysis include: (1) finding data relevant to a specific topic or purpose; (2) pre-processing collected data, e.g., summarizing data into single words and extracting relevant information from them; and (3) identifying the sentiment surrounding a product or service [15]. Sentiment analysis technologies, a special type of text mining, can be applied for extracting opinions and sentiments from unstructured human-authored documents [16]. Thus, NLP can be an excellent tool for handling many business intelligence tasks including reputation management, public relations, defect prediction and resolutions, tracking public viewpoints, as well as market trend prediction. In NLP, sentiment analysis takes the challenge of classifying the orientation of texts either into positive or negative to help the machines understand texts similar to human. The texts are analyzed at different levels, such as, word or phrase, sentence, document level or user level. Word level sentiment analysis explore the orientation of the words or phrases in the text as well as their effect on the overall sentiment, while sentence level expresses a single opinion and tries to define its orientation from sentences. The document level opinion mining looks at the overall sentiment of the whole document, and user level sentiment searches for the possibility that connected users on the social network could have the same opinion [17]. Three different approaches, namely machine learning approach, lexicon based, and linguistic analysis are found to be applied in sentiment analysis to classify texts. Machine learning methods are based on training an algorithm, mostly classification on a set of selected features for a specific mission and then test on another set whether it is able to detect the right features and give the right classification. Naïve Bayes, maximum entropy and SVM are used as sentiment 3

classifiers in this method. A lexicon based method depends on a predefined list or corpus of words with a certain polarity. An algorithm is then searching for those words, counting them or estimating their weight and measuring the overall polarity of the text. Lastly, the linguistic approach uses the syntactic characteristics of the words or phrases, the negation, and the structure of the text to determine the text orientation. This approach is usually combined with a lexicon based method [17, 18]. A study was conducted to find the relationship between public sentiment and stock market price using Twitter streams. They proposed an active learning approach using Support Vector Machine (SVM) classifier to query the news feed of the Twitter streams as an active learning process for the sentiment analysis [19]. Their proposed model was able to predict the stock market price movements a few days in advance. Another study also applied SVM to classify the topics for sentiment analysis [18]. The authors claimed that pre-processing of texts using SVM can improve the accuracy of the results. Many studies can be found on defect prediction and resolution that applied structured data in risk management. However, there are limited studies available considering both structured and unstructured data format for model development. To the best of the authors knowledge, none of the previous study applied both data format for defect prediction and resolution. Therefore, in this study, an initiative is taken to propose a defect analytics framework for defect prediction and resolution considering structured and unstructured data format. 3. Proposed Framework for Defect Analytics The proposed framework for defect analytics utilizes both structured and unstructured data for defect characterization and assessment. Figure 2 shows the proposed framework in which analytics models that are used to predict and resolve the defects. For the unstructured data, the individual defect files are kept together to form the corpus, which is a collection of documents. Text analytics models are then used to characterize the defects. Predictive analytics models are also used to characterize the defects based on the structured data. The output both text analytics and predictive analytics models are then used to predict defect root cause and potential solutions. Figure 2: Proposed defect analytics framework 3.1 Unstructured Data Analytics Unstructured data analytics is used to characterize and classify the defects. The proposed framework for defect unstructured data analytics is shown in Figure 3. The unstructured data framework consists of the following steps: 4

1. Documents collection step collects documents that include the unstructured data on defects 2. Text analysis and concept extraction step analyze text using NLP 3. Text link analysis step identifies relationships between the concepts using pattern matching 4. Building defect categories relies on the extracted concepts from the text link analysis. In this step, a clustering method is used to cluster the defect into categories based on the similarities in the extracted concepts 5. Defect characterization step in which the defects are characterized based on the concepts in each category Figure 3: Unstructured data analytics for defect assessment 3.2 Structured Data Analytics Structured data are used to predict defect root causes and potential solutions using the ANN. Structured data analytics consists of two main steps: 1) predicting the root cause of the defect and 2) predicting the potential solutions of the defect. For predicting the root causes, the main defect attributes that are used as inputs include: defect type, product characteristics, production environment variables, and the defect categories obtained by the text analytics model. For predicting the potential solutions, the attributes considered as inputs are the resource attributes and the predicted root causes. The proposed ANN structure for the defect root cause and solution prediction is shown in Figure 4. Examples of defect attributes that are used as inputs for root cause predictions include: part type, part size or capacity, and part supplier. Examples of production environment variables include: production stage and time-to-failure. Examples of resource availability that is used as an input for solution prediction include: available spare parts for repair, cost of disposal, etc. Figure 4: ANN based approach for predicting defect root cause and solution 4. Case Study: Defect Analytics in High-End Server Manufacturing Environment Server manufacturing environment is relatively complex and it is prone to many quality problems that could be caused by external suppliers and internal processes. Since the server manufacturing environment requires extremely high reliability and quality assurance, thus their test processes are expected to be very accurate and downtimes free. Figure 5

5 shows a high level overview of the main stages of the high-end server production process that is considered in this study. In this production process, there are three test stages: panel test, assembly or fabrication test, and fulfillment test. Structured and unstructured data of 5,000 defects data points of the three test stages were collected from different databases. Figure 5: Process flow of high-end server manufacturing The part considered in this study is the Memory Card which is also known as known as Dual In-line Memory Module (DIMM). The process flow of the DIMM inspection, test, and assembly is shown in Figure 6. Non-value added processes were highlighted with red frame while value-added processes were lighted with green frame. The figure shows the assembly and test processes that are performed on the DIMMs and the different movements of the DIMMS between the inventory and production area locations. The defect analytics framework is implemented using IBM SPSS Modeler software. The analytics models for defect root cause and solution prediction are shown in Figure 7. The unstructured data were characterized using the concept of text mining analytics. The text mining analytics model uses linguistic and frequency techniques of NLP methods to extract the key concepts from the unstructured data and categorize the data according to its concepts and patterns. The text mining model extracted 479 concepts by analyzing the unstructured data. By careful observation of extracted concepts usage percentage and technical importance, 179 key concepts were selected for cluster analysis. The concept-wise categorized unstructured data were clustered using the two-step clustering method. Two-step clustering algorithm was used due to its ability to handle mixed data types and larger data sets efficiently. In addition, the twostep clustering algorithm has the advantage of automatically decide the optimal number of clusters. Therefore, the clustering algorithm clustered 179 key concepts into 15 clusters. The selection of 15 clusters gives the best cluster quality which is measured by the Silhouette index. The obtained value of Silhouette index was 0.7 which means a good clustering quality. Root cause and solution prediction models are developed using ANN models. The output (15 clusters) of two-step clustering algorithm (obtained from the concept extraction of unstructured data using text mining) along with structured data were combined using Defect IDs. The combined data were used as inputs to train and test the ANN models for the root cause and solution prediction. Figure 8 shows that the accuracy rates of ANN models for both root cause and solution predictions are 86% and 74.4%, respectively. However, in absence of unstructured data, the accuracy rates of the ANN models for root cause and solution predictions are 75.7% and 50%, respectively. Therefore, inclusion of unstructured data for defect assessment increased the root cause and solution predictions accuracies by 14% and 49%, respectively. 6

Figure 6: Process flow for DIMMs Figure 7: Analytics model for defect assessment 7

Figure 8: Accuracy of the ANN models for the root cause and solution predictions 5. Conclusions and Future Work In this study, an analytics based framework is proposed for defect assessment in a high-end server manufacturing environment. Both structured and unstructured data were utilized to build prediction and assessment models for the defects. Identifying causes of defects and proposing solutions using the proposed framework is found to be very effective for sever manufacturing environment for early detection of production related faults. The performance levels of the analytics used in this framework are 86% and 74.4% for root cause prediction and solution prediction, respectively. The proposed defect analytics framework replaces the current manual defect analysis method which is based on trial and error. It plays a significant role to predict the defect characteristics and root causes using historical data and could be incorporated into the decision support system of the server manufacturing environment. There are several avenues that future research could follow to overcome the limitations of the proposed models. Efforts can be made to increase the accuracy levels of the model parameters by conducting Design of Experiment. Furthermore, a larger set of data as well as data from other defect prone sectors can be analyzed by adjusting the proposed framework model parameters. References 1. Ramakrishnan, S., Tsai, P.-F., Srihari, K., and Foltz, C., 2008, Using Design of Experiments and Simulation Modeling to Study the Facility Layout for a Server Assembly Process, Proc. of the 2008 Industrial Engineering Research Conference, May 17-21, Vancouver, BC, 601-606. 2. Cao, H., Xi, H., and Smith, S.F., 2003, A Reinforcement Learning Approach to Production Planning in the Fabrication/Fulfillment Manufacturing Process, Proc. of the 35th Winter Simulation Conference, December 7-10, New Orleans, LA, 1417-1423. 3. Lendermann, P., 2006, About the Need for Distributed Simulation Technology for the Resolution of Real- World Manufacturing and Logistics Problems, Proc. of the 2006 Winter Simulation Conference, December 3-6, Monterey, CA, 1119-1128. 4. Efendigil, T., Önüt, S., and Kahraman, C., 2009, A Decision Support System for Demand Forecasting with Artificial Neural Networks and Neuro-fuzzy Models: A Comparative Analysis, Expert Systems with Applications, 36(1), 6697-6707. 5. Siglaz, 2011, Intelligent Defect Analysis, Framework for Integrated Data Management, Available at http://www.siglaz.com/newsroom/whitepapers/pdf/whitepaper.pdf, Accessed Decemmber 26, 2014. 6. Zheng, J., 2010, Cost-sensitive Boosting Neural Networks for Software Defect Prediction, Expert Systems with Applications, 37(6), 4537-4543. 7. Yousef, A.H., 2014, Extracting Software Static Defect Models Using Data Mining, Ain Shams Engineering Journal, 6(1), 13-144. 8

8. Tsai, C.-Y., Chiu, C-C., and Chen, J.-S., 2005, A Case-based Reasoning System for PCB Defect Prediction, Expert Systems with Applications, 28(4), 813-822. 9. Tosun, A., Bener, A., Turhan, B., and Menzies, T., 2010, Practical Considerations in Deploying Statistical Methods for Defect Prediction: A Case Study within the Turkish Telecommunications Industry, Information and Software Technology, 52(11), 1242-1257. 10. Mirapeix, J., García-Allende, P.B., Cobo, A., Conde, O.M., López-Higuera, J.M., 2007, Real-Time Arc- Welding Defect Detection and Classification with Principal Component Analysis and Artificial Neural Networks, NDT & E International, 40(4), 315-323. 11. Leung, H. C., 1995, Neural Networks in Supply Chain Management, Proc. of IEEE Annual International Engineering Management Conference, June 28-30, 347-352. 12. George, G., Haas, M.R., and Pentland, A., 2014, Big Data and Management, Academy of Management Journal, 57(2), 321 326. 13. Aiden, E., and Michel, J.-B., Dec 25 2013, The Predictive Power of Big Data, Newsweek, Available at http://www.newsweek.com/predictive-power-big-data-225125, Accessed November 15, 2014. 14. Pang, B. and Lee, L., 2008, Opinion Mining and Sentiment Analysis, Foundations and Trends in Information Retrieval, 2(1-2), 1 135. 15. Schmunk, S., Höpken, W., Fuchs, M., and Lexhagen, M., 2013, Sentiment Analysis: Extracting Decisionrelevant Knowledge from UGC. In: Xiang, Z., Tussyadiah, I. (Eds.), Information and Communication Technologies in Tourism 2014. Springer Inter-national Publishing, New York, NY, 253 265. 16. Choudhary, A.K., Oluikpe, P.I., Harding, J.A., and Carrillo, P.M., 2009, The Needs and Benefits of Text Mining Applications on Post-Project Reviews, Computers in Industry, 60(9), 728-740. 17. Tan, C., Lee, L., Tang, J., Jiang, L., Zhou, M., and Li, P., 2011, User-level Sentiment Analysis Incorporating Social Networks, Proc. of the 17 th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, 1397-1405. 18. Haddi, E, Liu, X., and Shi, Y., 2013, The Role of Text Pre-processing in Sentiment Analysis, Information and Quantitative Management, 17(1), 26-32. 19. Smailović, J., Grčar, M., Lavrač, N., and Žnidaršič, M., 2014, Stream-based Active Learning for Sentiment Analysis in the Financial Domain, Information Sciences, 285(1), 181-203. 9