MACIEJ MAJEWSKI Research Monograph nr 172 FUNDAMENTALS OF BUILDING INTELLIGENT INTERACTION SYSTEMS OF TECHNOLOGICAL DEVICES AND THEIR OPERATORS Abstract The book concerns fundamentals of building intelligent interaction systems of technological devices and their operators. The interaction systems are equipped with intelligent mechanisms for operator biometric identification, command meaning analysis, command effect analysis, command safety assessment, and technological process supervision.
The book introduces the principles of the work and its scope in the first chapter. The second chapter of the book discusses modelling of decision processes, particularly analysis of methods and models of decision processes, mathematical modelling of technological process accuracy, decision models of expert systems in the conditions of incoherent knowledge states, and selected problems of decision process modelling in technological processes of treated machine elements. Fundamentals of interaction of technical devices and their operators are the main focus in the third chapter. Included is a classification of human activity models, a mathematical model of information gathering by an operator, an information perception model and an assessment of human information burden. The decision process formalization, general models of operator activities in control systems, a model of realization of diagnostics tasks in human-technology systems is presented. Selected aspects of intelligent decision aiding systems in technological processes are presented in the fourth chapter. Discussed are the basis and methodology for the selection of tools, conditions and machining parameters through the application of artificial intelligence methods. The chapter also presents examples of automated grinding processes of small ceramic elements. Furthermore, discussion of problems of process quality assessment and decisions concerning tools and parameters of the process are included. In the following chapter, there is an analysis of usability of different methods of operator identification, and discussed are methods of biometric identification including methods consisting of the typing in of passwords. Selected problems of information processing are investigated in the sixth chapter, which among other things include: information entropy, relay of information between operator and technical devices, problems of operator efficiency assessment, processing of uncertain information by operator, and modelling of the interaction of operator and technical devices. In addition, fundamentals of intelligent methods of recognition of operator s handwriting are presented. The most important part of the book is the seventh chapter concerning development of fundamentals of building intelligent two-way voice communication systems between humans and technical devices. In the introduction, aspects of voice communication between humans and technical devices in a natural language are emphasized, and an analysis of the state of the art in the field of voice communication has been carried out. A concept of an intelligent layer of two-way voice communication of technological devices with their operators is presented, and the concept of modern supervision systems of automated production processes, also for mobile technologies. A detailed architecture of the intelligent two-way voice communication system of technological devices with the operator is
shown. Problems of intelligent assessment system of operator s ability for efficient processing of information streams from multiple sources are introduced. Problems involving recognition of commands issued by voice, and synthesis of voice messages to operator are characterized. Achievements in the field of recognition of operator s command meaning are presented. The chapter also describes operator s command effect analysis and safety assessment. Principles concerning execution of operator s commands, and experimental research on the system are also presented. In the conclusion, a summary is given which emphasizes the scientific significance of the work, and future directions for further development of interaction systems between humans and technical devices are described. Table of contents Table of abbreviations... 11 1. INTRODUCTION... 15 1.1. Principles of the work... 15 1.2. Scope of the work... 16 2. MODELLING OF DECISION PROCESSES... 19 2.1. Analysis of known methods and models of decision processes... 19 2.2. Mathematical modelling of technological process accuracy... 22 2.2.1. Modelling methods... 22 2.2.2. Effects of random disturbances and condition changes in the grinding process... 26 2.3. Decision models of expert systems in the conditions of incoherent knowledge states... 31 2.3.1. Consensus choice problem... 31 2.3.2. Quality analysis of expert information... 38 2.4. Selected problems of decision process modelling in technological processes... 46 2.4.1. Principal problems of decision process modelling in technological processes of treated machine elements... 46 2.4.2. Problems of decision process modelling in aspects of process quality with the use of probabilistic neural networks... 49
3. FUNDAMENTALS OF INTERACTION OF TECHNICAL DEVICES AND THEIR OPERATORS... 55 3.1. Classification of human activity models... 55 3.2. Mathematical model of information gathering by an operator... 63 3.3. Information perception model and an assessment of human information burden... 68 3.4. Decision process formalization... 74 3.5. General models of operator activities in control systems... 80 3.6. Model of realization of diagnostics tasks in human-technology systems... 86 4. INTELLIGENT DECISION AIDING SYSTEMS IN TECHNOLOGICAL PROCESSES... 93 4.1. Basis for the selection of tools, conditions and machining parameters through the application of artificial intelligence methods... 93 4.2. Automated grinding processes of small ceramic elements... 98 4.2.1. Operator s commands and messages of technological devices... 105 4.2.2. Example of machining method application... 110 4.3. Analysis of grinding processes of small ceramic elements in process supervision by an operator... 111 4.4. Methodology of tool selection through the application of artificial intelligence methods in grinding processes... 117 4.5. Methodology of parameter selection through the application of artificial intelligence methods in grinding processes of small ceramic elements... 122 4.6. Problems of process quality assessment... 124 4.7. Decisions concerning tools and parameters of the process... 127
5. OPERATOR IDENTIFICATION METHODS... 133 5.1. Analysis of usability of different methods of identification... 133 5.2. Operator identification with use of methods consisting of the typing in of passwords... 134 5.3. Methods of operator biometric identification... 138 6. SELECTED PROBLEMS OF INFORMATION PROCESSING BY HUMAN. 149 6.1. Information entropy... 149 6.2. Information relay between operator and technical devices... 153 6.3. Problems of operator efficiency assessment... 157 6.4. Operator processing of uncertain information... 162 6.5. Modelling of the interaction of operator and technical devices... 163 6.6. Operator s handwriting recognition... 165 7. INTELLIGENT TWO WAY VOICE COMMUNICATION SYSTEMS BETWEEN HUMANS AND TECHNICAL DEVICES... 173 7.1. Aspects of voice communication between humans and technical devices in a natural language... 173 7.2. Analysis of the state of the art in the field of voice communication... 175 7.3. Concept of modern supervision systems of technological devices and automated production processes... 177 7.3.1. Concept of an intelligent layer of two-way voice communication of technological devices with their operators... 177 7.3.2. Concept of modern supervision systems of automated production processes... 179 7.4. Architecture of intelligent two-way voice communication systems... 184 7.4.1. Fundamentals of building an intelligent two-way voice communication system of technological devices with the operator... 184 7.4.2. Intelligent assessment system of operator s ability for efficient processing of information streams from multiple sources... 188 7.5. Voice communication between operator and technological devices.. 193 7.5.1. Recognition of commands issued by voice... 193 7.5.2. Synthesis of voice messages to operator... 196 7.6. Recognition of operator s command meaning... 199 7.7. Operator s command effect analysis and safety assessment... 213
7.8. Command safety assessment in grinding process of small ceramic elements... 228 7.8.1. Statistical analysis of parameters for command safety assessment... 228 7.8.2. Safety analysis of selected commands considering realistic grinding process features... 233 7.9. Execution of operator s commands... 235 7.10. Simulation research on the system... 238 7.10.1. Research on recognition of operator s natural speech... 238 7.10.2. Research on recognition of operator s voice commands... 240 7.10.3. Research on the system using headset and directional microphone... 243 7.10.4. Research on the system with background noise... 244 7.10.5. Recognition of operator s command meaning... 245 7.10.6. Developed intelligent two-way voice communication system of technological devices with the operator... 247 8. SUMMARY... 261 8.1. Scientific significance of the work... 261 8.2. Application significance of the work... 262 8.3. Future directions for further development of interaction systems between humans and technical devices... 265 BIBLIOGRAPHY... 267 Abstract in Polish... 289 Abstract in English... 291 Department of Mechanical Engineering Koszalin University of Technology, Koszalin, Poland Koszalin 2010 ISSN 0239-7129 ISBN 978-83-7365-203-3 Copyright Wydawnictwo Uczelniane Politechniki Koszalińskiej