Turing Test: Contribution, Criticism and Variations

Transcription

1 Turing Test: Contribution, Criticism and Variations Reffat Sharmeen Department of Computer Science University of New Mexico 1 Introduction In 1950, Alan Turing proposed an answer to the question Can (or could) a machine think? by replacing it with a precise question Can (or could) a machine pass a certain type of test? In his imitation game, a person acts as an interrogator and interacts with two interlocutors through text message in computerized connection. Success of the program is achieved when, after allotted time for the test, interrogator cannot differentiate between human and machine interlocutors. Turing claimed that behavioral responses in the test demonstrate a machine indistinguishable from a human being. Since then, debates are going on and research is conducted from different perspectives. This paper is motivated by recent findings and developments of Turing Test which finally reveals its contribution in the area of Artificial Intelligence even with criticisms. 2 Philosophy of Computer Science The author argues that, whether a machine passing Turing Test(TT) should be built or not, this question widely depends on the open questions in philosophy of computer science (CS). Darren [1] shows that when extended Church-Turing theses are true, then a machine can be built in principle. When either or both of these theses are false, counterexamples can be shown by having physically possible conditions. The first open issue discussed in the paper is, if a machine can reproduce purely verbal behavior. It has been argued that Descartes language test and Turings imitation game are nearly identical. Language test distinguishes between automaton and being with a soul by measuring the ability to converse in natural language. According to Descartes, objects that are able to respond meaningfully must also have a soul; this idea supported TT. Author argued that passing TT does not depend on human 1

2 verbal behavior; rather it depends on particular open questions in the philosophy of CS. Church-Turing thesis has two modified versions; both of them are part of foundation and philosophy of CS. If they both are true, then in principle, it is possible and this paper describes how to develop a Turing-equivalent computer passing TT. This new method uses only a single person; input of the program is the questions asked and responses from subjects provide output. The author also argues that no computer can pass TT as they consider embedded and embodied cognition. A research shows that physical experience of human beings affect cognitive processing and asking salient questions to the subjects demonstrates cognitive and physical difference. But this characterization is rejected by Turing himself because computers can only do what we tell them to do. Computers cannot experience the world as human beings, so they cannot pass TT; writer shows that this claim is not followed from other argument. The paper notes that if a computer is equivalent to a universal Turing Machine, ability to simulate another computer system depends on processor speed, computer architecture etc. Results in philosophy of CS define possibility of a computer passing TT and possibility of a program having time profile indistinguishable from persons. Thus passing TT is not dependent on verbal human behavior, but it is dependent on open questions in the philosophy of CS. At the end, the author concluded by presenting preliminary remarks on the idea that machines, passing TT, have experience which reflect embedded experience of humans. 3 Can Machines Think? According to the TT, the only observable aspect of intelligence is human behavior, but argument came in that, notion of algorithm is not sufficient to develop an intelligent machine and human minds are stronger than algorithms in case of computing which questions the possibility of Artificial Intelligence(AI). Claims have been proposed that AI is not intelligent in every situation because the system needs to be algorithmic. Though optimistic researchers say that, AI is in principle possible, the AI debate fails to address this possibility because of using a very general notion of algorithm. Hoffmann [2] discusses about the confusing notions and restrictions of algorithm with concept of algorithmic information theory and revises the question on limitation of AI. As an obstacle of future AI, research shows that practically we address a finite number of requests to a system; so if the system can respond to them, it is sufficient. On the contrary, for a finite number of responses, an algorithm must exist which is able to deliver all desired responses. This problem of non-algorithmic nature of human mind is because of general notion of algorithm. If we consider intuition as a formal requirement, a large collection of possible responses can satisfy all requests, but this is not effective for AI debate because human beings have limited lifetime and they cannot act, react, absorb on infinite facts. Thus the classical Turing notion 2

3 of algorithm can only shift the AI debate away from the original intuition. Regarding Algorithmic Information Theory, Hoffmann [2] restricts TT that agents will communicate with environment through finite strings of symbols from a finite alphabet which makes an agent pass TT only if it gives certain outputs to the inputs. An agent will be said to learn if his response to the same pattern of input, changes over time. If the length of input stream is finite throughout an agents lifetime, then it is unrestricted which indicates that there must exist a Turing Machine which can model the intelligent behavior. The qualitative question, whether intelligent behavior is algorithmic or not, is replaced by this paper [2] in quantitative manner, whether a small set of rules can produce intelligent behavior which will pass TT. In spite of having technical details, this reformulation directs the AI debate to consider length of algorithm, at least the order of magnitude, to pass TT and it allows differentiating between rule-governed system and look-up table for a finite set of expected behaviors. 4 Criticisms of Turing Test While discussing drawbacks of TT, the author [4] first addressed minor issues which dealt with subtleties of the machine and interrogators output. To pass TT, a machine should be virtually indistinguishable from a human which drives it to make mistakes. But mistakes usually come in consequence of the machines intelligence algorithm. Also, developing machine which is slightly articulate than humans, can give us insight so that we become more intelligent about ourselves. Another issue pointed here is, TT says if a machine is intelligent or not but it does not allow the interrogator specifying a level of intelligence. Author argues that even if a machine learns at a similar rate and similar ways as human, it does not essentially change the nature of test. These criticisms can only be revealed when we build machines that are much intelligent. Chinese Room argument, a famous philosophical criticism of TT, says that TT can be passed by an unintelligent machine, but another study showed that the book in Chinese Room cannot exist in our universe. Philosophers believed that TT is not a necessary condition for intelligence, but the author confirms that it tests for general intelligence which makes it worth pursuing. The interrogator interacts with subjects through language which might seem unfair because we cannot confirm that language is sufficient to capture all types of intelligence. Toe-stepping game was considered analogous to the TT, but the author claims it as a false analogy because ability to step on toes indicates virtually no other abilities. Researchers once claimed that TT cannot capture sub articulate thoughts; a fraction of intelligence is missed in the test and at the end, machines are incapable to attain sub articulate intelligence. According to the author, if the last part is true, we can distinguish between a human and a machine, disregarding intelligence level of the machine because machines can learn from large datasets like internet and there is no need on programmers to write programs for every rule. To conclude, the writer 3

4 talks about nave psychology and Total Turing Test (TTT) as replacements of the basic TT. TTT combines language with motor skills but is proved as redundant. Inverted Turing Test is efficient for nave psychology but still it needs to pass TT. Truly Total Turing Test (TTTT) required entire specification of robots which makes it even harder without starting with a single robot that passes TT. The final criticism writer addresses is, TT can only capture human intelligence because the interrogator is a human and machines compete against another human. As humans cannot recognize non-human intelligence, we should not expect any test to do so. TT, in a sense, tests intelligence of man-made machines. To argue with Seagull Test, author says if interrogator asks multiple questions about human experience and machine answers intelligently but negatively, then there might be a way to identify the machine. Thus there is not much way to know about criticisms without knowing much about intelligence and TT is still a very good test. 5 Judgment of the Humanness To create convincing human-like autonomous agent for virtual reality applications, ability to imitate a human is a rapidly growing need. But attempts to display advanced cognitive ability have remained unsuccessful. As focusing on good robots was not fruitful, bad humans who were perceived as non-human by independent judges can rather be focused. Catherine, Matthieu [5] are interested to find out parameters responsible to guarantee positive identification of humans within social interaction. They used cognitive and emotional dimensions in a global behavioral characterization and suggested importance of multimodal, emotional and cognitive parameters. To experiment, authors extracted dialogues from the Loebner Prize in AI and analyzed characteristics of dialogues between human subjects who have been judged at least once as robots. Parameters are classified into three categories as descriptive, cognitive and indicator of interest. Numbers of word, sentence, post, mistakes are considered as descriptive parameters whereas number of self-reference, compliments, occurrence of aggressiveness and emotions are described as cognitive. Number of questions, cognitive words is taken as indicator of interest. The five dimensions, occurrence of aggressiveness, self-references, references to relatives, compliments, occurrence of emotions are selected to build patterns of behavioral expressiveness. Final result indicated that subjects judged as robots used fewer words, less articles, less compliments per post than those judged as humans. Number of posts and words per post made a significant difference as descriptive parameters but social words, emotional words, number of mistakes were not the same. Cognitive parameters show effects for number of compliments but does not show for self-reference or emotions. Judges displayed more interest for subjects who were judged as human and difference is considered in the count of number of questions. Thus authors concluded that, to be judged as human, robots must keep a balanced response to their human interlocutor within each behavioral dimension which eventually means that 4

5 final judgment of humanness, to some extent, lies in the eye of the judge himself. 6 Social and Emotional AI Turing Test(TT) focused on the capability of thinking rationally, but generic notion of a general intelligence appears to have notions of social intelligence and emotional intelligence [8]. Social intelligence is defined as the human capacity to interpret social relationship in terms of beliefs, desires so that humans can manage complex social changes by predicting others behavior and thoughts. Emotional intelligence is a part of social intelligence that enables human to reason about own and others feelings and predict possible future actions by understanding the information in them. In machine intelligence, the role of social and emotional intelligence depends on the machine perspective in general. From a pragmatic point of view, if a machine is built to make human life easier and it is able to serve the purpose, then it is defined as intelligent. Several metaphors can be used to conceptualize machine which eventually claims that social and emotional intelligence are required in machines. From the tools metaphor, either simple or complex, when humans feel that the operating device which was in control, is not working properly, then the underneath mechanism can be handled. The second metaphor, machines as a social entity, replaced cause and effect model as a complex mental model. Whether people will use or interact with the machine is governed by the predictable machine behavior. As human beings intend to control machines, simple ones can be conceptualized as tool metaphor, but for machines of high technology, social entity metaphor adapts mental model of human even if unpredictable behavior is added to it. Sensitive artificial listener (SAL) is a current approach to build socially interactive machines which focuses on soft skills required for natural feelings of the users. The real time dialogue system analyzes users voice, face and then upgrades its own state to generate behavior through 3d head and synthetic voice. Cheerful Poppy, pragmatic Prudence, aggressive Spike and gloomy Obadiah are the four characters of different personality who gives variety of system response based on users current emotion. SEMAINE is considered as the first complete and standard autonomous implementation of SAL concept whereas OpenSMILE tracks users emotion from voice. A database of human to human dialogues is maintained to keep the system fully functional. To assess the user interactions with SAL agents, a number of evaluations are embedded into SEMAINE project which creates strong picture of success and failure of global or domain specific aspects. Yuck button approach and self report sub dialogues are built up to minimize verbal processing or cases that are hard to verbalize. Thus machines can be conceptualized as social entities when they are able to predict human behavior on given states. 5

6 7 Impact in Computational Creativity Three issues motivate computational creativity (CC); providing computational perspective on human creativity, enabling machines to be creative and producing tools which enhance human creativity [7]. In order to address these, CC has two notions to evaluate; judgment to determine if an idea is valuable and judgment to define if a system is acting creatively. To understand creativity in humans, testing is important but they do not affect humans to be creative. TT has been tried in CC because they can help to evaluate candidate programs in CC; also using an imitation based test to evaluate thought based area has philosophical problems. A related study, the discrimination test, was relevant for musical creativity within a specific style where subjects are played segments of both machine and human generated music and they were asked to distinguish them. A research shows that several computer generated systems pass TT as they model exploratory creativity where a style is explored. But the author argues that it presented a different version of TT with no interaction with system and disjunctive relationship between evaluation criteria allows systems passing TT in a different way. This kind of assessment can validate art generating software but is inappropriate for computational creativity context. To be considered as creative, software might not have a complete set of behaviors to exhibit but several features immediately disregard them as uncreative. Three kinds of criticisms are proposed; software that do not exhibit enough skill, do not appreciate others and do not show imagination in their processing. Hence software in CC context should present skillful, appreciative and imaginative behavior. In this context, attempts to pass TT might cause to lose valuable styles of creativity; it might not focus to the background and contextual information for creative acts; instead TT encourages pastiche and superficial uninteresting advances in front end and fail to reward genuine advances which make it impractical for CC research. To develop and evaluate creative software, authors introduced two descriptive models, FACE and IDEA, which are frameworks inspired by human creativity but are not aimed to capture all of them. The first model proposes creative acts as prominent units to be assessed in creative systems and the second one describes ways of evaluating those. The FACE model has eight kinds of generative acts which aid to describe the acts undertaken by computer but it is not sufficient to cover all creative software. The Iterative Development Execution Appreciation (IDEA) assumes a cycle which enables software to be engineered and expose its behavior to an audience. The authors suggested six stages to develop creative software; developmental, finetuning, re-invention, discovery, disruption and disorientation; which aid the analysis of software to compare and contrast creative programs by providing an acute tool. 8 Behavioral Turing Test Turing Test(TT) uses natural language to communicate between the agents which make it hard to be useful in the field of artificial intelligence. Evaluating intelligence appropriately through TT varies with evaluators background knowledge and context. 6

7 Real agents like robots need long term interaction and more intelligent behavior to themselves and also to the users. Hence simple behaviors like motions or gestures get focus to shrink rather than expand the features of TT [6]. In behavioral TT, the authors first determined the critical elements in communication between user and robot through a preliminary experiment where human manipulator controlled robot box with video image as input and human users put objects into the box. Analysis of preliminary test results led them to build a behavioral model for robots to automatically interact with human without human interference to copy the human manipulators behavior. The combined effect is called behavioral TT where users followed robot instruction and then identified either human or program controlled the robot. A manipulator of the robot and a user of the robot were the participants for the preliminary experiment who took turns as manipulator. The manipulators allowed the users moving objects into the robot box and sit down or stand up from chair. Eight persons participated into the experiment and the results indicated that humans are able to understand robot intention even in a restricted environment. While building an interactive model, shape and behavior of robot with initialization, observation and action phase play important roles. In the experiment, though some users were not successful to understand that front side of box indicates intentional direction of robot, it was generalized that distinctive shape and texture of object are helpful for the purpose. The behavior of robot found in the experiment is categorized as intentional and regulative gesture. Nodding and shaking, the intentional gestures are more time restrictive whereas action repeat or interruption during action is flexible. In the initialization phase, robot shows its intention to user who realizes the information in observation phase and performs the intended action in action phase where wrong action is pointed by negative gesture. The experimental result suggests that when robot box is controlled by human, all participants gave correct answers, even when a program controlled robot box was used, most users successfully identified computer as controller. Operator was totally hidden form users and the system used only head and hand positions to interact in a motion capture system. Thus the behavioral model extends TT in interactive real world and proves the test successful even in a restricted level. 9 Gestural Turing Test Classical AI focuses verbal language as ultimate indicator of human intelligence but to create believable characters in collaborative virtual environment, situated agent must adapt with environment where only textual communication might not be the best representation of intelligence. The open problem, simulating adaptive human like motion, recently approaches to use located intelligence or gestural energy. In this stream, this paper [10] uses TT with an alphabet of three moving dots, instead of alphabet of text characters, and used no verbal communication; thus agents could interact and generate spontaneous body language from interactions. Though there exists way to increase believability, this research concentrates on first principle of 7

8 motion behavior for primitive communication and silent co presence instead of addressing issues with co verbal gesture. According to the authors, Gestural TT does not need model for entire human body, rather three dots for head and hands can be used as locus of communication because they are the most motion expressive points of body. More points might be more useful in the experiment but that would be more complex for physical modeling and also for AI. In the experiment, the human observer, with a motion capture marker, had a projected screen with two sets of three dots. On the other side of screen, three dots are moved by either with another human, with similar marker or software program which can simulate human made motion of dots. Sign language, text, sound between the agents were prohibited. The paper presented two algorithms; both of them analyzed the detected energy of humans motion in response to three moving points and the variation with threshold indicates which type of gesture was produced. The first algorithm (AI1) used a set of pre recorded human gesture consisting of ambient and dynamic motions. While playing dynamic gestures, AI1 played ambient gestures for smaller movements and did not consider smooth transition intentionally so that subjects have base level non-believability to judge other motions. The second algorithm (AI2) used procedurally generated motion through sine and cosine oscillations, and imitative motions generated from positions of subjects motions; combined effects gives AI2 smoother blending than AI1. The experimental data reveals that most people guessed correct and AI2 performed well in case of believability where the performance was measured in duration. In spite of technical limitation and internet latency, the results showed that agents can imitate human gesture which is useful in gestural AI algorithm for creating better believable characters. 10 Improved CAPTCHA Method With high development in computer technology, internet usage for personal and professional purpose has increased. Often we are in need of registering ourselves for getting services which helps anonymous users to do false enrollment. To prevent abuse from these bots or automated programs, the Completely Automated Public Turing Test to tell Computers and Humans Apart(CAPTCHA) method is discovered for fighting against computer generated inputs by determining whether its user is human or not [9]. Currently OCR based and non-ocr based CAPTCHA methods are used; authors of this paper used non-ocr based approach. In native CAPTCHA, a distorted English word is presented to user who is asked to type the word as bots cannot identify distorted word. OCR approach uses image of distorted word and its pictorial effect to the user so that humans can understand but computer will face problems to detect the word. Low success of OCR based CAPTCHA lead scientists to think about non-ocr based idea which, in reality, was found easier to work with. Divide and conquer approach from neural network, segmentation and character recognition technique, font tricks, noise, color model are 8

9 examples of early experiment. This paper proposed an improved procedure for the Collage CAPTCHA which uses a set of slightly rotated objects and let users find out the goal object from them. The new algorithm shows a bunch of different types of object on left screen and an image of an object to the right. User is asked to select the shown object from left and type the name. User is considered as human being if his guess is right for choosing and writing the same object. Resistance level of this experiment could be increased by adding mirror image in the object set, using less distance among them or overlapping few images. The program is easy to implement in PDA; easy for using even for disabled people that makes this method worthy to research more. 11 Ultimate Strengthening of Turing Test With advent and progress of computers, philosophy as well as our life has turned into a new direction. Artificial thinking is still considered impossible but our philosophy cannot reject the available possibilities. AI is becoming a threat to our unique perception, language and even to our mind in reality, but viewers believe that artificially intelligent machine can do decision-making, free will but they are not capable to do the same for humor, morality or spirituality [3]. TT is a modest approach to identify the natural and exclusively human with a description of being human. It emerged with possibility of automatic complex data transfer and data processing. The current AI debate started in the paper with computer issue that computerized machines cannot interpret, they can only respond to the running programs. Computers do not understand they cannot transfer information; they can merely transfer data. On the contrary, sufficiently complex programs might be able to transmit or understand information. To resolve the debate, this paper suggests differentiating between data transmission and information transfer. As different situations play different roles to answer even a simple question, background knowledge should be included while creating AI machines. In TT, conversation was conducted without direct contact and at a distance. To extend the test step by step, general intelligence about being human can be experimented first. Contact method can be varied via voice or face-to-face. This functional humanness can be justified if the artificial creature acts in a way that is inseparable from human. Practical problems in AI, which might occur in the test is, how robots will classify important information without having complete idea of the domain. At the end, the author suggests a hypothetical variation of TT. A machine will be judged as human if it can bring up a child from birth till adulthood using all of its programs and material assets but no human support at all. Successful implementation of this test will bring up possibility of artificially developed humanity which will shed light into human intelligence. 9

10 12 Conclusion The Turing Test is a famous operational way to check if robots can pretend to be indistinguishable from human beings. Since proposed, it set up many avenues for research and discussion whether it is sufficient for the future of AI and if it can be evaluated in several areas of computer science. In addition to philosophical aspect, motion and computational activities can be included in the test. A reverse way to the Turing Test might be effective to the internet security. Though criticized in manifold directions, Turing Test is still the best indicator of intelligence. In this context, this paper has investigated research to the Turing Test where the roles of imitating a human go beyond the fundamental issues in AI. References [1] Darren Abramson. Philosophy of mind is (in part) philosophy of computer science. Minds and Machines, 21(2): , [2] Achim Hoffmann. Can machines think? an old question reformulated. Minds and Machines, 20(2): , [3] Stanisaw Krajewski. The ultimate strengthening of the turing test. page , [4] Katrina LaCurts. Criticisms of the turing test and why you should ignore (most of) them. Official Blog of MITs Course: Philosophy and Theoretical Computer Science, [5] Catherine L. Lortie and Matthieu J. Guitton. Judgment of the humanness of an interlocutor is in the eye of the beholder. PLoS ONE, 6(9):e25085, [6] Hirotaka Osawa et al. Behavioral turing test using two-axis actuators. The 21st IEEE International Symposium on Robot and Human Interactive Communication, [7] Alison Pease and Simon Colton. On impact and evaluation in computational creativity: A discussion of the turing test and an alternative proposal. Proceedings of the AISB symposium on AI and Philosophy, [8] Marc Schröder and Gary McKeown. Considering social and emotional artificial intelligence. Proc. AISB 2010 Symposium Towards a Comprehensive Intelligence Test, [9] Rituraj Soni and Devendra Tiwari. Improved captcha method. International Journal of Computer Applications IJCA, 1(25): , [10] Jeffrey Ventrella et al. Gestural turing test. International Workshop on Interacting with ECAs as Virtual Characters,