Measuring Universal Intelligence By: Tyler Staudinger

Transcription

1 [1] Measuring Universal Intelligence By: Tyler Staudinger

2 Overview How Do we Define Intelligence? Current Tests of Machine Intelligence The Ideal Test for Intelligence Components of an Intelligence Test An Example Algorithmic Information Theory The Universal Intelligence Test The Anytime Universal Intelligence Test The Future of Artificial Intelligence

3 How Do We Define Intelligence? Intelligence is the ability of a rational agent to perceive its environment, and take actions that maximize its chances of success in that environment. A rational agent is an entity capable of logical decision making processes. Create intelligence, not evaluate it. [2]

4 Landmarks in Artificial Intelligence The IBM supercomputer Deep Blue beat a reigning world chess champion, Gary Kasparov A Stanford robot won the DARPA Grand Challenge by driving autonomously for 131 miles along a desert trail 2011 The IBM question answering system Watson defeats two champions in the game Jeopardy! [8]

5 Current Tests of Machine Intelligence What is a Turing Test? Such a test has a number of shortcomings: It is anthropomorphic It does not give a score It is relatively easy to cheat It requires a human judge. [5] CAPTCHA ( Completely Automated Public Turing Test to tell Computers and Humans Apart) CAPTCHA s are easy questions for humans hard for A.I., i.e. Character Recognition In years CAPTCHA s will be hard for humans

6 The Current State of Machine Intelligence There is no system which is able to adapt to many environments, this is commonly called a Strong A.I. All the progress in Machine Intelligence has only addressed specific problems Even when considering the abilities of Watson, there is no true understanding behind its actions, it is simply a highly advanced search engine There is also no true test of Intelligence that has been developed Until Now

7 The Ideal Test for Intelligence Universal (Biological or Artificial) Quantitative ( Based upon Information Theory and probability) Can evaluate any past, present, or future system Can evaluate any intelligence level (brilliant/inept), or time scale (fast/slow) Time relative, longer test gives better estimate [3]

8 Components of an Intelligence Test Three parameters to make an intelligence measurement A subject to be examined, this is known as an agent. A structure that the agent operates in, this is known as an Environment A protocol from which a performance score can be derived Turing Test and CATCHA address the first two but do not give a score This can be addressed by adding a reward when the agent performs the desired action

9 The Interaction Between Agent and Environment Observation The agent inputs an action to the environment The environment outputs: An observation, and a reward This is an example of reinforcement learning Agent Reward Action Environment

10 Example: The Monkey and The Banana Consider a simple game 3 Buttons each under a cell meant to hold a ball One cell has a ball in it If the monkey hits the button under the cell with a ball in it, they get a banana From the definitions we just developed the rewards in this environment are: Now to evaluate the monkeys performance Performance is the Expected Value of all the rewards:

11 Hypothetical Agent Behaviors Some agents are able to learn an environment quickly while others cannot Some receive more reward in the beginning but receive less reward overall, while the opposite may be true for other agents. [1]

12 Problems with this Model Not every environment is the same, need quantify the complexity of an environment Should use many environments of varying complexity Complex environments should be weighted more Rewards must be balanced to penalize random guessing (more on this later) Time needs to be taken into account [6]

13 Quantifying Environment Complexity Algorithmic Information Theory relates the notions of computation and complexity Kolgmogorov complexity is the length of the shortest program p that outputs a given string x over a machine U. [1] Lets look at an example: This idea can easily be applied to an environment, in an environment the complexity is the shortest sequence of actions that will generate a reward.

14 Incorporating Environment Complexity into The Intelligence Measure Now we can come up with a definition of intelligence, that incorporates multiple environments with varying complexity [1]

15 Penalizing Random Behavior To ensure that an agents behavior is not simply random a balanced environment can be used. This can be accomplished by including penalties as well as rewards This allows for the quick identification of agents with random behavior

16 Incorporating Time into the Formulation The time to complete the test is finite so this must be taken into account The more time that elapses the better the estimate given [1]

17 The Final Anytime Intelligence Test Algorithm [1]

18 The Potential Results of the Test for Various Agents This system has all of the characteristics we stated for an ideal test It can be used for humans or an agent more intelligent than humans [1]

19 Problems with the Test and the Future of Machine Intelligence The test developed is much more quantitative and rigorous than its predecessors Environments would have to be defined by humans, this can introduce a source of error. There aren t any agents yet, that are more intelligent than humans As of yet there is no machine intelligence even close to that of a human Machines intelligences are specialized at one task and are not very versatile in a variety of environments like humans are. Perhaps one day a machine intelligence will be surpass humans but that will not happen in the near future. It may be possible to pass the Turing Test but true intelligence is a long way off [7]

20 ECE Core Topics: Algorithms, Probability, and Discrete Math An Algorithm is a set of well defined steps to accomplish a task The Conditional Probability (A B) is the probability of observing A given that B has occurred For example if we have 2 dice and we roll a 5 on one of them the conditional probability of obtaining a 7 when the second die is rolled is 1/6. Discrete Math: the realm of mathematics that deals with non-continuous functions and variables

21 Questions? [4]

22 Works Cited [1] Measuring universal intelligence: Towards an anytime intelligence test,artificial Intelligence Volume 174, Issue 18, December 2010, Pages [2]A.M. Turing, Computing machinery and intelligence, Mind 59 (1950), pp Full Text via CrossRef [3]M. Li and P. Vitányi, An Introduction to Kolmogorov Complexity and Its Applications (3rd ed.), Springer-Verlag, New York (2008). [4]D.L. Dowe, A.R. Hajek, A computational extension to the Turing test, in: Proceedings of the 4th Conference of the Australasian Cognitive Science Society, University of Newcastle, NSW, Australia, [5]D.L. Dowe and A.R. Hajek, A computational extension to the Turing test Technical report #97/322, Dept. Computer Science, Monash University, Melbourne, Australia, 9 pp. Images: [1] [2] [3] Intelligence.aspx [4] [5] [6] [7] [8]