Agent Simulation of Hull s Drive Theory

Size: px
Start display at page:

Download "Agent Simulation of Hull s Drive Theory"

Transcription

1 Agent Simulation of Hull s Drive Theory Nick Schmansky Department of Cognitive and Neural Systems Boston University March 7, 4 Abstract A computer simulation was conducted of an agent attempting to survive in a D environment, given existence of food, water and shock objects. The behavior of the agent was governed by Grossberg s [3] neural model of instrumental and classical conditioning that captures the concepts of Hull s drive theory [4]. The drives of pain, hunger and thirst were simulated, and acted as reinforcement signals to successfully learn approach-avoidance behavior. Introduction In 943, Hull [4] proposed a theory of motivated behavior where internal drivestates direct action to reduce the level of the primary drive in the human or animal. Hull expressed his theory in the form of an equation, where the potential for reaction to a particular stimulus is a multiplicative law combining habits and drive. Reward and inhibition factors were added later, the result is equation (): SE R = D K H I () In this equation, the likelihood that a given behavior will occur is S E R. D represents drive level, K represents reward, H is the habit strength, and I represents inhibition. The important idea behind the equation is that all these factors must work synergistically in order for a behavior to have a high probability of being emitted, resulting in selecting the most appropriate behavior in a given situation, while suppressing less adequate behavior. A habit hierarchy emerges. [] Later, Grossberg [3] proposed a neural model of instrumental and classical conditioning that captures the concepts of Hull s drive theory (among many other theories of motivated behavior). In the model, neurons simulate sensory stimuli, internal drive signals, and motor effectors. Reinforcement acts to focus

2 attention on important sensory stimuli, and learning is captured in the neuron connection weights. Drives and incentives combine in a multiplicative manner. Grossberg s model is schematized in figure. Each box represents a collection of neurons. Modifiable connections (subject to learning) are shown as filled half-circles. Of importance is the gating function (depicted as an AND gate symbol) whereby drive units, such as pain, hunger or thirst, control whether stimuli may pass-through to a modulation block (depicted as a filled-box with an arrow through it), which drives the behavior units. Notice that stimuli may take direct control over behavior, shown as the UCS UCR pathway. Also note that UCS are directed toward the drive units, which may pair with CS as the basis of the learning process. Conditioned Stimuli (CS) Gated Conditioned Stimuli Drive Units (Reward/Punishment) Unconditioned Stimuli (UCS) Behavior Generation (UCR/CR) Figure : Schematic of Grossberg s neural model of classical and operant conditioning To explore Hull s drive theory within the context of Grossberg s neural model, a computer simulation of an agent surviving in a D environment was conducted. Observations were made on whether the agent s behavior was properly modulated by it s internal drive states, and whether the agent could adapt, by learning associations, to the stimuli encountered in its environment. It is expected that the agent should learn to avoid shock (painful) stimuli, and to seek food and water (satiating) stimuli.

3 Methods The MATLAB toolkit was used to develop and execute the simulation. The simulated components included a bounded D arena (5x5 pixels in size) containing three object-types and the agent. A snapshot of the arena is shown in figure. The objects include a shock device (red Xs), food (green circles), and water (blue squares). A random number of each object-type is distributed on the arena at startup. To provide variability in the environment, at random intervals, the simulation will re-activate and move consumed objects (food and water), as well as moving some small number of shock objects. 5 Arena:: Agent + Food O Water [ ] Shock X Figure : Simulated agent and environment. The agent has a 6 o field of view. A water object is the closest object in this snapshot. The brain of the agent is a simple implementation of Grossberg s [3] neural model shown in figure. The factors of Hull s drive equation () are implicitly encoded in Grossberg s model and this implementation. A schematic of the implemented model is shown in figure 3. Referring to figure 3, the agent is endowed with vision receptors for red, green and blue stimuli. It s visual field (in it s direction of gaze) is 6 o wide (as shown in figure ). There are 5 neurons 3

4 associated with each color, each encoding 4 o of the visual field. The activity of the neuron correlates with range to the object. Thus, close objects produce high neural activity in the neuron upon which that object falls in the visual field. Distant objects induce lower activity. The activity within each color channel (a bank of 5 neurons per color) is normalized. A plot of receptor intensity level from a sample run is shown in figure 4. The center unit (#8 in figure 4) corresponds to the direction of gaze. Lower numbered units code a progressively left-ward field of view, and higher numbered units code a progressively rightward field of view. max channel selector three channel vision receptor drive learning units (weights) R G B pain visual field R G B R G B hunger hunger level R G B thirst max drive unit selector shock water food contact detector thirst level turn angle and forward motion control shock food water Figure 3: Implemented form of the Grossberg neural model The agent possesses an object contact and sense detector, thus, when it is within a pixel of an object, that object will induce an unconditioned stimulus (US). Red objects produce shock, inducing pain. Green objects represent food, inducing hunger satiation. Blue objects represent water, inducing thirst satiation. 4

5 .9 Red Green Blue RGB Receptors Intensity Visual Field Figure 4: Simulated visual field receptor. The center unit (8) corresponds to the direction of gaze, spreading outward right and left to the edge units, which detect the edge of the visual field. The field is 6 o wide. In this instance, the receptors indicate the objects detected in the arena shown in figure. The agent is endowed with three drives: pain, hunger and thirst. The pain drive is always present, that is, the agent always does not like pain (delivered as a shock in the environment). The hunger and thirst drives are initially zero, and increase monotonically. Once a threshold is reached on either drive, then the agent is motivated to behave. The hunger threshold is three times greater than the threshold for thirst. A graph of these drives from a sample run is shown in figure 5. The agent s innate behavior (akin to its unconditioned responses), is to seek the closest object in its field of view, and make a small random turn if no object is sighted. Upon contact with food, its hunger drive is reset to zero. Upon contact with water, its thirst drive is reset to zero. If a food or water object is consumed, the water or food object s color changes to yellow, indicating it is no longer a consumable (an arena mechanism will periodically re-activate this consumable). The agent will not consume food if it s hunger drive level is below threshold, nor will it consume water if it s thirst drive is below threshold. Upon contact with a shock object, the agent will make a sharp turn and move a few pixels away. In addition to the innate behavior, the agent contains modifiable weights which factor into the control the direction of gaze. There are three sets of weights, corresponding to pain, hunger and thirst. Each set is composed of three sets of 5 neurons, which map to the color channels from the vision receptor. Upon contact with an object, depending on the drive associated with the contact stimulus, the weights are increased by the intensity detected by the vision receptor. When shocked, the pain drive weights are modified. Having 5

6 4 3.5 Hunger Thirst Agent Drive Nodes Figure 5: Internal hunger and thirst drive states of the agent eaten, the hunger drive weights are modified. Having drank, the thirst drive weights are modified. The conditioned response learned for shock stimuli is to turn away when the stimulus found in the receptor at the time of the shock was detected. When food or water is contacted, the learned conditioned response is to turn toward whatever the receptor field detected at the time. Thus, classic CS-US pairing takes place, based on the color stimulus of the object in the visual field. This is the mechanism by which the agent is supposed to learn avoidance behavior of objects producing shock (irrespective of color), and approach behavior to food and hunger objects (again, irrespective of color). The agent initially has no knowledge of the content of each colored object. It must learn the meaning of the object through reinforced experience. In a direct implementation of Hull s theory, drives are nonspecific, that is, activation of the thirst drive may elicit approach of a food object. However, this agent s implementation allows for specific drive activation. Thus, if the agent is not hungry, then the hunger drive unit s learned cues are not activated. Observations during experimentation were made to determine whether this mechanism holds true. Lastly, at some predetermined timestep within a session (once it appears that a fair degree of learning has taken place) the simulation will perform a contingency change in the form of red objects delivering food, green objects water, and blue objects a shock. In this way, observations can be made on how the agent responds to this change in stimulus/response characteristics. 6

7 3 Results The simulation was successful in adequately modelling the innate tendency of the agent to move toward the closest object, and to make random turns if nothing was sighted. Movement was initiated only if the hunger or thirst drive had incremented to it s threshold. This behavior is not surprising, however, as it is reflected in straightforward coding within the agent. Learning, via repeated weight adjustment following contact with an object, proved to be adequate to teach the agent to turn away from red objects in future encounters. Conversely, for food and water objects, the agent learned to turn toward the proper stimulus color. Figure 6 depict the weight changes after timesteps, where initial weights are zero. The changes clearly indicate learning a tendency to direct toward the center (the middle units) given the appropriate colored stimulus. For each drive, the expected stimulus color corresponding to the object primarily affecting that drive has increased (red pain, green food, blue water). Two effects are evident in these plots. Firstly, the position of the stimulus at the time of learning is slightly off the center axis (node #8) in each case. This is due to the randomness of the approach angle to each object. Secondly, at the time of object contact (when stimulus learning takes place) random numbers of different object types are inevitably in the distance, and some small association value is often included. This is evident in each plot in figure 6. Along these same lines, an interesting observation made in some scenarios was that if a food or water object happened to be very close and within the field of view when contact was made with a red shock object, then the agent would strongly negatively associate green stimuli (food objects), or blue stimuli (water objects, as the case may be) with pain, and would avoid food or water objects (green or blue stimulus) in the future. Only by chance could this weighting be undone by positive pairings between food (water) and green (blue) stimulus. Another observation made was that, given a particular grouping of shock objects, upon learning to avoid such objects, the agent could get caught in a corner of the arena, or swing away from any food or water in its field of view if a shock object was also closely in view. Thus, the agent could be scared to the point of not being able eat or drink. Unexpected behavior emerged in the contingency change experiment. Recall that in this experiment, after a predetermined number of timesteps, in this case steps, the red objects contain food instead of shock, green objects deliver water instead of food, and blue objects deliver a shock instead of water. Thus, it would be expected that following the steps of learning (enough for the agent to learn to associate red with shock, green with food, and blue with water), that the agent would either go hungry because (red) food objects would be avoided, or that the incorrect associations would become quickly unlearned. In fact, what was observed was that indeed the agent went hungry, rarely touching a food object, due to avoiding the red food objects. Figure 7, a plot of the hunger and thirst drive levels over time, clearly demonstrates the lack of hunger drive fulfillment following the contingency change at timestep. 7

8 Figures 8, 9, and are plots of the changes in drive weights before and after the contingency change. In each, the top plot, taken at timestep, indicates that the agent has learned a normal color stimulus/drive reaction association. The bottom plots were taken at timestep, considered an adequate amount of time to retrain. The bottom plot in figure 8 reveals why the agent was unable to feed, as the learned red stimulus pattern is still fairly strong alongside the newly learned aversion to blue stimuli. Comparison of plots in figure 9 show that very little, if any, new learning has taken place to unlearn the former association of green with food. Figure reveals that some learning of green to water association has taken place, but tends to be off-axis (to the agents left). Notice the former association of blue to water is still present, thus the agent remains attracted to blue objects. Given this situation, it is doubtful that even given ample training time whether this agent could ever regain proper associations. 8

9 .5 Pain Drive Hunger Drive Thirst Drive Figure 6: controlling motor units associated with pain, hunger, and thirst drive units following timesteps of activity. For each drive, the expected stimulus color (shown as a solid line) corresponding to the object primarily affecting that drive has increased (red pain, green food, blue water). The middle units encode the agent s view directly in front of it. 9

10 4 3.5 Hunger Thirst Agent Drive Nodes Figure 7: Internal hunger and thirst drive states of the agent during the course of normal learning (up to timestep ) and following a contingency change (at timestep ). The contingency change causes the hunger drive to rarely achieve satiation.

11 5 5.9 Pain Drive Pain Drive Figure 8: Effect of contingency change on pain drive weights. The top plot is before the change, the bottom plot is after. The plots are shown properly scaled to clearly show the relative weight changes between objects. The plots show that the agent has learned the new blue-to-shock association, but retains a fairly strong aversion to red (formerly a shock object, now a necessary food object).

12 Hunger Drive Hunger Drive Figure 9: Effect of contingency change on hunger drive weights. The top plot is before the change, the bottom plot is after. The plots are shown properly scaled to clearly show the relative weight changes between objects. The plots show that the hunger drive retains the green-to-food association, whereas it should have learned a new red-to-food association.

13 5 5.9 Thirst Drive Thirst Drive Figure : Effect of contingency change on thirst drive weights. The top plot is before the change, the bottom plot is after. The plots are shown properly scaled to clearly show the relative weight changes between objects. The plots indicate an learning an association between green and water, but off-axis somewhat. This would have the effect of not undoing the old blue-to-water association, which accounts for the agent s retained attraction to blue (and subsequent shock). 3

14 4 Discussion It is difficult to say whether the simulation provided any new insight into whether Hull s drive theory, embedded in Grossberg s neural model, is a sufficient model of human or animal survival or learning function. The reason is that the simulation essentially encoded all the necessary behavior, where the adaptable weights were somewhat transparent in their operation. A model of this type is really more appropriate in mobile robotics (Chang []). Future work on this simulation could incorporate finer resolution across each component, thus allowing some possibility of making predictions, or observing ever more unusual behaviors. However, the danger is always present where incorporation of functionality may inadvertently incorporate the very features or predictions sought. References [] C. Chang. Biomimetic robotics: Application of biological learning theories to mobile robot behaviors, 999. [] C. Dorman and P. Gaudiano. Motivation. In M.A. Arbib, editor, Handbook of brain theory and neural networks. MIT Press, Cambridge, MA, 994. [3] S. Grossberg. A psychophysiological theory of reinforcement, drive, motivation and attention. Journal of Theoretical Neurobiology, :86 369, 98. [4] C. Hull. Principles of behavior,

Introduction to Psychology, 7th Edition, Rod Plotnik Module 9: Classical Conditioning. Module 9. Classical Conditioning

Introduction to Psychology, 7th Edition, Rod Plotnik Module 9: Classical Conditioning. Module 9. Classical Conditioning Module 9 Classical Conditioning THREE KINDS OF LEARNING 1. Classical conditioning a kind of learning in which a neutral stimulus acquires the ability to produce a response that was originally produced

More information

MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL

MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL G. Maria Priscilla 1 and C. P. Sumathi 2 1 S.N.R. Sons College (Autonomous), Coimbatore, India 2 SDNB Vaishnav College

More information

Okami Study Guide: Chapter 7

Okami Study Guide: Chapter 7 1 Chapter in Review 1. Learning is difficult to define, but most psychologists would agree that: In learning the organism acquires some new knowledge or behavior as a result of experience; learning can

More information

Classical vs. Operant Conditioning

Classical vs. Operant Conditioning Classical vs. Operant Conditioning Operant conditioning (R S RF ) A voluntary response (R) is followed by a reinforcing stimulus (S RF ) The voluntary response is more likely to be emitted by the organism.

More information

A. Learning Process through which experience causes permanent change in knowledge or behavior.

A. Learning Process through which experience causes permanent change in knowledge or behavior. Woolfolk, A. (2010). Chapter 6: Behavioral Views of Learning. In A. Woolfook (Ed.), Educational psychology (11th ed.). Columbus, OH: Pearson/Allyn & Bacon. This chapter begins by defining learning and

More information

NEUROEVOLUTION OF AUTO-TEACHING ARCHITECTURES

NEUROEVOLUTION OF AUTO-TEACHING ARCHITECTURES NEUROEVOLUTION OF AUTO-TEACHING ARCHITECTURES EDWARD ROBINSON & JOHN A. BULLINARIA School of Computer Science, University of Birmingham Edgbaston, Birmingham, B15 2TT, UK e.robinson@cs.bham.ac.uk This

More information

THE HUMAN BRAIN. observations and foundations

THE HUMAN BRAIN. observations and foundations THE HUMAN BRAIN observations and foundations brains versus computers a typical brain contains something like 100 billion miniscule cells called neurons estimates go from about 50 billion to as many as

More information

Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk

Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems

More information

Behavioural Therapy A GUIDE TO COUNSELLING THERAPIES (DVD) Published by: J & S Garrett Pty Ltd ACN 068 751 440

Behavioural Therapy A GUIDE TO COUNSELLING THERAPIES (DVD) Published by: J & S Garrett Pty Ltd ACN 068 751 440 Behavioural Therapy A GUIDE TO COUNSELLING THERAPIES (DVD) Published by: J & S Garrett Pty Ltd ACN 068 751 440 All Case Histories in this text are presented as examples only and any comparison which might

More information

IMPORTANT BEHAVIOURISTIC THEORIES

IMPORTANT BEHAVIOURISTIC THEORIES IMPORTANT BEHAVIOURISTIC THEORIES BEHAVIOURISTIC THEORIES PAVLOV THORNDIKE SKINNER PAVLOV S CLASSICAL CONDITIONING I. Introduction: Ivan Pavlov (1849-1936) was a Russian Physiologist who won Nobel Prize

More information

Designing Behavior-Based Systems

Designing Behavior-Based Systems Designing Behavior-Based Systems Objectives Use schema theory to design and program behaviors Design a complete behavioral system Understand how to develop a behavioral table for behaviors Understand how

More information

Section 2 - Behavior Modification Section 2.2 - Reinforcement

Section 2 - Behavior Modification Section 2.2 - Reinforcement Section 2 - Behavior Modification Section 2.2 - Reinforcement Positive versus Negative Reinforcement Wiki - reinforcement is an increase in the strength [or frequency] of a response following the change

More information

Learning. Learning. Associations 2/2/11. Classical Conditioning

Learning. Learning. Associations 2/2/11. Classical Conditioning Learning Classical Conditioning Learning Any relatively permanent change in the behavior, thoughts, and feelings of an organism Learning differs from reflex Our minds naturally connect events appearing

More information

Autodesk Fusion 360: Assemblies. Overview

Autodesk Fusion 360: Assemblies. Overview Overview In this module you will learn how different components can be put together to create an assembly. We will use several tools in Fusion 360 to make sure that these assemblies are constrained appropriately

More information

LEARNING. Chapter 6 (Bernstein), pages 194-229

LEARNING. Chapter 6 (Bernstein), pages 194-229 LEARNING Chapter 6 (Bernstein), pages 194-229 What is LEARNING? LEARNING is the adaptive process through which experience modifies preexisting behavior and understanding; relatively permanent change in

More information

Video-Based Eye Tracking

Video-Based Eye Tracking Video-Based Eye Tracking Our Experience with Advanced Stimuli Design for Eye Tracking Software A. RUFA, a G.L. MARIOTTINI, b D. PRATTICHIZZO, b D. ALESSANDRINI, b A. VICINO, b AND A. FEDERICO a a Department

More information

Growing Up With Epilepsy

Growing Up With Epilepsy Teaching Students with Epilepsy: Children with epilepsy often experience learning issues as a result of their seizures. These may include ongoing problems with motor skills or cognitive functions, as well

More information

Programmed Learning Review

Programmed Learning Review Programmed Learning Review L-HO1-121907 Take another sheet of paper and cover the answers located in the right hand column. Then read through the unit filling in the blanks as you go. After filling in

More information

Analecta Vol. 8, No. 2 ISSN 2064-7964

Analecta Vol. 8, No. 2 ISSN 2064-7964 EXPERIMENTAL APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS IN ENGINEERING PROCESSING SYSTEM S. Dadvandipour Institute of Information Engineering, University of Miskolc, Egyetemváros, 3515, Miskolc, Hungary,

More information

Classical Conditioning. Classical and Operant Conditioning. Basic effect. Classical Conditioning

Classical Conditioning. Classical and Operant Conditioning. Basic effect. Classical Conditioning Classical Conditioning Classical and Operant Conditioning January 16, 2001 Reminder of Basic Effect What makes for effective conditioning? How does classical conditioning work? Classical Conditioning Reflex-basic

More information

A Brief Explanation of Applied Behavior Analysis. conditioning to identify the contingencies affecting a student s behavior and the functions of the

A Brief Explanation of Applied Behavior Analysis. conditioning to identify the contingencies affecting a student s behavior and the functions of the A Brief Explanation of Applied Behavior Analysis Applied Behavior Analysis is the procedure for using the principles of operant conditioning to identify the contingencies affecting a student s behavior

More information

FUNCTIONAL ASSESSMENT: HYPOTHESIZING PREDICTORS AND PURPOSES OF PROBLEM BEHAVIOR TO IMPROVE BEHAVIOR-CHANGE PLANS

FUNCTIONAL ASSESSMENT: HYPOTHESIZING PREDICTORS AND PURPOSES OF PROBLEM BEHAVIOR TO IMPROVE BEHAVIOR-CHANGE PLANS APDT Journal Sept 2009 Susan G. Friedman, Ph.D. Word Count: 2,427 Biographical Sketch: Susan G. Friedman, Ph.D., is a psychology professor at Utah State University, specializing in applied behavior analysis

More information

Appendix 4 Simulation software for neuronal network models

Appendix 4 Simulation software for neuronal network models Appendix 4 Simulation software for neuronal network models D.1 Introduction This Appendix describes the Matlab software that has been made available with Cerebral Cortex: Principles of Operation (Rolls

More information

Module1. x 1000. y 800.

Module1. x 1000. y 800. Module1 1 Welcome to the first module of the course. It is indeed an exciting event to share with you the subject that has lot to offer both from theoretical side and practical aspects. To begin with,

More information

Classical and Operant Conditioning as Roots of Interaction for Robots

Classical and Operant Conditioning as Roots of Interaction for Robots Classical and Operant Conditioning as Roots of Interaction for Robots Jean Marc Salotti and Florent Lepretre Laboratoire EA487 Cognition et Facteurs Humains, Institut de Cognitique, Université de Bordeaux,

More information

How To Run Statistical Tests in Excel

How To Run Statistical Tests in Excel How To Run Statistical Tests in Excel Microsoft Excel is your best tool for storing and manipulating data, calculating basic descriptive statistics such as means and standard deviations, and conducting

More information

Learning is a relatively permanent change in behaviour that occurs through experience. It is a continuous process. It is a gradual process.

Learning is a relatively permanent change in behaviour that occurs through experience. It is a continuous process. It is a gradual process. Learning is a relatively permanent change in behaviour that occurs through experience. It is a continuous process. It is a gradual process. Process of Learning A stimulus could be an event, situation,

More information

RESCORLA-WAGNER MODEL

RESCORLA-WAGNER MODEL RESCORLA-WAGNER, LearningSeminar, page 1 RESCORLA-WAGNER MODEL I. HISTORY A. Ever since Pavlov, it was assumed that any CS followed contiguously by any US would result in conditioning. B. Not true: Contingency

More information

MINITAB ASSISTANT WHITE PAPER

MINITAB ASSISTANT WHITE PAPER MINITAB ASSISTANT WHITE PAPER This paper explains the research conducted by Minitab statisticians to develop the methods and data checks used in the Assistant in Minitab 17 Statistical Software. One-Way

More information

PRIMING OF POP-OUT AND CONSCIOUS PERCEPTION

PRIMING OF POP-OUT AND CONSCIOUS PERCEPTION PRIMING OF POP-OUT AND CONSCIOUS PERCEPTION Peremen Ziv and Lamy Dominique Department of Psychology, Tel-Aviv University zivperem@post.tau.ac.il domi@freud.tau.ac.il Abstract Research has demonstrated

More information

Operant Conditioning. Classical vs. Operant Conditioning. Classical vs. Operant Conditioning. Human Development and Learning

Operant Conditioning. Classical vs. Operant Conditioning. Classical vs. Operant Conditioning. Human Development and Learning Operant Conditioning EDS 248 Stephen E. Brock, Ph.D.,NCSP Classical vs. Operant Conditioning Operant conditioning (R S RF ) A voluntary response (R) is followed by a reinforcing stimulus (S RF ) As a result,

More information

Phobias: Irrational fears of specific objects or situations, often the result of a type of learning called classical conditioning

Phobias: Irrational fears of specific objects or situations, often the result of a type of learning called classical conditioning PSYC 1001 Page 1 Chapter 6 Phobias: Irrational fears of specific objects or situations, often the result of a type of learning called classical conditioning Learning: A relatively durable change in behaviour

More information

Instrumental Conditioning II

Instrumental Conditioning II Instrumental Conditioning II The shaping of behavior Conditioned reinforcement Response chaining Biological constraints The Stop-Action Principle Guthrie and Horton (1946) puzzle box experiment showed

More information

CHAPTER 6 PRINCIPLES OF NEURAL CIRCUITS.

CHAPTER 6 PRINCIPLES OF NEURAL CIRCUITS. CHAPTER 6 PRINCIPLES OF NEURAL CIRCUITS. 6.1. CONNECTIONS AMONG NEURONS Neurons are interconnected with one another to form circuits, much as electronic components are wired together to form a functional

More information

The operations performed to establish Pavlovian conditioned reflexes

The operations performed to establish Pavlovian conditioned reflexes ~ 1 ~ Pavlovian Conditioning and Its Proper Control Procedures Robert A. Rescorla The operations performed to establish Pavlovian conditioned reflexes require that the presentation of an unconditioned

More information

Handout 12: Understanding Randomness Evaluating a Claim of Hearing Loss STAT 100 Fall 2015

Handout 12: Understanding Randomness Evaluating a Claim of Hearing Loss STAT 100 Fall 2015 Consider the case study presented in an article by Pankratz, Fausti, and Peed titled A Forced-Choice Technique to Evaluate Deafness in the Hysterical or Malingering Patient. Source: Journal of Consulting

More information

Combining associative learning and nonassociative learning to achieve robust reactive navigation in mobile robots

Combining associative learning and nonassociative learning to achieve robust reactive navigation in mobile robots Combining associative learning and nonassociative learning to achieve robust reactive navigation in mobile robots Carolina Chang cchang@ldc.usb.ve Grupo de Inteligencia Artificial, Departamento de Computación

More information

Blender Notes. Introduction to Digital Modelling and Animation in Design Blender Tutorial - week 9 The Game Engine

Blender Notes. Introduction to Digital Modelling and Animation in Design Blender Tutorial - week 9 The Game Engine Blender Notes Introduction to Digital Modelling and Animation in Design Blender Tutorial - week 9 The Game Engine The Blender Game Engine This week we will have an introduction to the Game Engine build

More information

LEARNING MEYERS AND DEWALL CHAPTER 6

LEARNING MEYERS AND DEWALL CHAPTER 6 LEARNING MEYERS AND DEWALL CHAPTER 6 LEARNING OVERVIEW HOW DO WE LEARN? CLASSICAL CONDITIONING OPERANT CONDITIONING BIOLOGY, COGNITION, AND LEARNING LEARNING BY OBSERVATION LEARNING Learning the process

More information

Processing the Image or Can you Believe what you see? Light and Color for Nonscientists PHYS 1230

Processing the Image or Can you Believe what you see? Light and Color for Nonscientists PHYS 1230 Processing the Image or Can you Believe what you see? Light and Color for Nonscientists PHYS 1230 Optical Illusions http://www.michaelbach.de/ot/mot_mib/index.html Vision We construct images unconsciously

More information

Sensory-motor control scheme based on Kohonen Maps and AVITE model

Sensory-motor control scheme based on Kohonen Maps and AVITE model Sensory-motor control scheme based on Kohonen Maps and AVITE model Juan L. Pedreño-Molina, Antonio Guerrero-González, Oscar A. Florez-Giraldo, J. Molina-Vilaplana Technical University of Cartagena Department

More information

Life Cycle Asset Allocation A Suitable Approach for Defined Contribution Pension Plans

Life Cycle Asset Allocation A Suitable Approach for Defined Contribution Pension Plans Life Cycle Asset Allocation A Suitable Approach for Defined Contribution Pension Plans Challenges for defined contribution plans While Eastern Europe is a prominent example of the importance of defined

More information

Gas Dynamics Prof. T. M. Muruganandam Department of Aerospace Engineering Indian Institute of Technology, Madras. Module No - 12 Lecture No - 25

Gas Dynamics Prof. T. M. Muruganandam Department of Aerospace Engineering Indian Institute of Technology, Madras. Module No - 12 Lecture No - 25 (Refer Slide Time: 00:22) Gas Dynamics Prof. T. M. Muruganandam Department of Aerospace Engineering Indian Institute of Technology, Madras Module No - 12 Lecture No - 25 Prandtl-Meyer Function, Numerical

More information

Savvy software agents can encourage the use of second-order theory of mind by negotiators

Savvy software agents can encourage the use of second-order theory of mind by negotiators CHAPTER7 Savvy software agents can encourage the use of second-order theory of mind by negotiators Abstract In social settings, people often rely on their ability to reason about unobservable mental content

More information

Self Organizing Maps: Fundamentals

Self Organizing Maps: Fundamentals Self Organizing Maps: Fundamentals Introduction to Neural Networks : Lecture 16 John A. Bullinaria, 2004 1. What is a Self Organizing Map? 2. Topographic Maps 3. Setting up a Self Organizing Map 4. Kohonen

More information

ATTITUDE TO RISK. In this module we take a look at risk management and its importance. MODULE 5 INTRODUCTION PROGRAMME NOVEMBER 2012, EDITION 18

ATTITUDE TO RISK. In this module we take a look at risk management and its importance. MODULE 5 INTRODUCTION PROGRAMME NOVEMBER 2012, EDITION 18 INTRODUCTION PROGRAMME MODULE 5 ATTITUDE TO RISK In this module we take a look at risk management and its importance. NOVEMBER 2012, EDITION 18 CONTENTS 3 6 RISK MANAGEMENT 2 In the previous module we

More information

Hierarchical Clustering Analysis

Hierarchical Clustering Analysis Hierarchical Clustering Analysis What is Hierarchical Clustering? Hierarchical clustering is used to group similar objects into clusters. In the beginning, each row and/or column is considered a cluster.

More information

Chapter Outline. Learning. Ch. 6 Learning and Behavior General Psychology Jeffrey D. Leitzel, Ph.D.

Chapter Outline. Learning. Ch. 6 Learning and Behavior General Psychology Jeffrey D. Leitzel, Ph.D. Ch. 6 Learning and Behavior General Psychology Jeffrey D. Leitzel, Ph.D. Chapter Outline Types of Learning Classical Conditioning Operant Conditioning Observational Learning 2 Learning Learning: Relatively

More information

Identifying Market Price Levels using Differential Evolution

Identifying Market Price Levels using Differential Evolution Identifying Market Price Levels using Differential Evolution Michael Mayo University of Waikato, Hamilton, New Zealand mmayo@waikato.ac.nz WWW home page: http://www.cs.waikato.ac.nz/~mmayo/ Abstract. Evolutionary

More information

Today. Learning. Learning. What is Learning? The Biological Basis. Hebbian Learning in Neurons

Today. Learning. Learning. What is Learning? The Biological Basis. Hebbian Learning in Neurons Today Learning What is Learning? Classical conditioning Operant conditioning Intro Psychology Georgia Tech Instructor: Dr. Bruce Walker What is Learning? Depends on your purpose and perspective Could be

More information

Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham

Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham kms@cs.stir.ac.uk 1 What are Neural Networks? Neural Networks are networks of neurons, for example, as found in real (i.e. biological)

More information

A Spectral Clustering Approach to Validating Sensors via Their Peers in Distributed Sensor Networks

A Spectral Clustering Approach to Validating Sensors via Their Peers in Distributed Sensor Networks A Spectral Clustering Approach to Validating Sensors via Their Peers in Distributed Sensor Networks H. T. Kung Dario Vlah {htk, dario}@eecs.harvard.edu Harvard School of Engineering and Applied Sciences

More information

AP Psychology 2008-2009 Academic Year

AP Psychology 2008-2009 Academic Year AP Psychology 2008-2009 Academic Year Course Description: The College Board Advanced Placement Program describes Advanced Placement Psychology as a course that is designed to introduce students to the

More information

2-1 Position, Displacement, and Distance

2-1 Position, Displacement, and Distance 2-1 Position, Displacement, and Distance In describing an object s motion, we should first talk about position where is the object? A position is a vector because it has both a magnitude and a direction:

More information

Updated 2013 (Mathematica Version) M1.1. Lab M1: The Simple Pendulum

Updated 2013 (Mathematica Version) M1.1. Lab M1: The Simple Pendulum Updated 2013 (Mathematica Version) M1.1 Introduction. Lab M1: The Simple Pendulum The simple pendulum is a favorite introductory exercise because Galileo's experiments on pendulums in the early 1600s are

More information

Learning. Learning. relatively permanent change in an organism s behavior due to experience

Learning. Learning. relatively permanent change in an organism s behavior due to experience Chapter 7 Learning Learning Learning relatively permanent change in an organism s behavior due to experience Association We learn by association Our minds naturally connect events that occur in sequence

More information

Classical vs. Operant Conditioning

Classical vs. Operant Conditioning Classical vs. Operant Conditioning Operant conditioning (R S RF ) A voluntary response (R) is followed by a reinforcing stimulus (S RF ) The voluntary response is more likely to be emitted by the organism.

More information

Classroom Activities. These educational materials were developed by the Carnegie Science Center .

Classroom Activities. These educational materials were developed by the Carnegie Science Center <http://www.carnegiesciencecenter.org/>. Classroom Activities These materials are designed to increase students' understanding about what makes a robot a robot, how robots sense, think, and act as well as the uses and limitations of working robots.

More information

Factoring Patterns in the Gaussian Plane

Factoring Patterns in the Gaussian Plane Factoring Patterns in the Gaussian Plane Steve Phelps Introduction This paper describes discoveries made at the Park City Mathematics Institute, 00, as well as some proofs. Before the summer I understood

More information

Learning and Memory. Learning is a Process. The Learning Process. Behavioral Learning Theories. Classical Conditioning

Learning and Memory. Learning is a Process. The Learning Process. Behavioral Learning Theories. Classical Conditioning Learning and Memory The Learning Process Learning: A relatively permanent change in behavior caused by experience Incidental Learning: Casual, unintentional acquisition of knowledge Learning is an Ongoing

More information

A BEHAVIORAL VIEW OF LEARNING

A BEHAVIORAL VIEW OF LEARNING Chapter 10 Classical Conditioning Classical Conditioning: The Story of Dogs and Little Albert A BEHAVIORAL VIEW OF LEARNING As you read below you may come to think that behavioral learning theories seem

More information

Lab M1: The Simple Pendulum

Lab M1: The Simple Pendulum Lab M1: The Simple Pendulum Introduction. The simple pendulum is a favorite introductory exercise because Galileo's experiments on pendulums in the early 1600s are usually regarded as the beginning of

More information

Motivation What is it? Jay Gould 11/24/03

Motivation What is it? Jay Gould 11/24/03 Motivation What is it? Jay Gould 11/24/03 Motivation literally means: that which produces motion. It is often used interchangeably with the term drive. Drives have been thought of as internal states of

More information

Best Teaching Practices Conference. Teacher ID: BTPC07_07 SUBJECT: BIOLOGY. Class: X. TOPIC: Exploring our Nervous System

Best Teaching Practices Conference. Teacher ID: BTPC07_07 SUBJECT: BIOLOGY. Class: X. TOPIC: Exploring our Nervous System Best Teaching Practices Conference Teacher ID: BTPC07_07 SUBJECT: BIOLOGY Class: X TOPIC: Exploring our Nervous System OBJECTIVES: Use Information Technology to enable the students to: Explain the general

More information

Assessment of Camera Phone Distortion and Implications for Watermarking

Assessment of Camera Phone Distortion and Implications for Watermarking Assessment of Camera Phone Distortion and Implications for Watermarking Aparna Gurijala, Alastair Reed and Eric Evans Digimarc Corporation, 9405 SW Gemini Drive, Beaverton, OR 97008, USA 1. INTRODUCTION

More information

Chapter 5 OC (operant conditioning) quiz practice Name Period Date

Chapter 5 OC (operant conditioning) quiz practice Name Period Date Chapter 5 OC (operant conditioning) quiz practice Name Period Date MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question. 1) Thorndike was known for his

More information

Electrical Properties of Biological Systems

Electrical Properties of Biological Systems Laboratory 3 Electrical Properties of Biological Systems All cells are capable of maintaining a charge separation, or POTENTIAL DIFFERENCE, across their membranes. This is due to the diffusion of sodium

More information

The Impact of Potassium Concentration on Refractory Period in the Hodgkin Huxley Model

The Impact of Potassium Concentration on Refractory Period in the Hodgkin Huxley Model The Impact of Potassium Concentration on Refractory Period in the Hodgkin Huxley Model Margaret Douglass and Michael Salib December 7, 2001 Abstract We investigated the role of external potassium concentration

More information

IAI : Biological Intelligence and Neural Networks

IAI : Biological Intelligence and Neural Networks IAI : Biological Intelligence and Neural Networks John A. Bullinaria, 2005 1. How do Humans do Intelligent Things? 2. What are Neural Networks? 3. What are Artificial Neural Networks used for? 4. Introduction

More information

Facts about Weighted Blankets and Directions to Make Your Own by Linda McInnis

Facts about Weighted Blankets and Directions to Make Your Own by Linda McInnis www.complexchild.com Facts about Weighted Blankets and Directions to Make Your Own by Linda McInnis What Children May Benefit from a Weighted Blanket and Why: My daughter with Rett Syndrome and Mitochondrial

More information

Trial 9 No Pill Placebo Drug Trial 4. Trial 6.

Trial 9 No Pill Placebo Drug Trial 4. Trial 6. An essential part of science is communication of research results. In addition to written descriptions and interpretations, the data are presented in a figure that shows, in a visual format, the effect

More information

Online simulations of models for backward masking

Online simulations of models for backward masking Online simulations of models for backward masking Gregory Francis 1 Purdue University Department of Psychological Sciences 703 Third Street West Lafayette, IN 47907-2004 11 July 2002 Revised: 30 January

More information

Presents. Superstition in the Pigeon

Presents. Superstition in the Pigeon www.all-about-psychology.com Presents Superstition in the Pigeon By B. F. Skinner (1948) First Published in Journal of Experimental Psychology, 38, 168-172 To say that a reinforcement is contingent upon

More information

CHAPTER 11: Flip Flops

CHAPTER 11: Flip Flops CHAPTER 11: Flip Flops In this chapter, you will be building the part of the circuit that controls the command sequencing. The required circuit must operate the counter and the memory chip. When the teach

More information

Operant Conditioning

Operant Conditioning Operant Conditioning! Definition! Classical vs. Operant Conditioning! Four Basic Operant Procedures! Factors that make Operant procedures more effective! Primary and Secondary Reinforcers! Shaping Conditioning!

More information

Behaviorism: Laws of the Observable

Behaviorism: Laws of the Observable Behaviorism: Laws of the Observable The Backdrop to Watson: Functionalism at the Univ. of Chicago John Dewey, like James, was influenced by both Peirce and Darwin Moved to the University of Chicago in

More information

Chapter 5: Learning I. Introduction: What Is Learning? learning Conditioning II. Classical Conditioning: Associating Stimuli Ivan Pavlov

Chapter 5: Learning I. Introduction: What Is Learning? learning Conditioning II. Classical Conditioning: Associating Stimuli Ivan Pavlov Chapter 5: Learning I. Introduction: What Is Learning? A. Psychologists define learning as a process that produces a relatively enduring change in behavior or knowledge as a result of an individual s experience.

More information

A Cognitive Approach to Vision for a Mobile Robot

A Cognitive Approach to Vision for a Mobile Robot A Cognitive Approach to Vision for a Mobile Robot D. Paul Benjamin Christopher Funk Pace University, 1 Pace Plaza, New York, New York 10038, 212-346-1012 benjamin@pace.edu Damian Lyons Fordham University,

More information

SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING

SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING AAS 07-228 SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING INTRODUCTION James G. Miller * Two historical uncorrelated track (UCT) processing approaches have been employed using general perturbations

More information

The Capacity of Visual Short- Term Memory Is Set Both by Visual Information Load and by Number of Objects G.A. Alvarez and P.

The Capacity of Visual Short- Term Memory Is Set Both by Visual Information Load and by Number of Objects G.A. Alvarez and P. PSYCHOLOGICAL SCIENCE Research Article The Capacity of Visual Short- Term Memory Is Set Both by Visual Information Load and by Number of Objects G.A. Alvarez and P. Cavanagh Harvard University ABSTRACT

More information

Strategic Plan Proposal: Learning science by experiencing science: A proposal for new active learning courses in Psychology

Strategic Plan Proposal: Learning science by experiencing science: A proposal for new active learning courses in Psychology Strategic Plan Proposal: Learning science by experiencing science: A proposal for new active learning courses in Psychology Contacts: Jacob Feldman, (jacob.feldman@rutgers.edu, 848-445-1621) Eileen Kowler

More information

The Counterpropagation Network

The Counterpropagation Network 214 The Counterpropagation Network The Counterpropagation Network " The Counterpropagation network (CPN) is the most recently developed of the models that we have discussed so far in this text. The CPN

More information

Module 3 Crowd Animation Using Points, Particles and PFX Linker for creating crowd simulations in LightWave 8.3

Module 3 Crowd Animation Using Points, Particles and PFX Linker for creating crowd simulations in LightWave 8.3 Module 3 Crowd Animation Using Points, Particles and PFX Linker for creating crowd simulations in LightWave 8.3 Exercise 2 Section A Crowd Control Crowd simulation is something you see in movies every

More information

Lecture 1: Introduction to Reinforcement Learning

Lecture 1: Introduction to Reinforcement Learning Lecture 1: Introduction to Reinforcement Learning David Silver Outline 1 Admin 2 About Reinforcement Learning 3 The Reinforcement Learning Problem 4 Inside An RL Agent 5 Problems within Reinforcement Learning

More information

Where On Earth Will Three Different Satellites Provide Simultaneous Coverage?

Where On Earth Will Three Different Satellites Provide Simultaneous Coverage? Where On Earth Will Three Different Satellites Provide Simultaneous Coverage? In this exercise you will use STK/Coverage to model and analyze the quality and quantity of coverage provided by the Earth

More information

Beginner Modeling Exercises Section 2 Mental Simulation of Simple Positive Feedback

Beginner Modeling Exercises Section 2 Mental Simulation of Simple Positive Feedback D-4487 Beginner Modeling Exercises Section 2 Mental Simulation of Simple Positive Feedback Stock Flow Growth Factor Prepared for the MIT System Dynamics in Education Project Under the Supervision of Dr.

More information

Animal Environmental Internal Response Types

Animal Environmental Internal Response Types Animal Environmental Internal Response Types Stimuli Stimuli Panting (P) Sweating (S) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. What happens to you internally (in your body) when you run as fast as you can around

More information

RELEVANT TO ACCA QUALIFICATION PAPER P3. Studying Paper P3? Performance objectives 7, 8 and 9 are relevant to this exam

RELEVANT TO ACCA QUALIFICATION PAPER P3. Studying Paper P3? Performance objectives 7, 8 and 9 are relevant to this exam RELEVANT TO ACCA QUALIFICATION PAPER P3 Studying Paper P3? Performance objectives 7, 8 and 9 are relevant to this exam Business forecasting and strategic planning Quantitative data has always been supplied

More information

Classical Conditioning

Classical Conditioning PSYCHOLOGY (8th Edition, in Modules) David Myers PowerPoint Slides Aneeq Ahmad Henderson State University Worth Publishers, 2007 1 Classical Conditioning Module 21 2 Classical Conditioning How Do We Learn?

More information

Fish Chomp. Level. Activity Checklist Follow these INSTRUCTIONS one by one. Test Your Project Click on the green flag to TEST your code

Fish Chomp. Level. Activity Checklist Follow these INSTRUCTIONS one by one. Test Your Project Click on the green flag to TEST your code Introduction: We re going to make a game! Guide the large Hungry Fish and try to eat all the prey that are swimming around. Activity Checklist Follow these INSTRUCTIONS one by one Click on the green flag

More information

A Game of Numbers (Understanding Directivity Specifications)

A Game of Numbers (Understanding Directivity Specifications) A Game of Numbers (Understanding Directivity Specifications) José (Joe) Brusi, Brusi Acoustical Consulting Loudspeaker directivity is expressed in many different ways on specification sheets and marketing

More information

COMPUTATIONAL MODELS OF CLASSICAL CONDITIONING: A COMPARATIVE STUDY

COMPUTATIONAL MODELS OF CLASSICAL CONDITIONING: A COMPARATIVE STUDY COMPUTATIONAL MODELS OF CLASSICAL CONDITIONING: A COMPARATIVE STUDY Christian Balkenius Jan Morén christian.balkenius@fil.lu.se jan.moren@fil.lu.se Lund University Cognitive Science Kungshuset, Lundagård

More information

WHICH TYPE OF GRAPH SHOULD YOU CHOOSE?

WHICH TYPE OF GRAPH SHOULD YOU CHOOSE? PRESENTING GRAPHS WHICH TYPE OF GRAPH SHOULD YOU CHOOSE? CHOOSING THE RIGHT TYPE OF GRAPH You will usually choose one of four very common graph types: Line graph Bar graph Pie chart Histograms LINE GRAPHS

More information

Models of Cortical Maps II

Models of Cortical Maps II CN510: Principles and Methods of Cognitive and Neural Modeling Models of Cortical Maps II Lecture 19 Instructor: Anatoli Gorchetchnikov dy dt The Network of Grossberg (1976) Ay B y f (

More information

What is Visualization? Information Visualization An Overview. Information Visualization. Definitions

What is Visualization? Information Visualization An Overview. Information Visualization. Definitions What is Visualization? Information Visualization An Overview Jonathan I. Maletic, Ph.D. Computer Science Kent State University Visualize/Visualization: To form a mental image or vision of [some

More information

PSY402 Theories of Learning. Chapter 10 Stimulus Control of Behavior

PSY402 Theories of Learning. Chapter 10 Stimulus Control of Behavior PSY402 Theories of Learning Chapter 10 Stimulus Control of Behavior The Role of Environmental Stimuli In operant conditioning, the stimulus becomes associated with the reinforcer or punishment. Reward

More information

9/14/2015. Innate behavior. Innate behavior. Stimuli that trigger innate behaviors are called releasers.

9/14/2015. Innate behavior. Innate behavior. Stimuli that trigger innate behaviors are called releasers. & & has a fixed, genetic component. Despite differing environments, the behavior develops in all individuals. Learned behavior is acquired and modified over development. s can be triggered or enhanced

More information

Lecture 2, Human cognition

Lecture 2, Human cognition Human Cognition An important foundation for the design of interfaces is a basic theory of human cognition The information processing paradigm (in its most simple form). Human Information Processing The

More information

3 Introduction to Assessing Risk

3 Introduction to Assessing Risk 3 Introduction to Assessing Risk Important Question. How do we assess the risk an investor faces when choosing among assets? In this discussion we examine how an investor would assess the risk associated

More information

How do we Learn? How do you know you ve learned something? CLASS OBJECTIVES: What is learning? What is Classical Conditioning? Chapter 6 Learning

How do we Learn? How do you know you ve learned something? CLASS OBJECTIVES: What is learning? What is Classical Conditioning? Chapter 6 Learning How do we Learn? Chapter 6 Learning CLASS OBJECTIVES: What is learning? What is Classical Conditioning? How do you know you ve learned something? 1 Can our beliefs and attitudes be a result of learning??

More information