Dynamic Cognitive Modeling IV CLS2010 - Computational Linguistics Summer Events University of Zadar 23.08.2010 27.08.2010 Department of German Language and Linguistics Humboldt Universität zu Berlin
Overview 1. Introduction to linear algebra and calculus 2. Dynamical systems and neural networks 3. Dynamic automata: dynamic recognizers, fractal automata, nonlinear dynamical automata, and quantum automata 4. Language processing with neural networks: context-free and minimalist grammars 5. Dynamic field theory: functional representations, logics, and brain dynamics Literature: beim Graben, P. & Potthast, R. (2009). Inverse problems in dynamic cognitive modeling. Chaos: An Interdisciplinary Journal of Nonlinear Science, 19, 015103; and references therein: http://www.beimgraben.info/pbgpub/grabenpotthast.chaos19.pdf
Dynamic Cognitive Modeling
Dynamic Cognitive Modeling
Recursive Filler/Role Binding
Filler/Role Binding labeled trees left subtree: right subtree: complex fillers whole tree:
Dynamic Cognitive Modeling
Tensor Product Representations Let be a filler/role binding for some symbolic data structures with (simple) fillers and roles. A mapping to a vector space is a tensor product representation if 1. is a subspace of. 2. is a subspace of. 3.
Tensor Product Representations Let be a filler/role binding for some symbolic data structures with (simple) fillers and roles and a tensor product representation for the filler/role binding. The concatenation product is then the tensor product representation for the symbolic structures.
Fock Space As a consequence, is the Fock space used in quantum field theory
Computations and Processes Let be a set of symbolic data structures. Symbolic computations are partial functions. Let be a data structure. If the computations process can be concatenated to a Then, where P denotes the set of processes, is a semigroup.
Filler/Role Unbinding Let be a filler/role binding for some symbolic data structures with (simple) fillers and roles and a tensor product representation for the filler/role binding. A mapping for some is called unbinding if
Filler/Role Unbinding Unbinding can be achieved, e.g. by means of linear forms. Let, and be the dual space of the role subspace. Then, the linear form implements an unbinding via where denotes the identity map at filler subspace.
Realizing Computations Let be symbolic computations on. Two linear maps are called realizations of the computations in Fock space, if there is a tensor product representation such that mediates between semigroup homomorphisms
Example: String Processing strings: symbol alphabet: fillers: roles:
Example: String Processing tensor product representation fillers: roles:
Example: String Processing computations like in symbolic dynamics
Example: String Processing realization: Proof:
labeled trees Example: Tree Processing symbol alphabet: fillers: roles:
Example: Tree Processing tensor product representation fillers: roles:
computations Example: Tree Processing
Example: Tree Processing Passivization á la Smolensky (2006): passive sentence logical form
realization: Example: Tree Processing
Continuous Time Symbolic processes take place in discrete time: Brain dynamics takes place in continuous time! Language-related brain potentials for Die Rednerin hat der Berater gesucht the speaker has sought the advisor...
Order Parameter continuoustime dynamics in Fock space amplitude of k-th representation, between 0 and 1 tensor product representation of symbolic process in p time steps
Amplitude Dynamics initial state decayed state
Amplitude Dynamics excitation
Amplitude Dynamics
Fock Space Dynamics
Fock Space Dynamics repulsion saddle point attraction
Stable Heteroclinic Sequences SHS
Particular Representations Fillers and roles can be chosen from different vector spaces: number fields: Gödel representations arithmetic spaces: finite-dimensional dynamical systems combinations: fractal representations function spaces: dynamic fields
Dotted Sequences Turing machine state description: 1 a dotted sequences :
Symbolic Dynamics phase space dynamics symbolic dynamics
Cylinder Sets s k s 1 s 1 s k s a i1 a i2... a 2 in s 2 s n s n
Generalized Shifts current time domain of dependence: equivalent to Turing machine
Dotted Sequences obviously: filler roles however:
Dotted Sequences Decompose into left- and right substrings: tensor product representation:
Gödel Representations filler: Gödel numbers roles
Symbologram
Domains of Dependence cylinder set: domain of dependence: domain of dependence
Nonlinear Dynamical Automata The symbologram representation of a generalized shift is called nonlinear dynamical automaton.
Example: Parsing Process sentence: the dog chased the cat. context-free grammar (1) Gödel code (2)
Algorithm: Top-Down Recognizer time stack input operation 1 S NP V NP predict by rule (1) 2 NP VP NP V NP attach 3 VP V NP predict by rule (2) 4 V NP V NP attach 5 NP NP attach 6 ε ε accept
Domains of Dependence State descriptions provide a partition of the unit square, the domains of dependence. predict : if there is a rule : attach : if : do not accept : if : analogous to the Bernoulli map. piecewise affine-linear maps:
Symbologram domains of dependence images
Microstate Dynamics phase space initial condition
Microstate Dynamics
Ensemble Dynamics cloud of initial conditions ( ensemble )
Ensemble Dynamics
Contextual Partition
Dynamical Parsing Preparation: randomly distributed initial conditions. Predict: squeeze and shift horizontally. Attach: expand to unit square. Accept: unit square