# Threshold Logic. 2.1 Networks of functions

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 2 Threshold Logic 2. Networks of functions We deal in this chapter with the simplest kind of computing units used to build artificial neural networks. These computing elements are a generalization of the common logic gates used in conventional computing and, since they operate by comparing their total input with a threshold, this field of research is known as threshold logic. 2.. Feed-forward and recurrent networks Our review in the previous chapter of the characteristics and structure of biological neural networks provides us with the initial motivation for a deeper inquiry into the properties of networks of abstract neurons. From the viewpoint of the engineer, it is important to define how a network should behave, without having to specify completely all of its parameters, which are to be found in a learning process. Artificial neural networks are used in many cases as a black box : a certain input should produce a desired output, but how the network achieves this result is left to a self-organizing process. x n... F... y y 2 y m Fig. 2.. A neural network as a black box In general we are interested in mapping an n-dimensional real input (,,..., x n ) to an m-dimensional real output (y,y 2,..., y m ). A neural R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

2 3 2 Threshold Logic network thus behaves as a mapping machine, capable of modeling a function F : IR n IR m. If we look at the structure of the network being used, some aspects of its dynamics must be defined more precisely. When the function is evaluated with a network of primitive functions, information flows through the directed edges of the network. Some nodes compute values which are then transmitted as arguments for new computations. If there are no cycles in the network, the result of the whole computation is well-defined and we do not have to deal with the task of synchronizing the computing units. We just assume that the computations take place without delay. x g g(x) f f (g (x)) Fig Function composition If the network contains cycles, however, the computation is not uniquely defined by the interconnection pattern and the temporal dimension must be considered. When the output of a unit is fed back to the same unit, we are dealing with a recursive computation without an explicit halting condition. We must define what we expect from the network: is the fixed point of the recursive evaluation the desired result or one of the intermediate computations? To solve this problem we assume that every computation takes a certain amount of time at each node (for example a time unit). If the arguments for a unit have been transmitted at time t, its output will be produced at time t +. A recursive computation can be stopped after a certain number of steps and the last computed output taken as the result of the recursive computation. x t f f (x t, f (x t!, f( x t!2,...)...) Fig Recursive evaluation In this chapter we deal first with networks without cycles, in which the time dimension can be disregarded. Then we deal with recurrent networks and their temporal coordination. The first model we consider was proposed in 943 by Warren McCulloch and Walter Pitts. Inspired by neurobiology they put forward a model of computation oriented towards the computational capabilities of real neurons and studied the question of abstracting universal concepts from specific perceptions [299]. R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

3 2. Networks of functions 3 We will avoid giving a general definition of a neural network at this point. So many models have been proposed which differ in so many respects that any definition trying to encompass this variety would be unnecessarily clumsy. As we show in this chapter, it is not necessary to start building neural networks with high powered computing units, as some authors do [384]. We will start our investigations with the general notion that a neural network is a network of functions in which synchronization can be considered explicitly or not The computing units The nodes of the networks we consider will be called computing elements or simply units. We assume that the edges of the network transmit information in a predetermined direction and the number of incoming edges into a node is not restricted by some upper bound. This is called the unlimited fan-in property of our computing units. f f (,,..., x n ) x n Fig Evaluation of a function of n arguments The primitive function computed at each node is in general a function of n arguments. Normally, however, we try to use very simple primitive functions of one argument at the nodes. This means that the incoming n arguments have to be reduced to a single numerical value. Therefore computing units are split into two functional parts: an integration function g reduces the n arguments to a single value and the output or activation function f produces the output of this node taking that single value as its argument. Figure 2.5 shows this general structure of the computing units. Usually the integration function g is the addition function. g f f (g(,,..., x n )) x n Fig Generic computing unit McCulloch Pitts networks are even simpler than this, because they use solely binary signals, i.e., ones or zeros. The nodes produce only binary results R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

4 32 2 Threshold Logic and the edges transmit exclusively ones or zeros. The networks are composed of directed unweighted edges of excitatory or of inhibitory type. The latter are marked in diagrams using a small circle attached to the end of the edge. Each McCulloch Pitts unit is also provided with a certain threshold value θ. At first sight the McCulloch Pitts model seems very limited, since only binary information can be produced and transmitted, but it already contains all necessary features to implement the more complex models. Figure 2.6 shows an abstract McCulloch Pitts computing unit. Following Minsky [3] it will be represented as a circle with a black half. Incoming edges arrive at the white half, outgoing edges leave from the black half. Outgoing edges can fan out any number of times. "... x n Fig Diagram of a McCulloch Pitts unit The rule for evaluating the input to a McCulloch Pitts unit is the following: Assume that a McCulloch Pitts unit gets an input,,...,x n through n excitatory edges and an input y,y 2,...,y m through m inhibitory edges. If m and at least one of the signals y,y 2,...,y m is, the unit is inhibited and the result of the computation is. Otherwise the total excitation x = x n is computed and compared with the threshold θ of the unit (if n = then x = ). If x θ the unit fires a, if x<θ the result of the computation is. This rule implies that a McCulloch Pitts unit can be inactivated by a single inhibitory signal, as is the case with some real neurons. When no inhibitory signals are present, the units act as a threshold gate capable of implementing many other logical functions of n arguments. Figure 2.7 shows the activation function of a unit, the so-called step function. This function changes discontinuously from zero to one at θ. When θ is zero and no inhibitory signals are present, we have the case of a unit producing the constant output one. If θ is greater than the number of incoming excitatory edges, the unit will never fire. R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

5 2.2 Synthesis of Boolean functions 33 " Fig The step function with threshold θ In the following subsection we assume provisionally that there is no delay in the computation of the output. 2.2 Synthesis of Boolean functions The power of threshold gates of the McCulloch Pitts type can be illustrated by showing how to synthesize any given logical function of n arguments. We deal firstly with the more simple kind of logic gates Conjunction, disjunction, negation Mappings from {, } n onto {, } are called logical or Boolean functions. Simple logical functions can be implemented directly with a single McCulloch Pitts unit. The output value can be associated with the logical value true and with the logical value false. It is straightforward to verify that the two units of Figure 2.8 compute the functions AND and OR respectively. AND OR 2 Fig Implementation of AND and OR gates A single unit can compute the disjunction or the conjunction of n arguments as is shown in Figure 2.9, where the conjunction of three and four arguments is computed by two units. The same kind of computation requires several conventional logic gates with two inputs. It should be clear from this simple example that threshold logic elements can reduce the complexity of the circuit used to implement a given logical function. R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

6 34 2 Threshold Logic AND x 2 3 x 4 OR Fig Generalized AND and OR gates As is well known, AND and OR gates alone cannot be combined to produce all logical functions of n variables. Since uninhibited threshold logic elements are capable of implementing more general functions than conventional AND or OR gates, the question of whether they can be combined to produce all logical functions arises. Stated another way: is inhibition of McCulloch Pitts units necessary or can it be dispensed with? The following proposition shows that it is necessary. A monotonic logical function f of n arguments is one whose value at two given n-dimensional points x =(,...,x n ) and y =(y,..., y n ) is such that f(x) f(y) whenever the number of ones in the input y is a subset of the ones in the input x. An example of a non-monotonic logical function of one argument is logical negation. Proposition. Uninhibited threshold logic elements of the McCulloch Pitts type can only implement monotonic logical functions. Proof. An example shows the kind of argumentation needed. Assume that the input vector (,,..., ) is assigned the function value. Since no other vector can set more edges in the network to than this vector does, any other input vector can also only be evaluated to. In general, if the ones in the input vector y are a subset of the ones in the input vector x, then the first cannot set more edges to than x does. This implies that f(x) f(y), as had to be shown. AND NOR NOT x Fig. 2.. Logical functions and their realization The units of Figure 2. show the implementation of some non-monotonic logical functions requiring inhibitory connections. Logical negation, for example, can be computed using a McCulloch Pitts unit with threshold and an inhibitory edge. The other two functions can be verified by the reader. R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

7 2.2 Synthesis of Boolean functions 35 Fig. 2.. Function values of a logical function of three variables x + + = Fig Separation of the input space for the OR function Geometric interpretation It is very instructive to visualize the kind of functions that can be computed with McCulloch Pitts cells by using a diagram. Figure 2. shows the eight vertices of a three-dimensional unit cube. Each of the three logical variables, and can assume one of two possible binary values. There are eight possible combinations, represented by the vertices of the cube. A logical function is just an assignment of a or a to each of the vertices. The figure shows one of these assignments. In the case of n variables, the cube consists of 2 n vertices and admits 2 2n different binary assignments. McCulloch Pitts units divide the input space into two half-spaces. For a given input (,, ) and a threshold θ the condition + + θ is tested, which is true for all points to one side of the plane with the equation + + = θ and false for all points to the other side (without including the plane itself in this case). Figure 2.2 shows this separation for the case in R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

8 36 2 Threshold Logic + + = 2 x + + = Fig Separating planes of the OR and majority functions which θ =, i.e., for the OR function. Only those vertices above the separating plane are labeled. The majority function of three variables divides input space in a similar manner, but the separating plane is given by the equation + + = 2. Figure 2.3 shows the additional plane. The planes are always parallel in the case of McCulloch Pitts units. Non-parallel separating planes can only be produced using weighted edges Constructive synthesis Every logical function of n variables can be written in tabular form. The value of the function is written down for every one of the possible binary combinations of the n inputs. If we want to build a network to compute this function, it should have n inputs and one output. The network must associate each input vector with the correct output value. If the number of computing units is not limited in some way, it is always possible to build or synthesize a network which computes this function. The constructive proof of this proposition profits from the fact that McCulloch Pitts units can be used as binary decoders. Consider for example the vector (,, ). It is the only one which fulfills the condition. This condition can be tested by a single computing unit (Figure 2.4). Since only the vector (,, ) makes this unit fire, the unit is a decoder for this input. Assume that a function F of three arguments has been defined according to the following table: To compute this function it is only necessary to decode all those vectors for which the function s value is. Figure 2.5 shows a network capable of computing the function F. R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

9 2.2 Synthesis of Boolean functions 37 2 Fig Decoder for the vector (,, ) input vectors F (,,) (,,) all others F Fig Synthesis of the function F The individual units in the first layer of the composite network are decoders. For each vector for which F is a decoder is used. In our case we need just two decoders. Components of each vector which must be are transmitted with inhibitory edges, components which must be with excitatory ones. The threshold of each unit is equal to the number of bits equal to that must be present in the desired input vector. The last unit to the right is a disjunction: if any one of the specified vectors can be decoded this unit fires a. It is straightforward to extend this constructive method to other Boolean functions of any other dimension. This leads to the following proposition: Proposition 2. Any logical function F : {, } n {, } can be computed with a McCulloch Pitts network of two layers. No attempt has been made here to minimize the number of computing units. In fact, we need as many decoders as there are ones in the table of function values. An alternative to this simple constructive method is to use harmonic analysis of logical functions, as will be shown in Sect We can also consider the minimal possible set of building blocks needed to implement arbitrary logical functions when the fan-in of the units is bounded R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

10 38 2 Threshold Logic in some way. The circuits of Figure 2.4 and Figure 2.5 use decoders of n inputs. These decoders can be built of simpler cells, for example, two units capable of respectively implementing the AND function and negation. Inhibitory connections in the decoders can be replaced with a negation gate. The output of the decoders is collected at a conjunctive unit. The decoder of Figure 2.4 can be implemented as shown in Figure 2.6. The only difference from the previous decoder are the negated inputs and the higher threshold in the AND unit. All decoders for a row of the table of a logical function can be designed in a similar way. This immediately leads to the following proposition: Proposition 3. All logical functions can be implemented with a network composed of units which exclusively compute the AND, OR, and NOT functions. The three units AND, NOT and OR are called a logical basis because of this property. Since OR can be implemented using AND and NOT units, these two alone constitute a logical basis. The same happens with OR and NOT units. John von Neumann showed that through a redundant coding of the inputs (each variable is transmitted through two lines) AND and OR units alone can constitute a logical basis [326]. 3 Fig A composite decoder for the vector (,, ) 2.3 Equivalent networks We can build simpler circuits by using units with more general properties, for example weighted edges and relative inhibition. However, as we show in this section, circuits of McCulloch Pitts units can emulate circuits built out of high-powered units by exploiting the trade-off between the complexity of the network versus the complexity of the computing units Weighted and unweighted networks Since McCulloch Pitts networks do not use weighted edges the question of whether weighted networks are more general than unweighted ones must be answered. A simple example shows that both kinds of networks are equivalent. Assume that three weighted edges converge on the unit shown in Figure 2.7. The unit computes R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

11 2.3 Equivalent networks Fig Weighted unit But this is equivalent to , and this computation can be performed with the network of Figure Fig Equivalent computing unit The figure shows that positive rational weights can be simulated by simply fanning-out the edges of the network the required number of times. This means that we can either use weighted edges or go for a more complex topology of the network, with many redundant edges. The same can be done in the case of irrational weights if the number of input vectors is finite (see Chap. 3, Exercise 3) Absolute and relative inhibition In the last subsection we dealt only with the case of positive weights. Two classes of inhibition can be identified: absolute inhibition corresponds to the one used in McCulloch Pitts units. Relative inhibition corresponds to the case of edges weighted with a negative factor and whose effect is to lower the firing threshold when a is transmitted through this edge. Proposition 4. Networks of McCulloch Pitts units are equivalent to networks with relative inhibition. R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

12 4 2 Threshold Logic Proof. It is only necessary to show that each unit in a network where relative inhibition is used is equivalent to one or more units in a network where absolute inhibition is used. It is clear that it is possible to implement absolute inhibition with relative inhibitory edges. If the threshold of a unit is the integer m and if n excitatory edges impinge on it, the maximum possible total excitation for this unit is n m. If m n the unit never fires and the inhibitory edge is irrelevant. It suffices to fan out the inhibitory edge n m + times and make all these edges meet at the unit. When a is transmitted through the inhibitory edges the total amount of inhibition is n m + and this shuts down the unit. To prove that relative inhibitory edges can be simulated with absolute inhibitory ones, refer to Figure 2.9. The network to the left contains a relative inhibitory edge, the network to the right absolute inhibitory ones. The reader can verify that the two networks are equivalent. Relative inhibitory edges correspond to edges weighted with. We can also accept any other negative weight w. In that case the threshold of the unit to the right of Figure 2.9 should be m + w instead of m +. Therefore networks with negative weights can be simulated using unweighted McCulloch Pitts elements. relative inhibition equivalent circuit with absolute inhibition x n... m... x n y m y m+ Fig Two equivalent networks As shown above, we can implement any kind of logical function using unweighted networks. What we trade is the simplicity of the building blocks for a more convoluted topology of the network. Later we will always use weighted networks in order to simplify the topology Binary signals and pulse coding An additional question which can be raised is whether binary signals are not a very limited coding strategy. Are networks in which the communication channels adopt any of ten or fifteen different states more efficient than channels which adopt only two states, as in McCulloch Pitts networks? To give an R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

13 2.3 Equivalent networks 4 answer we must consider that unit states have a price, in biological networks as well as in artificial ones. The transmitted information must be optimized using the number of available switching states. e number of representable values base Fig Number of representable values as a function of the base Assume that the number of states per communication channel is b and that c channels are used to input information. The cost K of the implementation is proportional to both quantities, i.e., K = γbc, where γ is a proportionality constant. Using c channels with b states, b c different numbers can be represented. This means that c = K/γb and, if we set κ = K/γ, we are seeking the numerical base b which optimizes the function b κ/b. Since we assume constant cost, κ is a constant. Figure 2.2 shows that the optimal value for b is the Euler constant e. Since the number of channel states must be an integer, three states would provide a good approximation to the optimal coding strategy. However, in electronic and biological systems decoding of the signal plays such an important role that the choice of two states per channel becomes a better alternative. Wiener arrived at a similar conclusion through a somewhat different argument [452]. The binary nature of information transmission in the nervous system seems to be an efficient way to transport signals. However, in the next chapters we will assume that the communication channels can transport arbitrary real numbers. This makes the analysis simpler than when we have to deal explicitly with frequency modulated signals, but does not lead to a minimization of the resources needed for a technical implementation. Some researchers prefer to work with so-called weightless networks which operate exclusively with binary data. R. Rojas: Neural Networks, Springer-Verlag, Berlin, 996

### 4.1 Learning algorithms for neural networks

4 Perceptron Learning 4.1 Learning algorithms for neural networks In the two preceding chapters we discussed two closely related models, McCulloch Pitts units and perceptrons, but the question of how to

### The equation for the 3-input XOR gate is derived as follows

The equation for the 3-input XOR gate is derived as follows The last four product terms in the above derivation are the four 1-minterms in the 3-input XOR truth table. For 3 or more inputs, the XOR gate

### Artificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence

Artificial Neural Networks and Support Vector Machines CS 486/686: Introduction to Artificial Intelligence 1 Outline What is a Neural Network? - Perceptron learners - Multi-layer networks What is a Support

### CS311 Lecture: Sequential Circuits

CS311 Lecture: Sequential Circuits Last revised 8/15/2007 Objectives: 1. To introduce asynchronous and synchronous flip-flops (latches and pulsetriggered, plus asynchronous preset/clear) 2. To introduce

### Counters and Decoders

Physics 3330 Experiment #10 Fall 1999 Purpose Counters and Decoders In this experiment, you will design and construct a 4-bit ripple-through decade counter with a decimal read-out display. Such a counter

### Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk

Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems

### 3.Basic Gate Combinations

3.Basic Gate Combinations 3.1 TTL NAND Gate In logic circuits transistors play the role of switches. For those in the TTL gate the conducting state (on) occurs when the baseemmiter signal is high, and

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### Biological Neurons and Neural Networks, Artificial Neurons

Biological Neurons and Neural Networks, Artificial Neurons Neural Computation : Lecture 2 John A. Bullinaria, 2015 1. Organization of the Nervous System and Brain 2. Brains versus Computers: Some Numbers

### A TUTORIAL. BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY

ARTIFICIAL NEURAL NETWORKS: A TUTORIAL BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY Contents Introduction Origin Of Neural Network Biological Neural Networks ANN Overview

### THE HUMAN BRAIN. observations and foundations

THE HUMAN BRAIN observations and foundations brains versus computers a typical brain contains something like 100 billion miniscule cells called neurons estimates go from about 50 billion to as many as

### Self Organizing Maps: Fundamentals

Self Organizing Maps: Fundamentals Introduction to Neural Networks : Lecture 16 John A. Bullinaria, 2004 1. What is a Self Organizing Map? 2. Topographic Maps 3. Setting up a Self Organizing Map 4. Kohonen

### 6.080/6.089 GITCS Feb 12, 2008. Lecture 3

6.8/6.89 GITCS Feb 2, 28 Lecturer: Scott Aaronson Lecture 3 Scribe: Adam Rogal Administrivia. Scribe notes The purpose of scribe notes is to transcribe our lectures. Although I have formal notes of my

### CHAPTER 3 Boolean Algebra and Digital Logic

CHAPTER 3 Boolean Algebra and Digital Logic 3.1 Introduction 121 3.2 Boolean Algebra 122 3.2.1 Boolean Expressions 123 3.2.2 Boolean Identities 124 3.2.3 Simplification of Boolean Expressions 126 3.2.4

### Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승

Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승 How much energy do we need for brain functions? Information processing: Trade-off between energy consumption and wiring cost Trade-off between energy consumption

### Applications of Methods of Proof

CHAPTER 4 Applications of Methods of Proof 1. Set Operations 1.1. Set Operations. The set-theoretic operations, intersection, union, and complementation, defined in Chapter 1.1 Introduction to Sets are

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### 2. Compressing data to reduce the amount of transmitted data (e.g., to save money).

Presentation Layer The presentation layer is concerned with preserving the meaning of information sent across a network. The presentation layer may represent (encode) the data in various ways (e.g., data

### Artificial neural networks

Artificial neural networks Now Neurons Neuron models Perceptron learning Multi-layer perceptrons Backpropagation 2 It all starts with a neuron 3 Some facts about human brain ~ 86 billion neurons ~ 10 15

### Computationally Complete Spiking Neural P Systems Without Delay: Two Types of Neurons Are Enough

Computationally Complete Spiking Neural P Systems Without Delay: Two Types of Neurons Are Enough Rudolf Freund 1 and Marian Kogler 1,2 1 Faculty of Informatics, Vienna University of Technology Favoritenstr.

### Biological Neurons and Neural Networks, Artificial Neurons

Biological Neurons and Neural Networks, Artificial Neurons Introduction to Neural Networks : Lecture 2 John A. Bullinaria, 2004 1. Organization of the Nervous System and Brain 2. Brains versus Computers:

### Recurrent Neural Networks

Recurrent Neural Networks Neural Computation : Lecture 12 John A. Bullinaria, 2015 1. Recurrent Neural Network Architectures 2. State Space Models and Dynamical Systems 3. Backpropagation Through Time

### MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL

MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL G. Maria Priscilla 1 and C. P. Sumathi 2 1 S.N.R. Sons College (Autonomous), Coimbatore, India 2 SDNB Vaishnav College

### CHAPTER 6 PRINCIPLES OF NEURAL CIRCUITS.

CHAPTER 6 PRINCIPLES OF NEURAL CIRCUITS. 6.1. CONNECTIONS AMONG NEURONS Neurons are interconnected with one another to form circuits, much as electronic components are wired together to form a functional

### Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham

Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham kms@cs.stir.ac.uk 1 What are Neural Networks? Neural Networks are networks of neurons, for example, as found in real (i.e. biological)

### Module 3: Floyd, Digital Fundamental

Module 3: Lecturer : Yongsheng Gao Room : Tech - 3.25 Email : yongsheng.gao@griffith.edu.au Structure : 6 lectures 1 Tutorial Assessment: 1 Laboratory (5%) 1 Test (20%) Textbook : Floyd, Digital Fundamental

### Digital Fundamentals

Digital Fundamentals Tenth Edition Floyd hapter 8 2009 Pearson Education, Upper 2008 Pearson Saddle River, Education NJ 07458. All Rights Reserved ounting in Binary As you know, the binary count sequence

### CHAPTER 5 SIGNALLING IN NEURONS

5.1. SYNAPTIC TRANSMISSION CHAPTER 5 SIGNALLING IN NEURONS One of the main functions of neurons is to communicate with other neurons. An individual neuron may receive information from many different sources.

### Shift registers. 1.0 Introduction

Shift registers 1.0 Introduction Shift registers are a type of sequential logic circuit, mainly for storage of digital data. They are a group of flip-flops connected in a chain so that the output from

COPYRIGHT NOTICE: David A. Kendrick, P. Ruben Mercado, and Hans M. Amman: Computational Economics is published by Princeton University Press and copyrighted, 2006, by Princeton University Press. All rights

### Take-Home Exercise. z y x. Erik Jonsson School of Engineering and Computer Science. The University of Texas at Dallas

Take-Home Exercise Assume you want the counter below to count mod-6 backward. That is, it would count 0-5-4-3-2-1-0, etc. Assume it is reset on startup, and design the wiring to make the counter count

### Module-3 SEQUENTIAL LOGIC CIRCUITS

Module-3 SEQUENTIAL LOGIC CIRCUITS Till now we studied the logic circuits whose outputs at any instant of time depend only on the input signals present at that time are known as combinational circuits.

### The Backpropagation Algorithm

7 The Backpropagation Algorithm 7. Learning as gradient descent We saw in the last chapter that multilayered networks are capable of computing a wider range of Boolean functions than networks with a single

### 2. Methods of Proof Types of Proofs. Suppose we wish to prove an implication p q. Here are some strategies we have available to try.

2. METHODS OF PROOF 69 2. Methods of Proof 2.1. Types of Proofs. Suppose we wish to prove an implication p q. Here are some strategies we have available to try. Trivial Proof: If we know q is true then

### 2011, The McGraw-Hill Companies, Inc. Chapter 4

Chapter 4 4.1 The Binary Concept Binary refers to the idea that many things can be thought of as existing in only one of two states. The binary states are 1 and 0 The 1 and 0 can represent: ON or OFF Open

### Designing a Divide-by-Three Logic Circuit

April 28, 2013 Designing a Divide-by-hree Logic Circuit c 2005, C. Bond. All rights reserved. http://www.crbond.com Purpose his document provides a detailed description of each step in the design of an

### THE DIMENSION OF A VECTOR SPACE

THE DIMENSION OF A VECTOR SPACE KEITH CONRAD This handout is a supplementary discussion leading up to the definition of dimension and some of its basic properties. Let V be a vector space over a field

### Introduction to Artificial Neural Networks

POLYTECHNIC UNIVERSITY Department of Computer and Information Science Introduction to Artificial Neural Networks K. Ming Leung Abstract: A computing paradigm known as artificial neural network is introduced.

### Performance Level Descriptors Grade 6 Mathematics

Performance Level Descriptors Grade 6 Mathematics Multiplying and Dividing with Fractions 6.NS.1-2 Grade 6 Math : Sub-Claim A The student solves problems involving the Major Content for grade/course with

### Geometrical Approaches for Artificial Neural Networks

Geometrical Approaches for Artificial Neural Networks Centre for Computational Intelligence De Montfort University Leicester, UK email:elizondo@dmu.ac.uk http://www.cci.dmu.ac.uk/ Workshop on Principal

### LAB 7 MOSFET CHARACTERISTICS AND APPLICATIONS

LAB 7 MOSFET CHARACTERISTICS AND APPLICATIONS Objective In this experiment you will study the i-v characteristics of an MOS transistor. You will use the MOSFET as a variable resistor and as a switch. BACKGROUND

### Padma Charan Das Dept. of E.T.C. Berhampur, Odisha, India

Volume 5, Issue 3, March 2015 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Measuring Quality

### Notes on Complexity Theory Last updated: August, 2011. Lecture 1

Notes on Complexity Theory Last updated: August, 2011 Jonathan Katz Lecture 1 1 Turing Machines I assume that most students have encountered Turing machines before. (Students who have not may want to look

### Lecture 5: Gate Logic Logic Optimization

Lecture 5: Gate Logic Logic Optimization MAH, AEN EE271 Lecture 5 1 Overview Reading McCluskey, Logic Design Principles- or any text in boolean algebra Introduction We could design at the level of irsim

### 3 An Illustrative Example

Objectives An Illustrative Example Objectives - Theory and Examples -2 Problem Statement -2 Perceptron - Two-Input Case -4 Pattern Recognition Example -5 Hamming Network -8 Feedforward Layer -8 Recurrent

### Handout #1: Mathematical Reasoning

Math 101 Rumbos Spring 2010 1 Handout #1: Mathematical Reasoning 1 Propositional Logic A proposition is a mathematical statement that it is either true or false; that is, a statement whose certainty or

### 1. Realization of gates using Universal gates

1. Realization of gates using Universal gates Aim: To realize all logic gates using NAND and NOR gates. Apparatus: S. No Description of Item Quantity 1. IC 7400 01 2. IC 7402 01 3. Digital Trainer Kit

### We now explore a third method of proof: proof by contradiction.

CHAPTER 6 Proof by Contradiction We now explore a third method of proof: proof by contradiction. This method is not limited to proving just conditional statements it can be used to prove any kind of statement

### Edge-Triggered D-type Flip-flop

Edge-Triggered D-type Flip-flop The transparent D-type flip-flop is written during the period of time that the write control is active. However there is a demand in many circuits for a storage device (flip-flop

### Digital System Design. Digital System Design with Verilog

Digital System Design with Verilog Adapted from Z. Navabi Portions Copyright Z. Navabi, 2006 1 Digital System Design Automation with Verilog Digital Design Flow Design entry Testbench in Verilog Design

### PA Common Core Standards Standards for Mathematical Practice Grade Level Emphasis*

Habits of Mind of a Productive Thinker Make sense of problems and persevere in solving them. Attend to precision. PA Common Core Standards The Pennsylvania Common Core Standards cannot be viewed and addressed

### Efficient Data Structures for Decision Diagrams

Artificial Intelligence Laboratory Efficient Data Structures for Decision Diagrams Master Thesis Nacereddine Ouaret Professor: Supervisors: Boi Faltings Thomas Léauté Radoslaw Szymanek Contents Introduction...

### Induction. Margaret M. Fleck. 10 October These notes cover mathematical induction and recursive definition

Induction Margaret M. Fleck 10 October 011 These notes cover mathematical induction and recursive definition 1 Introduction to induction At the start of the term, we saw the following formula for computing

### Circuit and System Representation. IC Designers must juggle several different problems

Circuit and System Representation IC Designers must juggle several different problems Multiple levels of abstraction IC designs requires refining an idea through many levels of detail, specification ->

### Series and Parallel Resistive Circuits

Series and Parallel Resistive Circuits The configuration of circuit elements clearly affects the behaviour of a circuit. Resistors connected in series or in parallel are very common in a circuit and act

### 6. Feed-forward mapping networks

6. Feed-forward mapping networks Fundamentals of Computational Neuroscience, T. P. Trappenberg, 2002. Lecture Notes on Brain and Computation Byoung-Tak Zhang Biointelligence Laboratory School of Computer

### 13.1 Synchronous and asynchronous networks

13 The Hopfield Model One of the milestones for the current renaissance in the field of neural networks was the associative model proposed by Hopfield at the beginning of the 1980s. Hopfield s approach

### INCIDENCE-BETWEENNESS GEOMETRY

INCIDENCE-BETWEENNESS GEOMETRY MATH 410, CSUSM. SPRING 2008. PROFESSOR AITKEN This document covers the geometry that can be developed with just the axioms related to incidence and betweenness. The full

### Creating, Solving, and Graphing Systems of Linear Equations and Linear Inequalities

Algebra 1, Quarter 2, Unit 2.1 Creating, Solving, and Graphing Systems of Linear Equations and Linear Inequalities Overview Number of instructional days: 15 (1 day = 45 60 minutes) Content to be learned

### Lecture 8: Flip-flops

Points Addressed in this Lecture Properties of synchronous and asynchronous sequential circuits Overview of flip-flops and latches Lecture 8: Flip-flops Professor Peter Cheung Department of EEE, Imperial

### Offline 1-Minesweeper is NP-complete

Offline 1-Minesweeper is NP-complete James D. Fix Brandon McPhail May 24 Abstract We use Minesweeper to illustrate NP-completeness proofs, arguments that establish the hardness of solving certain problems.

### APPLICATION NOTE - 017

APPLICATION NOTE - 017 PWM Motor Drives Theory and Measurement Considerations Pulse Width Modulated (PWM) power electronic techniques represent a large and increasing proportion of modern power electronics.

### Chapter 3. Distribution Problems. 3.1 The idea of a distribution. 3.1.1 The twenty-fold way

Chapter 3 Distribution Problems 3.1 The idea of a distribution Many of the problems we solved in Chapter 1 may be thought of as problems of distributing objects (such as pieces of fruit or ping-pong balls)

### Chapter - 5 FLIP-FLOPS AND SIMPLE FLIP-FLOP APPLICATIONS

Chapter - 5 FLIP-FLOPS AND SIMPLE FLIP-FLOP APPLICATIONS Introduction : Logic circuit is divided into two types. 1. Combinational Logic Circuit 2. Sequential Logic Circuit Definition : 1. Combinational

### CHAPTER TEN. 10.1 New Truth Table Symbols. 10.1.1 Edges/Transitions. Memory Cells

CHAPTER TEN Memory Cells The previous chapters presented the concepts and tools behind processing binary data. This is only half of the battle though. For example, a logic circuit uses inputs to calculate

### 1. True or False? A voltage level in the range 0 to 2 volts is interpreted as a binary 1.

File: chap04, Chapter 04 1. True or False? A voltage level in the range 0 to 2 volts is interpreted as a binary 1. 2. True or False? A gate is a device that accepts a single input signal and produces one

### Part 2: Operational Amplifiers

Part 2: Operational Amplifiers An operational amplifier is a very high gain amplifier. Op amps can be used in many different ways. Two of the most common uses are a) as comparators b) as amplifiers (either

### 5. Sequential CMOS Logic Circuits

5. Sequential CMOS Logic Circuits In sequential logic circuits the output signals is determined by the current inputs as well as the previously applied input variables. Fig. 5.1a shows a sequential circuit

### APPLICATION OF FILTER BANK THEORY TO SUBBAND CODING OF IMAGES

EC 623 ADVANCED DIGITAL SIGNAL PROCESSING TERM-PROJECT APPLICATION OF FILTER BANK THEORY TO SUBBAND CODING OF IMAGES Y. PRAVEEN KUMAR 03010240 KANCHAN MISHRA 03010242 Supervisor: Dr. S.R.M. Prasanna Department

### Cartesian Products and Relations

Cartesian Products and Relations Definition (Cartesian product) If A and B are sets, the Cartesian product of A and B is the set A B = {(a, b) :(a A) and (b B)}. The following points are worth special

### Lecture 8: Synchronous Digital Systems

Lecture 8: Synchronous Digital Systems The distinguishing feature of a synchronous digital system is that the circuit only changes in response to a system clock. For example, consider the edge triggered

### 8-bit Digital to Analog converter (DAC)

8-bit Digital to Analog converter (DAC) Posted on February 28, 2008, by Ibrahim KAMAL, in General electronics, tagged This article aims to introduce to beginners and intermediate readers a simple solution

### Stock Prediction using Artificial Neural Networks

Stock Prediction using Artificial Neural Networks Abhishek Kar (Y8021), Dept. of Computer Science and Engineering, IIT Kanpur Abstract In this work we present an Artificial Neural Network approach to predict

### Elementary Number Theory We begin with a bit of elementary number theory, which is concerned

CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete

### TRAINING A 3-NODE NEURAL NETWORK IS NP-COMPLETE

494 TRAINING A 3-NODE NEURAL NETWORK IS NP-COMPLETE Avrim Blum'" MIT Lab. for Computer Science Cambridge, Mass. 02139 USA Ronald L. Rivest t MIT Lab. for Computer Science Cambridge, Mass. 02139 USA ABSTRACT

### 7. Latches and Flip-Flops

Chapter 7 Latches and Flip-Flops Page 1 of 18 7. Latches and Flip-Flops Latches and flip-flops are the basic elements for storing information. One latch or flip-flop can store one bit of information. The

### Neural Networks and Support Vector Machines

INF5390 - Kunstig intelligens Neural Networks and Support Vector Machines Roar Fjellheim INF5390-13 Neural Networks and SVM 1 Outline Neural networks Perceptrons Neural networks Support vector machines

### Basic Logic Gates Richard E. Haskell

BASIC LOGIC GATES 1 E Basic Logic Gates Richard E. Haskell All digital systems are made from a few basic digital circuits that we call logic gates. These circuits perform the basic logic functions that

### Representing Reversible Cellular Automata with Reversible Block Cellular Automata

Discrete Mathematics and Theoretical Computer Science Proceedings AA (DM-CCG), 2001, 145 154 Representing Reversible Cellular Automata with Reversible Block Cellular Automata Jérôme Durand-Lose Laboratoire

### The 104 Duke_ACC Machine

The 104 Duke_ACC Machine The goal of the next two lessons is to design and simulate a simple accumulator-based processor. The specifications for this processor and some of the QuartusII design components

A New Paradigm for Synchronous State Machine Design in Verilog Randy Nuss Copyright 1999 Idea Consulting Introduction Synchronous State Machines are one of the most common building blocks in modern digital

### The last three chapters introduced three major proof techniques: direct,

CHAPTER 7 Proving Non-Conditional Statements The last three chapters introduced three major proof techniques: direct, contrapositive and contradiction. These three techniques are used to prove statements

### Lecture Notes on Linear Search

Lecture Notes on Linear Search 15-122: Principles of Imperative Computation Frank Pfenning Lecture 5 January 29, 2013 1 Introduction One of the fundamental and recurring problems in computer science is

### Gray Code Generator and Decoder by Carsten Kristiansen Napier University. November 2004

Gray Code Generator and Decoder by Carsten Kristiansen Napier University November 2004 Title page Author: Carsten Kristiansen. Napier No: 04007712. Assignment title: Design of a Gray Code Generator and

### IE1204 Digital Design F12: Asynchronous Sequential Circuits (Part 1)

IE1204 Digital Design F12: Asynchronous Sequential Circuits (Part 1) Elena Dubrova KTH / ICT / ES dubrova@kth.se BV pp. 584-640 This lecture IE1204 Digital Design, HT14 2 Asynchronous Sequential Machines

### PRINCIPLES OF PROBLEM SOLVING

PRINCIPLES OF PROBLEM SOLVING There are no hard and fast rules that will ensure success in solving problems. However, it is possible to outline some general steps in the problem-solving process and to

### Digital Logic Design

Digital Logic Design: An Embedded Systems Approach Using VHDL Chapter 1 Introduction and Methodology Portions of this work are from the book, Digital Logic Design: An Embedded Systems Approach Using VHDL,

### Models of Cortical Maps II

CN510: Principles and Methods of Cognitive and Neural Modeling Models of Cortical Maps II Lecture 19 Instructor: Anatoli Gorchetchnikov dy dt The Network of Grossberg (1976) Ay B y f (

### (Refer Slide Time: 00:01:16 min)

Digital Computer Organization Prof. P. K. Biswas Department of Electronic & Electrical Communication Engineering Indian Institute of Technology, Kharagpur Lecture No. # 04 CPU Design: Tirning & Control

### Karnaugh Maps. Example A B C X 0 0 0 0 0 0 1 0 0 1 0 0 0 1 1 1 1 0 0 1 1 0 1 0 1 1 0 1 1 1 1 1. each 1 here gives a minterm e.g.

Karnaugh Maps Yet another way of deriving the simplest Boolean expressions from behaviour. Easier than using algebra (which can be hard if you don't know where you're going). Example A B C X 0 0 0 0 0

### This representation is compared to a binary representation of a number with N bits.

Chapter 11 Analog-Digital Conversion One of the common functions that are performed on signals is to convert the voltage into a digital representation. The converse function, digital-analog is also common.

### Section 1.1. Introduction to R n

The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to

### Chapter 4 Register Transfer and Microoperations. Section 4.1 Register Transfer Language

Chapter 4 Register Transfer and Microoperations Section 4.1 Register Transfer Language Digital systems are composed of modules that are constructed from digital components, such as registers, decoders,

### Integer Operations. Overview. Grade 7 Mathematics, Quarter 1, Unit 1.1. Number of Instructional Days: 15 (1 day = 45 minutes) Essential Questions

Grade 7 Mathematics, Quarter 1, Unit 1.1 Integer Operations Overview Number of Instructional Days: 15 (1 day = 45 minutes) Content to Be Learned Describe situations in which opposites combine to make zero.

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### Determinants, Areas and Volumes

Determinants, Areas and Volumes Theodore Voronov Part 2 Areas and Volumes The area of a two-dimensional object such as a region of the plane and the volume of a three-dimensional object such as a solid