VLSI FOR ARTIFICIAL INTELLIGENCE
THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE VLSI, COMPUTER ARCHITECTURE AND DIGITAL SIGNAL PROCESSING Consulting Editor Jonathan Allen Other books in the series: Logic Minimization Algorithms jor VLSI Synthesis. R.K. Brayton, O.D. Hachtel, C.T. McMullen, and A.L. Sangiovanni-Vincentelli. ISBN 0-89838-164-9. Adaptive Filters: Structures, Algorithms, and Applications. M.L. Honig and D.O. Messerschmitt. ISBN 0-89838-163-0. Introduction to VLSI Silicon Devices: Physics, Technology and Characterization. B. El-Kareh and R.J. Bombard. ISBN 0-89838-210-6. Latchup in CMOS Technology: The Problem and Its Cure. R.R. Troutman. ISBN 0-89838-215-7. Digital CMOS Circuit Design. M. Annaratone. ISBN 0-89838-224-6. The Bounding Approach to VLSI Circuit Simulation. C.A. Zukowski. ISBN 0-89838-176-2. Multi-Level Simulation jor VLSI Design. D.D. Hill and D.R. Coelho. ISBN 0-89838-184-3. Relaxation Techniques jor the Simulation oj VLSI Circuits. J. White and A. Sangiovanni-Vincentelli. ISBN 0-89838-1i:S-X. VLSI CAD Tools and Applications. W. Fichtner and M. Morf, editors. ISBN 0-89838-193-2. A VLSI Architecture jor Concurrent Data Structures. W.J. Dally. ISBN 0-89838-235-1. Yield Simulation jor Integrated Circuits. D.M.H. Walker. ISBN 0-89838-244-0. VLSI Specification, Verification and Synthesis. o. Birtwistle and P.A. Subrahmanyam. ISBN 0-89838-246-7. Fundamentals oj Computer-Aided Circuit Simulation. W.J. McCalla. ISBN 0-89838-248-3. Serial Data Computation. s.o. Smith and P.B. Denyer. ISBN 0-89838-253-X. Phonological Parsing in Speech Recognition. K.W. Church. ISBN 0-89838-250-5. Simulated Annealing jor VLSI Design. D.F. Wong, H.W. Leong, and C.L. Liu. ISBN 0-89838-256-4. Polycrystalline Silicon jor Integrated Circuit Applications. T. Kamins. ISBN 0-89838-259-9. FET Modeling jor Circuit Simulation. D. Divekar. ISBN 0-89838-264-5. VLSI Placement and Global Routing Using Simulated Annealing. C. Sechen. ISBN 0-89838-281-5. Adaptive Filters and Equalisers. B. Mulgrew, C.F.N. Cowan. ISBN 0-89838-285-8. Computer-Aided Design and VLSI Device Development, Second Edition. K.M. Cham, S-Y. Dh, J.L. Moll, K. Lee, P. Vande Voorde, D. Chin. ISBN: 0-89838-277-7. Automatic Speech Recognition. K-F. Lee. ISBN 0-89838-296-3. Speech Time-Frequency Representations. M.D. Riley. ISBN 0-89838-298-X A Systolic Array Optimizing Compiler. M.S. Lam. ISBN: 0-89838-300-5. Algorithms and Techniquesjor VLSI Layout Synthesis. D. Hill, D. Shugard, J. Fishburn, K. Keutzer. ISBN: 0-89838-301-3. Switch-Level Timing Simulation oj MOS VLSI Circuits. V.B. Rao, D.V. Dverhauser, T.N. Trick, I.N. Hajj. ISBN 0-89838-302-1
VLSI FOR ARTIFICIAL INTELLIGENCE edited by Jose G. Delgado-Frias Department of Electrical Engineering State University of New York at Binghamton Will R. Moore Department of Engineering Science University of Oxford KLUWER ACADEMIC PUBLISHERS BOSTON/DORDRECHT/LONDON
Distributors for North America: Kluwer Academic Publishers 101 Philip Drive Assinippi Park Norwell, Massachusetts 02061 USA Distributors for the UK and Ireland: Kluwer Academic Publishers Falcon House, Queen Square Lancaster LAI lrn, UNITED KINGDOM Distributors for all other countries: Kluwer Academic Publishers Group Distribution Centre Post Office Box 322 3300 AH Dordrecht, THE NETHERLANDS Library of Congress Cataloging-in-Publication Data VLSI for artificial intelligence. (The Kluwer international series in engineering and computer science; 68) Includes bibliographies and index. 1. Artificial intelligence-data processing. 2. Integrated circuits-very large scale integration. I. Delgado-Frias, Jose G. II. Moore, Will R. III. Series. Q336.V57 1989 006.3 88-37254 ISBN-13: 978-1-4612-8895-4 e-isbn-13: 978-1-4613-1619-0 DOl: 10.1007/978-1-4613-1619-0 Copyright 1989 by Kluwer Academic Publishers Softcover reprint of the hardcover 1 st edition 1989 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher, Kluwer Academic Publishers, 101 Philip Drive, Assinippi Park, Norwell, Massachusetts 02061.
Contents List of Contributors Preface Programme Committee Prologue viii ix x xi 1 Prolog Machines 1 1.1 From Low Level Semantic Description of Prolog to Instruction Set and VLSI Design 3 Reinhard Eck 1.2 A 32 Bit Processor for Compiled Prolog 13 Pierluigi Civera, Dante Del Corso, Gianluca Piccinini and Maurizio Zamboni 1.3 CARMEL-I: A VLSI Architecture for Flat Concurrent Prolog 27 Ran Ginosar and Arie Harsat 1.4 VLSI for Parallel Execution of Prolog 38 Jeff Reynolds and Sergio Delgado-Rannauro 2 Functional Programming Oriented Architectures 47 2.1 Supporting Functional and Logic Programming Languages through a Data Parallel VLSI Architecture 49 John O'Donnell 2.2 Translating Declaratively Specified Knowledge and Usage Requirements into a Reconfigurable Machine Richard Frost, Subir Bandyopadhyay and Dimitris Phoukas 61 3 Garbage Collection 3.1 VLSI-Appropriate Garbage Collection Support Steven Krueger 3.2 A Self-timed Circuit for a Prolog Machine Yves Bekkers, Louis Chevallier, Serge Le Huitouze, Olivier Ridoux and Lucien Ungaro 73 75 85
vi Contents 4 Content-Addressable Memory 4.1 VLSI and Rule-Based Systems Peter Kogge, John Oldfield, Mark Brule and Charles Stormon 4.2 Unify with Active Memory Yan Ng, Raymond Glover and Chew-Lye Chng 4.3 The Pattern Addressable Memory: Hardware for Associative Processing Ian Robinson 93 95 109 119 5 Knowledge Based Systems 5.1 A High Performance Relational Algebraic Processor for Large Knowledge Bases Simon Lavington, Jerome Robinson and Kai-Yau Mok 5.2 A WSI Semantic Network Architecture Jose Delgado-Frias and Will Moore 6 Neural Architectures 6.1 A VLSI Implementation of Multilayered Neural Networks Bernard Faure and Guy Mazare 6.2 A Fully Digital Integrated CMOS Hopfield Network Including the Learning Algorithm Michel Weinfeld 6.3 A Neural Network for 3-D VLSI Accelerator Tadashi Ae and Reiji Aibara 6.4 Shift Invariant Associative Memory Donald Prados and Subhash Kak 7 Digital and Analog VLSI Neural Networks 7.1 VLSI Bit-Serial Neural Networks Zoe Butler, Alan Murray and Anthony Smith 7.2 A New CMOS Architecture for Neural Networks Michel Verleysen, Bruno Sirletti and Paul Jespers 7.3 A Limited-Interconnect, Highly Layered Synthetic Neural Architecture Lex Akers, Mark Walker, David Ferry and Robert Grondin 7.4 VLSI-Design of Associative Networks Ulrich Ruckert and Karl Goser 7.5 Fully-Programmable Analogue VLSI Devices for the Implementation of Neural Networks Alan Murray, Anthony Smith and Lionel Tarassenko 131 133 144 157 159 169 179 189 199 201 209 218 227 236
Contents vii 8 Architectures for Neural Computing 245 8.1 Are Special Chips Necessary for Neural Computing? Igor Aleksander 8.2 A VLSI Systolic Array Dedicated to Hopfield Neural Network Frant;ois Blayo and Philippe Hurat 8.3 An Integrated System for Neural Network Simulations Simon Garth and Danny Pike Index 247 255 265 273
List of Contributors T Ae, Hiroshima (Japan) R Aibara, Hiroshima (Japan) L A Akers, Arizona State (USA) I Aleksander, Imperial College (UK) S Bandyopadhyay, Windsor (Canada) Y Bekkers, INRIA (France) F Blayo, LGI (France) M Brule, Syracuse (USA) Z F Butler, Edinburgh (UK) L Chevallier, INRIA (France) C L Chng, Nanyang (Taiwan) P Civera, Torino (Italy) J G Delgado-Frias, Oxford (UK) S Delgado-Rannauro, Essex (UK) D Del Corso, Torino (Italy) REck, Erlangen-Nuernberg (FRG) B Faure, IMAG (France) D K Ferry, Arizona State (USA) R A Frost, Windsor (Canada) S Garth, Texas Instruments (UK) R Ginosar, Technion (Israel) R J Glover, Brunei (UK) K Goser, Dortmund (FRG) R 0 Grondin, Arizona State (USA) A Harsat, Technion (Israel) P Hurat, LGI (France) P G A Jespers, Louvain (Belgium) S Kak, Louisiana State (USA) P M Kogge, IBM (USA) S D Krueger, TI (USA) S H Lavington, Essex (UK) S Le Huitouze, INRIA (France) G Mazare, IMAG (France) K-Y Mok, Essex (UK) W R Moore, Oxford (UK) A F Murray, Edinburgh (UK) Y H Ng, Imperial College (UK) J T O'Donnell, Glasgow (UK) J Oldfield, Syracuse (USA) D Phoukas, Windsor (Canada) D Pike, Cambridge (UK) G L Piccinini, Torino (Italy) D Prados, Louisiana State (USA) J Reynolds, Essex (UK) o Ridoux, INRIA (France) I N Robinson, HP (USA) J Robinson, Essex (UK) U Ruckert, Dortmund (FRG) B Sirletti, Louvain (Belgium) A V W Smith, Edinburgh (UK) C Stormon, Syracuse (USA) L Tarassenko, Oxford (UK) L Ungaro, INRIA (France) M Verleysen, Louvain (Belgium) M R Walker, Arizona State (USA) M Weinfeld, Poly technique (France) M Zamboni, Torino (Italy)
Preface This book is an edited selection of the papers presented at the International Workshop on VLSI for Artiflcial Intelligence which was held at the University of Oxford in July 1988. Our thanks go to all the contributors and especially to the programme committee for all their hard work. Thanks are also due to the ACM-SIGARCH, the Alvey Directorate, the lee and the IEEE Computer Society for publicising the event and to Oxford University for their active support. We are particularly grateful to David Cawley and Paula Appleby for coping with the administrative problems. Jose Delgado-Frias Will Moore October 1988
Programme Committee Igor Aleksander, Imperial College (UK) Yves Bekkers, IRISA/INRIA (France) Michael Brady, University of Oxford (UK) Jose Delgado-Frias, University of Oxford (UK) Steven Krueger, Texas Instruments Inc. (USA) Simon Lavington, University of Essex (UK) Will Moore, University of Oxford (UK) Philip Treleaven, University College London (UK) Benjamin Wah, University of Illinois (USA)
Prologue Research on architectures dedicated to artificial intelligence (AI) processing has been increasing in recent years, since conventional data- or numerically-oriented architectures are not able to provide the computational power and/or functionality required. For the time being these architectures have to be implemented in VLSI technology with its inherent constraints on speed, connectivity, fabrication yield and power. This in turn impacts on the effectiveness of the computer architecture. The aim of this book is to present the state-of-the-art and future trends on VLSI implementations of machines for AI computing. In order to achieve this objective the papers are drawn from a number of research communities spanning the subjects of VLSI design through computer architectures to AI programming and applications. This book has eight chapters which have been grouped into three major categories: hardware support for artificial intelligence programming languages, computer architectures for knowledge oriented systems, and neural network hardware implementations. This grouping covers the complete range from purely programmable systems to learning systems and from symbolic manipulation to connectionism. Hardware support for artificial intelligence programming languages Logic-oriented programming languages -such as Prolog- and functional languages -such as pure Lisp and Miranda- have been widely used as high-level languages for artificial intelligence applications. As a consequence, much research has been carried out to develop high performance computers for these programming languages. Chapter 1 contains papers which examine the implementations of Prolog machines. Although, the majority of these machines are based on the Warren abstract machine (WAM), there is a wide range of architectures: from reduced instruction set computers (RISC) to complex instruction set computers (CISC) and from uni-processor to multiprocessor architectures. Chapter 2 presents two functional programming oriented VLSI architectures. Chapter 3 looks at hardware support for programming languages to overcome memory limitations. Garbage collection (GC) helps to reclaim memory space that is no longer used by the program. In this chapter two garbage collectors are discussed; the first is for Lisp-like machines and the second for Prolog computers. Computer architectures for knowledge oriented systems Knowledge representation and manipulation tasks are frequently required in AI systems. These tasks have inherent parallelism which must be exploited in order to obtain reasonable execution times. Chapter 4 deals with content-addressable memory (CAM) circuits. CAM circuits are useful for applications such as production systems and logic programming. The CAM implementations presented in this chapter illustrate the effective use of parallelism. In Chapter 5, two architectures for
xii Prologue knowledge bases are described. The multiprocessor architectures are based on relational algebraic operations and semantic networks. Neural network hardware implementations In recent years many computer scientists have become interested in neural network models. Such models are believed to have a potential for new architectures for computing systems; such systems may be able to achieve human-like performance in some fields. Chapter 6 looks at architectural implementations of neural networks which Il re based on the Hopfield model. Chapter 7 presents several digital and analog circuits to implement these networks. The implementations reveal contrasting approaches to exploiting the VLSI capabilities and for overcoming the limitations imposed by this technology. Chapter 8 gives some alternative designs for neural network computations. The computers presented here are not themselves based on a neural network model but they do, through more conventional conventional architectures provide high computational power for neural computing applications.
VLSI FOR ARTIFICIAL INTELLIGENCE