A VLSI Implementation of Mixed-Signal mode Bipolar Neuron Circuitry

Transcription

1 A LS mplementation of Mixed-Signal mode Bipolar Neuron Circuitry ong Pan, and Bogdan. M. Wilamowski University of daho, College of Engeerg 800 Park Blvd. Suite 200 Boise, Abstract Neuron circuits have parallel operation features. LS implemented Neuron networks are suitable for high speed and low power consumption applications. igital implementations have good noise immunity while analog neuron circuits have smaller size. This paper presents a mixed-signal neuron design. t uses digital put, put, and weight signals while keeps analog ternal operation. Thus, this circuit has both good noise immunity and small size features. Clock signal is used to synchronize the neuron circuit operation. Simulation shows it has sigmoid activation function. This circuit is suitable for beg used feed-forward type neural networks.. NTROUCTON Although most neural networks can be implemented by software through microprocessors, LS based neural network shows promisg future the high speed and low power consumption fields due to its parallel operation nature. Meanwhile, many terests were paid to tegrate the entire neural network circuit to CMOS LS chip, which will effectively reduce both cost and size of the implementation. As the basic component of the neural network, a sgle neuron can be designed as digital circuits, as analog circuits, or as mixed-signal mode circuits. The digital neuron circuit[1] uses many adders and multipliers. t performs mathematics operation like a simple microprocessor with the digital put and weight signals. The digital neuron circuit can achieve accurate activation function while the complex structure may limit its application. The analog neuron circuit [2][3][4] uses analog put and analog weight signals. t generates analog put signals. Comparg to its digital counterpart, the analog design is more compact. However, the analog signals are easily affected by noise and it is dficult to store the analog weight signals. The mixed-signal neuron circuit [5] keeps the analog put and put signal while uses digital weight signal. This solution is much more practical the circuit implementation. However, the noise terference and the size issue are still not solved. Another type of mixedsignal neuron circuit uses digital pulses as the put/put signals [6][7]. t can be designed with compact size and good noise immunity. But it is hard to achieve good activation functions. This paper presents a neuron circuit with digital put, digital put and digital weight signals. Unlike the pure digital neuron circuit, which uses digital adder and multiplier blocks, the ternal operation of this circuit is analog. Therefore it has good noise immunity and compact size. The preferred activation function can be obtaed. Signal defition and circuit diagram are presented the section. After illustratg the basic operation mechanism, detail circuits of each block are discussed Sections and. Then, the simulation results of the neuron circuit and the entire neural network are presented.. CRCUT AGRAM AN SGNAL FNTON The circuit diagram of the neuron cell is shown Fig.1. t cludes synapse circuits, a weight control circuit, an itialization circuit, a capacitor, a comparator, and a circuit. The number of the synapse circuits is same Weight 1 Weight 2 Weight n n 1 n 2 n n Weight Control Circuit Synapse 1 Synapse 2 Synapse n Feedback nitial Circuit Capnode REF C Comparator Fig. 1. Circuit diagram of a neuron cell /03/$ EEE 971

2 as the number the put signals of the neuron. The operation of this circuit is based on chargg and dischargg the capacitor C, which is similar to the operation of dual slope A/ converter. The voltage or current levels usually carry put and put values of the analog or mixed signal neuron circuits. The voltage or current levels can be easily processed the neuron with pre-defed activation function. However, the put and the put values can also be described by other variables that can be applied the neuron circuit. n this paper, the relative timg of the selected digital signal with the clock signal is used to defe the magnitude of the values while the itial value (high or low) of the select signal is used as the sign. As the signals shown Fig.2, the magnitude of the put and put value are proportional to the timg relative to CLOCK, which is t and t. The sign of the bipolar put and put signals are defed by its itial value ( _it and _it ) when the CLOCK changes. Thus, we can defe the put and put as follows n1 (c) n2 (d) 1 (e) 2 (f) Capnode t = T / 2 t T / 2 set weight nitialization t t 2 nput t 2 T/2 T/2 t 1 _ it _ it Feedback/ = = = t /(T/2) (1) 2 = - t 2/(T/2) 1 = t 1/(T/2) 2 = - t 2/(T/2) GN Fig.2.Signal defition of the neuron circuit: positive put, (c) negative put, (d) positive put, (e) negative put, and the operation of a neuron circuit,,(c), (d), and (f). Out t = T / 2 t T / 2 _ it _ it = = (2) which T is the period of the clock signal (ab 1-2ms). f the signal does not change durg the certa clock phase (put signal when clock is high and put signal when clock is low), t and t will be recognized as T/2. Thus, the maximum and the mimum value of and will be 1 and 1.. NEURON OPERATON Similar to dual slope A/ converter, the operation of the neuron circuit is based on chargg and dischargg the capacitor, which is synchronized by the CLOCK signal. The operation of a neuron circuit with two synapses, a positive put (N1), a negative put (N2) and positive weight signals is shown Fig.2. t can be divided as the followg steps: 1). Set weight: When the CLOCK is low, the weight signals of all the synapses are generated by the weight control circuit. 2). nitilization: When the CLOCK rises, the voltage of the top plate of the capacitor ( CAPNOE ) will be set to the predefed value REF. The example shown Fig.2 uses GN as REF level. 3). nput chargg/dischargg: When the CLOCK is high, all the synapses start to charge or discharge the capacitor through their own variable current sources, which are controlled by the weight signals. The sign of the put and the weight determe whether to charge or discharge the capcitor. The chargg and dischargg will be termated when the put changes (either risg or fallg) or the CLOCK becomes low. The voltage of the put termal(out1) is generated by the comparator, which compares the voltage of CAPNOE and GN. 4). Negative chargg/dischargg: This stage is trigered by the fallg edge of the CLOCK signal. A fixed current source will charge or discharge the capcitor. _it determe whether to charge or discharge the capacitor. The CAPNOE will fally achieve GN becasue of the negative nature. Once the CAPNOE reachs GN, the OUT1 state will change and it will further stop the process of chargg/discharge the capaciator. Sce the weight signals are set the phase before put chargg and dischargg, it can be hiden the phase. Thus, the entire operation can be completed one cycle of clock. The next set of put data will be evaluated from the risg edge of next clock cycle. 972

3 Considerg the put chargg/dischargg stage, we assume the put current source is stable, we can get: t + 2t 2 + L n t n = + (3) C _ fall ref C which C_fall is the CAPNOE value when the CLOCK is fallg., 2,, and are the current sources of the synapse 1, synapse 2,, and synapse n. t, t 2,, and t are the put time of the synapse1, synapse2,, and synapse n. C is the capacitor value. f the current source ( ) is fixed, then the magnitude of the put value will be: t C Out = = T / 2 Cref + t + = = = C C ref ref + /( T / 2) + C _ fall T / T / 2 2t2 + L T / L T / 2 + L t T / 2 (4) put signal (voltage level while CLOCK is risg), and the sign of the weight signal W sign (H is negative and L is positive). Fig. 4 is the truth table of the PUP_ and PN. Fig. 4 shows the PUP_ and PN signals generation circuit. The current generated by MP1 and will not keep constant when the voltage of CAPNOE approaches and. MP1 and will operate non-saturation region when the ds < gs - t. Furthermore, even saturation region, the existence of the channel length Weight Control Circuit WTN C1 PUP_ PN MP1 CAPNOE Considerg TH =-C ref /(T/2) and SUM= + +, the above equation can be written as Out = ( 1 / ) ( SUM TH ) (5) The put value is proportional to the SUM case the TH is zero (when REF is equal to GN). However, due to the and voltage limitation the real circuit, the maximum will be the less than C*/( *(T/2)) and 1. At the same time, the value of put controlled current source will change due to the channel length modulation effect and the non-saturation operation as shown the next section. Thus, the activation function has sigmoid feature, which is required by the neuron operation network circuits. Fig.3. Synapse circuit. nitial put when risg nput signal MSB of Weight PUP_ PN L X X X H L H L L L L L H L L H H H H L H X H L H H L X H L H H H L H H H H H H L L. ETAL CRCUT CONFGURATON SET PUP_ A. Synapse Synapse circuit is shown Fig.3. t accepts the analog weight control signal WTN and stores the capacitor C1. WTN controls MP1 and, which operate like current sources. Control signals PUP_ and PN are used to turn on or turn off these current sources. As mentioned before, these current sources are used to charge or discharge the followg capacitor when CLOCK is high. Besides CLOCK signal, the states of the PUP_ and PN signals are also controlled by the put signal, the itial N W MSB CLR PN Fig.4. Synapse control: signal true table and circuit 973

4 modulation parameter λ will make the current variation when the CAPNOE changes as shown the follow: β 2 = ( gs t ) [1 + λ ( ds ds, sat ) (6) 2 ferent transfer curves can be obtaed by choosg dferent lengths of the CMOS transistors, which changes the value of λ. As shown (5), is proportional to CAPNOE. At the same time, CAPNOE affects the value of the MP1 and. Thus, it is possible to get a sigmoid activation function. B. Weight Control Circuit igital signals are preferred weight control circuits than analog signals due to the robustness. Pure digital weight signals can easily be adopted by replacg the MP1 and Fig.3 by transistor stacks as shown Fig.5. However, for a normal 10 bit weight control signals, the largest transistor used will be 2 10 * m size. A neural network usually has at least several tens of neurons and each neuron has several synapses. The total size of the neural network circuit will be too large to tegrate to a LS chip. A weight control circuit is used this neuron circuit to reduce the transistor size. As shown Fig.6, a weight control circuit is a simple /A converter. t converts the digital signals to a voltage control signal WTN. WTN will be stored the capacitor the synapse. Switch S1 can be used to determe whether this synapse circuit is beg selected or not. One or several the weight control circuits can be used to set the weights for all neurons they can charge the capacitor C1 much faster than the clock period. Thus, the total size of the neural network will be signicantly reduced. Leakage is the biggest problems for this type of the weight control circuit. CAP Fig.7 is equal to WTN when the transmission gate is turned on. When the transmission gate is off, possible leakage current may flow through gate of (1), through the capacitor (2), and through transmission gate (3). The leakage current 1 and 2 are usually much less than 3, which is caused by the sub-threshold current. Fig.7 shows a improved circuit to reduce the sub-threshold leakage current. 1 is force to GN when the transmission gate is off. Sub-threshold leakage current will thus be mimized sce both gs and sg of the MN2 and MP2 are less than zero. EN_ EN W0 W1 Wn WTN Synapse Synapse Fig. 6. Weight Control Circuit. EN_ WTN EN CAP 1 MP2 MN2 Synapse S1 Fig.7. Switch circuit: simplied diagram, and actual diagram. C1 MP1 FPUP_ M. 2 * M. 2 n * M. W0 W1 Wn FPN CAPNOE W0_ W1_ Wn_ M. 2 * M. 2 n * M. Fig.5. Method of usg digital weight signals. 974

5 REF LY 6ns N+ N- OUT CAPNOE Fig.10. nitialization circuit. Fig. 8. Feedback circuit, and comparator circuit. C. Feedback Circuit The circuit is shown the Fig.8. Unlike the synapse circuit, which uses weight controlled current sources, the circuit uses constant current source to charge or discharge the capacitor. The negative will fally charge the capacitor to GN and further change the comparator put signal state. Control signals FPUP_ and FPN are generated by the CLOCK, put signal of the comparator, and the itial put signal (put state while CLOCK is fallg). As the truth table and the circuit shown Fig.9, FPUP_ and FPN will disables the charge or discharge process once the comparator put signal toggles. nitial OUT when fallg OUT signal FPUP_ FPN H X X H L L L L H H L L H H L L H L H L L H H L L. Comparator and nitialization Circuit A simple 2-stage comparator is used for comparg the CAPNOE and GN level as shows Fig.8. The itialization circuit (Fig.10) resets the CAPNOE to REF before starts the normal operation. t is a simple one-pulse generation circuit triggered by the CLOCK risg edge. Sce the pulse width (5-10ns) is much shorter than the CLOCK cycle (1-2ms), the time used for itialization can be ignored.. SMULATON RESULT AN NETWORK OPERATON Fig.11 shows the simulation result of the activation function for the proposed bipolar neuron circuit. The put and put are defed (1) and (2). The limit of both and are +/-1. t has sigmoid transfer function which is required for the proper operation of the neuron. The simulation uses 2ms clock period as the operation frequency. The put chargg and dischargg will happen the first half cycle and the put will be generated the next half cycle. Because all the puts of the neuron must be available at the same time, this type of neuron is not suitable for related neural network circuit. SET FPUP_ N CLR FPN Fig. 9. Feedback control: truth table, and circuit. Fig.11. Simulation result of the activation function. 975

6 First layer Second layer Third layer 1 and dischargg capacitor (10pf-30pf) and several weight storage capacitors (1 pf each). t is suitable for beg tegrated to a LS chip because of the compact size.. CONCLUSON 2 CLOCK 2 3 A mixed signal mode bipolar neuron circuit usg digital weight, put, and put signals is presented. t features both good noise immunity and small size. The simulation result dicates that the circuit has sigmoid activation function. t is suitable for beg used the feed-forward neural networks circuit. ACKNOWLEGEMENT Fig. 12. iagram of a feed-forward neural network circuit. Fig.12 is the diagram of a feed-forward type neural network, which is suitable for usg proposed neuron circuits. As shown Fig.13, all the operation is synchronized by the CLOCK signal. The operation of the neural network has pipele features. All the neurons the same layer are operatg at the same time. The neurons the next layer will operate half cycle later. They receive the puts of neurons the previous layer as their puts. Besides the put and the chargg/dischargg operation, weight signals are generated by the weight control circuit the phase previous to the put chargg and dischargg (may overlapped with the chargg/dischargg phase). Thus, the time for processg the first set of put data is 2.5 clock cycles. After that, only half cycle is needed for processg one set of data. CLOCK Neurons first layer Neurons second layer Neurons third layer Phase 1 Phase 2 Phase 3 Phase 4 Phase 5 Phase1 Phase2 Phase3 Phase4 Phase5 Set weights put Set weights, put, set put Set weights new put, put, set put, put, set put, put, set Fig. 13. Operation timg of feed-forward neural network. The size of analog or mixed type neuron circuit is usually determed by the size of the passive components it doesn t use huge size MOSFETs. This circuit uses only one chargg This work was supported by NSF-daho EPSCoR and National Science Foundation grant EPS REFERENCES [1] Tim Schoenauer, Sah Atasoy, Nasser Mehrtash, and Herich Klar, NeuroPipe-Chip: A igital Neuro-Processor for Spikg Neural Networks, EEE Transactions on Neural Networks, ol. 13, No. 1, Jan. 2002, pp [2] B. M. Wilamowski, J. Bfet, and M. O. Kaynak, "LS mplementation of Neural Networks," nternational Journal of Neural Systems (JNS), ol. 10, No. 3, pp , June, [3] J Liu, Mart A. Brooke, and Kenichi Hirotsu, A CMOS Feedforward Neural-Network Chip With On-Chip Parallel Learng for Oscillation cancellation, EEE Transactions on Neural Networks, ol. 13, No. 5, Sep. 2002, pp [4] C.Lu, and B.Shi, "Circuit Realization of an Programmable Neuron Transfer Function and ts erivative," Proceedgs of the 2000 ternational Jot Conference on Neural networks, pp [5] cent F.Koosh and Rodney M. Goodman, Analog LS Neural Network With igital Perturbative Learng, EEE Transactions on Circuits ans Systems, ol. 49, No.5, May. 2002, pp [6] Yasuhiro Ota, and Bogdan M. Wilamowski, Analog mplementation of Pluse-Coupled Neural, EEE Transactions on Neural Networks, ol. 10, No.3, May. 1999, pp [7] Bo Liu, and James F. Frenzel, A CMOS Neuron for LS Circuit mplementation of Pulsed Neural Networks, Proceedgs of the 28 th annual conference of EEE dustrial Electronics society,