Preview. Spectral Associative Memories: Neural Encoders and Decoders for Analog & Digital Communications. Practical Features

Transcription

1 Spectral Associative Memories: Neural Ecoders ad Decoders for Aalog & Digital Commuicatios Ro Specer, Ph.D. Texas A&M Uiversity Aalog & Mixed Sigal Ceter, Electrical Egieerig Dept. Preview Overview Backgroud Associative amplitude modulatio (AAM) Spectral associative memories (SAMs) ) Codig ) Decodig 3) Bad structures 4) Ati-aliasig Costraits Applicatios Performace Implemetatio Coclusios Refereces Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity Bipolar-Valued Patters Overview of the SAM Family Applicatio. Sigle-Patter - Autoassociative Digital modulators w/codig gai - Heteroassociative Digital modulators w/recofigurable codig gai. Multi-Patter - Autoassociative Spectral CAMs - Heteroassociative Spectral CAMs Real-Valued Patters. Sigle-Patter - Autoassociative Aalog/Digital MODEMs w/codig gai - Heteroassociative Aalog/Digital MODEMs w/recofigurable codig gai. Multi-Patter - Autoassociative Spectral CAMs Practical Features Codig Gai at the level of Modulatio: - Extrisic redudacy of eural etworks combied with the i-phase, quadrature, ad complex modulatio schemes of telecommuicatios. Applicatios: - Digital patters over a oisy chael - Aalog patters over a oisy chael Performace: - BER of less tha ^-3 for db SNR for code rates aroud /5. Implemetatio: - Covetioal blocks - summers, mixers, & filters. - No ad hoc expoetial log-likelihood fuctios Scalability: - Liear with patter dimesio Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 3 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 4 Backgroud o Hopfield Networks p ( m) ( m) T W s s I p m The etwork is iitialized with a oisy, ukow patter which coverges to oe of the store attractors that is earest i terms of Hammig distace. Flip-Flop (-D Hopfield) The flip-flop is the simplest autoassociative memory (=). Bistable eergy ladscape: State Trasitio Diagrams Sychroous Asychroous (Images from Jacek Zurada s book) Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 5 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 6

2 A() S ( ) i Si S k Associative Memories - Spatial vs. Spectral Autoassociative vs. Heteroassociative Memories Spatial - Restores stored vectors from oisy iitial coditios - Simple architecture - Attractors stored spatially - Superpositio of attractors, but ot superpositio of memories Autoassociative Spatial Spectral Spectral - Attractors maifested as waves - Attractor exists separately from eural decoder - Badwidth scales as - Area scales as - No-local eural coectivity is made i the frequecy domai by spectral covolutio Heteroassociative Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 7 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 8 Spectral Associative Codig & Recall Associative amplitude modulatio may be used to achieve o-local eural coectivity i the frequecy domai by spectral covolutio. Data patter, or codeword to be ecoded f ( f ( F ( ) F ( ) Recalled data patter Spectral Codig Associative amplitude modulatio ecodes data patters ito a attractor wave.. Sythesis:. Covolutio: 3. Kerel Geeratio:. Sythesis:. Covolutio: 3. Kerel Geeratio: I-Phase (I): Quadrature (I/Q): s I () ( x cos( S() i s Q ( x () i si( S() i i R R i i M ( M ( k s ( ) ( ) M ( M SSB( km si t si t sq t sq t DSB m I t si t W ( kw I( M ( I ( IDSB( cos ILi i Complex: () ( ) j t s t ks () x e ; k i s () i x () ( ) ( ) ( ) jt M t km s t s t km x ixke i k ( k ( ( W w I M W ( kw I( M ( cos( cos( I ( I ( cos cos( cos j t IL ( k IL i e i k i cos i IUi SSB i ILi T x x x x Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 9 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity Codig Coefficiets for Normalized Sigal Power Spectral Recall Recurret associative amplitude modulatio focuses extrisic iformatio distributed all over the attractor bad for subsequet spectral aalysis, or dow coversio.. Spectral Sythesis: j t v ( t ) v v lie li l i. Spectral Covolutio: a ( v ( W( 3. Spectral Aalysis (Dow Coversio): du() dt LPF Re ( e j t a (spectral gradie 4. State Update: v f (u) u udt Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity

3 Bad Structure Chaels are separated by iteger multiples of a beat frequecy,, ad have row-colum form. Spectral Codig (AAM) Spectral Decodig (Recurret AAD) Typical spectrum: Ati-Aliasig Costraits: Side-bad ati-aliasig costrait: (Lowest sum must be higher tha highest differece) ==========> Alias-free memory formatio Alias-free recall S.L BS Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 3 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 4 Telecommuicatio Applicatios Sigle-Patter Telecommuicatios Digital modulators w/codig gai (built-i oise immuity) Aalog modulators w/codig gai Aalog modulators for digital codewords ested iside a Turbo wrapper Cotet addressable telecommuicatios - Itelliget Bluetooth - Collective commuicatios - Merge computatio ad commuicatio ito oe Ecodig multiple attractors ad trasmittig the codeword as iitial coditios would ot work. Why ot But sigle patters may be trasmitted, oe at a time, across a oisy chael, to a spectral eural decoder with spurious-free recall due to a lack of iterferece betwee patters. Code rate depeds o virtual architecture (auto/hetero), which allows dyamic recofiguratio. Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 5 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 6 Performace: Noise Immuity (RV-SAMs) Trasiet respose of a -D autoassociative etwork for a sampled sie wave iput: (a) what the origial sigal would have bee if set directly over the oisy chael with db SNR, (b) oisy attractor wave of the first data sample, ad (c) recalled sie wave. Ecodig ad recall of multiple, simultaeous 8x8 biary patters Iterferece ad spurious attractio are still a problem for small etworks Ecoded patters => Recalled patters => Performace: Multi-Patter Recall Comparig (c) with (a) illustrates the built-i oise immuity. Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 7 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 8

4 Performace: Real-Valued Recall Performace: Settlig Time & Accuracy of Recall Real-valued codig ad recall of Maryli Moroe i time Butterworth recall (B) of various order, with ad without otch filter (N). mi =4dB Chebyshev recall (C) of various order, with ad without otch filter (N). mi =4dB Bits B3 B4 B5 B6 Ts ( s) Ts ( s) Ts ( s) Accuracy Ts (s) If If Bits B3 + N B4 + N B5 + N B6 + N Accuracy Ts ( s) Ts ( s) Ts ( s) Ts (s) If 93 If 8 Bits C3 C4 C5 C6 Ts ( s) Ts ( s) Ts (s) Accuracy Ts ( s) If If If 48 If 6 If If If If Bits C3 + N C4 + N C5 + N C6 + N Accuracy Ts ( s) Ts ( s) Ts (s) Ts ( s) If If If If If Isertig a otch filter at i cascade with the LPF allows the begiig of the stopbad to be doubled, which icreases the width of the passbad ad icreases the speed of recall. Butterworth recall filters appear to give the fastest respose for higher accuracy of recall. Data rates: ~ 3kHz/ch (Nyquis or 6kHz/ch. (baud) for or bits accuracy ~ khz/ch (Nyquis or khz/ch. (baud) for 5 or 6 bits accuracy All results were obtaied with a beat frequecy of = 5k rad/sec. Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 9 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity Performace: Bit Error Rate (BER) BER is affected by: () Redudacy: Codeword or patter dimesio () () Decodig Time, T c, or equivaletly, data rate, f data (3) SNR (4) Spectrum Spread, or equivaletly, beat frequecy, f (5) Computatioal Oversamplig (f comp /f data ) Decodig time is the most effective variable i reducig BER. Computatioal Oversamplig is sigificat i extremely oisy coditios. Double Sidebad Attractor Wave (I-phase codig) Sigle Sidebad Attractor Wave (Quadrature codig).improved BER for additioal decodig time Performace: Sigal-To-Noise Ratio Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity Performace: Decodig Time / Data Rate The largest improvemet is from -db -db ad -db db SNR. Not early as much improvemet from db +db SNR. Performace: Spectrum Spread / Beat Frequecy The more spread-out the spectrum is, the higher the oise immuity. The higher the beat frequecy (color separatio), the more spread-out the spectrum is. Icreasig beat frequecy Decreasig BER Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 3 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 4

5 Implemetatio: Bipolar SAM System Level CMOS Implemetatio: Decoder Additio may be implemeted i curret domai Mixers may be implemeted with Gilbert cells Itegrators may be implemeted by OTA-C stages Oscillators may be geerated off-chip or implemeted by rig oscillators (must be phaselocked) Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 5 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 6 CMOS Mixer P-type mixer i a -well process esures o body effect, or modulatio of the threshold voltage Disadvatage: lower mobility for PFETs => larger devices => more area CMOS Hebbia Filter: Mixer + LPF A additioal trascoductace stage followed by a capacitor ad two iverters Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 7 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 8 CMOS Decoder Chael 3D CMOS Decoder Techology:.35 TSMC double-poly, quadruple metal -well, 5u x 4u Output Buffer Sythesis Mixer Techology:.35 TSMC double-poly, quadruple metal -well,.6mm x mm Package: 9 pis coected to 8 PGA 3D Decoder: Bypass caps Output Buffers Sigle Chael Sigle Chael Quatizer w/hysteresis Aalysis Mixer & LPF Global Coectivity Mixer Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 9 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 3

6 CMOS Implemetatio: Trasiet Respose Bit ecoded low, Bit ecoded high, Bit ecoded high Bits & Bitx Attractor Wave State Wave Bitx Bit Coclusios SAMs combie the extrisic redudacy of eural etworks with the i-phase, quadrature, ad complex modulatio schemes of telecommuicatios Data patters may be ecoded ito a attractor wave, which exists separately from the eural decoder, by associative amplitude modulatio (AAM) ad recalled by recurret associative amplitude demodulatio (AAD) SAMs are multi-chael multi-carrier MODEM-CODECs SAMs have built-i oise immuity - BER < -3 may be obtaied for db SNR for block legths of oly 5 bits SAMs require oly covetioal buildig blocks: mixers, summers, itegrators, & comparators. Dyamic recofiguratio of chael codig gai is possible. Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 3 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 3 Aalogies Solid-state - Bad structures - Bad gaps - Collective eergy states Electromagetic - Photo emissio & absorptio => differece eergy ito LSB - Attractor wave - Focus & defocus - Mass-actio & spectral covolutio Quatum mechaical - Classical aalog of pure Bell states ad etaglemet - Attractor space scales geometrically, but QPS scales expoetially - Iterferece of attractors - Classical aalog of o-local coectivity Refereces Joural Papers: R. Specer, "Bipolar Spectral Associative Memories," IEEE Tras. o Neural Networks. To appear. R. Specer, Spectral Associative Memories: Cotet-Addressable Commuicatios," It'l. J. of Smart Egieerig System Desig. To appear. Coferece Papers: R. Specer, "Noliear Heteroassociative Spectral Memories: Virtually Partitioed Neural Ecoders ad Decoders for Digital Commuicatios," It'l. Symposium o Computatioal Itelligece, It'l. Cogress o Itelliget Systems ad Applicatios, Wollogog, Australia, Dec. -5,. R. Specer, "Noliear Spectral Associative Memories: Neural Ecoders ad Decoders for Digital Commuicatios," I: Itelliget Egieerig Systems Through Artificial Neural Networks, C.H. Dagli, A.L. Buczac, J. Ghosh, M.J. Embrechts, O. Ersoy, ad S. Kercel (Eds.), ASME Press, Vol.., pp ,. ISBN Best Paper Award. Submitted/Workig Papers: R. Specer, H. Dic, & T. Sumesaglam, "Associative Amplitude Modulatio with Built-I Noise Immuity," submitted to ICECS, Malta. R. Specer, Multi-Patter Real-Valued Spectral Associative Memories," submitted to IJCNN. Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 33 Aalog & Mixed-Sigal Ceter, Electrical Egieerig Dept., Texas A&M Uiversity 34