Recurrent Neural Networks A Brief Overview

Size: px

Start display at page:

Download "Recurrent Neural Networks A Brief Overview"

Alberta York
7 years ago
Views:

1 Recurrent Neural Networks A Brief Overview Douglas Eck University of Montreal RNN Overview Oct p.1/33

2 RNNs versus FFNs Feed-Forward Networks (FFNs, left) can learn function mappings (FFN==>FIR filter) Recurrent Neural Networks (RNNs, right) use hidden layer as memory store to learn sequences (RNN==>IIR filter) RNNs can (in principle at least) exhiit virtually unlimited temporal dynamics RNN Overview Oct p.2/33

3 Several Methods SRN Simple Recurrent Network (Elman, 1990) BPTT Backpropagation Through Time (Rumelhart, Hinton & Williams, 1986) RTRL Real Time Recurrent Learning (Williams & Zipser, 1989) LSTM Long Short-Term Memory (Hochreiter & Schmidhuer, 1996) RNN Overview Oct p.3/33

4 SRN (Elman Net) Simple Recurrent Networks Hidden layer activations copied into a copy layer Cycles are eliminated, allowing use of standard ackpropagation RNN Overview Oct p.4/33

5 Some Oservations y Elman Some prolems change nature when expressed in time Example: Temporal XOR (Elman, 1990) [ ] RNN learns Frequency Detectors Error can give information aout temporal structure of input Increasing sequential dependencies does not necessarily make task harder Representation of time is task-dependent RNN Overview Oct p.5/33

6 SRN Strengths Easy to train Potential for complex and useful temporal dynamics Can induce hierarchical temporal structure Can learn, e.g., simple natural language grammar RNN Overview Oct p.6/33

7 SRN Shortcomings We need to compute weight changes at every timestep t0 < t t1: t1 t=t0+1 w ij(t) Thus we need to compute at every t: E(t) w ij = k U (t) y k (t) y k (t) w ij = k U e k(t) y k(t) w ij Since we know e k (t) we need only compute y k(t) w ij SRN Truncates this derivative Long-timescale (and short-timescale) dependencies etween error signals are lost RNN Overview Oct p.7/33

8 SRNs Generalized: BPTT Generalize SRN to rememer deeper into past Trick: unfold network to represent time spatially (one layer per discrete timestep) Still no cycles, allowing use of standard ackpropagation RNN Overview Oct p.8/33

9 BPTT( ) For each timestep t Current state of network and input pattern is added to history uffer (stores since time t=0) Error e k (t) is injected; ǫs and δs for times (t 0 < τ t) are computed: ǫ k (t) = e k (t) δ k (τ) = f k (s k(τ))ǫ k (τ) ǫ k (τ 1) = l inu w lkδ l (τ) Weight changes are computed as in standard BP: = t τ=t 0 +1 δ i(τ)x j (τ 1) E(t) w ij RNN Overview Oct p.9/33

10 Truncated/Epochwise BPTT When training data is in epochs, can limit size of history uffer to h, length of epoch When training data not in epochs, can nonetheless truncate gradient after h timesteps For n units and O(n 2 ) weights, epochwise BPTT has space complexity O(nh) and time complexity O(n 2 h) Compares favoraly to BPTT (sustitute L, length of input sequence, for h) RNN Overview Oct p.10/33

11 BPTT s Gradient Recall that BPTT( ) computes = t τ=t 0 +1 δ i(τ)x j (τ 1) E(t) w ij Thus errors at t take into account equally δ values from the entire history of computation Truncated/Epochwise BPTT cuts off the gradient: = t τ=t h δ i(τ)x j (τ 1) E(t) w ij If data is naturally organized in epochs, this is not a prolem If data is not organized in epochs, this is not a prolem either, provided h is ig enough RNN Overview Oct p.11/33

12 RTRL RTRL=Real Time Recurrent Learning Instead of unfolding network ackward in time, one can propagate error forward in time Compute directly p k ij (t) = y k(t) w ij p k ij (t + 1) = f k (s k(t)) l U w klp l ij (t) + δ ikx j (t) w ij = α E(t) w ij = α k U e k(t)p k ij (t) RNN Overview Oct p.12/33

13 RTRL vs BPTT RTRL saves from executing ackward dynamics Temporal credit assignment solved during forward pass RTRL is painfully slow: for n units and n 2 weights, O(n 2 ) space complexity and O(n 4 )(!) time complexity Because BPTT is faster, in general RTRL is only of theoretical interest RNN Overview Oct p.13/33

14 Training Paradigms Epochwise training: reset system at fixed stopping points Continual training: never reset system Epochwise training provides arrier for credit assignment Unnatural for many tasks Distinct from whether weights are atchwise or iteratively updated RNN Overview Oct p.14/33

15 Teacher Forcing Continually-trained networks can move far from desired trajectory and never return Especially true if network enters region where activations ecome saturated (gradients go to 0) Solution; replace during training actual output y k (t) with teacher signal d k (t). Necessary for certain prolems in BPTT/RTRL networks (e.g. generating square waves) RNN Overview Oct p.15/33

16 Puzzle: How Much is Enough? Recall that for Truncated BPTT, the true gradient is not eing computed How many steps do we need to get close enough? RNN Overview Oct p.16/33

17 Puzzle: How Much is Enough? Recall that for Truncated BPTT, the true gradient is not eing computed How many steps do we need to get close enough? Answer: certainly not more than 50; Proaly not more than 10(!) RNN Overview Oct p.17/33

18 Credit Assignment is Difficult In BPTT, error gets diluted with every susequent layer (credit assignment prolem): ǫ k (t) = e k (t) δ k (τ) = f k (s k(τ))ǫ k (τ) ǫ k (τ 1) = l inu w lkδ l (τ) The logistic sigmoid 1.0/(1.0 + (exp( x))) has maximum derivative of When w < 4.0, error is always<1.0 RNN Overview Oct p.18/33

19 Vanishing Gradients Bengio, Simard & Frasconi (1994) Rememering a it requires creation of attractor asin in state space Two cases: either system overly sensitive to noise or error gradient vanishes exponentially General prolem (HMMs suffer something similar) RNN Overview Oct p.19/33

20 Solutions and Alternatives Non-gradient learning algorithms (including gloal search) Expectation Maximization (EM) training (e.g. Bengio IO/HMM) Hyrid architectures to aid in preserving error signals RNN Overview Oct p.20/33

21 LSTM memory lock with single cell y[t] u[t] Hyrid recurrent neural network Make hidden units linear (derivative is then 1.0) Linear units unstale Place units in a memory lock protected y multiplicative gates RNN Overview Oct p.21/33

22 Inside an LSTM Memory Block g(net c ) s c h(s c ) y c y in y ϕ y out Gates are standard sigmoidal units Input gate y in protects linear unit s c from spurious inputs Forget gate y φ allows s c to empty own contents Output gate y out allows lock to take itself offline and ignore error RNN Overview Oct p.22/33

23 LSTM Learning For linear unit s c, a truncated RTRL approach is used (tales of partial derivatives) Everywhere else, standard ackpropagation is used Rationale: due to vanishing gradient prolem, errors would decay anyway RNN Overview Oct p.23/33

24 Properties of LSTM Very good at finding hierarchical structure Can induce nonlinear oscillation (for counting and timing) But error flow among locks truncated Difficult to train: weights into gates are sensitive RNN Overview Oct p.24/33

25 Formal Grammars LSTM solves Emedded Reer Grammar faster, more relialy and with smaller hidden layer than RTRL/BPTT LSTM solves CSL A n B n C n etter than RTRL/BPTT networks Trained on examples with n < 10 LSTM generalized to n > 1000 BPTT-trained networks generalize to n 18 in est case (Bodén & Wiles, 2001) a T S a a a a a a a a a a a c c c c c c c c c c c c T : input : target s y s y c 1 c 1 c2 c 2 RNN Overview Oct p.25/33

Recurrent Neural Networks

Recurrent Neural Networks Neural Computation : Lecture 12 John A. Bullinaria, 2015 1. Recurrent Neural Network Architectures 2. State Space Models and Dynamical Systems 3. Backpropagation Through Time