topic connectionist ★ seed
Recurrent Neural Networks
Networks with feedback connections for processing sequences. Capable of modeling temporal dependencies but prone to vanishing gradients in long sequences.
#rnn
#recurrent
#sequence-modeling
Sub-topics
LSTM (1997) concept
Hochreiter and Schmidhuber's 1997 Long Short-Term Memory solved the vanishing gradient problem with gated memory cells. Dominated sequence modeling for 20 years.
GRU (2014) concept
Kyunghyun Cho's 2014 Gated Recurrent Unit simplified LSTM with fewer parameters (two gates instead of three) while achieving comparable performance.