Overview

• Recurrent-NN
  • Feedback from outputs to inputs
• Hopfield NN
  • No self-feedback
  • Can store a set of fundamental memories if the weight matrix is symmetrical & zeros in its main diagonal

Algorithm

• $$Y$$: fundamental memories
• $$W$$: weights
• $$M$$: number of fundamental memories
• $$n$$: number of neurons

1. Storage of fundamental memories
   1. $$W = \sum_{i=1}^M Y_iY_i^T - MI$$
2. Testing the recall of fundamental memories
   1. $$X_i = Y_i$$
   2. $$Y_i = sign(WX_i)$$
3. Retrieval of incomplete/corrupted version of a fundamental memory
   1. $$Y(0) = sign(WX)$$
   2. $$Y(t+1) = sign(WY(t))$$ (neurons for updating selected asynchronously)

Behavior

1. Always converge to a stable state
2. Not necessarily retrieve one of the fundamental memories

Storage Capacity

1. Experimental result: $$M_{max} = 0.15n$$ (small)
2. All fundamental memories can be retrieved perfectly: $$M_{max} = \frac{n}{4\ln{n}}$$
3. Auto-associative memory