Notes for Sep. 26 lecture

................................................................................

Any questions about the previous lecture?


associative memory:
  basic idea:  store some vectors, retrieve based on partial vector
  hetero-associator vs. auto-associator
  auto-associators can do pattern completion
  Willshaw's holographic associative memory model:  based on correlation matrix
  memories as eigenvectors (the eigenvalue is the number of bits in the probe)
  capacity?  can store up to N orthogonal vectors, or < N non-orthogonal ones
  show how Willshaw's model can be implemented by neurons with binary weights and
    a threshold set by the number of ones in the probe vector
  many other associative models, most of which use feedback (Willshaw does not)
  Kohonen's face completion model

interactive activation model:
  combination of spreading activation (mentioned in 1st lecture) and
    lateral inhibition:  the two major connectionist processes
  does not do learning
  the Jets & Sharks model - show the table of data
  show the architecture; the PDP handbook includes the IA simulator
  assoc retrieval:  activate name, get properties; activate properties, get name
  classification of novel instances as Jet or Shark based on properties
  filling in default values for a novel instance
  prototype generation:  activate "Jet" and read off the default properties
  lots of psychological modeling possible, e.g., equate settling time
    with reaction time:  more "typical" members recognized quicker
  relation to Lakoff's notion of radial categories
  note: settling depends on good choice of activation and inhibition parameters
  this is classic connectionism:  modeling qualitative properties of human
    reasoning via parllel settling, without claiming this is how the brain does it

competitive learning:
  talk about supervised vs. unsupervised learning
  the point of competitive learning is to divide the the input space into clusters
  it is an incremental learning technique, but has no error signal
  often used in pattern recognition, e.g., phoneme classification
  special tricks:  prevent units from starving if they aren't close to ANY points,
      by adjusting the losers' weights a teeny bit in addition to the winner's.
      This drags the losers over to where the points are
  another trick:  vary the diameter (magnitude) of a unit so that all units can have
      roughly equal numbers of points
  stability of this learning procedure is discused in PDP chapter 5
  Grossberg's Adaptive Resonance model is a variant
