15-883 Homework #4
Issued: October 12, 2015. Due: October 26, 2013.
Computational Models of Neural Systems
A Simple Matrix Memory
In this exercise you will calculate some statistics for a matrix
memory that maps input patterns into "simple representations" (Marr's
term) in a higher-dimensional space. Read the 1994 paper by O'Reilly
and McClelland before proceeding further.
The input space contains 50 units. Each input pattern has 6 of these
units turned on.
The output space contains 100 units. Each output unit receives
connections from 15 input units, selected at random. The output
units' thresholds are set so that roughly 5 units turn on.
The Matlab function
nchoosek(N,k) returns the number of
ways of selecting k elements out of a set of N. You will want to use
it in this exercise. You also need to know how to define simple
functions of your own. You can do it by creating a file in the
current directory, using the Matlab editor or any other text editor.
But you can also create simple functions right at the keyboard.
Example: how many ways are there to flip a coin 20 times and get equal
numbers of heads and tails? The expression below creates a function
faircoin that takes a value N as input, and returns the
value of the expression
faircoin(20) will solve our problem.
>> faircoin = @(N) nchoosek(N,N/2)
- How many possible input patterns are there for our little memory?
Give a precise number.
- What is the sparseness of the input code, i.e., what percentage
of units are active? What about the output code?
- Write a Matlab function to calculate the probability that an
output unit receives exactly H hits from active input units.
- Calculate the probability distribution for values of H, and write
it down in tabular form.
- What is the expected value of H? (a) Show how to calculate this from
the probability distribution table above. (b) Show how to calculate this
directly, using the formula on this page:
- What is the variance of the distribution for H? Write a Matlab
expression to compute the result, and give the numerical value. To
answer this question, read about the hypergeometric distribution here:
- What value should be used for the thresholds of output units in
order to make it likely that the output pattern will contain 5 active
- Assume that input patterns A and B overlap by 3 bits. Consider
an output unit that has 5 hits for pattern A. What is the likelihood
that this unit has exactly 5 hits for pattern B? You must use the
formula for Pb from O'Reilly and McClelland, and consider the
different possible values for Hab, the number of bits that are hits
for both pattern A and pattern B. Note: Hab cannot be greater than
the overlap; it also can't be too small, since we know that the unit
has 5 hits for pattern A, we're assuming it also has 5 hits for
pattern B, and there are only 6 bits in each pattern. Once you
define a function for Pb, you can write a one-line expression
to calculate the answer.
Last modified: Mon Oct 12 02:57:07 EDT 2015