Computational Models of Neural Systems

The input space contains 50 units. Each input pattern has 6 of these units turned on.

The output space contains 100 units. Each output unit receives connections from 15 input units, selected at random. The output units' thresholds are set so that roughly 5 units turn on.

`nchoosek(N,k)`

returns the number of
ways of selecting k elements out of a set of N. You will want to use
it in this exercise. You also need to know how to define simple
functions of your own. You can do it by creating a file in the
current directory, using the Matlab editor or any other text editor.
But you can also create simple functions right at the keyboard.
Example: how many ways are there to flip a coin 20 times and get equal
numbers of heads and tails? The expression below creates a function
`faircoin`

that takes a value N as input, and returns the
value of the expression `nchoosek(N,N/2)`

. Computing
`faircoin(20)`

will solve our problem.
>> faircoin = @(N) nchoosek(N,N/2) faircoin = @(N) nchoosek(N,N/2) >> faircoin(20) 184756

- How many possible input patterns are there for our little memory?
Give a precise number.
- What is the sparseness of the input code, i.e., what percentage
of units are active? What about the output code?
- Write a Matlab function to calculate the probability that an
output unit receives exactly H hits from active input units.
- Calculate the probability distribution for values of H, and write
it down in tabular form.
- What is the expected value of H? (a) Show how to calculate this from
the probability distribution table above. (b) Show how to calculate this
directly, using the formula on this page:
http://mathworld.wolfram.com/HypergeometricDistribution.html
- What is the variance of the distribution for H? Write a Matlab
expression to compute the result, and give the numerical value. To
answer this question, read about the hypergeometric distribution here:
http://mathworld.wolfram.com/HypergeometricDistribution.html
- What value should be used for the thresholds of output units in
order to make it likely that the output pattern will contain 5 active
units?
- Assume that input patterns A and B overlap by 3 bits. Consider an output unit that has 5 hits for pattern A. What is the likelihood that this unit has exactly 5 hits for pattern B? You must use the formula for Pb from O'Reilly and McClelland, and consider the different possible values for Hab, the number of bits that are hits for both pattern A and pattern B. Note: Hab cannot be greater than the overlap; it also can't be too small, since we know that the unit has 5 hits for pattern A, we're assuming it also has 5 hits for pattern B, and there are only 6 bits in each pattern. Once you define a function for Pb, you can write a one-line expression to calculate the answer.

Dave Touretzky Last modified: Mon Oct 12 02:57:07 EDT 2015