15-883 Homework #3
Issued: September 28, 2015. Due: October 7, 2015
This assignment makes use of files in the course's matlab
subdirectory, which is linked from the course home page. From this
directory you can download zip files containing any of the demo
programs used in the course.
Computational Models of Neural Systems
Part I: Run the Simple Matrix Demo
This Matlab script demonstrates creation of a matrix memory using the
outer product rule. The memory stores three items correctly using
orthogonal keys; it fails catastrophically when a fourth item is added
whose key is not orthogonal to the others. However, the four keys are
linearly independent, so a new memory is constructed using the matrix
inverse technique instead of outer product, and it is verified to work
The file script1.log shows the output you should see if you
execute the script correctly. You do not have to hand anything in for
this part. But make sure you understand what the script is doing, because
you will be performing similar computations later in this assignment.
- Go to the matlab/simplemat directory.
- Execute the file script1.
- Each time you see the word pause, press the space bar to proceed.
Part II: Run the Graphical Matrix Memory Demo
You should cd to the directory matlab/matmem, or download the
file matmem.zip and unzip it. Read the README file before
When you're ready to begin, type "matlab" to start up Matlab. Then
type "matmem" to start the matrix memory demo.
Experiment with the matmem demo:
Hand in your answers to the questions above.
- Click on the Random button to generate a set of random memory
items. Then start clicking on memory item buttons. As you click on a
button, it turns pink, and the result box to its right should also
turn pink to show you that the stored item is successfully retrieved.
As you click on additional memory items, the memory fills up, and
eventually one of the corresponding result boxes will turn blue,
indicating that retrieval of that item is not completely correct.
Questions: How many items can you store before the first
retrieval error occurs? What percentage of memory bits are active
when the first retrieval error occurs? Hit the New button and then
Random to generate another set of data items. Try the experiment
several times and report average values.
- Switch from a dense to a sparse representation by clicking on the
Dense button. The button label will change from "Dense" to "Sparse".
Now instead of using a 5-bit binary code, each letter is stored using
a 2-of-8 bit code. This requires 24 input units instead of 15.
Re-run the previous experiment a few times, but using sparse
Questions: How many items can you store using the sparse
representation? What percentage of memory bits are turned on when the
first retrieval error occurs?
Part III: Building Your Own Associative Memory
Consider an associative memory using binary keys with a randomly
chosen 3 out of 10 bits true. Here is a function for generating
Complete this Matlab program to fill a memory with key-value pairs
until the memory capacity is exceeded. The keys will be random. For
the values to be stored, we'll use 1 for the first pattern, 2 for the
second, and so on. Since the keys are not orthogonal, we will use the
pseudo-inverse method to calculate the weight matrix. Here is the
program; you need to fill in some missing parts.
function Z = keyvec(N,B)
% Generate an N-bit binary vector with a random B bits true
P = randperm(N);
Z(P(1:B)) = 1;
N = 10; % length of a key
B = 3; % number of 1 bits in a key
Keys = ;
Values = 0; % values to store
Results = 0; % results of retrieval using Keys
M = zeros(1,N); % weight matrix
NumPatterns = 0;
while max(abs(Values-Results)) < 1e-10
NumPatterns = NumPatterns + 1;
k = keyvec(N,B) / B; % normalized random key
Keys = [Keys k]
Values = 1:NumPatterns;
M = ... fill this in to construct the weight matrix
Results = ... fill this in to retrieve the values given the keys
fprintf('Stored %d patterns without error.\n',NumPatterns-1);
- Run the program a bunch of times. How many patterns can you
typically store in the memory?
- Occasionally you'll have a "bad" run where only 2-4 patterns are
stored before an error occurs. What is the cause of this?
- If you change the value of B from 3 to 7, the keys are less
sparse and less nearly orthogonal. What effect does this have on the
memory capacity? (Don't just guess; try it and see.) Explain the
reason for your result.
Hand in your source code plus the answers to all questions.
Last modified: Mon Sep 28 04:39:17 EDT 2015