Assignment 3. In this problem, we will experiment with a standard back-prop example, the auto-encoder. The basic idea is very simple, get Tlearn to produce output that is as close as possible to its input. As is conventional, we will restrict ourselves to UNARY strings, that is strings with just one "1". The catch is that the networks will all have a hidden layer that is significantly smaller than the input and output layers. In principle, we should be able to encode all UNARY strings of length K using a hidden layer of size log(2) K with no direct links from input to output. More generally and more interestingly, if the set of UNARY patterns is not complete, back-prop should be able to learn an even more compact representation. The PDP approach to cognitive modeling relies heavily these compact representations.

a) The first part of the problem is trivial, just set up and use the auto3 project that is provided in the Tlearn distribution. Experiment with different settings for learning rate, momentum and training style. Your goal should be to find a training regime that will reliably do auto-association from as small a training sample as possible.

b) For the second part of the problem, try to get the network to learn to auto-encode only those UNARY strings with ones in the EVEN POSITIONS. Just don't present any data with ones in ODD POSITIONS. Does this make the problem easier or harder for the network? How does the even-ones trained net perform on strings with ones in the ODD POSITIONS?

c) As you have seen, it is not easy to interpret the encoding produced by back-prop training. Tlearn has two powerful mechanisms for helping to do this: hierarchical cluster analysis and principle components analysis. Decide which will be more informative for this problem and use it on both parts.

There are problems with some versions of Tlearn wrt these functions. At least you will need to edit some files to get rid of the text header information. We will try to find fixes or at least keep you informed.

This assignment is due in class on Thursday, September 18 or by earlier electronic submission.