Examples of Learning by Neural Networks
Learning errors by back propagation, competitive learning, and the Boltzmann machines explained in Chapter 10 are the most popular learning algorithms using neural networks. In this Appendix we will show sample programs written in C for each of these algorithms. Each program describes only the basic algorithm. Although an interface that shows the execution process and the input-output data of a program using neural networks for learning is important, we have not included the program for such interfaces here. The following programs can easily be executed on a personal computer running MS-DOS.
The examples of back propagation error learning and competitive learning are for the following problem:
Problem 1 (back propagation error learning and competitive learning)
Construct a neural network that learns how to distinguish the four training patterns shown in Figure A.1.
Next, input the eight test patterns shown in Figure A.2 to the resulting neural network and check which test patterns belong to each of the categories of the training patterns.
The results for both programs are
test patterns 5, 6 ⇒ training pattern 1
test patterns 7, 8 ⇒ training pattern 2
The example of a Boltzmann machine is for the following problem:
Problem 2 (Boltzmann machine)
Suppose we have the neural network shown in Figure A.3. Construct the neural network that outputs (0, 1), (1, 0) for the input (0, 1), (1, 0), respectively. (This problem is called the 2-1-2 symbolization problem.)
Execution Example A.2
(The two columns on the right are the numbers of the units that output 1 at the output clusters 1 and 2.)
Execution Example A.3
How the sum of the errors in the probability of occurence changes for each pattern (the vertical axis shows the learning cycle; the horizontal axis shows the sum of the errors).
The weights of a unit and the threshold after learning (the rows show the unit number; the columns show the weight of each link and the input bias value (with thresholds in reverse).
The probability of the occurence of the output pattern for each input pattern after learning (the rows show the input pattern number; the columns show the output pattern number; error shows the error from the ideal output result).