Preface

Neural networks are interesting and useful tools for researchers working on a wide range of problems from various disciplines and the networks continue to attract new users. Figure P.1 shows increasing profits from sales of neural network software.

As the figure indicates, the first commercially available software tools using neural networks emerged in the early 1990s. The dynamic growth of this portion of the market took place in or around 2000, and throughout the past decade interest in neural networks has increased steadily although not as fast as in the early years of development. In addition to the practical value of information tools based on neural networks, it is interesting that these networks are mapped into parts of the brains of humans and animals (Figure P.2).

The work of these neural network researchers initiated by the Russian scholar Pavlov led to a whole series of discoveries honored with the most valuable trophy a scholar can earn: the Nobel Prize. Table P.1 lists the Nobel Prize winners who contributed to our understanding of the principles according to which the biological brain functions and thus created the base on which we can build technical neural networks.

Table P.1

Nobel Prizes Associated with Studies of the Nervous System, Results of Which Were Used Directly or Indirectly in Neural Networks

As you can see from the table, biologists in the early 1990s knew very much about brain function. The bio-cybernetics experts built cyber models (mathematical and computer simulations) of all the brain mechanisms. It was proven beyond a reasonable doubt that the brain collects and processes information more efficiently and economically than computer systems although computers dominate in the areas of speed and storage capacity.

Let us now see where and how neural network concepts apply to IT problems and you will see why they are so interesting. Figure P.3 depicts how classification tasks are performed by different information systems. Clearly, some tasks are easy and some are very difficult. The horizontal axis of the chart illustrates the degree of task difficulty.

The difficulty of a task is not the only measure of trouble faced by the person who wants to complete it. A second dimension of a problem is the availability of knowledge on which the solver can depend while reaching the solution. The rules for some tasks are specified although the number of rules or their complexity can continue to make a task more difficult. For example, a software developer for a large bank has a lot of work in a situation where tolerance of ignorance of the rules is zero. This is because the rules are consistent and known and can be used to construct tools required to solve the task.

We know how to use computers to solve tasks characterized by complete knowledge of the rules. On the basis of the rules, we can build an algorithm; on the basis of the algorithm, we can write a suitable program. In practice, the situation may be more complicated. We are often called upon to solve problems for which no rules are known. If we want to solve a task, we may have to make decisions based observations of related phenomena or processes—we must deal with a situation symbolically (see center section of Figure P.3). Knowledge of the rules is insufficient in this case and the rules may not be known until the task is complete. Despite these shortcomings, we can use a convenient technique for deduction. By trying to develop a general rule (or more often a set of general rules), we can start to solve any problem.

However, we may face an even more complex situation that bears little resemblance to known rules. We may have to work with a number of examples of tasks that have been resolved correctly. The correct solution may come from observing the behavior of a system whose properties we want to model, although we do not know the internal structure of the system or why it works.

At first you might think that such situations do not exist. You are wrong! The human brain regularly solves such tasks, for example, tasks involving perception. A simple example might be recognizing people based on appearance—a fairly easy task for a human and a very difficult one for a computer.

The required approach to problem solving in this case is called induction and we know of no general rules that apply to this specific detailed problem. Instead, however, we have a handful of examples of specific tasks that are known to have correct solutions. The human mind is capable of analyzing generalized examples and drawing conclusions from them. Consider the example of a rat in a maze. Its brain is capable of learning the rules for finding the right way around the maze. The rat manages to learn how to navigate the maze by analyzing the degrees of similarity in the situations it encounters. In essence, the rat is using a neural network.

This book will give you the skills to explore the properties of neural networks. A series of highly specific programs form integral parts of this book. Following the procedures and using the programs will allow you to learn how to work with neural networks and evaluate your progress. You can download the programs legally (and for free!) from http://home.agh.edu.pl/~tad//index.php?page=programy&lang=en

More information about how these programs work and how to download them and use them appear in Chapter 4. The aforementioned website also contains complete source codes for all the applications along with a set of tools that will you to view and analyze these codes. This book and the website can be used in three ways:

If you are interested only in neural network information and do not want to “play” with software, you can simply read the book.

If you are curious about the workings of neural networks, you can download a ready-to-use program that will allow you to build and test neural networks on your computer. This way you can connect theory from the book with practice by using our programs and learn how to apply the methods.

If you are in the learning states and are passionate about programming, you will be able to see exactly how our programs were built, change them, and even improve them. We have nothing to hide!