In remembrance of Dr. Peter Kemp, Notary (ret.), Bonn, Germany.
A small preface "Originally, this work has been prepared in the framework of a seminar of the University of Bonn in Germany, but it has been and will be extended (after being presented and published online under www.dkriesel.com on 5/27/2005). First and foremost, to provide a comprehensive overview of the subject of neural networks and, second, just to acquire more and more knowledge about LATEX . And who knows – maybe one day this summary will become a real preface!" Abstract of this work, end of 2005 The above abstract has not yet become a preface but at least a little preface, ever since the extended text (then 40 pages long) has turned out to be a download hit. Ambition and intention of this manuscript The entire text is written and laid out more effectively and with more illustrations than before. I did all the illustrations myself, most of them directly in LATEX by using XYpic. They reflect what I would have liked to see when becoming acquainted with the subject: Text and illustrations should be memorable and easy to understand to offer as many people as possible access to the field of neural networks. Nevertheless, the mathematically and formally skilled readers will be able to understand the definitions without reading the running text, while the opposite holds for readers only interested in the subject matter; everything is explained in both colloquial and formal language. Please let me know if you find out that I have violated this principle. The sections of this text are mostly independent from each other The document itself is divided into different parts, which are again divided into chapters. Although the chapters contain cross-references, they are also individually acces- v