Although there are many different neural networks for classification, clustering, and modeling, the most popular one is by far the Multi-Layer-Perceptron (MLP) for classification using the ``backpropagation'' learning algorithm. In this short introduction I will concentrate my attention on history and ideas of this method.
In general the concept of ``Neural Networks'' just means using the metaphor of interacting neurons. Each neuron is a relatively simple structure, computing some kind of function from inputs and delivering the result as an output. In a network several neurons are connected, each one using outputs of other neurons for its inputs. Normally there are also some general inputs from outside of the NN-system and outputs by some neurons are also used as general outputs. In the example of classification the observed data may be the input to the system and the classification-category the output.
The metaphor and idea about Neural Networks was introduced 1943 by the neurophysiologist Warren McCulloch and the logician Walter Pitts in the connection with brain research [33]. The brain is viewed as consisting of billions of interacting neurons. General inputs to the brain are delivered by the senses: seeing, hearing, feeling, etc. Actions and Decisions then can be seen as outputs of the neural network ``brain''.
All the interactions between neurons can make a neural network a fairly complex system. The advantage of this is that NN are known to be ``universal classifiers''. This means that in theory they can approximate every classification function as closely as required. But this complexity has also a significant disadvantage. Using NN for classification it is often not comprehensible how the network came up with its decision, or what the important evaluated classification criteria are. For most business decisions this is unacceptable. Often the only way to get a ``feeling'' for the classification process lies in doing a sensitivity analysis.
For practical computing purposes the connectivity of neurons is usually restricted. For avoiding ``circles of dependencies'' (feedbacks) where the output from one neuron goes through several other neurons but then ends up again as an input for itself, neurons are organized in ``layers''. The outputs of neurons in one layer are only used as inputs for the following layer (also called feedforward).