Introduction to Artificial Neural Networks

“Artificial Neural Networks (ANNs)” is a collective name for a number of methods (computer algorithms) that can be used for a variety of tasks, such as:

  • Pattern recognition (e.g. handwritten character recognition, medical image classification and classification of pork meat)
  • Time series prediction (e.g. energy consumption prediction for buildings and predictions in the stock market)
  • Function approximation
  • Combinatorial optimization

The name suggests that the ANNs technology has something to do with neural networks (i.e. the human brain). It is by no means a simulation of the brain, however. Artificial Neural Networks is based on the way neurons in the brain are organized and on the idea of connecting small computational units together to form large, complex networks. From a mathematical point of view, ANNs is yet another statistical analysis method.

Let’s describe an ANN in the context of medical image classification. Suppose we have a medical image that we want to be classified as either healthy or not healthy. We can feed the image to a trained ANN that will output one of two possible answers, “healthy” or “not healthy”.



 

This is the end-user picture of how an ANN is used. Below we highlight three questions:

1. What is meant by a trained ANN and how do you train an ANN?
2. What is in the black box?
3. How do you feed images into the ANN? Let’s start with the second question. An ANN can be visualized in the following way: 

This network has four input nodes (represented by the squares) containing input numbers. The three nodes in the middle (circles) are called “hidden nodes” and each of them receives a weighted sum of all inputs. The weights are represented by the arrows between the layers. Each hidden node outputs a number (in relation to its weighted sum on the input side) and these are in turn used to create a weighted sum that feeds the single output node. The output value from the output node is to be regarded as the output from the ANN.

In summary, numbers on the input side give a new number on the output side. The output value depends on the values of all the weights in the ANN. Usually the output number is between 0 and 1, where 1 means “not healthy” and 0 means “healthy”, and can be regarded as a probability.

With regard to Question 1, What is meant by a trained ANN and how do you train an ANN?, conceptually the training is simple. If the values of all the weights in the network are adjusted, when inputting a “healthy image” the ANN outputs 0, while for the “not healthy image” it shows 1. To accomplish this, trained data are needed that consist of many images with true status (either healthy or not healthy). The training process can be pictured as follows:



 



When the adjustment loop is complete we evaluate how well the trained ANN is performing. This evaluation is performed on another dataset, which is not part of the training data.

Question 3, How do we feed images into the input side of the ANN?, is simple. We know that the ANN wants plain numbers (not jpeg images). The idea is to represent each image by a set of characteristic numbers. These numbers should of course be such that they have something to do with that status of the image, i.e. representing “healthy” or “not healthy”. The process of finding these numbers can be called feature selection and is usually complicated. A great deal of knowledge has to go into this feature selection process. Good features give good ANN classifiers.

So back to question 2. What is in the black box? Usually we do not train only one ANN, but rather, an ensemble of single ANNs, which collectively produce and output. From the point of view of the end-user, it is not really important what is in the black box, as long as its performance is good and reliable.