What is Artificial Neural Network (ANN)?

[Youtube: https://youtu.be/LnsVmZQ2IL8]

Artificial Neural Network (ANN) model is commonly used in machine learning as a way of learning approach. The reason why it is called neural networks NN is because it refers to the principle of biological neural networks, as shown in Figure 1. One of the key features of biological neural network is the synapse, which is the connection between neurons. Dendrites transmit input signals to the neuron and an axon transmits the output signal over the synapses towards the dendrites of other neurons. the connection is controlled by the strength or amplitude of a connection between both nodes.

Figure 1 Biological neural network

What is an artificial neural network?

Walczak & Cerpa (2003) explain ANN as layers of neurodes as follows:

“ANN models simulate the electrical activity of the brain and nervous system. Processing elements (neurode) are connected to other processing elements. Typically the neurodes are arranged in a layer, with the output of one layer serving as the input to the next layer and possibly other layers. A neurode may be connected to all or a subset of the neurodes in the subsequent layer, with these connections simulating the synaptic connections of the brain. Weighted data signals entering a neurode simulate the electrical excitation of a nerve cell and consequently the transference of information within the network or brain. The input values to a processing element, in, are multiplied by a connection weight, wn,m, that simulates the strengthening of neural pathways in the brain. It is through the adjustment of the connection strengths or weights that learning is emulated in ANNs.”

Hence, usually an ANN is divided into Input Layer, Hidden Layer, and Output Layer, as shown in Figure 2. First, assuming that there are only three input parameters for this prediction, instead of providing a decision instruction to the machine, a large number of cases (data) is inputted to train the machine. The machine guesses the answers by imposing different weights on the parameters, and the answers are compared with the actual correct answers provided in the training dataset. The ANN provides a feedback loop to adjust the weights (i.e. learning) to improve the accuracy of the predictions.

For example, when a set of parameters is input for a training session (the arrow on the left input layer), the machine adjusts the weights (0.7, 0.6, 1.4) for them, then sums them up and adds a bias value, and then tries to activate through a synapse (activate). If the difference between the obtained value and the actual value (answer) is small enough, then the output can pass through as 1. A feedback loop is provided to adjust the weights until the predictions can be sufficiently accurate.

Figure 2 an Illustration of an Artificial neural network concept

Synaptic Activation

Learning from mistakes is the learning process of human beings, and the same is true for machine learning (Yiu & Yau, 2006). Machine learning decisions are bound to make mistakes, and data analysis must have errors. The predictions of an ANN rely on a set of Activation Functions to decide go (1) or no-go (0). Commonly used activation functions, such as a S-function or a tanh function (Figure 3), decide a passing threshold.

Figure 3a Sigmoid Function

Figure 3b Tanh Function

One of the key points of machine learning is the training and testing processes! A training process is to compare the predictions of the machine with the correct answer provided to see how accurate the predictions of the machine is. It works like training kids to learn, which requires an open atmosphere with an honest and reliable coach to encourage children to make mistakes in a safe place!

An Example: machine learns to estimate property prices

Taking the property price appraisals as an example, suppose a machine learning model estimates property price based on three parameters of a residential unit (other parameters are kept the same), namely the number of rooms (BR), the year of completion (YEAR BUILT) and the floor area (GFA), some training data are as shown in Figure 4.

Figure 4 Training data for property price estimation

The parameters of different residential units are different. For example, the first unit is a one-bedroom unit with an area of ​​23 square meters. It was built in 1990. The price is $205,500. If the machine puts the weights 0.7, 0.6 and 1.4 to estimate the price, it will get $2800.5, which is far from the correct answer $205,500. The error is fed back to adjust the weights to improve the estimate. After many rounds of training sessions, the machine can reach a predetermined accuracy in the data of the training group ​​(Figure 6).

Now it can be tested using the testing dataset, as shown in the last row in Figure 5. What is the selling price of a 2-bedroom, 50-square-meter unit built in 2000, how much do you estimate it to be?

Figure 6. Weighted values ​​and hidden layer values ​​after training.

For more details of the calculations of an ANN using Excel, readers are recommended to watch the following youtube: Machine Learning MNIST using a Neural Network in Excel, https://www.youtube.com/watch?v=kCL065_0zTY&t=3224s

There are also some youtubes that provide more ANN examples, such as the following Youtube, which introduces how to use ANN to train a machine to judge handwritten 0, 1, 2, …, 9, but it is only available in English: 3Blue1Brown, But what is a neural network? | Chapter 1, Deep learning.

References:
Walczak, S. & Cerpa, N. (2003) I.A Biological Basis of Artificial Neural Networks, in Encyclopedia of Physical Science and Technology (Third Edition). https://www.sciencedirect.com/topics/computer-science/artificial-neural-network

Yiu, C.Y. and Yau, Y. (2006) A learning model of intelligent home, Facilities, Vol. 24 №9/10, pp. 365–375. https://doi.org/10.1108/02632770610677646