There are many algorithms and ways to accomplish the recognition of handwritten character. Although none of them can claim to provide cent percent accuracy in character recognition, many of the algorithms can provide good results. Computer engineers and mathematicians are still working to find a perfect solution for the problem.
In this article, Handwritten character recognition using artificial neural networks is discussed. The article is written in a manner to enable readers to write programs using artificial neural networks instantly. Even though the article emphasizes character recognition, the ideas can be applied to other problems also.
The human brain is able to solve many cognitive and classification problems that are hard for even the most powerful supercomputers. We now know that the human brain is nothing but a large network of cells called neurons. The idea of artificial neural network emerged from the efforts to simulate the working of the human brain.
Human Neurons
First let us look how the human neurons work. This will help us in the study of artificial neural networks. Human neurons are biological structures that receive signals from other neurons and pass them to some other neurons.
The signals that dendrites receive are passed to the other neurons through the axon. But it is not actually a simple transfer of signals. There is some processing of signals involved before passing it to other neurons. A neuron collects the signals through the dendrites and transfers the signal only if the signals received add up to a specific threshold (If the signals are strong enough). Thus noise and other unwanted signals are removed.
The connector between dendrites of a neuron and the axon of another neuron is called the synaptic connector. When the electrical signals from the dendrites are strong enough, the neuron activates and fires a signal through its axon.
The most interesting thing to note here is that the synaptic connections and the 'threshold' amount of signals needed to activate the neuron changes every time. This helps the neuron to 'learn'. It learns when to fire signal by changing its synaptic connections and the threshold.
Introduction to Artificial Neural Networks (ANNs)
We begin with some questions posed by beginners in the field, and the answers to those questions.
What is an artificial neural network?
An artificial neural network is a software or hardware that tries to simulate the working of the human brain. An artificial neural network is an interconnected network of many artificial neurons. These artificial neurons are objects used to simulate the neurons in the human brain.
What are artificial neural networks used for?
Artificial neural networks are used widely in machine learning applications. They are usually used for classification problems or for problems that require some type of cognitive ability. Some examples are: Optical Character Recognition (OCR), Speech Synthesis, Data Classification etc. Neural Networks are used for many more exciting applications and it is an active research field.
How does a neural network work?
It is very important to understand how a neural network works. If we understand the basics correctly, it is easy to implement a working neural network. Here is how our character recognition neural network work:
The image pattern containing a single character is given to the neural network as the input. This input is received by the input layer of the network. Each neuron in the input layer processes its inputs and produces one output each. The outputs from the neurons in the input layer becomes the inputs to the next layer in the neural network. So this process continues till the output from the output layer is obtained. This output is expected to be the character contained in the image.
When running a neural network for the first time, the output to an arbitrary input may not be correct. There will be errors in the output. That is, the neural network may recognize a character incorrectly. So to increase the accuracy of recognition, we must train a neural network before actual use.
What is meant by 'Training a neural network'?
A neural network before training can be thought of as the brain of a child who does not know to read alphabets. So to teach the child, we show him some alphabets and tell him what they are. We do this again and again to increase the ability of the child to read the alphabets. Likewise, an untrained neural network does not know how to recognize characters. If we give it an input, it most probably will give the wrong output. So we train the neural network with patterns which we would like it to recognize. A neural network learns from this training and becomes good in recognition.
What is the algorithm used for handwriting recognition?
Many beginners have the misconception that there should be some algorithm to recognize a character in an OCR program using neural networks. Actually, using a neural network saves us from having to find out an algorithm for recognition. Thus there is no need for any complicated recognition algorithm if we are using neural networks. A neural network works by itself to help us recognize the character. We just have to implement the neural network and train it properly. This is why neural networks are considered as demonstrating traits of real Artificial Intelligence (AI).
Note that there may be different algorithms or methods to implement a neural network. These are usually simple algorithms that are part of the neural network and they are not directly related to the problem in hand. For example, the type of neural network we use in this article is called Multilayer Feedforward Neural Network. The algorithm which we are going to use for training the neural net is called the BackPropagation algorithm.
Artificial Neuron
The artificial neuron is designed to simulate the working of a human neuron in mind. Let us discuss the details of an artificial neural network. After that we will discuss how to implement this neural networks in actual program.
The artificial neural network will consist of artificial neurons. Artificial neurons are thus the building blocks of any neural network. We can design a neuron in many ways. We will discuss only the type of neuron and neural network which we need for our handwriting recognition software.
An artificial neuron will have many inputs and one output. Each input will be given different priorities. This priority is denoted by weights. Look at the figure below for more details:
In the figure, Xi denotes the ith input to the neuron. Wi denotes the weight of the ith input. In effect the value of the ith input will become: Wi * Xi
The weights are part of a neuron. They are property of a neuron and not a property of the input values. So whatever Xi arrives at an input of a neuron, the Xi is multiplied by the corresponding Wi at that input node.
Each neuron receives many inputs and gives out a single output. The output must reflect the values of the input. So we add all the inputs (Wi * Xi) and then if the total value of the inputs is more than a threshold, the output is fired. This function is called the activation function, since the value of this function decides whether the output is activated or not. So the output will be 0 (if sum(Wi * Xi) < Threshold) or 1 (if sum(Wi * Xi) > Threshold) if we are using this activation function. Since we are comparing the sum of signals to a threshold, this activation function is called the threshold activation function.
Now according to our definition, the output of a neuron will be either 0 or 1. But this is not very good for practical implementations. We would like to have an output that can vary its value continuously between 0 and 1 instead of an abrupt jump from 0 to 1. This is why we use a squeezing function or sigmoid function as the activation function rather than the threshold activation function.
Next Page >> The sigmoid function (squeezing function)