How to set up a Neural Network for Handwriting/Character Recognition using Back-Propagation Algorithm – Text with source code
There are many algorithms and ways to accomplish the recognition of handwritten character. Although none of them can claim to provide cent percent accuracy in character recognition, many of the algorithms can provide good results. Computer engineers and mathematicians are still working to find a perfect solution for the problem.
In this article, Handwritten character recognition using artificial neural networks is discussed. The article is written in a manner to enable readers to write programs using artificial neural networks instantly. Even though the article emphasizes character recognition, the ideas can be applied to other problems also.
Introduction to Artificial Neural Networks (ANNs)
We begin with some questions posed by beginners in the field, and the answers to those questions.
What is an artificial neural network?
An artificial neural network is a software or hardware that tries to simulate the working of the human brain. An artificial neural network is an interconnected network of many artificial neurons. These artificial neurons are objects used to simulate the neurons in the human brain.
What are artificial neural networks used for?
Artificial neural networks are used widely in machine learning applications. They are usually used for classification problems or for problems that require some type of cognitive ability. Some examples are: Optical Character Recognition (OCR), Speech Synthesis, Data Classification etc. Neural Networks are used for many more exciting applications and it is an active research field.
How does a neural network work?
It is very important to understand how a neural network works. If we understand the basics correctly, it is easy to implement a working neural network. Here is how our character recognition neural network work:
The image pattern containing a single character is given to the neural network as the input. This input is received by the input layer of the network. Each neuron in the input layer processes its inputs and produces one output each. The outputs from the neurons in the input layer becomes the inputs to the next layer in the neural network. So this process continues till the output from the output layer is obtained. This output is expected to be the character contained in the image.
When running a neural network for the first time, the output to an arbitrary input may not be correct. There will be errors in the output. That is, the neural network may recognize a character incorrectly. So to increase the accuracy of recognition, we must train a neural network before actual use.
What is meant by ‘Training a neural network’?
A neural network before training can be thought of as the brain of a child who does not know to read alphabets. So to teach the child, we show him some alphabets and tell him what they are. We do this again and again to increase the ability of the child to read the alphabets. Likewise, an untrained neural network does not know how to recognize characters. If we give it an input, it most probably will give the wrong output. So we train the neural network with patterns which we would like it to recognize. A neural network learns from this training and becomes good in recognition.
What is the algorithm used for handwriting recognition?
Many beginners have the misconception that there should be some algorithm to recognize a character in an OCR program using neural networks. Actually, using a neural network saves us from having to find out an algorithm for recognition. Thus there is no need for any complicated recognition algorithm if we are using neural networks. A neural network works by itself to help us recognize the character. We just have to implement the neural network and train it properly. This is why neural networks are considered as demonstrating traits of real Artificial Intelligence (AI).
Note that there may be different algorithms or methods to implement a neural network. These are usually simple algorithms that are part of the neural network and they are not directly related to the problem in hand. For example, the type of neural network we use in this article is called BackPropagation Neural Network.
First let us look how the human neurons work. This will help us in the study of artificial neural networks. Human neurons are biological structures that receive signals from other neurons and pass them to some other neurons.
The signals that dendrites receive are passed to the other neurons through the axon. But it is not actually a simple transfer of signals. There is some processing of signals involved before passing it to other neurons. A neuron collects the signals through the dendrites and transfers the signal only if the signals received add up to a specific threshold. Thus noise and other unwanted signals are removed.
The artificial neuron is designed to simulate the working of a human neuron in mind. Let us discuss the details of an artificial neural network. After that we will discuss how to implement this neural networks in actual program.
The artificial neural network will consist of artificial neurons. Artificial neurons are thus the building blocks of any neural network. We can design a neuron in many ways. We will discuss only the type of neuron and neural network which we need for our handwriting recognition software.
An artificial neuron will have many inputs and one output. Each input will be given different priorities. This priority is denoted by weights. Look at the figure below for more details:
In the figure, Xi denotes the ith input to the neuron. Wi denotes the weight of the ith input. In effect the value of the ith input will become: Wi * Xi
So, each neuron receives many inputs and gives out a single output. The output must reflect the values of the input. So we add all the inputs (Wi * Xi) and then if the total value of the inputs is more than a threshold, the output is fired. So the output will be 0 (if sum(Wi * Xi) < Threshold) or 1 (if sum(Wi * Xi) > Threshold) if we are using this policy. The weights are part of a neuron. They are property of a neuron and not a property of the input values. So whatever Xi arrives at an input of a neuron, the Xi is multiplied by the corresponding Wi at that input node.
Now according to our definition, the output of a neuron will be either 0 or 1. But this is not very good for practical implementations. WE would like to have an output that
The rest of the article coming soon…Check back after some time.