Understand Recurrent Neural Networks with this Easy Guide

Suppose you are a chef and you want to decide every day what you cook based on what you cooked yesterday (of course, you don’t want to repeat) and how today’s weather is!

You decide to take the help of a neural network for everyday planning

Here’s a simple neural network with an input layer having nine inputs, a hidden layer and an output layer.

Image by Author

Notice a couple of things in this neural network:

• Here, we define the number of inputs beforehand, and this remains constant.
• Also, all inputs are fed to the network at the same time in the beginning.

How can we use this kind of architecture to decide the food to cook based on yesterday’s meal?

Introducing RNNs

For that, we need a memory of previous output, which is not present in neural networks. Also, we cannot provide input on today’s weather for each day as inputs are passed beforehand.

This is a simple problem where our Recurrent Neural Networks can come to the rescue. An RNN has a memory which takes decisions based on previous outputs and current input.

In the below diagram, at each time step, an input is given, an output is produced, and the same output is given to the same network for the next time step.

Image by Author

Thus, using the previous output and current input, an output is produced, and this process goes on.

The network at all steps is the same, i.e. the same activation function, bias and weights are used, and inputs are given one after another in a sequential manner.

So, for our chef example, say the first day the weather is hot, so you make an ice cream sundae, then the next day the weather is hot only, but the previous output is an ice cream sundae, so we cook a different dish, say salad.

Image by Author