Recurrent Neural Networks

Introduction

Recurrent Neural Networks (RNNs) are designed to handle sequential and time-series data, allowing models to learn from previous steps while processing sequences.

They are widely used in:

✅ Natural Language Processing (NLP).
✅ Time-series forecasting.
✅ Speech recognition and audio analysis.

Why RNNs?

Unlike feedforward networks, RNNs maintain hidden states that capture previous information, enabling the model to learn temporal dependencies and patterns in sequences.

Structure of an RNN Cell

An RNN cell takes:

• Current input (x_t)
• Previous hidden state (h_t-1)

and computes:

• New hidden state (h_t) using an activation function.

Challenges with RNNs

• Vanishing and exploding gradients during training.
• Difficulty learning long-term dependencies.
• Computational inefficiency for long sequences.

Solutions:

✅ Use advanced variants like LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) to handle long-term dependencies effectively.

Building a Simple RNN with Keras

We will build an RNN for predicting sine wave values as a sequence modeling example.

1️⃣ Install and Import Libraries

pip install tensorflow

import numpy as np
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import SimpleRNN, Dense
import matplotlib.pyplot as plt

2️⃣ Prepare Sequence Data

# Generate sine wave data
x = np.linspace(0, 100, 1000)
y = np.sin(x)

# Prepare dataset
seq_length = 50
X, Y = [], []
for i in range(len(y) - seq_length):
    X.append(y[i:i+seq_length])
    Y.append(y[i+seq_length])

X = np.array(X)
Y = np.array(Y)

# Reshape for RNN [samples, timesteps, features]
X = X.reshape((X.shape[0], X.shape[1], 1))

3️⃣ Define the RNN Model

model = Sequential([
    SimpleRNN(50, activation='tanh', input_shape=(seq_length, 1)),
    Dense(1)
])

4️⃣ Compile and Train the Model

model.compile(optimizer='adam', loss='mse')
history = model.fit(X, Y, epochs=20, batch_size=32)

5️⃣ Make Predictions and Visualize

pred = model.predict(X)

plt.figure(figsize=(10,5))
plt.plot(Y, label='Actual')
plt.plot(pred, label='Predicted')
plt.legend()
plt.title('RNN Sine Wave Prediction')
plt.show()

Conclusion

✅ RNNs are effective for sequence modeling by maintaining memory across timesteps.
✅ They are foundational for NLP, time-series forecasting, and sequential data analysis.
✅ Keras simplifies building and training RNNs for experimentation.

What’s Next?

✅ Explore LSTM and GRU for better handling of long-term dependencies.
✅ Apply RNNs to NLP tasks like text generation.
✅ Learn about sequence-to-sequence models and attention mechanisms.

Join our SuperML Community to share your RNN projects and learn advanced deep learning together.

Happy Learning! ⏳