Building a Hybrid Oscillatory Hyperdimensional RNN for Time Series Forecasting

Foreword

In today’s era of deep learning, sequence modelling and time series forecasting remain at the forefront of research and industrial applications. In this article, I present a detailed walkthrough of a novel hybrid recurrent neural network (RNN) architecture I made that leverages oscillatory dynamics, hyperdimensional bundling and complex-valued operations for improved time series forecasting.

I focus on predicting sunspot numbers using a publicly available dataset, but the techniques discussed here can be generalised to many other applications. This article is intended for technical audiences and researchers who have a background in machine learning and deep learning.

I will cover the following sections in detail:

1. Mathematical Foundations — A deep dive into the theory behind oscillatory dynamics, hyperdimensional computing and complex-valued RNNs.
2. Code Breakdown — A line-by-line analysis of my implementation, including data pre-processing, custom RNN cell design and model training.
3. Use Cases — An exploration of where and how this hybrid architecture can be applied in real-world scenarios.
4. Benefits — A discussion on the advantages of combining these advanced techniques into one model.
5. Experimental Results — A comprehensive review of my training experiments, performance metrics and insights derived from the results.

Below, I embark on this technical journey to unravel the inner workings of my hybrid model and discuss how each component contributes to better time series forecasting.

Introduction

Recurrent Neural Networks (RNNs) have long been used to model sequential data due to their inherent ability to capture temporal dependencies. However, standard RNN architectures — such as vanilla RNNs or even LSTM and GRU networks — often struggle when dealing with complex periodic patterns or long-term dependencies. To overcome these challenges, my recent research has introduced alternative approaches that incorporate oscillatory dynamics, hyperdimensional representations and complex-valued computations.