Skip to main content

Riverflow forecasting

River flow prediction from precipitation using LSTMs or GRU**

The task is to predict the river flow by analyzing the precipitation. There is a causal relationship between the river flow and precipitation. A causal relationship meaning the river flow is being affected due to the cause of the precipitation. A causal relationship between two events exists if the occurrence of the first cause the other. Dependency on rainfall on the current day and previous days as well. Problems are related to grouping the data, group by hour, day, seconds, months, etc. There is a physics-based approach/representation where these relationships can be expressed in terms of equations and other formulae.

Timeseries in this case could be univariate or multivariate. There are various implementations of LSTMs, GRUs and XGBoost and SVMs.

Univariate: Only one variable is varying over time. For example, data collected from a sensor measuring the temperature of a room every second. Therefore, each second, you will only have a one-dimensional value, which is the temperature.

Multivariate: Multiple variables are varying over time. For example, a tri-axial accelerometer. There are three accelerations, one for each axis (x, y, z) and they vary simultaneously over time.

Here is a link. The link discusses important points about additive and multiplicative seasonality. Additive seasonality meaning, the data

Basic definitions and characteristics of a time series: Taken from this medium article here​

  1. Trend: The trend is showing the general tendency of the data to increase or decrease with time.
  2. Seasonality: Seasonality in a time series is a regular pattern of changes that repeats over S time periods, where S defines the number of periods until the pattern repeats.
  3. Stationary: Stationarity means that the statistical properties of a time series which are mean, variance and covariance do not change over time.
  4. White noise: Simply the “white” means all frequencies are equally represented and “noise” is because there’s no pattern, just random variation. A time series is a white noise if it distributed with a mean of zero, constant variance and a zero correlation between lags. Gaussian white noise, Binary white noise and Sinusoidal white noise are examples for white noise.

Time series forecasting​

A wonderful library is made by AWS labs for time series forecasting. The link is over here. The library is built on top of mxnet. There is an important issue to be discussed with all of this. When should one use LSTMs or Bi-LSTMs or GRUs? Reference link.

GRUs have a simpler design than LSTMs. GRU has 2 gates, while LSTMs have 3 gates. Due to the complex design of an LSTM, it takes longer to train an LSTM than GRU. There is no specific answer to where should one use any of the models, their application changes from problem to problem. The best is to try to use both the models, compare their training times and their accuracy.

Splitting data into train/valid/test​

Splitting Time Series Data into Train/Test/Validation Sets - Cross Validated (stackexchange.com)

Training LSTM: chooses tanh as an activation function. Reference link. Here is the notebook for trying a Keras implementation in Colab. It is done on a univariate time series.

GRUs for time series forecasting​

Here is a link for a Kaggle notebook.

It stands for eXtreme gradient boost. It uses regularization: ridge regression. The entire playlist for XGboost, gradient boosting and regularization is available over here. The entire playlist is approximately 11 hours, and it goes over everything. The description in the YouTube videos, includes links to original research papers and supplementary reading material.

Need to check implementations of time series forecasting in Keras; is the implementation for univariate or multivariate time series.

A wonderful library maintained by a student of fastai over here. The installation and usage instructions are available over here.

Multi-step time series forecasting:

In time series forecasting, value is usually predicted for a single step in the future. With multi-step forecasting. There are multiple approaches to solve the multi-step forecasting problem. They are:

  1. Direct Multi-step Forecast Strategy.
  2. Recursive Multi-step Forecast Strategy.
  3. Direct-Recursive Hybrid Multi-Step Forecast Strategies.
  4. Multiple Output Forecast Strategy.

Reference links:

  1. Machine Learning Strategies for Time Series Forecasting | SpringerLink, rectify.pdf (robjhyndman.com).
  2. These links were taken from this source
  3. Multi-step forecasting is also achieved in

An implementation of multivariate multi step forecasting in done using linear models over here. This is the outline of all the steps that are done in the notebook. The dataset used was of Household electricity consumption.

  1. Data cleaning:

    1. Filling missing values: There is a lot of work done on this. The most easy and convenient way to handle missing values is using backward fill. Meaning, if there is a missing value for a particular hour in a day, then you would put the value from the same hour of the previous day. Other methods include, replacing with mean, median and mode. Here is a Kaggle notebook which explains these things in detail.

  2. Data preprocessing: If one has huge time series data, say which is varying minute by minute over a course of multiple years, the size of the data becomes an issue to deal with. So, you would need to resample the data based on hours, days, months and years depending on the size of the original data.

Multivariate multi-step prediction using LSTMs​

  • Overview of time series forecasting models: link
  • Definitions for terms used in time series analysis: A very nice medium article summarizing the important concepts in a few words. Link
  • Removing seasonality from time series: link

Things to write about​

  1. Time series association: with different variables (using association mining rules) Time delay
  2. Discovering phase shift through association
  3. Math behind XGBoost with forecasting – working – internal working
  4. Math behind LSTM with forecasting
  5. Multi step time forecasting
  6. Converting the time series into a supervised learning problem