# Granger Causality in Time Series – Explained using Chicken and Egg problem

**Introduction**

The purpose of this article is to understand what is granger causality and its application in Time series forecasting for better prediction.

Image Source: https://www.wallpaperflare.com

**Overview of Time series:**

Time series is a set of observations on the values that the variable takes at different times. The time interval at which the data is collected may be hourly, daily, weekly, monthly, quarterly, annually, etc.

Time series data can be categorized as follows:

- Univariate time series – Consists of a single variable that changes over time. Eg Account balance over a span of ten years.
- Multivariate time series – Consists of more than one variable which varies over time and there may be dependency among the variables. Eg Different expenditures of a family over a span of ten years.

Time series analysis provides insight into the pattern or characteristics of time series data. Time series data can be decomposed into three components:

- Trend – This shows the tendency of the data over a long period of time, it may be upward, downward, or stable.
- Seasonality – It is the variation that occurs in a periodic manner and repeats each year.
- Noise or Random – Fluctuations in the data that are erratic.

Image Source: https://www.researchgate.net

Forecasting the data for a particular time in the future is known as Time Series Forecasting. It is one of the powerful machine learning models widely used in the fields like finance, weather forecasting, the health sector, environmental studies, business, retail, etc to arrive at strategic decisions.

In real-time, most of the data consists of multiple variables, in which independent variables might depend on other independent variables, these relations will have an impact on the predictions or forecasting. Most of the time people generally get misled and build multilinear regression models in such cases. A High R square value will further mislead and make a poor prediction.

**Spurious Regression**

Linear regression might indicate a strong relationship between two or more variables, but these variables may be totally unrelated in reality. Predictions fail when it comes to domain knowledge, this scenario is known as spurious regression.

There is a strong relationship between chicken consumption and crude oil exports in the below graph even though they are unrelated.

Source: http://www.tylervigen.com/spurious-correlations

Strong trend / nonstationary and higher R square are observed in spurious regression. Spurious regression has to be eliminated while building the model since they are unrelated and have no causal relationship.

Multilinear regression makes use of a correlation matrix to check the dependency between all the independent variables. If the correlation coefficient value is high between two variables any one variable is retained and another one is discarded to remove the dependency. In the dataset when time is a confounding factor multilinear regression fails, the correlation coefficient which is used to eliminate the variable is not time-bound instead it just gives the correlation between the two variables.

Consider the above time-series graph, variable X has a direct influence on variable Y but there is a lag of 5 between X and Y in which case we cant use the correlation matrix. For eg. an increase in coronavirus positive cases in the city and an increase in the number of people getting hospitalized. For better forecasting, here we would like to know if there is a causal relationship.

**Granger Causality comes to Rescue**

Prof. Clive W.J. Granger, recipient of the 2003 Nobel Prize in Economics developed the concept of causality to improve the performance of forecasting.

It is basically an econometric hypothetical test for verifying the usage of one variable in forecasting another in multivariate time series data with a particular lag.

A prerequisite for performing the Granger Causality test is that the data need to be stationary i.e it should have a constant mean, constant variance, and no seasonal component. Transform the non-stationary data to stationary data by differencing it, either first-order or second-order differencing. Do not proceed with the Granger causality test if the data is not stationary after second-order differencing.

Let us consider three variables *X** _{t}* ,

*Y*

*, and*

_{t}*W*

_{t }preset in time series data.Case 1: Forecast X

_{t+1}based on past values X_{t }.Case 2: Forecast X

_{t+1}based on past values X_{t}and Y_{t}._{ }Case3 : Forecast X

_{t+1}based on past values X_{t}, Y_{t}, and W_{t}, where variable Y_{t }has direct dependency on variable W_{t}.

Here Case 1 is univariate time series also known as the autoregressive model in which there is a single variable and forecasting is done based on the same variable lagged by say order p.

**The equation for the Auto-regressive model of order p (RESTRICTED MODEL, RM)
**

X

_{t}= α + 𝛾_{1}X_{𝑡−1}+ 𝛾_{2}X_{𝑡−2}+ ⋯ + 𝛾_{𝑝}X_{𝑡−𝑝 }where p parameters (degrees of freedom) to be estimated.

In Case 2 the past values of Y contain information for forecasting X_{t+1}. Y_{t} is said to “Granger cause” X_{t+1} provided Y_{t} occurs before X_{t+1 }and it contains data for forecasting X_{t+1}.

**Equation using a predictor Y _{t} (UNRESTRICTED MODEL, UM)**

X

_{t}= α + 𝛾_{1}X_{𝑡−1}+ 𝛾_{2}X_{𝑡−2}+ ⋯ + 𝛾_{𝑝}X_{𝑡−𝑝 }+ α_{1}Y_{t-1}+ ⋯ + α_{𝑝 }Y_{t-p}

2p parameters (degrees of freedom) to be estimated.

If Y* _{t}* causes X

*, then Y must precede X which implies:*

_{t}

- Lagged values of Y should be significantly related to X.
- Lagged values of X should not be significantly related to Y.

Case 3 can not be used to find Granger causality since variable Y_{t} is influenced by variable W_{t}.

**Hypothesis test**

Null Hypothesis (H_{0}) : *Y** _{t}* does not “Granger cause”

*X*

_{t+1}i.e., 𝛼

_{1}= 𝛼

_{2}= ⋯ = 𝛼

_{𝑝}= 0

Alternate Hypothesis(H_{A}): *Y** _{t}* does “Granger cause”

*X*

_{t+1}, i.e., at least one of the lags of Y is significant.

**Calculate the f-statistic **

F

_{p,n-2𝑝−1}= (𝐸𝑠𝑡𝑖𝑚𝑎𝑡𝑒 𝑜𝑓 𝐸𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒) / (𝐸𝑠𝑡𝑖𝑚𝑎𝑡𝑒 𝑜𝑓 𝑈𝑛𝑒𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒)F

_{p,n-2𝑝−1}= ( (𝑆𝑆𝐸_{𝑅𝑀}−𝑆𝑆𝐸_{𝑈𝑀}) /𝑝) /(𝑆𝑆𝐸_{𝑈𝑀}/𝑛−2𝑝−1)

where n is the number of observations and

SSE is Sum of Squared Errors.

If the p-values are less than a significance level (0.05) for at least one of the lags then reject the null hypothesis.

Perform test for both the direction X_{t}->Y_{t} and Y_{t}->X_{t}.

Try different lags (p). The optimal lag can be determined using AIC.

**Limitation**

- Granger causality does not provide any insight on the relationship between the variable hence it is not true causality unlike ’cause and effect’ analysis.
- Granger causality fails to forecast when there is an interdependency between two or more variables (as stated in Case 3).
- Granger causality test can’t be performed on non-stationary data.

**Resolving Chicken and Egg problem**

Let us apply Granger causality to check whether the egg came first or chicken came first.

#### Importing libraries

import matplotlib.pyplot as plt import seaborn as sns import numpy as np import pandas as pd

#### Loading Dataset

Data is from the U.S. Department of Agriculture. It consists of two-time series variables from 1930 to 1983, one of U.S. egg production and the other the estimated U.S. chicken population.

df = pd.read_csv('chickegg.csv')

#### Exploring the Dataset

df.head()

df.dtypes

df.shape

(53, 3)

df.describe()

Check if the data is stationary, if not make it stationary to proceed.

# Draw Plot def plot_df(df, x, y, title="", xlabel='Date', ylabel='Value', dpi=100): plt.figure(figsize=(16,5), dpi=dpi) plt.plot(x, y, color='tab:red') plt.gca().set(title=title, xlabel=xlabel, ylabel=ylabel) plt.show() plot_df(df, x=df.Year, y=df.chicken, title='Polulation of the chicken across US') plot_df(df, x=df.Year, y=df.egg, title='Egg Produciton')

By visual inspection, both the chicken and egg data are not stationary. Let us confirm this by running Augmented Dickey-Fuller Test (ADF Test).

**Augmented Dickey-Fuller Test (ADF test)**

ADF test is a popular statistical test for checking whether the Time Series is stationary or not which works based on the unit root test. The number of unit roots present in the series indicates the number of differencing operations that are required to make it stationary

Consider the hypothesis test where:

Null Hypothesis (H_{0}): Series has a unit root and is non-stationary.

Alternative Hypothesis (H_{A}): Series has no unit root and is stationary.

from statsmodels.tsa.stattools import adfuller result = adfuller(df['chicken']) print(f'Test Statistics: {result[0]}') print(f'p-value: {result[1]}') print(f'critical_values: {result[4]}')

if result[1] > 0.05: print("Series is not stationary") else: print("Series is stationary")

result = adfuller(df['egg'])

print(f'Test Statistics: {result[0]}') print(f'p-value: {result[1]}') print(f'critical_values: {result[4]}') if result[1] > 0.05: print("Series is not stationary") else: print("Series is stationary")

p-values of both the egg and chicken variables are greater than the significant value (0.05), the Null hypothesis is valid and the series is not stationary.

## Data Transformation

Granger causality test is carried out only on stationary data hence we need to transform the data by differencing it to make it stationary. Let us perform the first-order differencing on chicken and egg data.

df_transformed = df.diff().dropna() df = df.iloc[1:] print(df.shape) df_transformed.shape

df_transformed.head()

plot_df(df_transformed, x=df.Year, y=df_transformed.chicken, title='Polulation of the chicken across US') plot_df(df_transformed, x=df.Year, y=df_transformed.egg, title='Egg Produciton')

result = adfuller(df_transformed['chicken']) print(f'Test Statistics: {result[0]}') print(f'p-value: {result[1]}') print(f'critical_values: {result[4]}') if result[1] > 0.05: print("Series is not stationary") else: print("Series is stationary")

result = adfuller(df_transformed['egg']) print(f'Test Statistics: {result[0]}') print(f'p-value: {result[1]}') print(f'critical_values: {result[4]}') if result[1] > 0.05: print("Series is not stationary") else: print("Series is stationary")

Transformed chicken and egg data are stationary, hence there is no need to go for second-order differencing.** **

**Test the Granger Causality**

There are several ways to find the optimal lag but for simplicity let’s consider 4th lag as of now.

Do eggs granger cause chickens?

Null Hypothesis (H_{0}) : eggs do not granger cause chicken.

Alternative Hypothesis (H_{A}) : eggs granger cause chicken.

from statsmodels.tsa.stattools import grangercausalitytests

grangercausalitytests(df_transformed[['chicken', 'egg']], maxlag=4)

p-value is very low, Null hypothesis is rejected hence eggs are granger causing chicken.

That implies eggs came first.

Now repeat the Granger causality test in the opposite direction.

Do chickens granger cause eggs at lag 4?

Null Hypothesis (H_{0}) : chicken does not granger cause eggs.

Alternative Hypothesis (H_{A}) : chicken granger causes eggs.

grangercausalitytests(df_transformed[['egg', 'chicken']], maxlag=4)

The p-value is considerably high thus chickens do not granger cause eggs.

The above analysis concludes that the egg came first and not the chicken.

Once the analysis is done the next step is to begin forecasting using time series forecasting models.

**EndNote**

*Whenever you come across time-bound data having multiple variables be suspicious about high R ^{2} and possible spurious regression. Make use of the time series forecasting for better performance. Check for bidirectional Granger causality between each variable and eliminate the variable based on test results before proceeding with forecasting techniques. I hope you enjoyed the article and gained insight into the Granger causality concept. Please drop your suggestions or queries in the comment section.*

*Thank you for reading!*

### About Author

Hello, Pallavi Padav from Mangalore holding PG degree in Data Science from INSOFE. Passionate about participating in Data Science hackathons, blogathons and workshops.

Would love to catch you on Linkedin. Mail me here for any queries.