Vijay Kumar G — June 16, 2021
Beginner Data Science Libraries Programming Python Statistics Time Series Time Series Forecasting

This article was published as a part of the Data Science Blogathon

Introduction

In this article, I will be talking through the Augmented Dickey-Fuller test (ADF Test) and Kwiatkowski-Phillips-Schmidt-Shin test (KPSS test) that are the most common statistical tests used to test whether a given Time series is stationary or not. The 2 tests are the most commonly used statistical tests when it comes to analyzing the stationary of a series. Stationary is a very important factor in time series. In ARIMA time series forecasting, the first step is to determine the number of differencing required to make the series stationary because a model cannot forecast on non-stationary time series data. let’s try to understand a little bit in-depth.

What is a Stationary Series?

A Stationary series is one whose statistical properties like mean, variance, covariance do not vary with time or these stats properties are not the function of time.

In other words, stationarity in Time Series also means series without a Trend or Seasonal components.

Why should time series be stationary?

Stationary series is easier for statistical models to predict effectively and precisely.

Types of Stationary Series

  1. Strict Stationary – Satisfies the mathematical definition of a stationary process and mean, variance & covariance are not the function of time.
  2. Seasonal Stationary – Series exhibiting seasonality.
  3. Trend stationary – Series exhibiting trend.

Note: Once the seasonality and trend is removed, series will be strict stationary

How to check Stationarity?

  • Visualizations 

The most basic methods for stationarity detection rely on plotting the data, and visually checking for trend and seasonal components.

Trying to determine whether a time series was generated by a stationary process just by looking at its plot is a dubious task. However, there are some basic properties of non-stationary data that we can look for.

Let’s take an example the following nice plots from [Hyndman & Athanasopoulos, 2018]:

stationarity in Time Series examples

Figure 1: Nine examples of time series data; (a) Google stock price for 200 consecutive days; (b) Daily change in the Google stock price for 200 consecutive days; (c) Annual number of strikes in the US; (d) Monthly sales of new one-family houses sold in the US; (e) Annual price of a dozen eggs in the US (constant dollars); (f) Monthly total of pigs slaughtered in Victoria, Australia; (g) Annual total of lynx trapped in the McKenzie River district of north-west Canada; (h) Monthly Australian beer production; (i) Monthly Australian electricity production. [Hyndman & Athanasopoulos, 2018]

  • Seasonality can be observed in series (d), (h), and (i)
  • The trend can be observed in series (a), (c), (e), (f), and (i)
  • Series (b) and (g) are stationary
  • Statistical Tests

Two statistical tests which we will be discussing are

  1. Augmented Dickey-Fuller (ADF) Test
  2. Kwiatkowski-Phillips-Schmidt-Shin (KPSS) Test

Augmented Dickey-Fuller Test

Statistical tests make strong assumptions about your data. They can only be used to inform the degree to which a null hypothesis can be rejected or fail to be rejected. The result must be interpreted for a given problem to be meaningful.

However, they provide a quick check and confirmatory evidence that the time series is stationary or non-stationary.

The Augmented Dickey-Fuller test is a type of statistical test called a unit root test.

In probability theory and statistics, a unit root is a feature of some stochastic processes (such as random walks) that can cause problems in statistical inference involving time series models. In a simple term, the unit root is non-stationary but does not always have a trend component.

ADF test is conducted with the following assumptions.

Null Hypothesis (HO): Series is non-stationary or series has a unit root.

Alternate Hypothesis(HA): Series is stationary or series has no unit root.

If the null hypothesis is failed to be rejected, this test may provide evidence that the series is non-stationary.

Conditions to Reject Null Hypothesis(HO)

  • If Test statistic < Critical Value and p-value < 0.05 – Reject Null Hypothesis(HO) i.e., time series does not have a unit root, meaning it is stationary. It does not have a time-dependent structure.

Dickey-Fuller Test

Before going into ADF test, let’s first understand what is the Dickey-Fuller test.

A Dickey-Fuller test is a unit root test that tests the null hypothesis that α=1 in the following model equation. alpha is the coefficient of the first lag on Y.

Null Hypothesis (H0): alpha=1

where,

  • y(t-1) = lag 1 of time series
  • delta Y(t-1) = first difference of the series at time (t-1)

Fundamentally, it has a similar null hypothesis as the unit root test. That is, the coefficient of Y(t-1) is 1, implying the presence of a unit root. If not rejected, the series is taken to be non-stationary.

The Augmented Dickey-Fuller test evolved based on the above equation and is one of the most common form of Unit Root Test.

How does Augmented Dickey-Fuller (ADF) Test work?

As the name suggest, the ADF test is an ‘augmented’ version of the Dickey-Fuller test.

The ADF test expands the Dickey-Fuller test equation to include high order regressive process in the model.

ADF test stationarity in Time Series

If you notice, we have only added more differencing terms, while the rest of the equation remains the same. This adds more thoroughness to the test.

The null hypothesis however is still the same as the Dickey-Fuller test.

A key point to remember here is: Since the null hypothesis assumes the presence of unit root, that is α=1, the p-value obtained should be less than the significance level (say 0.05) in order to reject the null hypothesis. Thereby, inferring that the series is stationary.

However, this is a very common mistake analysts commit with this test. That is, if the p-value is less than significance level, people mistakenly take the series to be non-stationary.

ADF Test in Python

So, how to perform an Augmented Dickey-Fuller test in Python?

The statsmodels package provides a reliable implementation of the ADF test via the adfuller() function in statsmodels.tsa.stattools. It returns the following outputs:

  1. The p-value
  2. The value of the test statistic
  3. Number of lags considered for the test
  4. The critical value cut-offs.

When the test statistic is lower than the critical value shown, you reject the null hypothesis and infer that the time series is stationary.

Alright, let’s run the ADF test on the sunspots dataset from the statsmodels library of python.

As see earlier, the null hypothesis of the test is the presence of unit root, that is, the series is non-stationary.

Let’s run the ADF test on Time series data and analyze the result.

# Load the libraries
import numpy as np
import pandas as pd
# Load Statsmodels 
import statsmodels.api as sm
# Load Matplotlib for visualization
import matplotlib.pyplot as plt
%matplotlib inline
# Load the dataset
df = sm.datasets.sunspots.load_pandas().data
# Check the dimensionality of the dataset
df.shape
print("Dataset has {} records and {} columns".format(df.shape[0], df.shape[1]))
# Changing the YEAR data type and setting it as index
df['YEAR'] = pd.Index(sm.tsa.datetools.dates_from_range('1700', '2008'))
df.index = df['YEAR']
# Check the data type
del df['YEAR']
# View the dataset
df.head()
# Plotting the Data
# Define the plot size
plt.figure(figsize=(16,5))
# Plot the data
plt.plot(df.index, df['SUNACTIVITY'], label = "SUNACTIVITY")
plt.legend(loc='best')
plt.title("Sunspot Data from year 1700 to 2008")
plt.show()
# ADF Test
# Function to print out results in customised manner
from statsmodels.tsa.stattools import adfuller
def adf_test(timeseries):
    print ('Results of Dickey-Fuller Test:')
    dftest = adfuller(timeseries, autolag='AIC')
    dfoutput = pd.Series(dftest[0:4], index=['Test Statistic','p-value','#Lags Used','Number of Observations Used'])
    for key,value in dftest[4].items():
        dfoutput['Critical Value (%s)'%key] = value
    print (dfoutput)
# Call the function and run the test

adf_test(df['SUNACTIVITY'])

The ADF tests gives the following results – test statistic, p-value and the critical value at 1%, 5% , and 10% confidence intervals.

Results of Dickey-Fuller Test:
Test Statistic                   2.837781
p-value                          0.053076
#Lags Used                       8.000000
Number of Observations Used    300.000000
Critical Value (1%)             -3.452337
Critical Value (5%)             -2.871223
Critical Value (10%)            -2.571929
dtype: float64
Test Statistic is 2.837781 is greater than any critical values.

p-value is 0.053076

The p-value is obtained is greater than significance level of 0.05 and the ADF statistic is higher than any of the critical values.

Clearly, there is no reason to reject the null hypothesis. So, the time series is in fact non-stationary.

Kwiatkowski-Phillips-Schmidt-Shin (KPSS) Test

Introduction

The KPSS test, short for, Kwiatkowski-Phillips-Schmidt-Shin (KPSS), is a type of Unit root test that tests for the stationarity of a given series around a deterministic trend.

In other words, the test is somewhat similar in spirit with the ADF test.

A common misconception, however, is that it can be used interchangeably with the ADF test. This can lead to misinterpretations about the stationarity, which can easily go undetected causing more problems down the line.

In this article, you will see how to implement KPSS test in python, how it is different from ADF test and when and what all things you need to take care when implementing a KPSS test.

How to implement KPSS test

In python, the statsmodel package provides a convenient implementation of the KPSS test.

A key difference from ADF test is the null hypothesis of the KPSS test is that the series is stationary.

So practically, the interpretation of p-value is just the opposite to each other.

That is, if p-value is < significance level (say 0.05), then the series is non-stationary. Whereas in ADF test, it would mean the tested series is stationary.

Alright let’s implement the test on the ‘sunspots’ dataset.

KPSS test is conducted with the following assumptions.

Null Hypothesis (HO): Series is trend stationary or series has no unit root.

Alternate Hypothesis(HA): Series is non-stationary or series has a unit root.

Note: Hypothesis is reversed in KPSS test compared to ADF Test.

If the null hypothesis is failed to be rejected, this test may provide evidence that the series is trend stationary.

Conditions to Fail to Reject Null Hypothesis(HO)

  • If Test statistic < Critical Value and p-value < 0.05 – Fail to Reject Null Hypothesis(HO) i.e., time series does not have a unit root, meaning it is trend stationary.

To implement the KPSS test, we’ll use the kpss function from the statsmodel. The code below implements the test and prints out the returned outputs and interpretation from the result.

Let’s run the KPSS test on Time series data and analyze the result.

# Function to print out results in customised manner
from statsmodels.tsa.stattools import kpss
def kpss_test(timeseries):
    print ('Results of KPSS Test:')
    kpsstest = kpss(timeseries, regression='c', nlags="auto")
    kpss_output = pd.Series(kpsstest[0:3], index=['Test Statistic','p-value','#Lags Used'])
    for key,value in kpsstest[3].items():
        kpss_output['Critical Value (%s)'%key] = value
    print (kpss_output)
# Call the function and run the test
kpss_test(df[‘SUNACTIVITY’])
Results of KPSS Test:
Test Statistic           0.669866
p-value                  0.016285
Lags Used                7.000000
Critical Value (10%)     0.347000
Critical Value (5%)      0.463000
Critical Value (2.5%)    0.574000
Critical Value (1%)      0.739000

How to interpret KPSS test results

The output of the KPSS test contains 4 things:

  1. The KPSS statistic
  2. p-value
  3. Number of lags used by the test
  4. Critical values

The p-value reported by the test is the probability score based on which you can decide whether to reject the null hypothesis or not. If the p-value is less than a predefined alpha level (typically 0.05), we reject the null hypothesis.

The KPSS statistic is the actual test statistic that is computed while performing the test. For more information, no the formula, the references mentioned at the end should help.

In order to reject the null hypothesis, the test statistic should be greater than the provided critical values. If it is in fact higher than the target critical value, then that should automatically reflect in a low p-value.

That is, if the p-value is less than 0.05, the kpss statistic will be greater than the 5% critical value.

Finally, the number of lags reported is the number of lags of the series that was actually used by the model equation of the kpss test. By default, the statsmodels kpss() uses the ‘legacy’ method. In legacy method, int(12 * (n / 100)**(1 / 4)) a number of lags are included, where n is the length of the series.

Test Statistic is 0.669866

Critical Value (5%) is 0.463000

p-value is 0.016285

Test Statistic > Critical Value and p-value < 0.05. As a result, we reject the Null hypothesis in favor of an Alternative.

Hence we conclude series is non-stationary

When to choose ADF or KPSS test?

There could be a lot of confusion on when should one use the ADF test or KPSS test and which test would give a correct result.
A better solution is to apply/run both the tests and makes sure that the series is truly stationary.
The following are the possible outcomes of applying both the tests.
  • Case 1: Both tests conclude that the given series is stationaryThe series is stationary
  • Case 2: Both tests conclude that the given series is non-stationary The series is non-stationary
  • Case 3: ADF concludes non-stationary and KPSS concludes stationaryThe series is trend stationary. To make the series strictly stationary, the trend needs to be removed in this case. Then the detrended series is checked for stationarity.
  • Case 4: ADF concludes stationary and KPSS concludes non-stationaryThe series is difference stationary. Differencing is to be used to make series stationary. Then the differenced series is checked for stationarity.

Conclusion

Two tests for checking the stationarity of a time series are used, namely the ADF test and the KPSS test. Detrending is carried out by using differencing technique and the same will be covered in future articles on Statistical tests to check stationarity in Time Series.

About the Author

Our Top Authors

  • Analytics Vidhya
  • Guest Blog
  • Tavish Srivastava
  • Aishwarya Singh
  • Aniruddha Bhandari
  • Abhishek Sharma
  • Aarshay Jain

Download Analytics Vidhya App for the Latest blog/Article

Leave a Reply Your email address will not be published. Required fields are marked *