# Statistical tests to check stationarity in Time Series – Part 1

This article was published as a part of the Data Science Blogathon

## Introduction

*In this article, I will be talking through the **Augmented Dickey-Fuller test (ADF Test)** and *** Kwiatkowski-Phillips-Schmidt-Shin test (KPSS test)** that are the most common statistical tests used to test whether a given Time series is stationary or not. The 2 tests are the most commonly used statistical tests when it comes to analyzing the stationary of a series. Stationary is a very important factor in time series. In ARIMA time series forecasting, the first step is to determine the number of

*differencing*required to make the series stationary because a model cannot forecast on non-stationary time series data. let’s try to understand a little bit in-depth.

**What is a Stationary Series?**

A Stationary series is one whose statistical properties like mean, variance, covariance do not vary with time or these stats properties are not the function of time.

In other words, stationarity in Time Series also means series without a *Trend* or *Seasonal* components.

**Why should time series be stationary?**

Stationary series is easier for statistical models to predict effectively and precisely.

**Types of Stationary Series**

**Strict Stationary**– Satisfies the mathematical definition of a stationary process and mean, variance & covariance are not the function of time.**Seasonal Stationary**– Series exhibiting seasonality.**Trend stationary**– Series exhibiting trend.

**Note: Once the seasonality and trend is removed, series will be strict stationary**

**How to check Stationarity?**

**How to check Stationarity?**

*Visualizations*

The most basic methods for stationarity detection rely on plotting the data, and visually checking for trend and seasonal components.

Trying to determine whether a time series was generated by a stationary process just by looking at its plot is a dubious task. However, there are some basic properties of non-stationary data that we can look for.

Let’s take an example the following nice plots from [Hyndman & Athanasopoulos, 2018]:

Figure 1: Nine examples of time series data; (a) Google stock price for 200 consecutive days; (b) Daily change in the Google stock price for 200 consecutive days; (c) Annual number of strikes in the US; (d) Monthly sales of new one-family houses sold in the US; (e) Annual price of a dozen eggs in the US (constant dollars); (f) Monthly total of pigs slaughtered in Victoria, Australia; (g) Annual total of lynx trapped in the McKenzie River district of north-west Canada; (h) Monthly Australian beer production; (i) Monthly Australian electricity production. [Hyndman & Athanasopoulos, 2018]

- Seasonality can be observed in series (d), (h), and (i)
- The trend can be observed in series (a), (c), (e), (f), and (i)
- Series (b) and (g) are stationary
*Statistical Tests*

Two statistical tests which we will be discussing are

- Augmented Dickey-Fuller (ADF) Test
- Kwiatkowski-Phillips-Schmidt-Shin (KPSS) Test

**Augmented Dickey-Fuller Test**

**Augmented Dickey-Fuller Test**

Statistical tests make strong assumptions about your data. They can only be used to inform the degree to which a null hypothesis can be rejected or fail to be rejected. The result must be interpreted for a given problem to be meaningful.

However, they provide a quick check and confirmatory evidence that the time series is stationary or non-stationary.

The *Augmented Dickey-Fuller* test is a type of statistical test called a *unit root test*.

In probability theory and statistics, a **unit root** is a feature of some stochastic processes (such as random walks) that can cause problems in statistical inference involving time series models. **In a simple term, the unit root is non-stationary **but does not always have a trend component**.**

ADF test is conducted with the following assumptions.

*Null Hypothesis (H _{O})*: Series is non-stationary or series has a unit root.

*Alternate Hypothesis(H _{A})*: Series is stationary or series has no unit root.

If the null hypothesis is failed to be rejected, this test may provide evidence that the series is non-stationary.

Conditions to Reject Null Hypothesis(H_{O})

- If Test statistic < Critical Value and p-value < 0.05 –
time series does not have a unit root, meaning it is stationary. It does not have a time-dependent structure.*Reject Null Hypothesis(H*_{O}) i.e.,

## Dickey-Fuller Test

Before going into ADF test, let’s first understand what is the Dickey-Fuller test.

A Dickey-Fuller test is a unit root test that tests the null hypothesis that α=1 in the following model equation. `alpha`

is the coefficient of the first lag on Y.

Null Hypothesis (H0): alpha=1

where,

- y(t-1) = lag 1 of time series
- delta Y(t-1) = first difference of the series at time (t-1)

Fundamentally, it has a similar null hypothesis as the unit root test. That is, the coefficient of Y(t-1) is 1, implying the presence of a unit root. If not rejected, the series is taken to be non-stationary.

The Augmented Dickey-Fuller test evolved based on the above equation and is one of the most common form of Unit Root Test.

## How does Augmented Dickey-Fuller (ADF) Test work?

As the name suggest, the ADF test is an ‘augmented’ version of the Dickey-Fuller test.

The ADF test expands the Dickey-Fuller test equation to include high order regressive process in the model.

If you notice, we have only added more differencing terms, while the rest of the equation remains the same. This adds more thoroughness to the test.

The null hypothesis however is still the same as the Dickey-Fuller test.

A key point to remember here is: Since the null hypothesis assumes the presence of unit root, that is α=1, the p-value obtained should be less than the significance level (say 0.05) in order to reject the null hypothesis. Thereby, inferring that the series is stationary.

However, this is a very common mistake analysts commit with this test. That is, if the p-value is less than significance level, people mistakenly take the series to be non-stationary.

## ADF Test in Python

So, how to perform an Augmented Dickey-Fuller test in Python?

The `statsmodels`

package provides a reliable implementation of the ADF test via the `adfuller()`

function in `statsmodels.tsa.stattools`

. It returns the following outputs:

- The p-value
- The value of the test statistic
- Number of lags considered for the test
- The critical value cut-offs.

When the test statistic is lower than the critical value shown, you reject the null hypothesis and infer that the time series is stationary.

Alright, let’s run the ADF test on the sunspots dataset from the statsmodels library of python.

As see earlier, the null hypothesis of the test is the presence of unit root, that is, the series is non-stationary.

*Let’s run the ADF test on Time series data and analyze the result.*

# Load the libraries import numpy as np import pandas as pd

# Load Statsmodels import statsmodels.api as sm

# Load Matplotlib for visualization import matplotlib.pyplot as plt %matplotlib inline

# Load the dataset df = sm.datasets.sunspots.load_pandas().data # Check the dimensionality of the dataset df.shape print("Dataset has {} records and {} columns".format(df.shape[0], df.shape[1])) # Changing the YEAR data type and setting it as index df['YEAR'] = pd.Index(sm.tsa.datetools.dates_from_range('1700', '2008')) df.index = df['YEAR'] # Check the data type del df['YEAR'] # View the dataset df.head() # Plotting the Data # Define the plot size plt.figure(figsize=(16,5)) # Plot the data plt.plot(df.index, df['SUNACTIVITY'], label = "SUNACTIVITY") plt.legend(loc='best') plt.title("Sunspot Data from year 1700 to 2008") plt.show() # ADF Test # Function to print out results in customised manner from statsmodels.tsa.stattools import adfuller def adf_test(timeseries): print ('Results of Dickey-Fuller Test:') dftest = adfuller(timeseries, autolag='AIC') dfoutput = pd.Series(dftest[0:4], index=['Test Statistic','p-value','#Lags Used','Number of Observations Used']) for key,value in dftest[4].items(): dfoutput['Critical Value (%s)'%key] = value print (dfoutput) # Call the function and run the test adf_test(df['SUNACTIVITY'])

The ADF tests gives the following results – test statistic, p-value and the critical value at 1%, 5% , and 10% confidence intervals.

`Results of Dickey-Fuller Test:`

Test Statistic 2.837781 p-value 0.053076 #Lags Used 8.000000 Number of Observations Used 300.000000 Critical Value (1%) -3.452337 Critical Value (5%) -2.871223 Critical Value (10%) -2.571929 dtype: float64

p-value is 0.053076

The p-value is obtained is greater than significance level of 0.05 and the ADF statistic is higher than any of the critical values.

Clearly, there is no reason to reject the null hypothesis. So, the time series is in fact non-stationary.

**Kwiatkowski-Phillips-Schmidt-Shin (KPSS) Test**

**Kwiatkowski-Phillips-Schmidt-Shin (KPSS) Test**

**Introduction**

**Introduction**

The KPSS test, short for, Kwiatkowski-Phillips-Schmidt-Shin (KPSS), is a type of Unit root test that tests for the stationarity of a given series around a deterministic trend.

In other words, the test is somewhat similar in spirit with the ADF test.

A common misconception, however, is that it can be used interchangeably with the ADF test. This can lead to misinterpretations about the stationarity, which can easily go undetected causing more problems down the line.

In this article, you will see how to implement KPSS test in python, how it is different from ADF test and when and what all things you need to take care when implementing a KPSS test.

### How to implement KPSS test

In python, the `statsmodel`

package provides a convenient implementation of the KPSS test.

A key difference from ADF test is the null hypothesis of the KPSS test is that the series is stationary.

So practically, the interpretation of p-value is just the opposite to each other.

That is, if p-value is < significance level (say 0.05), then the series is non-stationary. Whereas in ADF test, it would mean the tested series is stationary.

Alright let’s implement the test on the ‘sunspots’ dataset.

**KPSS test is conducted with the following assumptions.**

**KPSS test is conducted with the following assumptions.**

*Null Hypothesis (H _{O})*: Series is trend stationary or series has no unit root.

*Alternate Hypothesis(H _{A})*: Series is non-stationary or series has a unit root.

**Note: Hypothesis is reversed in KPSS test compared to ADF Test.**

If the null hypothesis is failed to be rejected, this test may provide evidence that the series is trend stationary.

Conditions to Fail to Reject Null Hypothesis(H_{O})

- If Test statistic < Critical Value and p-value < 0.05 – Fail to
time series does not have a unit root, meaning it is trend stationary.*Reject Null Hypothesis(H*_{O}) i.e.,

To implement the KPSS test, we’ll use the `kpss`

function from the `statsmodel`

. The code below implements the test and prints out the returned outputs and interpretation from the result.

*Let’s run the KPSS test on Time series data and analyze the result.*

# Function to print out results in customised manner from statsmodels.tsa.stattools import kpss

def kpss_test(timeseries): print ('Results of KPSS Test:') kpsstest = kpss(timeseries, regression='c', nlags="auto") kpss_output = pd.Series(kpsstest[0:3], index=['Test Statistic','p-value','#Lags Used']) for key,value in kpsstest[3].items(): kpss_output['Critical Value (%s)'%key] = value print (kpss_output)

`Results of KPSS Test:`

Test Statistic 0.669866 p-value 0.016285 Lags Used 7.000000 Critical Value (10%) 0.347000 Critical Value (5%) 0.463000 Critical Value (2.5%) 0.574000 Critical Value (1%) 0.739000

### How to interpret KPSS test results

The output of the KPSS test contains 4 things:

- The KPSS statistic
- p-value
- Number of lags used by the test
- Critical values

The p-value reported by the test is the probability score based on which you can decide whether to reject the null hypothesis or not. If the p-value is less than a predefined alpha level (typically 0.05), we reject the null hypothesis.

The KPSS statistic is the actual test statistic that is computed while performing the test. For more information, no the formula, the references mentioned at the end should help.

In order to reject the null hypothesis, the test statistic should be greater than the provided critical values. If it is in fact higher than the target critical value, then that should automatically reflect in a low p-value.

That is, if the p-value is less than 0.05, the kpss statistic will be greater than the 5% critical value.

Finally, the number of lags reported is the number of lags of the series that was actually used by the model equation of the kpss test. By default, the `statsmodels`

`kpss()`

uses the ‘legacy’ method. In `legacy`

method, `int(12 * (n / 100)**(1 / 4))`

a number of lags are included, where n is the length of the series.

Test Statistic is 0.669866

Critical Value (5%) is 0.463000

p-value is 0.016285

Test Statistic > Critical Value and p-value < 0.05. As a result, we **reject the Null hypothesis in favor of an Alternative.**

**Hence we conclude series is non-stationary**

**When to choose ADF or KPSS test?**

*Case 1*:**Both tests conclude that the given series is stationary**–**The series is stationary***Case 2*:**Both tests conclude that the given series is non-stationary**–**The series is non-stationary***Case 3*:**ADF concludes non-stationary and KPSS concludes stationary**–To make the series strictly stationary, the trend needs to be removed in this case. Then the detrended series is checked for stationarity.**The series is trend stationary.***Case 4*:**ADF concludes stationary and KPSS concludes non-stationary**–Differencing is to be used to make series stationary. Then the differenced series is checked for stationarity.**The series is difference stationary.**

**Conclusion**

**Conclusion**

Two tests for checking the stationarity of a time series are used, namely the ADF test and the KPSS test. Detrending is carried out by using * differencing technique* and the same will be covered in future articles on

*.*

**Statistical tests to check stationarity in Time Series**