Time Series Techniques You Should Know

Time series analysis is a crucial aspect of data science, particularly when dealing with datasets that are collected or recorded over time. Understanding time series data allows data scientists to identify trends, seasonal patterns, and potential anomalies to make predictions. So, if you want to improve your skills in working with time series problems, this article is for you. In this article, I’ll take you through some essential time series techniques you should know as a Data Scientist.

Time Series Techniques You Should Know

Below are some essential time series techniques you should know as a Data Scientist:

1. Understanding and preparing time series data
2. Time series decomposition
3. Moving Averages and Smoothing
4. Stationarity and Differencing
5. Autocorrelation and Partial Autocorrelation
6. Extracting Days Data from dates
7. ARIMA and SARIMA

Let’s go through all these techniques in detail with implementation using Python. To implement all these techniques, I’ll use time series data based on Netflix subscription growth. You can download the dataset from here.

Understanding and Preparing Time Series Data

Time series data consists of sequences of data points collected at consistent time intervals. Examples include daily stock prices, monthly sales data, and annual rainfall records. Understanding and preparing time series data involves identifying the time-related variables and converting them into an appropriate format for analysis.

Here’s how to convert a time series data into an appropriate format for analysis:

import pandas as pd

netflix_data = pd.read_csv('/content/Netflix-Subscriptions.csv')

# convert 'Time Period' to datetime
netflix_data['Time Period'] = pd.to_datetime(netflix_data['Time Period'])

# set 'Time Period' as the index
netflix_data.set_index('Time Period', inplace=True)

# display the first few rows to confirm changes
print(netflix_data.head())

             Subscribers
Time Period             
2013-01-04      34240000
2013-01-07      35640000
2013-01-10      38010000
2014-01-01      41430000
2014-01-04      46130000

Time Series Decomposition

Decomposing a time series into its components (trend, seasonality, and residual) helps in understanding the underlying patterns, which helps in selecting the right algorithm for forecasting a time series.

Here’s how to decompose a time series:

import statsmodels.api as sm
import matplotlib.pyplot as plt

# decomposing the time series
decomposition = sm.tsa.seasonal_decompose(netflix_data['Subscribers'], model='additive', period=4)
decomposition.plot()
plt.show()

Moving Averages and Smoothing

Moving averages help in smoothing out short-term fluctuations and highlighting longer-term trends. By calculating the average of the data points over a specified number of periods, moving averages reduce the noise and variability inherent in the raw data, which makes it easier to identify underlying patterns and trends.

Here’s how to calculate moving averages:

# calculating moving averages
netflix_data['MA4'] = netflix_data['Subscribers'].rolling(window=4).mean()

# plotting original data and moving average
netflix_data[['Subscribers', 'MA4']].plot()
plt.title('Netflix Subscribers with 4-Period Moving Average')
plt.show()

Stationarity and Differencing

A stationary time series has properties that do not depend on the time at which the series is observed. Checking for stationarity is crucial for many time series forecasting models.

Although, the data we are using is quite stationary, still here’s how to check stationarity and perform differencing to achieve stationarity:

from statsmodels.tsa.stattools import adfuller

# perform augmented dickey-fuller test
result = adfuller(netflix_data['Subscribers'].dropna())
print('ADF Statistic:', result[0])
print('p-value:', result[1])

# differencing to achieve stationarity
netflix_data['Subscribers_diff'] = netflix_data['Subscribers'].diff().dropna()

# re-perform ADF test on differenced data
result_diff = adfuller(netflix_data['Subscribers_diff'].dropna())
print('ADF Statistic (Differenced):', result_diff[0])
print('p-value (Differenced):', result_diff[1])

ADF Statistic: 0.20591078338329172
p-value: 0.972591170013349
ADF Statistic (Differenced): -4.523088428253382
p-value (Differenced): 0.00017858926729786176

Autocorrelation and Partial Autocorrelation

Autocorrelation (ACF) and partial autocorrelation (PACF) are tools to identify the correlation of the time series with its own past values. ACF measures the correlation between the time series and its lagged values, which provides insights into repeating patterns, such as seasonality or cyclic behaviour, across the entire series. PACF, on the other hand, isolates the direct effect of a lag on the series by removing the influence of intermediate lags, which makes it useful for identifying the actual order of autoregressive terms in a time series model.

Here’s how to plot the ACF and PACF plots using Python:

from statsmodels.graphics.tsaplots import plot_acf, plot_pacf

# plotting ACF and PACF
plot_acf(netflix_data['Subscribers_diff'].dropna(), lags=20)
plt.show()

plot_pacf(netflix_data['Subscribers_diff'].dropna(), lags=20)
plt.show()

Extracting Days Data from Dates

Extracting days data from the dates helps in a detailed analysis of the underlying patterns in a time series. Here’s how to extract day information from the dates:

# extract day of the month
netflix_data['DayOfMonth'] = netflix_data.index.day

# extract day of the week (Monday=0, Sunday=6)
netflix_data['DayOfWeek'] = netflix_data.index.dayofweek

# extract day name (e.g., Monday, Tuesday)
netflix_data['DayName'] = netflix_data.index.day_name()

# display the dataset with extracted day information
print(netflix_data.head())

             Subscribers         MA4  Subscribers_diff  DayOfMonth  DayOfWeek  \
Time Period                                                                     
2013-01-04      34240000         NaN               NaN           4          4   
2013-01-07      35640000         NaN         1400000.0           7          0   
2013-01-10      38010000         NaN         2370000.0          10          3   
2014-01-01      41430000  37330000.0         3420000.0           1          2   
2014-01-04      46130000  40302500.0         4700000.0           4          5   

               DayName  
Time Period             
2013-01-04      Friday  
2013-01-07      Monday  
2013-01-10    Thursday  
2014-01-01   Wednesday  
2014-01-04    Saturday  

ARIMA and SARIMA

ARIMA (AutoRegressive Integrated Moving Average) and SARIMA (Seasonal ARIMA) are powerful time series forecasting models. ARIMA combines three components: autoregression (AR), differencing (I), and moving average (MA), to model temporal dependencies and trends in non-stationary time series data. It is effective for capturing linear relationships and patterns over time. SARIMA extends ARIMA by incorporating seasonal components, making it suitable for time series data with repeating seasonal patterns.

You can learn about ARIMA and SARIMA in detail with implementation using Python here.

Summary

So, here are some essential time series techniques you should know as a Data Scientist:

1. Understanding and preparing time series data
2. Time series decomposition
3. Moving Averages and Smoothing
4. Stationarity and Differencing
5. Autocorrelation and Partial Autocorrelation
6. Extracting Days Data from dates
7. ARIMA and SARIMA

I hope you liked this article on time series techniques you should know as a Data Scientist. Feel free to ask valuable questions in the comments section below. You can follow me on Instagram for many more resources.