Learn everything about Analytics

Home » Complete Guide to Statistical Moments for Data Science

Complete Guide to Statistical Moments for Data Science

This article was published as a part of the Data Science Blogathon

Introduction

Statistical Moments plays a crucial role while we specifying our probability distribution to work with since, with the help of moments, we can describe the properties of statistical distribution. Therefore, they are helpful to describe the distribution.

In Statistical Estimation and Testing of Hypothesis, which all are based on the numerical values arrived for each distribution, we required the statistical moments.

So, In this article, we will be discussing primary statistical moments in a detailed manner.

 

Table of Contents

1. What is the Moment in Statistics?

2. Understanding Four Statistical Moments

  • The Expected Value or Mean
  • Variance and Standard Deviation
  • Skewness
  • Kurtosis

3. Different Types of Moments

  • Raw Moments
  • Centered Moments
  • Standardized Moments

What is the Moment in Statistics?

In Statistics, Moments are popularly used to describe the characteristic of a distribution.

Let’s say the random variable of our interest is X then, moments are defined as the X’s expected values.

For Example, E(X), E(X²), E(X³), E(X⁴),…, etc.

Figure 1: Moments in Statistics | Moment Generating Function for Probability Distribution with Python

                                                   Image Source: Google Images

What is the use of Moments?

  • These are very useful in statistics because they tell you much about your data.
  • The four commonly used moments in statistics are- the mean, variance, skewness, and kurtosis.

Understanding Four Statistical Moments

In Physics, moments refer to mass and inform us how the physical quantity is found or arranged. Similarly, in Mathematics, moments refer to something similar — the probability distribution, which is a function that represents how probable the different possible outcomes of an experiment are.

To be ready to compare different data sets we will describe them using the primary four statistical moments.

Let’s discuss each of the moment in an exceedingly detailed manner:

 

The First Moment

The first central moment is the expected value, known also as an expectation, mathematical expectation, mean, or average.

It measures the location of the central point.

Case-1: When all outcomes have the same probability of occurrence

It is defined as the sum of all the values the variable can take times the probability of that value occurring. Intuitively, we can understand this as the arithmetic mean:

Statistical Moments mean

Case-2: When all outcomes don’t have the same probability of occurrence

This is the more general equation that includes the probability of each outcome, and defined by,

Statistical Moments expected mean

Conclusion:

For equally probable events, the expected value is exactly the same as the Arithmetic Mean. This is one of the most popular measures of central tendency, which we also called Averages. But, there are some other common measures also like, Median and Mode.

  • Median — The middle value
  • Mode — The most likely value.

The Second Moment

The second central moment is “Variance”.

It measures the spread of values in the distribution OR how far from the normal.

Variance represents how a set of data points are spread out around their mean value. For n equally likely data points, the variance is defined as:

Statistical Moments expected variance

Where μ denotes the average value.

So, the variance is strongly impacted by the expected value.

Standard deviation

Standard deviation is just a square root of the variance and is commonly used since the unit of random variable X and Standard deviation is the same, so interpretation is easier.

Statistical Moments SD

In the given below plot, let’s see the clear visualization of Variance and standard deviation i.e, information about how strong data is spread around the mean:

Statistical Moments distributions

                                                               Image Source: Google Images

For Example, For a normal distribution:

  • Within 1st Standard Deviation: 68.27% of the data points lie
  • Within 2nd Standard Deviation: 95.45% of the data points lie
  • Within 3rd Standard Deviation: 99.73% of the data points lie

Now. let’s understand the answer to the given questions:

“Why Variance is preferred over Mean Absolute Deviation(MAD)?”

Variance is preferred over MAD due to the following reasons:

  • Mathematical properties: The function of variance is both Continuous and differentiable.
  • For a Population, the Standard Deviation of a sample is a more consistent estimate: If we picked the repeated samples from a normally distributed population, then the standard deviations of samples are less spread out as compared to mean absolute deviations.

NOTE:

At this point, we have a good understanding of both Mean and Variance. Let’s visualize these two moments with the help of the below diagram:

dist

                                                               Image Source: Google Images

From the above visualization, we can observe that:

If the variance or standard deviation is greater, then wider the spread of values around the mean. But in contrast, if the variance or standard deviation is lower, then the values are cumulated closer to the mean and the peak is higher.

The Third Moment

The third statistical moment is “Skewness”.

It measures how asymmetric the distribution is about its mean.

The formula for calculating skewness is as follows:

skewness

We can differentiate three types of distribution with respect to its skewness:

Symmetrical distribution: If both tails of a distribution are symmetrical, and the skewness is equal to zero, then that distribution is symmetrical.

Positively Skewed:  In these types of distributions, the right tail (with larger values) is longer. So, this also tells us about ‘outliers’ that have values higher than the mean. Sometimes, this is also referred to as:

  • Right-skewed
  • Right-tailed
  • Skewed to the Right

Negatively skewed: In these types of distributions, the left tail (with small values) is longer. So, this also tells us about ‘outliers’ that have values lower than the mean. Sometimes, this is also referred to as:

  • Left-skewed
  • Left-tailed
  • Skewed to the Left

types of skewness

                                                    Image Source: Google Images

For Example, For a Normal Distribution, which is Symmetric, the value of Skewness equals 0 and that distribution is symmetrical.

In general, Skewness will impact the relationship of mean, median, and mode in the described manner:

  • For a Symmetrical distribution: Mean = Median = Mode
  • For a positively skewed distribution: Mode < Median <Mean (large tail of high values)
  • For a negatively skewed distribution: Mean < Median <Mode (large tail of small values)

But the above generalization is not true for all possible distributions.

For Example, if one tail is long, but the other is heavy, this may not work. The best way to explore your data is to first compute all three estimators and then try to draw conclusions based on the results, rather than just focusing on the general rules.

Other Formula of Calculating Skewness:

Skewness = (Mean-Mode)/SD = 3*(Mean-Median)/SD

Since,

(Mode = 3*Median-2*Mean)

Some transformations to make the distribution normal:

For Positively skewed (right): Square root, log, inverse, etc.

For Negatively skewed (left): Reflect and square[sqrt(constant-x)], reflect and log, reflect and inverse, etc.

The Fourth Moment

The fourth statistical moment is “kurtosis”.

It measures the amount in the tails and outliers.

It focuses on the tails of the distribution and explains whether the distribution is flat or rather with a high peak. This measure informs us whether our distribution is richer in extreme values than the normal distribution.

kurtosis

For Example, For a normal distribution, the value of Kurtosis equals 3

For Kurtosis not equals to 3, there are the following cases:

  • Kurtosis<3 [Lighter tails]: Negative kurtosis indicates a broad flat distribution.
  • Kurtosis>3 [Heavier tails]: Positive kurtosis indicates a thin pointed distribution.

In general, we can differentiate three types of distributions based on the Kurtosis:

Mesokurtic: These types of distributions are having the kurtosis of 3 or excess kurtosis of 0. This category includes the normal distribution and some specific binomial distributions.

Leptokurtic: These types of distributions are having a kurtosis greater than 3, or excess kurtosis greater than 0. This is the distribution with fatter tails and a more narrow peak.

Platykurtic: These types of distributions are having the kurtosis smaller than 3 or excess kurtosis less than 0(negative). This is a distribution with very thin tails compared to the normal distribution.

Types of kurtosis

                                                          Image Source: Google Images

Now, let’s define what is Excess Kurtosis:

Excess Kurtosis = Kurtosis – 3

Understanding of Kurtosis related to Outliers:

  • Kurtosis is defined as the average of the standardized data raised to the fourth power. Any standardized values less than |1| (i.e. data within one standard deviation of the mean) will contribute petty to kurtosis.
  • The standardized values that will contribute immensely are the outliers.
  • Therefore, the high value of Kurtosis alerts about the presence of outliers.

 

Different Types of Moments

Raw Moments

The raw moment or the n-th moment about zero of a probability density function f(x) is the expected value of X n. It is also known as the Crude moment

We can generalize the definition of raw moments in the following ways:

Raw Moments

                                                        Image Source: Google Images

Centered Moments

central moment is a moment of a probability distribution of a random variable defined about the mean of the random variable’s i.e, it is the expected value of a specified integer power of the deviation of the random variable from the mean.

Centered Moments

                                                   Image Source: Google Images

Standardized Moments

A standardized moment of a probability distribution is a moment that is normally a higher degree central moment, but it is normalized typically by divide the standard deviation which renders the moment scale-invariant.

Standardized Moments

                                                         Image Source: Google Images

This ends our discussion!

Endnotes

Thanks for reading!

I hope you enjoyed the article and increased your knowledge about Statistical Moments in Statistics.

Please feel free to contact me on Email

Something not mentioned or want to share your thoughts? Feel free to comment below And I’ll get back to you.

For the remaining articles, refer to the link.

About the Author

Aashi Goyal

Currently, I am pursuing my Bachelor of Technology (B.Tech) in Electronics and Communication Engineering from Guru Jambheshwar University(GJU), Hisar. I am very enthusiastic about Statistics, and Data Science.

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.

You can also read this article on our Mobile APP Get it on Google Play