This article was published as a part of the Data Science Blogathon
Any data science project starts with exploring the data. When we perform an analysis on a sample through exploratory data analysis and inferential statistics we get information about the sample. Now, we want to use this information to predict values for the entire population.
Hypothesis testing is done to confirm our observation about the population using sample data, within the desired error level. Through hypothesis testing, we can determine whether we have enough statistical evidence to conclude if the hypothesis about the population is true or not.
To trust your model and make predictions, we utilize hypothesis testing. When we will use sample data to train our model, we make assumptions about our population. By performing hypothesis testing, we validate these assumptions for a desired significance level.
Let’s take the case of regression models: When we fit a straight line through a linear regression model, we get the slope and intercept for the line. Hypothesis testing is used to confirm if our beta coefficients are significant in a linear regression model. Every time we run the linear regression model, we test if the line is significant or not by checking if the coefficient is significant. I have shared details on how you can check these values in python, towards the end of this blog.
Key steps to perform hypothesis test are as follows:
Now let’s look into the steps in detail:
One of the key steps to do this is to formulate the below two hypotheses:
The null hypothesis represented as H₀ is the initial claim that is based on the prevailing belief about the population.
The alternate hypothesis represented as H₁ is the challenge to the null hypothesis. It is the claim which we would like to prove as True
One of the main points which we should consider while formulating the null and alternative hypothesis is that the null hypothesis always looks at confirming the existing notion. Hence, it has sign >= or , < and ≠
The significance level is the proportion of the sample mean lying in critical regions. It is usually set as 5% or 0.05 which means that there is a 5% chance that we would accept the alternate hypothesis even when our null hypothesis is true
Based on the criticality of the requirement, we can choose a lower significance level of 1% as well.
Hypothesis testing uses Test Statistic which is a numerical summary of a dataset that reduces the data to one value that can be used to perform the hypothesis test.
We choose the type of test statistic based on the predictor variable – quantitative or categorical. Below are a few of the commonly used test statistics for quantitative data
Type of predictor variable  Distribution type  Desired Test  Attributes 
Quantitative  Normal Distribution  Z – Test 

Quantitative  T Distribution  TTest 

Quantitative  Positively skewed distribution  F – Test 

Quantitative  Negatively skewed distribution  NA 

Categorical  NA  ChiSquare test 

Zstatistic is used when the sample follows a normal distribution. It is calculated based on the population parameters like mean and standard deviation.
One sample Z test is used when we want to compare a sample mean with a population mean
Two sample Z test is used when we want to compare the mean of two samples
Tstatistic is used when the sample follows a T distribution and population parameters are unknown. T distribution is similar to a normal distribution, it is shorter than normal distribution and has a flatter tail.
For samples involving three or more groups, we prefer the F Test. Performing Ttest on multiple groups increases the chances of Type1 error. ANOVA is used in such cases.
Analysis of variance (ANOVA) can determine whether the means of three or more groups are different. ANOVA uses Ftests to statistically test the equality of means.
Fstatistic is used when the data is positively skewed and follows an F distribution. F distributions are always positive and skewed right.
F = Variation between the sample means/variation within the samples
For negatively skewed data we would need to perform feature transformation
For categorical variables, we would be performing a chiSquare test.
Following are the two types of chisquared tests:
Test Statistic is then used to calculate PValue. A Pvalue measures the strength of evidence in support of a null hypothesis. If the Pvalue is less than the significance level, we reject the null hypothesis.
if the pvalue < α, then we have statistically significant evidence against the null hypothesis, so we reject the null hypothesis and accept the alternate hypothesis
if the pvalue > α then we do not have statistically significant evidence against the null hypothesis, so we fail to reject the null hypothesis.
As we make decisions, it is important to understand the errors that can happen while testing.
There are two possible types of error we could commit while performing hypothesis testing.
1) Type1 Error – This occurs when the null hypothesis is true but we reject it.The probability of type I error is denoted by alpha (α). Type 1 error is also known as the level of significance of the hypothesis test
2) Type 2 Error – This occurs when the null hypothesis is false but we fail to reject it. The probability of type II error is denoted by beta (β)
The stats model library has the unique ability to perform and summarize the outcomes of hypothesis tests on your model. Based on your feature variables, you can determine which test value is relevant for your model and make decisions accordingly.
import statsmodels.api as sm
To create a fitted model, I have used Ordinary least squares
lr = sm.OLS(y_train, X_train_lm).fit()
Once we have trained the model, we can see the summary of the tests using the command
print(lr.summary())
The model summary will look something like below.
From a hypothesis testing standpoint, you need to pay attention to the following values decide if you need to refine your model
This is all about hypothesis testing in this article.
Image source: All images in this blog have been created by the author
The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.
Lorem ipsum dolor sit amet, consectetur adipiscing elit,
this article very help full to understand about the hypothesis testing :)
such a good article easy to understand
What does the p value for each variable represent?
A pvalue is composed of summation of 3 cases: 1. The probability of getting a particular value in a distribution (eg. pdf, mdf or histogram) 2. The probability of getting a value which is equally rare in that distribution 3. The probability of getting values which are more rare than the observed value in that distribution. For example, If we tossing a coin 5 times and want to know the pvalue for 4 Head and 1 tails, the pvalue will be calculated as follows: total possible outcomes = 2^5 = 32 1. Prob. of getting 4 heads and 1 tail (HHHHT, HHHTH, HHTHH, HTHHH, THHHH); P1 = 5/32 2. Equally rare event is 4 tails and 1 head; P2. = 5/32 3. More extreme events are 5 heads or 5 tails (HHHHH, TTTTT); P3. = 1/32 + 1/32 = 2/32 Finally pvalue = P1 + P2 + P3 = 5/32 + 5/32 + 2/32 = 12/32 = 0.375 Pvalue = 0.375 Further more, this test is used to understand about hypothesis test. If we are using alpha value of 0.05 to generalize whether coin is biased or not. H0 = if pvalue of 4 head and 1 tails is less than than alpha, coin is biased H1 = else coin is not biased Now as pvalue is > 0.05, Coin is not biased and H0 is rejected.
The pvalue is the summation of 3 cases: 1. The probability of getting a value in a particular distribution (eg histogram, pdf, mdf). 2. The probability of getting equally rare value in that distribution 3. The summation of observing more extreme values in that distribution. You can understand it with the help of an example explained below. Let a coin is tossed 5 times and we want to know the pvalue of getting 4 heads and 1 tail. Total events = 2^5 = 32 The pvalue will be calculated in 3 steps as explained below: 1. Prob. of getting 4 heads and 1 tails (HHHHT, HHHTH, HHTHH, HTHHH, THHHH); P1 = 5/32 2. Prob. of equally rare event 4 tails and 1 head, P2 = 5/32 3. Sum of prob. of more rare events i.e. 5 heads or 5 tails; P3 = 1/32 + 1/32 = 2/32 pvalue = P1 + P2 + P3 = 5/32 + 5/32 + 2/32 = 12/32 = 0.375 pvalue = 0.375 Furthermore, we can have understanding of hypothesis test using this example. Lets say we are using a confidence interval of 95% to check whether coin is biased or not for the event of getting 4 heads and 1 tail. Now, alpha = 1  0.95 = 0.05 H0 = Coin is biased if pvalue 0.05, We reject H0 and the coin is not biased It means getting 4 heads and 1 tail in tossing a coin 5 times does not means that coin is special or biased which is also true as per our knowledge. This is how pvalue works. Hope this explanation makes sense.