Interview Prep
Career
GenAI
Prompt Engg
ChatGPT
LLM
Langchain
RAG
AI Agents
Machine Learning
Deep Learning
GenAI Tools
LLMOps
Python
NLP
SQL
AIML Projects

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details

Reading list

Machine Learning Basics for a Newbie
6 Steps of Machine learning Lifecycle Introduction to Predictive Modeling
Introduction to Exploratory Data Analysis & Data Insights Descriptive Statistics Inferential Statistics How to Understand Population Distributions?
Reading Data Files into Python Different Variable Datatypes
Probability for Data Science Basic Concepts of Probability Axioms of Probability Conditional Probability
Central Tendencies for Categorical Variables Understanding Discrete Distributions Performing EDA on Categorical Variables
Dealing with Missing Values Understanding Outliers Identifying Outliers in Data Outlier Detection in Python Outliers Detection Using IQR, Z-score, LOF and DBSCAN
Sample and Population Central Limit Theorem Confidence Interval and Margin of Error
Bivariate Analysis Introduction
Covariance Pearson Correlation Spearman's Correlation & Kendall's Tau Correlation versus Causation Tabular and Graphical methods for Bivariate Analysis Performing Bivariate Analysis on Continuous-Continuous Variables
Tabular and Graphical methods for Continuous-Categorical Variables Introduction to Hypothesis Testing P-value Two sample Z-test T-test T-test vs Z-test Performing Bivariate Analysis on Continuous-Catagorical variables
Chi-Squares Test Bivariate Analysis on Categorical Categorical Variables
Multivariate Analysis A Comprehensive Guide to Data Exploration The Data Science behind IPL
Supervised Learning vs Unsupervised Learning Reinforcement Learning Generative and Descriminative Models Parametric and Non Parametric model
Machine Learning Pipeline Preparing Dataset Build a Benchmark Model: Regression Build a Benchmark Model: Classification
Evaluation Metrics for Machine Learning Everyone should know Confusion Matrix Accuracy Precision and Recall AUC-ROC Log Loss R2 and Adjusted R2
Dealing with Missing Values Replacing Missing Values Imputing Missing Values in Data Working with Categorical Variables Working with Outliers Preprocessing Data for Model Building
Understanding Cost Function Understanding Gradient Descent Math Behind Gradient Descent Assumptions of Linear Regression Implement Linear Regression from Scratch Train Linear Regression in Python Implementing Linear Regression in R Diagnosing Residual Plots in Linear Regression Models Generalized Linear Models Introduction to Logistic Regression Odds Ratio Implementing Logistic Regression from Scratch Introduction to Scikit-learn in Python Train Logistic Regression in python Multiclass using Logistic Regression How to use Multinomial and Ordinal Logistic Regression in R ? Challenges with Linear Regression Introduction to Regularisation Implementing Regularisation Ridge Regression Lasso Regression
Introduction to K Nearest Neighbours Determining the Right Value of K in KNN Implement KNN from Scratch Implement KNN in Python
Bias Variance Tradeoff Introduction to Overfitting and Underfitting Visualizing Overfitting and Underfitting Selecting the Right Model What is Validation? Hold-Out Validation Understanding K Fold Cross Validation
Introduction to Feature Selection Feature Selection Algorithms Missing Value Ratio Low Variance Filter High Correlation Filter Backward Feature Elimination Forward Feature Selection Implement Feature Selection in Python Implement Feature Selection in R
Introduction to Decision Tree Purity in Decision Tree Terminologies Related to Decision Tree How to Select Best Split Point in Decision Tree? Chi-Squares Information Gain Reduction in Variance Optimizing Performance of Decision Tree Train Decision Tree using Scikit Learn Pruning of Decision Trees
Introduction to Feature Engineering Feature Transformation Feature Scaling Feature Engineering Frequency Encoding Automated Feature Engineering: Feature Tools
Introduction to Naive Bayes Conditional Probability and Bayes Theorem Introduction to Bayesian Adjustment Rating: The Incredible Concept Behind Online Ratings! Working of Naive Bayes Math behind Naive Bayes Types of Naive Bayes Implementation of Naive Bayes
Understanding how to solve Multiclass and Multilabled Classification Problem Evaluation Metrics: Multi Class Classification
Introduction to Ensemble Techniques Basic Ensemble Techniques Implementing Basic Ensemble Techniques Finding Optimal Weights of Ensemble Learner using Neural Network Why Ensemble Models Work well?
Introduction to Stacking Implementing Stacking Variants of Stacking Implementing Variants of Stacking Introduction to Blending Bootstrap Sampling Introduction to Random Sampling Hyper-parameters of Random Forest Implementing Random Forest Out-of-Bag (OOB) Score in the Random Forest IPL Team Win Prediction Project Using Machine Learning Introduction to Boosting Gradient Boosting Algorithm Math behind GBM Implementing GBM in python Regularized Greedy Forests Extreme Gradient Boosting Implementing XGBM in python Tuning Hyperparameters of XGBoost in Python Implement XGBM in R/H2O Adaptive Boosting Implementing Adaptive Boosing LightGBM Implementing LightGBM in Python Catboost Implementing Catboost in Python
Different Hyperparameter Tuning methods Implementing Different Hyperparameter Tuning methods GridsearchCV RandomizedsearchCV Bayesian Optimization for Hyperparameter Tuning Hyperopt
Understanding SVM Algorithm SVM Kernels In-depth Intuition and Practical Implementation SVM Kernel Tricks Kernels and Hyperparameters in SVM Implementing SVM from Scratch in Python and R
Introduction to Principal Component Analysis Steps to Perform Principal Compound Analysis Computation of Covariance Matrix Finding Eigenvectors and Eigenvalues Implementing PCA in python Visualizing PCA A Brief Introduction to Linear Discriminant Analysis Introduction to Factor Analysis
Introduction to Clustering Applications of Clustering Evaluation Metrics for Clustering Understanding K-Means Implementation of K-Means in Python Implementation of K-Means in R Choosing Right Value for K Profiling Market Segments using K-Means Clustering Hierarchical Clustering Implementation of Hierarchial Clustering DBSCAN Defining Similarity between clusters Build Better and Accurate Clusters with Gaussian Mixture Models
Understand Basics of Recommendation Engine with Case Study
8 Ways to Improve Accuracy of Machine Learning Models
Introduction to Dask Working with CuML
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to Machine Learning Interpretability Framework and Interpretable Models model Agnostic Methods for Interpretability Implementing Interpretable Model Understanding SHAP Out-of-Core ML Introduction to Interpretable Machine Learning Models Model Agnostic Methods for Interpretability Game Theory & Shapley Values
Introduction to AutoML Implementation of MLBox Introduction to PyCaret TPOT Auto-Sklearn EvalML
Pickle and Joblib Introduction to Model Deployment
Deploying Machine Learning Model using Streamlit Deploying ML Models in Docker Deploy Using Streamlit Deploy on Heroku Deploy Using Netlify Introduction to Amazon Sagemaker Setting up Amazon SageMaker Using SageMaker Endpoint to Generate Inference Deploy on Microsoft Azure Cloud Introduction to Flask for Model Deploying ML model using Flask
Model Deployment in Android Model Deployment in Iphone

Exploring The Different Types Of Probability Distribution Function!

Harika
Harika 09 Apr, 2024
10 min read

Introduction

Probability distribution function

In this article, we’ll look into Probability Distribution Functions (PDFs), Distribution Functions and their different types, and how they’re used. We’ll break down the formulas, understand their characteristics, and make it easy to grasp how they help with data. This guide is here to simplify probability and statistics for better decision-making.

This article was published as a part of the Data Science Blogathon.

Table of contents

What is Probability Distribution Function?

A Probability Distribution Function (PDF) is a mathematical way of showing how likely different outcomes are in a random event. It gives probabilities to each possible result, and when you add up all the probabilities, the total is always 1. The PDF helps us understand the chances of different outcomes in a random experiment.

Probability distribution function formula?

The Probability Distribution Function (PDF) is often denoted as f(x), and its formula varies based on the specific probability distribution. Here are a few examples for commonly used distributions:

Continuous Uniform Distribution:

f(x)=b−a1​for a≤x≤b

Normal Distribution (Gaussian Distribution):

f(x)=σ2π​1​⋅e−2σ2(x−μ)2​

Exponential Distribution:

f(x)=λe−λxfor x≥0

Binomial Distribution:

f(x)=(xn​)⋅px⋅(1−p)n−xfor x=0,1,2,…,n

Here,f(x) represents the probability density function,x is the variable, and the other symbols have specific meanings depending on the distribution. Note that these are just a few examples, and different probability distributions have different formulas.

What is a distribution function?

In statistical terms, a distribution function is a mathematical expression that describes the probability of different possible outcomes for an experiment.

It is denoted as Variable ~ Type (Characteristics)

Let us say we are running an experiment of tossing a fair coin. The possible events are Heads, Tails. And for instance, if we use X to denote the events, the probability distribution of X would take the value 0.5 for X=heads, and 0.5 for X=tails.

Data Types

At a higher level, we have Qualitative and Quantitative data. And in Quantitative data, we have Continuous and Discrete data types.

Continuous data is measured and can take any number of values in a given finite or infinite range. It can be represented in decimal format. And the random variable that holds continuous values is called the Continuous random variable.

Examples: A person’s height, Time, distance, etc.

Discrete data is counted and can take only a limited number of values. It makes no sense when written in decimal format. And the random variable that holds discrete data is called the Discrete random variable.

Example: The number of students in a class, number of workers in a company, etc.

Types of distribution functions:

Based on the types of data we deal with, we have two types of distribution functions.

For discrete data, we have discrete distributions; and for continuous data, we have continuous distributions.

Discrete distributions 

Continuous distributions

Uniform distributionNormal distribution
Binomial distributionStandard Normal distribution
Bernoulli distributionStudent’s T distribution
Poisson distributionChi-squared  distribution

Before deep-diving into the types of distributions, it is important to revise the fundamental concepts like Probability Density Function (PDF), Probability Mass Function (PMF), and Cumulative Density Function (CDF).

Probability Density Function (PDF): 

It is a statistical term that describes the probability distribution of a continuous random variable. The probability associate with a single value is always Zero. Below is the formula for PDF.

probability distribution function

The formula for PDF. 

Probability Mass Function (PMF)

It is a statistical term that describes the probability distribution of a discrete random variable.

probability mass function

The formula for PMF.

Cumulative Distribution Function (CDF)

It is another method to describe the distribution of a random variable (either continuous or discrete).

cumulative distribution function

The formula for CDF. 

Discrete distributions

Let us start with the easiest one – Uniform distribution.

Discrete Uniform distribution (U)

It is denoted as X ~ U (a, b). And is read as X is a discrete random variable that follows uniform distribution ranging from a to b.

Uniform distribution is when all the possible events are equally likely. For example, consider an experiment of rolling a dice. We have six possible events X = {1, 2, 3, 4, 5, 6} each having a probability of P(X) = 1/6.

The PMF graph of the above experiment is:

Probability Mass Function,

The formula for PMF, CDF of Uniform distribution function are:

fromula of pmf, cmf

The Mean and Variance of Uniform distribution are:

Mean = (a+b)/2

Variance = (n2-1)/12

Binomial distribution (B):

It is denoted as X ~ B(n, p). And is read as X is a discrete random variable that follows Binomial distribution with parameters n, p.
Where n is the no. of trials, and p is the success probability for each trial.

Binomial distribution is a discrete probability distribution of the number of successes in ‘n’ independent experiments sequence. The two outcomes of a Binomial trial could be Success/Failure, Pass/Fail/, Win/Lose, etc.

Generally, the outcome success is denoted as 1, and the probability associated with it is p.

And Failure is denoted as 0, and the probability associate with it is q = 1-p.

The formula for PMF, CDF of Binomial distribution are:

binomial distribution pMF and CMF

K in the above formula is the number of successes.

The mean and variance of a binomial distribution are given as:

Mean = np

Variance = npq

Now consider, we ran a Binomial experiment 10 times, and the probability of success = 0.25. Below are how PMF, CDF looks.

Binomial distribution | Probability Distribution Function

Image by Author

PMF, CDF for a Binomial experiment with Probability of success = Probability of failure look like below.

Probability Distribution Function |Binomial

Bernoulli distribution (Bern): 

It is denoted as X ~ Bern(p). And is read as X is a discrete random variable that follows Bernoulli distribution with parameter p.
Where p is the probability of the success.

Bernoulli can be represented as a Binomial experiment with a single trial.

X ~ Bern(p) —-> X ~ B(1, p)

The formula for PMF, CDF of Bernoulli distribution is:

PMF and CMF

The Mean and Variance of Bernoulli distribution are given as:

Mean = p

Variance = p(1-p) = pq

Example: Consider an example of tossing a fair. The two possible outcomes are Heads, Tails. The probability (p) associated with each of them is 1/2.

If we take an unfair coin, the probability associated with each of them need not be 1/2. Heads can have a probability of p = 0.8, then the probability of tail q = 1-p = 1-0.8 = 0.2

example | Probability Distribution Function

Bernoulli’s event suggests which outcome can be expected for a single trial. Whereas, a Binomial event suggests the no. of times a specific outcome can be expected. 

Poisson Distribution (Po):

It is denoted as X ~ Po(λ). And is read as X is a discrete random variable that follows Poisson Distribution with parameter λ.
Where λ is the expected rate of occurrences.

Poisson Distribution is a discrete probability distribution function that expresses the probability of a given number of events occurring in a fixed time interval.

Examples: 

  • The number of diners at a restaurant on a given day.
  • Calls per hour at a call centre.

The formula for PMF, CDF of poison distribution are:

pmf cmf poison distribution | Probability Distribution Function

The Mean and Variance of Poisson distribution are given as:

Mean = Variance = λ

A Poisson distribution with λ = 5 look like below

example poison distribution

Continuous Distributions

Normal or Gaussian Distribution (N)

It is denoted as X ~ N (μ, σ2). And is read as X is a continuous random variable that follows a Normal distribution with parameters μ, σ2.
Where μ is the mean, and σ2 is the variance. Mean, Variance together talks about shape statistics.

A normal distribution is a continuous distribution that describes the probability of a continuous random variable that takes real values.

Examples: Heights of people, exam scores of students, IQ Scores, etc follows Normal distribution.

Properties of Normal distribution:

  • The random variable takes values from -∞ to +∞
  • The probability associate with any single value is Zero.
  • looks like a bell curve and is symmetric about x=μ. 50% of data lies on the left-hand side and 50% of the data lies on the right-hand side.
  • The area under the curve (AUC) = 1
  • All the measures of central tendency coincide i.e., mean = median = mode

A normal distribution with different means, standard deviations look like below:

normal distribution | Probability Distribution Function

Normal distribution follows the 68-95-99.7 rule. This rule is also known as the empirical rule. According to it, 68% of data lies in the first standard deviation range, 95% of data lies in the second standard deviation range, and 99.7% of data lies in the third standard deviation range.

Probability Density. Probability distribution function

The formula for PDF, CDF of the normal distribution are:

pdf for normal distribution
Source

The Mean and Variance of a Normal distribution are given as:

Mean = μ

Variance = σ2

Let’s assume we have a height distribution with mean = 25, standard deviation = 2.48. Below is how the graph looks like.

example normal disribution

Standard Normal Distribution or SND:

It is denoted as Z ~ N(0, 1). And is read as X is a continuous random variable that follows  Normal distribution with mean 0 and variance 1.

It is a transformation of Normal distribution in such a way that Mean = 0, and standard deviation 1.

Transformation is a way in which we alter every element of distribution to get a new distribution with similar characteristics.

All the properties of a Normal distribution will be satisfied by a Standard Normal distribution.

SND | Probability Distribution Function

And in addition, there exists a table that summarizes the most commonly used values of a CDF of s Standard Normal Distribution. This table is known as a Z-score table.

The formula for standardisation is Z = (X-μ)/σ

The formula for PDF, CDF of Standard Normal distribution are given as:

formula PDF,CDF distribution function

The Mean and Variance of Standard Normal distribution are:

Mean = 0

Variance = 1

Normal distribution with mean 0 and variance 1 (SND) looks like below:

normal distribution with mean 0 and sd 1

Student’s T distribution or t-distribution (t)

It is denoted as X ~ t(k). And is read as X is a continuous random variable that follows Student’s T distribution with parameter k.
where k is the degrees of freedom. If the sample size is n, then k = n-1.

Student’s T distribution is a small sample size approximation of a normal distribution. As the degrees of freedom increase, t distribution tends to become Standard Normal distribution.

T distribution, probability distribution function

The formula for PDF, CDF of t-distribution are:

pdf and cdf for t distribution

(v in above formulae is degrees of freedom)

t-distribution can be used in Hypothesis testing (to test if there is any significant difference between two sample means), calculating confidence intervals with population standard deviation is unknown.

Like standard normal distribution, t-distribution also has a table of its own. This table is known as the t-table.

The mean and variance of Student’s T distribution are:

Mean = 0

variance = k/(k-2)

A student’s t-distribution with degrees of freedom = 25 looks like below:

example tdistribution | Probability Distribution Function

Chi-Square distribution

It is denoted as X~χ2(k). And is read as X is a continuous random variable that follows Chi-Square distribution with k degrees of freedom.

It is used in Hypothesis testing, computing confidence intervals, and for the goodness of fit.

It is a transformation of t-distribution. Finding the t-distribution to the power of 2 gives Chi-Square distribution and finding the square root of Chi-Square of distribution gives us t-distribution.

Chi-Square distribution has a chi-square table.

The formula for PDF, CDF of Chi-square distribution are:

chi square distribution

Source

The mean and variance of Chi-square distribution are:

Mean = k

Variance = 2k

A Chi-square distribution with degrees of freedom = 5 looks like below.

example chi sqaured , probabiltiy distribution function

Conclusion

In summary, Probability Distribution Functions (PDFs) and their formulas and Distribution Functions are key for understanding different data types. Explore Discrete Uniform, Binomial, Bernoulli, Poisson, and Continuous distributions to gain insights for effective data analysis and decision-making in various scenarios.

I hope this article is informative. Feel free to share it with your study buddies.

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.

Frequently Asked Questions?

Q1. What is probability distribution functions?

A. Probability distribution functions describe the probabilities of possible outcomes in a random phenomenon. They assign probabilities to various events or values that a random variable can take.

Q2. What is the state probability distribution function?

A. The state probability distribution function describes the probabilities of a system being in different states or conditions. It’s commonly used in fields like physics, chemistry, and engineering to model the behavior of systems in different states.

Q3. What is the expression for the probability distribution function?

A. The expression for the probability distribution function varies depending on the specific distribution being considered. For example, for a continuous distribution, it might be expressed as a probability density function (PDF), while for a discrete distribution, it might be expressed as a probability mass function (PMF).

Q4. What is the probability distribution function also called?

A. The probability distribution function is also commonly referred to as the probability density function (PDF) for continuous distributions or the probability mass function (PMF) for discrete distributions.

blogathondata scienceDistribution functionfollowing is a discrete probability distribution functionprobability distributionproperties of probability distribution functionstatistics
Harika
Harika 09 Apr, 2024

Hi, my name is Harika. I am a Data Engineer and I thrive on creating innovative solutions and improving user experiences. My passion lies in leveraging data to drive innovation and create meaningful impact.

Free Courses

Generative AI 4.7

Generative AI - A Way of Life

Explore Generative AI for beginners: create text and images, use top AI tools, learn practical skills, and ethics.
Large Language Models 4.5

Getting Started with Large Language Models

Master Large Language Models (LLMs) with this course, offering clear guidance in NLP and model training made simple.
Prompt Engineering 4.6

Building LLM Applications using Prompt Engineering

This free course guides you on building LLM apps, mastering prompt engineering, and developing chatbots with enterprise data.
LlamaIndex 4.8

Improving Real World RAG Systems: Key Challenges & Practical Solutions

Explore practical solutions, advanced retrieval strategies, and agentic RAG systems to improve context, relevance, and accuracy in AI-driven applications.
RAG systems using LlamaIndex 4.7

Microsoft Excel: Formulas & Functions

Learn Retrieval-Augmented Generation (RAG): learn how it works, the RAG framework, and use LlamaIndex for advanced systems.

Recommended Articles

Write for us

Write, captivate, and earn accolades and rewards for your work
  • Reach a Global Audience
  • Get Expert Feedback
  • Build Your Brand & Audience
  • Cash In on Your Knowledge
  • Join a Thriving Community
  • Level Up Your Data Science Game
imag
Rahul Shah img

Rahul Shah

27
Sion Chakrabarti img

Sion Chakrabarti

16
CHIRAG GOYAL img

CHIRAG GOYAL

87
kaustubh gupta img

kaustubh gupta

25
Barney Darlington img

Barney Darlington

5
Suvojit Hore img

Suvojit Hore

9
Arnab Mondal img

Arnab Mondal

15