What is Hierarchical Clustering in Python?

Pulkit Sharma 24 May, 2024 • 17 min read

Introduction

In the vast landscape of data exploration, where datasets sprawl like forests, hierarchical clustering acts as a guiding light, leading us through the dense thicket of information. Imagine a dendrogram, a visual representation of data relationships, branching out like a tree, revealing clusters and connections within the data. This is where machine learning meets the art of clustering, where Python serves as the wizard’s wand, casting spells of insight into the heart of datasets.

In this journey through the Python kingdom, we will unravel the mysteries of hierarchical clustering, exploring its intricacies and applications in data science. From dendrograms to distance matrices, from agglomerative to divisive clustering, we will delve deep into the techniques and methods that make hierarchical clustering a cornerstone of data analysis.

Join us as we embark on this adventure, where data points become nodes in a vast knowledge network, and clusters emerge like constellations in the night sky, guiding us toward the insights hidden within the data. Welcome to the world of hierarchical clustering in Python, where every cluster tells a story, and every dendrogram holds the key to unlocking the secrets of data science.

Study Material

Quiz Time

Welcome to the quiz on Hierarchical Clustering! Test your knowledge about this clustering technique and its key concepts.

What is Hierarchical Clustering?

Hierarchical clustering is an unsupervised learning technique for grouping similar objects into clusters. It creates a hierarchy of clusters by merging or splitting them based on similarity measures. It uses a bottom-up approach or top-down approach to construct a hierarchical data clustering schema.

Clustering Hierarchical groups similar objects into a dendrogram. It merges similar clusters iteratively, starting with each data point as a separate cluster. This creates a tree-like structure that shows the relationships between clusters and their hierarchy.

The dendrogram from hierarchical clustering reveals the hierarchy of clusters at different levels, highlighting natural groupings in the data. It provides a visual representation of the relationships between clusters, helping to identify patterns and outliers, making it a valuable tool for exploratory data analysis. For example, let’s say we have the below points, and we want to cluster them into groups:

initial points ,Hierarchical clustering

We can assign each of these points to a separate cluster:

multiple clusters

Now, based on the similarity of these clusters, we can combine the most similar clusters together and repeat this process until only a single cluster is left:

single cluster,Hierarchical clustering

We are essentially building a hierarchy of clusters. That’s why this algorithm is called hierarchical clustering. I will discuss how to decide the number of clusters later. For now, let’s look at the different types of hierarchical clustering.

Also Read: Python Interview Questions to Ace Your Next Job Interview in 2024

Types of Hierarchical Clustering

There are mainly two types of hierarchical clustering:

  • Agglomerative hierarchical clustering
  • Divisive Hierarchical clustering

Let’s understand each type in detail.

Agglomerative Clustering Hierarchical

We assign each point to an individual cluster in this technique. Suppose there are 4 data points. We will assign each of these points to a cluster and hence will have 4 clusters in the beginning:

multiple clusters

Then, at each iteration, we merge the closest pair of clusters and repeat this step until only a single cluster is left:

agglomerative clustering

We are merging (or adding) the clusters at each step, right? Hence, this type of clustering is also known as additive hierarchical clustering.

Divisive Hierarchical Clustering

Divisive Clustering Hierarchical works in the opposite way. Instead of starting with n clusters (in case of n observations), we start with a single cluster and assign all the points to that cluster.

So, it doesn’t matter if we have 10 or 1000 data points. All these points will belong to the same cluster at the beginning:

single cluster,Divisive Hierarchical clustering

Now, at each iteration, we split the farthest point in the cluster and repeat this process until each cluster only contains a single point:

multiple clusters

We are splitting (or dividing) the clusters at each step, hence the name divisive hierarchical clustering.

Agglomerative Clustering is widely used in the industry and will be the article’s focus. Divisive hierarchical clustering will be a piece of cake once we have a handle on the agglomerative type

Also Read: Python Tutorial to Learn Data Science from Scratch

Applications of Hierarchical Clustering

Here are some common applications of hierarchical clustering:

  1. Biological Taxonomy: Hierarchical clustering is extensively used in biology to classify organisms into hierarchical taxonomies based on similarities in genetic or phenotypic characteristics. It helps understand evolutionary relationships and biodiversity.
  2. Document Clustering: In natural language processing, hierarchical clustering groups similar documents or texts. It aids in topic modeling, document organization, and information retrieval systems.
  3. Image Segmentation: Hierarchical clustering segments images by grouping similar pixels or regions based on color, texture, or other visual features. It finds applications in medical imaging, remote sensing, and computer vision.
  4. Customer Segmentation: Businesses use hierarchical clustering to group customers into groups based on their purchasing behaviors, demographics, or preferences. This helps with targeted marketing, personalized recommendations, and customer relationship management.
  5. Anomaly Detection: Hierarchical clustering can identify outliers or anomalies in datasets by isolating data points that do not fit well into any cluster. It is useful in fraud detection, network security, and quality control.
  6. Social Network Analysis: Hierarchical clustering helps uncover community structures or hierarchical relationships in social networks by clustering users based on their interactions, interests, or affiliations. It aids in understanding network dynamics and identifying influential users.
  7. Market Basket Analysis: Retailers use hierarchical clustering to analyze transaction data and identify associations between products frequently purchased together. It enables them to optimize product placements, promotions, and cross-selling strategies.
  8. Genomic Data Analysis: In bioinformatics, hierarchical clustering analyzes gene expression data, DNA sequences, or protein interactions. It helps identify gene expression patterns, classify diseases, and understand molecular interactions.
  9. Ecological Modeling: Hierarchical clustering assists ecologists in identifying ecological communities or species assemblages based on similarities in environmental factors, species composition, or habitat characteristics. It aids in biodiversity conservation and ecosystem management.
  10. Stock Market Analysis: Financial analysts apply hierarchical clustering to group stocks based on their price movements, trading volumes, or financial indicators. It helps in portfolio diversification, risk management, and investment strategy formulation.

Advantages and Disadvantages of Hierarchical Clustering

Here are some advantages and disadvantages of hierarchical clustering:

Advantages of hierarchical clustering:

  1. Easy to interpret: Hierarchical clustering produces a dendrogram, a tree-like structure that shows the order in which clusters are merged. This dendrogram provides a clear visualization of the relationships between clusters, making it easy to interpret the results.
  2. No need to specify the number of clusters: Unlike other clustering algorithms, such as k-means, hierarchical clustering does not require you to specify the number of clusters beforehand. The algorithm determines the number of clusters based on the data and the chosen linkage method.
  3. Captures nested clusters: Hierarchical clustering captures the hierarchical structure in the data, meaning it can identify clusters within clusters (nested clusters). This can be useful when the data naturally forms a hierarchy.
  4. Robust to noise: Hierarchical clustering is robust to noise and outliers because it considers the entire dataset when forming clusters. Outliers may not significantly affect the clustering process, especially if a suitable distance metric and linkage method are chosen.

Disadvantages of hierarchical clustering:

  1. Computational complexity: Hierarchical clustering can be computationally expensive, especially for large datasets. The time complexity of hierarchical clustering algorithms is typically 𝑂(𝑛2log⁡𝑛)O(n2logn) or 𝑂(𝑛3)O(n3), where 𝑛n is the number of data points.
  2. Memory usage: Besides computational complexity, hierarchical clustering algorithms can consume much memory, particularly when dealing large datasets. Storing the entire distance matrix between data points can require substantial memory.
  3. Difficulty with large datasets: Due to its computational complexity and memory requirements, hierarchical clustering may not be suitable for large datasets. In such cases, alternative clustering methods, such as k-means or DBSCAN, may be more appropriate.
  4. Sensitive to noise and outliers: While hierarchical clustering is generally robust to noise and outliers, extreme outliers or noise points can still affect the clustering results, especially if they are not handled properly beforehand.
  5. Difficulty in merging clusters: Once clusters are formed in hierarchical clustering, merging or splitting them can be difficult, especially if the clustering uses a divisive method. This lack of flexibility can be a limitation in certain scenarios where cluster adjustments are needed.

Application of Hierarchical Clustering with Python

In Python, the scipy and scikit-learn libraries are often used to perform hierarchical clustering. Here’s how you can apply hierarchical clustering using Python:

  1. Import Necessary Libraries: First, you’ll need to import the necessary libraries: numpy for numerical operations, matplotlib for plotting, and scipy.cluster.hierarchy for hierarchical clustering.
  2. Generate or Load Data: You can either generate a synthetic dataset or load your dataset.
  3. Compute the Distance Matrix: Compute the distance matrix which will be used to form clusters.
  4. Perform Hierarchical Clustering: Use the linkage method to perform hierarchical clustering.
  5. Plot the Dendrogram: Visualize the clusters using a dendrogram.

Here’s an example of hierarchical clustering using Python:

import numpy as np

import matplotlib.pyplot as plt

from scipy.cluster.hierarchy import dendrogram, linkage

from scipy.cluster.hierarchy import fcluster

from sklearn.datasets import make_blobs

# Generate sample data

X, y = make_blobs(n_samples=100, centers=3, cluster_std=0.60, random_state=0)

# Compute the linkage matrix

Z = linkage(X, 'ward')

# Plot the dendrogram

plt.figure(figsize=(10, 7))

plt.title("Dendrogram")

plt.xlabel("Sample index")

plt.ylabel("Distance")

dendrogram(Z)

plt.show()

# Determine the clusters

max_d = 7.0 # this can be adjusted based on the dendrogram

clusters = fcluster(Z, max_d, criterion='distance')

# Plot the clusters

plt.figure(figsize=(10, 7))

plt.scatter(X[:, 0], X[:, 1], c=clusters, cmap='prism')

plt.title("Hierarchical Clustering")

plt.xlabel("Feature 1")

plt.ylabel("Feature 2")

plt.show()

Supervised vs Unsupervised Learning

Understanding the difference between supervised and unsupervised learning is important before we dive into the Clustering hierarchy. Let me explain this difference using a simple example.

Suppose we want to estimate the count of bikes that will be rented in a city every day:

regression problem

Or, let’s say we want to predict whether a person on board the Titanic survived or not:

classification problem

Examples

  • In the first example, we have to predict the number of bikes based on features like the season, holiday, working day, weather, temperature, etc.
  • In the second example, we are predicting whether a passenger survived. In th’ ‘Surviv’d’ variable, 0 represents that the person did not survive, and 1 means the person did make it out alive. The independent variables here include Pclass, Sex, Age, Fare, etc.

Let’s look at the figure below to understand this visually:

supervised learning
unsupervised learning,Independent variables and dependent variable

Here, y is our dependent or target variable, and X represents the independent variables. The target variable is dependent on X, also called a dependent variable. We train our model using the independent variables to supervise the target variable. Hence, the name supervised learning.

When training the model, we aim to generate a function that maps the independent variables to the desired target. Once the model is trained, we can pass new sets of observations, and the model will predict their target. This, in a nutshell, is supervised learning.

In these cases, we try to divide the entire data into a set of groups. These groups are known as clusters, and the process of making them is known as clustering.

clustering

This technique is generally used for clustering a population into different groups. A few common examples include segmenting customers, clustering similar documents, recommending similar songs or movies, etc.

There are many more applications of unsupervised learning. If you come across any interesting ones, feel free to share them in the comments section below!

Various algorithms help us make these clusters. The most commonly used clustering algorithms are K-means and Hierarchical clustering

Why Hierarchical Clustering?

We should first know how K-means works before we dive into hierarchical clustering. Trust me, it will make the concept of hierarchical clustering much easier.

Here’s a brief overview of how K-means works:

  • Decide the number of clusters (k)
  • Select k random points from the data as centroids
  • Assign all the points to the nearest cluster centroid
  • Calculate the centroid of newly formed clusters
  • Repeat steps 3 and 4

It is an iterative process. It will keep on running until the centroids of newly formed clusters do not change or the maximum number of iterations are reached.

But there are certain challenges with K-means. It always tries to make clusters of the same size. Also, we have to decide the number of clusters at the beginning of the algorithm. Ideally, we would not know how many clusters should we have, in the beginning of the algorithm and hence it a challenge with K-means.

This is a gap hierarchical clustering bridges with aplomb. It takes away the problem of having to pre-define the number of clusters. Sounds like a dream! So, let’s see what hierarchical clustering is and how it improves on K-means.

How Does Hierarchical Clustering Improve on K-means?

Hierarchical clustering and K-means are both popular clustering algorithms, but they have different strengths and weaknesses. Here are some ways in which hierarchical clustering can improve on K-means:

1. No Need to Pre-specify Number of Clusters

Hierarchical Clustering:

  • Does not require the number of clusters (k) to be specified in advance.
  • The dendrogram provides a visual representation of the hierarchy of clusters, and the number of clusters can be determined by cutting the dendrogram at a desired level.

K-means:

  • Requires the number of clusters (k) to be specified beforehand, which can be difficult if the optimal number of clusters is unknown.

2. Captures Nested Clusters

Hierarchical Clustering:

  • Can identify nested clusters, which means it can find clusters within clusters.
  • This is useful for datasets with a natural hierarchical structure (e.g., taxonomy of biological species).

K-means:

  • Assumes clusters are flat and does not capture hierarchical relationships.

3. Flexibility with Cluster Shapes

Hierarchical Clustering:

  • Can find clusters of arbitrary shapes.
  • The algorithm is not restricted to spherical clusters and can capture more complex cluster structures.

K-means:

  • Assumes clusters are spherical and of similar size, which may not be suitable for datasets with irregularly shaped clusters.

4. Distance Metrics and Linkage Criteria

Hierarchical Clustering:

  • Offers flexibility in the choice of distance metrics (e.g., Euclidean, Manhattan) and linkage criteria (e.g., single, complete, average).
  • This flexibility can improve clustering performance on different types of data.

K-means:

  • Typically uses the Euclidean distance, which may not be suitable for all types of data.

5. Handling Outliers

Hierarchical Clustering:

  • Outliers can be identified as singleton clusters at the bottom of the dendrogram.
  • This makes it easier to detect and potentially remove outliers.

K-means:

  • Sensitive to outliers, as they can significantly affect the position of cluster centroids.

6. Robustness to Initialization

Hierarchical Clustering:

  • Does not require random initialization of cluster centroids.
  • The clustering result is deterministic and does not depend on initial conditions.

K-means:

  • Requires random initialization of centroids, which can lead to different clustering results in different runs.
  • The algorithm may converge to local minima, depending on the initial placement of centroids.

7. Visual Interpretation

Hierarchical Clustering:

  • The dendrogram provides a visual and interpretable representation of the clustering process.
  • It helps in understanding the relationships between clusters and the structure of the data.

K-means:

  • Provides cluster labels and centroids, but does not offer a visual representation of the clustering process.

Practical Example

Let’s consider a practical example using hierarchical clustering and K-means on a simple dataset:

import numpy as np

import matplotlib.pyplot as plt

from scipy.cluster.hierarchy import dendrogram, linkage

from sklearn.datasets import make_blobs

from sklearn.cluster import KMeans

# Generate sample data

X, y = make_blobs(n_samples=100, centers=3, cluster_std=0.60, random_state=0)

# Hierarchical Clustering

Z = linkage(X, 'ward')

plt.figure(figsize=(10, 7))

plt.title("Hierarchical Clustering Dendrogram")

dendrogram(Z)

plt.show()

# K-means Clustering

kmeans = KMeans(n_clusters=3, random_state=0).fit(X)

labels = kmeans.labels_

plt.figure(figsize=(10, 7))

plt.scatter(X[:, 0], X[:, 1], c=labels, cmap='prism')

plt.title("K-means Clustering")

plt.show()

Steps to Perform Hierarchical Clustering

We merge the most similar points or clusters in hierarchical clustering – we know this. Now the question is – how do we decide which points are similar and which are not? It’s one of the most important questions in clustering!

Here’s one way to calculate similarity – Take the distance between the centroids of these clusters. The points having the least distance are referred to as similar points and we can merge them. We can refer to this as a distance-based algorithm as well (since we are calculating the distances between the clusters).

In hierarchical clustering, we have a concept called a proximity matrix. This stores the distances between each point. Let’s take an example to understand this matrix and the steps to perform hierarchical clustering.

Setting up the Example

Steps to perform hierarchical clustering

Suppose a teacher wants to divide her students into different groups. She has the marks scored by each student in an assignment and based on these marks, she wants to segment them into groups. There’s no fixed target here as to how many groups to have. Since the teacher does not know what type of students should be assigned to which group, it cannot be solved as a supervised learning problem. So, we will try to apply hierarchical clustering here and segment the students into different groups.

Let’s take a sample of 5 students:

sample student data

Creating a Proximity Matrix

First, we will create a proximity matrix which will tell us the distance between each of these points. Since we are calculating the distance of each point from each of the other points, we will get a square matrix of shape n X n (where n is the number of observations).

Let’s make the 5 x 5 proximity matrix for our example:

proximity matrix

The diagonal elements of this matrix will always be 0 as the distance of a point with itself is always 0. We will use the Euclidean distance formula to calculate the rest of the distances. So, let’s say we want to calculate the distance between point 1 and 2:

√(10-7)^2 = √9 = 3

Similarly, we can calculate all the distances and fill the proximity matrix.

Steps to Perform Hierarchical Clustering

Step 1: First, we assign all the points to an individual cluster:

proximity matrix

Different colors here represent different clusters. You can see that we have 5 different clusters for the 5 points in our data.

Step 2: Next, we will look at the smallest distance in the proximity matrix and merge the points with the smallest distance. We then update the proximity matrix:

smallest distance

Here, the smallest distance is 3 and hence we will merge point 1 and 2:

step 2 : hierarchical

Let’s look at the updated clusters and accordingly update the proximity matrix:

updated data

Here, we have taken the maximum of the two marks (7, 10) to replace the marks for this cluster. Instead of the maximum, we can also take the minimum value or the average values as well. Now, we will again calculate the proximity matrix for these clusters:

updated proximity matrix

Step 3: We will repeat step 2 until only a single cluster is left.

So, we will first look at the minimum distance in the proximity matrix and then merge the closest pair of clusters. We will get the merged clusters as shown below after repeating these steps:

final hierarchical clustering

We started with 5 clusters and finally have a single cluster. This is how agglomerative hierarchical clustering works. But the burning question still remains – how do we decide the number of clusters? Let’s understand that in the next section.

How to Choose the Number of Clusters in Hierarchical Clustering?

Ready to finally answer this question that’s been hanging around since we started learning? To get the number of clusters for hierarchical clustering, we make use of an awesome concept called a Dendrogram.

A dendrogram is a tree-like diagram that records the sequences of merges or splits.

Example

Let’s get back to teacher-student example. Whenever we merge two clusters, a dendrogram will record the distance between these clusters and represent it in graph form. Let’s see how a dendrogram looks:

dendrogram

We have the samples of the dataset on the x-axis and the distance on the y-axis. Whenever two clusters are merged, we will join them in this dendrogram and the height of the join will be the distance between these points. Let’s build the dendrogram for our example:

final hierarchical clustering

Take a moment to process the above image. We started by merging sample 1 and 2 and the distance between these two samples was 3 (refer to the first proximity matrix in the previous section). Let’s plot this in the dendrogram:

dendrogram

Here, we can see that we have merged sample 1 and 2. The vertical line represents the distance between these samples. Similarly, we plot all the steps where we merged the clusters and finally, we get a dendrogram like this:

final dendrogram

We can visualize the steps of hierarchical clustering. More the distance of the vertical lines in the dendrogram, more the distance between those clusters.

Now, we can set a threshold distance and draw a horizontal line (Generally, we try to set the threshold so that it cuts the tallest vertical line). Let’s set this threshold as 12 and draw a horizontal line:

dendrogram threshold

The number of clusters will be the number of vertical lines intersected by the line drawn using the threshold. In the above example, since the red line intersects 2 vertical lines, we will have 2 clusters. One cluster will have a sample (1,2,4) and the other will have a sample (3,5).

This is how we can decide the number of clusters using a dendrogram in Hierarchical Clustering. In the next section, we will implement hierarchical clustering to help you understand all the concepts we have learned in this article.

Solving the Wholesale Customer Segmentation Problem using Hierarchical Clustering

Time to get our hands dirty in Python!

We will be working on a wholesale customer segmentation problem. You can download the dataset using this link. The data is hosted on the UCI Machine Learning repository. The aim of this problem is to segment the clients of a wholesale distributor based on their annual spending on diverse product categories, like milk, grocery, region, etc.

Let’s explore the data first and then apply Hierarchical Clustering to segment the clients.

Required Libraries

Load the data and look at the first few rows:

Python Code

customer segmentation dataset

There are multiple product categories – Fresh, Milk, Grocery, etc. The values represent the number of units purchased by each client for each product. Our aim is to make clusters from this data that can segment similar clients together. We will, of course, use Hierarchical Clustering for this problem.

But before applying Hierarchical Clustering, we have to normalize the data so that the scale of each variable is the same. Why is this important? Well, if the scale of the variables is not the same, the model might become biased towards the variables with a higher magnitude like Fresh or Milk (refer to the above table).

So, let’s first normalize the data and bring all the variables to the same scale:

scaled_data

Here, we can see that the scale of all the variables is almost similar. Now, we are good to go. Let’s first draw the dendrogram to help us decide the number of clusters for this particular problem:

dendrogram


The x-axis contains the samples and y-axis represents the distance between these samples. The vertical line with maximum distance is the blue line and hence we can decide a threshold of 6 and cut the dendrogram:

dendrogram with threshold

We have two clusters as this line cuts the dendrogram at two points. Let’s now apply hierarchical clustering for 2 clusters:

clustering predictions

We can see the values of 0s and 1s in the output since we defined 2 clusters. 0 represents the points that belong to the first cluster and 1 represents points in the second cluster. Let’s now visualize the two clusters:

cluster visualization

Awesome! We can visualize the two clusters here. This is how we can implement hierarchical clustering in Python.

Conclusion

In our journey through the intricate realm of hierarchical clustering in Python, we’ve uncovered a powerful tool for unraveling the complexities of data relationships. From the conceptual elegance of dendrograms to their practical applications in diverse fields like biology, document analysis, cluster analysis, and customer segmentation, hierarchical cluster analysis emerges as a guiding light in the labyrinth of data exploration.

As we conclude this expedition, we stand at the threshold of possibility, where every cluster tells a story, and every dendrogram holds the key to unlocking the secrets of data science. In the ever-expanding landscape of Python and machine learning, hierarchical clustering stands as a stalwart companion, guiding us toward new horizons of discovery and understanding.

If you are still relatively new to data science, I highly recommend taking the Applied Machine Learning course. It is one of the most comprehensive end-to-end machine learning courses you will find anywhere. Hierarchical clustering is just one of the diverse topics we cover in the course.

What are your thoughts on hierarchical clustering? Do you feel there’s a better way to create clusters using less computational resources? Connect with me in the comments section below, and let’s discuss!

Frequently Asked Questions?

Q1. What is hierarchical K clustering?

A. Hierarchical K clustering is a method of partitioning data into K clusters where each cluster contains similar data points, and these clusters are organized in a hierarchical structure.

Q2. What is an example of a hierarchical cluster?

A. An example of a hierarchical cluster could be grouping customers based on their purchasing behavior, where clusters are formed based on similarities in purchasing patterns, leading to a hierarchical tree-like structure.

Q3. What are the two methods of hierarchical clustering?

A. The two methods of hierarchical clustering are:
1. Agglomerative hierarchical clustering: It starts with each data point as a separate cluster and merges the closest clusters together until only one cluster remains.
2. Divisive hierarchical clustering: It begins with all data points in one cluster and recursively splits the clusters into smaller ones until each data point is in its own cluster.

Q4. What is hierarchical clustering of features?

A. Hierarchical clustering of features involves clustering features or variables instead of data points. It identifies groups of similar features based on their characteristics, enabling dimensionality reduction or revealing underlying patterns in the data.

Pulkit Sharma 24 May 2024

My research interests lies in the field of Machine Learning and Deep Learning. Possess an enthusiasm for learning new skills and technologies.

Frequently Asked Questions

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Responses From Readers

Clear

Prabhakar Krishnamurthy
Prabhakar Krishnamurthy 27 May, 2019

Dear Pulakit Sharma, Thank you very much for your article. Is it possible to use R to get the same output. Your reply will be much appreciated.

Srinivas
Srinivas 27 May, 2019

Hi Thanks for sharing very informative and useful. I have one question once we cluster the data how do find the factor or variable which differentiate the cluster. For eg if age is a factor then all the cluster should have the different age bins this way we can say that age is a factor which differentiate the clusters. My question how do we find when we have n variables ? Thanks

Piotr Lorenc
Piotr Lorenc 27 May, 2019

Really well explained for beginners. As already mentioned in article clustering is very challenging for data analyst and always requires plenty of expertise

MD
MD 27 May, 2019

Hi Pulkit, I really applaud your efforts to effectively communicate the concepts of Machine Learning with visualisations. I am not a ML practitioner but I am a student and I have recently studied these subjects recently. I would like to add few points to the above article. Firstly, I think the scaling operation that you have performed has been done on the categorical variables. I dont think categorical variables need to undergo the scaling and transformation(normalisation). The above normalised data for region and channel do not make any sense. Secondly, the drawbacks of hierarchical clustering have not been posted here. The drawbacks of Hierarchical clustering is that they do not perform well with large datasets. I hope my inputs are helpful to you. Regards, MD

Anna
Anna 28 May, 2019

Amazing post, thanks for sharing.

Kumar Saurabh
Kumar Saurabh 06 Jun, 2019

can you please share the python code to do this clustering . what u have said is theoretical . How to do it in python notebook ???

Punaravasu
Punaravasu 22 Jul, 2019

Please explain how to perform clustering if the number of variables are more than 20.

Sree
Sree 16 Aug, 2019

Awesome! We can clearly visualize the two clusters here, in reality, we have to get the nest cluster out so that we can target this group. None of the articles or training shows that. All end up with visualizations.

Mary
Mary 29 Aug, 2019

Hi, thanks for this article, I still can't find the code. Can you please share that link for the code here.

Ankur Madharia
Ankur Madharia 16 Sep, 2019

How to do this for categorical variables?

Pierina Galvez
Pierina Galvez 03 Nov, 2019

Hi! I find it really interesting, I'll try to apply it with documents to cluster them.

Mauro
Mauro 05 Nov, 2019

Hi, great job. I only have binary values ​​for my variables, what can I do?

Michael Shparber
Michael Shparber 26 Nov, 2019

Dear Pulkit, Thank you very much for making this easy-to-follow demo! My question is - how do I append the calculated cluster labels to my original 'data' DataFrame? So I know which row (customer) belongs to each cluster. Thank you

Michael Shparber
Michael Shparber 26 Nov, 2019

Lets say I want to cut at 2.7, so I have 7 clusters. How will the clusters be numbered? Does that mean that (2,3) will be "closer" and (4,5) also will be closer, also (6,7) and (1) will be closer to (2,3)? How can I make sure that the numbering correctly corresponds to the hierarchy shown in the dendrogram? Thanks! Michael

Michael Shparber
Michael Shparber 04 Dec, 2019

Another question: Since the dendrogram linkage already shows the connections between the data - it must be calculating the Euclidean distances... So it is ACTUALLY doing the clustering already, not just the visualization... So why should I run AgglomerativeClustering at all? Can't I simply somehow output the connections that the dendrogram found - as data, not visual? And why AgglomerativeClustering result has to be consistent with dendrogram's? Is it the SAME algorithm? It feels to me that we are using two clusterings here... Thanks!

Elissa
Elissa 06 Dec, 2019

I am working on school project, vessel prediction from AIS data. the data corresponds to an observation of a single maritime vessel at a single point in time(more like a position report). We have to track the movements of the different vessels given these reports over time. Do you think Hierarchical clustering is the best choice for this problem? How can we find the minimum distance between each cluster? In your example you decided to take the maximum distance which was the blue line in the dendrogram. I followed your steps on the data I have and I feel like I need a way to figure out the min distance between the clusters not the max; cause I will always get 2 clusters which is not ideal here!! Thanks!

hazlan
hazlan 11 Dec, 2019

hi Pulkit, how you choose the x and y variable for plt.scatter?in your example you choose 'milk' as x and 'grocery' as y. in my case i have many variable. when i choose different x and y its give me different graph..and looks like the clustering not clear see as a group. thanks hazlan

Jonathan
Jonathan 05 Jan, 2020

Really good write-up!

Trinath
Trinath 30 Nov, 2021

How dendograms caluculates distance/linkage between two samples(having more than 2 features like region, fresh, milk, grocery). How does it do ?

Zaheer
Zaheer 01 Dec, 2021

I found the article very useful but the calculation part of the distance matrix has been done correctly. The concept was clear and I found the distance matrix has two same values[(1,2), (5,3)] at the initial. I don't know why you avoided this basic math in the proximity matrix that too at the very beginning of the description.

sheelz
sheelz 04 Dec, 2021

I really enjoyed your write-up!. I have few questions. I'm working on a fortune500 dataset. 1) How do you know when a problem is a prediction problem or an Interpretation problem? 2) Why did we have to cut the dendrogram at the threshold of the blue line? Why can't we just use the whole number of cluster? I'm a novice! I will really appreciate your explanations.

sheelz
sheelz 04 Dec, 2021

I really enjoyed your write-up!. I have few questions. I'm working on a fortune500 dataset. 1) How do you know when a problem is a prediction problem or an Interpretation problem? 2) Why did we have to cut the dendrogram at the threshold of the blue line? Why can't we just use the whole number of cluster? I'm a novice! I will really appreciate your explanations.

Jandy
Jandy 25 Feb, 2022

Dear Pulakit Sharma, Thank you very much for your article.I have a technical question, how should I do hierarchical clustering if I use my own obtained distance matrix? Also, how do I locate the time series I use into the original data after clustering?

Abubakar
Abubakar 26 Feb, 2022

Very interesting. Thank you

Harsha Reddy
Harsha Reddy 09 Feb, 2023

hello sir how did you label the "milk" and "Grocery" in the last step .

Abhinav
Abhinav 01 May, 2023

When calculating distance, you are just calculating the square root of a square. Is this the correct method?

Related Courses

image.name
0 Hrs 70 Lessons
5

Introduction to Python

Free

Machine Learning
Become a full stack data scientist