Â This article was published as a part of the Data Science Blogathon.

KNN stands for **K-Nearest Neighbors,** the **supervised machine learning** algorithm that can operate with both **classification** and **regression** tasks. KNN is often one of the hot topics in the interviews when the panel is interested in someone who can deal with **sparse** dataset situations or even **a curse of dimensionality** (we will discuss these things too) while working with the KNN model as creating a model is not a tedious task but dealing with its drawback and coming back with a solution is!

In this article, we will discuss some of the **most asked** and **tricky questions** so that one can answer not only specific questions but also cross-questions from the panel member. So let’s get started!

Suppose one is choosing KNN as their primary model. In that case, one needs to **have sufficient domain knowledge** of the problem statement he/she is working on, as the KNN algorithm can give us **a high-accuracy model,** but the same is not **human-readable**. Other than that, KNN can work accurately for **classification problems** where we need to find the data point (say X1) from two categories in the **sample space**.

Along with classification problems, KNN also **fits well with regression** tasks. Make sure that this model is not preferred when we have a way too **large a dataset** to deal with, as **KNN is a distance-based algorithm** which makes it **high in cost** when it comes to calculating the distance between two data points.

K-NN follows up a well-structured method to complete the assigned task, and I have tried to break it down into a few steps:

**Step-1:**The first step is to choose the**number of neighbors**i.e., the**K-variable,**which changes based on the requirements and different tasks**Step-2:**So, we already have selected the number of neighbors. Now we need to find the**Euclidean distance**of those neighbors.**Step 3:**After calculating the Euclidean distance between those points, choose the**nearest K neighbors**based on the previous calculation**Step-4:**Now count the**total number of data points**in both categories from the selected K-Neighbors**Step-5:**The last step is to give the new data points to those categories where the number of K-Neighbors is maximum.

First, we need to know what exactly is **K value** is; K is the numeric value that keeps the **count of nearest neighbors,** and we can’t have the hit, and trial method for multiple k values as the cost of **calculation is pretty expensive** hence we need to have certain **guidelines** to choose the optimal “K.”

- It is quite a
**domain-specific task**that also requires an**experience in a related field**to choose the optimal K-value for the different problem statements, widely the most preferred value for K is supposed to be**5**(not a hard-coded number). - If one is choosing a very
**small value for K**(say k=1,2) for reducing the cost of computation, then, it will lead to a**noisy model**which will surely be**prone to outliers**in the model. **Moderately large**values for K are preferred, but when it is too large, then, it will lead to**the underfitting**condition.

Choosing KNN over another classification algorithm solely depends on our requirements. Suppose we are working on a task that requires **flexibility** in the model then we can go for KNN, whereas if **efficiency** is the priority then we can go for other algorithms like **Gradient descent** or **Logistic Regression**.

**Note:** Above answer can backfire on another question, so be prepared for it: **How KNN is more flexible?**

The main logic behind KNN flexibility is that it is **a non-parametric** algorithm so it won’t have to make any **assumption** on the underlying dataset. Still, at the same time, it is expensive in computation, unlike other classification algorithms.

Both decision trees and KNN is non-parametric algorithms but are different in their way of delivering the results; some are as follows:

- When dealing with larger dataset, decision trees is
**faster**than KNN because of its**high computational**drawback (when distance is being calculated) - KNN is more
**accurate**than decision trees as it**scans the whole dataset**closely. - KNN is
**easy to implement**comparatively with decision trees.

For calculating distances in KNN, we have multiple options available like **Chi-square**, **Minkowsky**, **cosine similarity measure**, and so on. But **Euclidean distance** is a widely preferred method for calculating the distance as it returns us the **shortest distance** between two data points.

Before hitting the nail, let’s first understand which nail to hit i.e. what is normalization?

**Normalization** is the process where the whole dataset is scaled down **within a specific range,** mostly between **0 and 1**. This turned out to be a necessary step while dealing with the KNN algorithm as it is **a distance-based** algorithm, so if the data points are not within a specific range, then different magnitudes can **misclassify** the data points in the testing phase.

The curse of dimensionality is considered to be checked frequently while working with the KNN model, whereas the dimension tends to increase, then the data turns to be more **sparse,** i.e., we often **find tons of space** in the datasets, which leads to the state of **overfitting** also which makes algorithm incapable to find the nearest neighbors. Ideally, as the number of dimensions increases, the space in the dataset should also increase **exponentially** (both should positively complement each other).

**Linearization** is one of the best techniques to break the curse of dimensionality – This is a bonus point I’ll suggest having more understanding of this term.

There are a few conditions where KNN will not perform based on our expectations, listing them below:

- When the data is very
**noisy**or is not**linearly separable**. - KNN, at times, can be costly in terms of
**computations**for larger datasets. - KNN is least preferred when datasets have
**multiple dimensions,**as it leads to**a curse of dimensionality**.

Here we are in the last section of the article, by far we have discussed **the Top 9 questions** that are most frequently asked in an interview related to KNN, and this section will help you in **revision** and let you know what **things to cover** in brief regarding this hyper extensive algorithm.

- Firstly we saw
**where**the KNN algorithm is most likely to**be applicable**then we found out the**blueprint**of**how**it works in the background after that, we discussed the most interesting part related to KNN, i.e., the**K-value**and how to find the optimal K value. - In the next quarter of the article, we compared
**KNN**with**different algorithms,**like how it differs from another classification algorithm in terms of implementation. Also, we saw how KNN and**decision trees**are different species in terms of performance later; we also noted why**Euclidean distance**is preferred for calculating distance. - Last, we answered a few questions related to
**data normalization**and discussed why it is important. Then, we look at the hottest topic, i.e., the**curse of dimensionality**and the method to deal with it. Lastly, listed down some points where KNN can**underperform**

**The media shown in this article is not owned by Analytics Vidhya and is used at the Authorâ€™s discretion.**

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Become a full stack data scientist##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

Understanding Cost Function
Understanding Gradient Descent
Math Behind Gradient Descent
Assumptions of Linear Regression
Implement Linear Regression from Scratch
Train Linear Regression in Python
Implementing Linear Regression in R
Diagnosing Residual Plots in Linear Regression Models
Generalized Linear Models
Introduction to Logistic Regression
Odds Ratio
Implementing Logistic Regression from Scratch
Introduction to Scikit-learn in Python
Train Logistic Regression in python
Multiclass using Logistic Regression
How to use Multinomial and Ordinal Logistic Regression in R ?
Challenges with Linear Regression
Introduction to Regularisation
Implementing Regularisation
Ridge Regression
Lasso Regression

Introduction to Stacking
Implementing Stacking
Variants of Stacking
Implementing Variants of Stacking
Introduction to Blending
Bootstrap Sampling
Introduction to Random Sampling
Hyper-parameters of Random Forest
Implementing Random Forest
Out-of-Bag (OOB) Score in the Random Forest
IPL Team Win Prediction Project Using Machine Learning
Introduction to Boosting
Gradient Boosting Algorithm
Math behind GBM
Implementing GBM in python
Regularized Greedy Forests
Extreme Gradient Boosting
Implementing XGBM in python
Tuning Hyperparameters of XGBoost in Python
Implement XGBM in R/H2O
Adaptive Boosting
Implementing Adaptive Boosing
LightGBM
Implementing LightGBM in Python
Catboost
Implementing Catboost in Python

Introduction to Clustering
Applications of Clustering
Evaluation Metrics for Clustering
Understanding K-Means
Implementation of K-Means in Python
Implementation of K-Means in R
Choosing Right Value for K
Profiling Market Segments using K-Means Clustering
Hierarchical Clustering
Implementation of Hierarchial Clustering
DBSCAN
Defining Similarity between clusters
Build Better and Accurate Clusters with Gaussian Mixture Models

Introduction to Machine Learning Interpretability
Framework and Interpretable Models
model Agnostic Methods for Interpretability
Implementing Interpretable Model
Understanding SHAP
Out-of-Core ML
Introduction to Interpretable Machine Learning Models
Model Agnostic Methods for Interpretability
Game Theory & Shapley Values

Deploying Machine Learning Model using Streamlit
Deploying ML Models in Docker
Deploy Using Streamlit
Deploy on Heroku
Deploy Using Netlify
Introduction to Amazon Sagemaker
Setting up Amazon SageMaker
Using SageMaker Endpoint to Generate Inference
Deploy on Microsoft Azure Cloud
Introduction to Flask for Model
Deploying ML model using Flask