This article was published as a part of the Data Science Blogathon
OpenCV is an open-source computer vision library that provides privileges to play with different images and video streams and also helps in end-to-end projects like object detection, face detection, object tracking, etc.
CVzone is a computer vision package that makes us easy to run like face detection, hand tracking, pose estimation, etc., and also image processing and other AI functions. At the core, it uses OpenCV and MediaPipe libraries. Check here for more information.
For many reasons, the background of the video needs to be modified as there are so many other interruptions in the background or the background colour doesn’t suit the person due to which background or the color needs to be modified. So, we use the real-time background replacement technique to substitute the backgrounds and add replace them with the desired content.
Image clipping path – This technique is used if the subject of the image has sharp edges. All those elements that fall outside the path will be eliminated.
Image cut-out – Here we cut the required region or subject in a frame and remove the background.
Image masking – If the images have frills or fine edges we can use image masking techniques.
Erasing the background – Erasing the background of an image using any different tools
Many famous applications use a background removal technique and replace it with a custom one. Here we are going to implement something similar, but using OpenCV and CVzone.
Install the required modules.
-- pip install OpenCV-python -- pip install cvzone -- pip install mediapipe
First, let us check if our webcam is working fine.
import cv2 cap = cv2.VideoCapture(0) cap.set(3, 640) cap.set(4, 480) while True: success, img = cap.read() cv2.imshow("Image", img) if cv2.waitKey(1) & 0xFF == ord('q'): break
The above code pops up a window if you have a webcam, Here the frame size is 640 X 480. So we need to take a note here because the background replacing images should be of the same size as the frame, that is 640 X 480.
Now create the folder inside the project directory here, I am creating a folder with the name ‘BackgroundImages’. You can download any images or any number of images and place them in this directory.
The project structure will look like the image given below:
Let us write a small piece of code in a separate python file to resize all the images in the folder ‘BackgroundImages’ to 640 X 480.
import cv2 import os for root, subdirs, files in os.walk('D:/pycharmprojects/BackgroundRemover/BackgroundImages'): for f in files: if f.endswith('jpg'): # print(f) img = cv2.imread('D:/pycharmprojects/BackgroundRemover/BackgroundImages/' + f) img = cv2.resize(img, (640, 480)) cv2.imwrite('D:/pycharmprojects/BackgroundRemover/BackgroundImages/'+f, img) print(*["Image", f, "is resized to 640 X 480"])
The above code will read the image (jpg) files in the specified folder and resize all the images to 640 X480 at once.
Now we are all set to implement the background replacement technique.
Import the required modules
import cv2 import cvzone from cvzone.SelfiSegmentationModule import SelfiSegmentation import os
Here in the above module, ‘SelfiSegmentation’ is used to remove the background of the frame and replace it with our images in the directory.
cap = cv2.VideoCapture(0) cap.set(3, 640) cap.set(4, 480) # cap.set(cv2.CAP_PROP_FPS, 60) segmentor = SelfiSegmentation() fpsReader = cvzone.FPS()
# imgBG = cv2.imread("BackgroundImages/3.jpg") listImg = os.listdir("BackgroundImages") imgList = [] for imgPath in listImg: img = cv2.imread(f'BackgroundImages/{imgPath}') imgList.append(img) indexImg = 0
In the above code, we take input from the webcam and also set the frame width to 640 X 480. Then we call SelfiSegmentation() and assign it to a variable called segmentor, and in order to display the frames per second(fps) in the output frames, we use cvzone.FPS() function.
Then we create a list of images present in the BackgroundImages folder and we loop through that list and read each and every image and append it to an empty list. The initial index is set to zero.
while True: success, img = cap.read() # imgOut = segmentor.removeBG(img, (255,0,255), threshold=0.83) imgOut = segmentor.removeBG(img, imgList[indexImg], threshold=0.8) imgStack = cvzone.stackImages([img, imgOut], 2,1) _, imgStack = fpsReader.update(imgStack) print(indexImg) cv2.imshow("image", imgStack) key = cv2.waitKey(1) if key == ord('a'): if indexImg>0: indexImg -=1 elif key == ord('d'): if indexImg<len(imgList)-1: indexImg +=1 elif key == ord('q'): break
Now the main part, inside a while loop reads the frames from the webcam, and then we use segmentor.removeBG() function to remove the background from the frames and replace it with our images in the directory. In the above code, you can see we have passed three parameters to segmentor.removeBG() function, that is image frame from webcam (img), then the list of images present in the directory along with an index of image (imgList[indexImg]) and finally the threshold. The threshold cuts everything if it’s set to 1, here we set it to 0.8, for better edges, play with different threshold values.
Then we stack the images using cvzone.stackImages, here we will get the output of the background replaced image or frames. Then using a simple if statement we assign keys to change the background. For example, if we have 10 background images, as per the above code we can use key “a” or key “d” to change the background of the frames.
The entire code is given below.
import cv2 import cvzone from cvzone.SelfiSegmentationModule import SelfiSegmentation import os cap = cv2.VideoCapture(0) cap.set(3, 640) cap.set(4, 480) # cap.set(cv2.CAP_PROP_FPS, 60) segmentor = SelfiSegmentation() fpsReader = cvzone.FPS() # imgBG = cv2.imread("BackgroundImages/3.jpg") listImg = os.listdir("BackgroundImages") imgList = [] for imgPath in listImg: img = cv2.imread(f'BackgroundImages/{imgPath}') imgList.append(img) indexImg = 0 while True: success, img = cap.read() # imgOut = segmentor.removeBG(img, (255,0,255), threshold=0.83) imgOut = segmentor.removeBG(img, imgList[indexImg], threshold=0.8) imgStack = cvzone.stackImages([img, imgOut], 2,1) _, imgStack = fpsReader.update(imgStack) print(indexImg) cv2.imshow("image", imgStack) key = cv2.waitKey(1) if key == ord('a'): if indexImg>0: indexImg -=1 elif key == ord('d'): if indexImg<len(imgList)-1: indexImg +=1 elif key == ord('q'): break
The output screenshots are given below
The entire code is also available here:
https://github.com/BakingBrains/Real-Time_Background_remover
Reference:
https://www.youtube.com/watch?v=k7cVPGpnels
My LinkedIn
Thank you
The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.