SketchAR Application Predicts what you’re Trying to Draw, using Computer Vision
Overview
- This application needs a rough figure to understand what the user drew
- It uses neural networks and computer vision to predict the object being drawn
- Check out the video below to learn how the application works
Introduction
We covered NVIDIA’s python library recently that makes being an artist so much easier. While that requires programming skills and is on a different level altogether, there’s an application that makes sketching and drawing a piece of cake.
The SketchAR team has developed an application that uses machine learning to help you become an artist. All you have to do, is draw a rough figure of the picture and blindly follow the instructions. The application gets the idea and takes care of the rest.
Here is a video for you to understand how it works.
I bet you’re wondering how it works! What if you don’t use a white sheet? How does it differentiate between your hand and the sketch?
The developers have provided a detailed explanation of how the algorithm works here. They mentioned that the application uses machine learning and neural networks to analyze and train data. The multiple layers (hand, pen, sheet) are separated, and the algorithm, using computer vision, teaches the camera to distinguish between everything it sees. Check out the below series of images to get a general idea of the underlying process:
This provides a clear separation of the surface and other objects, enabling it to focus only on the drawn content. When a user starts to draw, the drawing is mapped to the original picture, placing a virtual object on the surface that the user can trace.
Our take on this
This isn’t the first attempt at using machine learning to predict what the user is drawing. Back in 2017, Google introduced AutoDraw which used machine learning algorithms to match your attempts to a professional art. An example of this is below:
And now Machine Learning is here to teach the newbies to draw, providing a canvas to the non-artists. It is incredible how researchers are using machine learning to influence artworks. Soon there will be a new generation of digital artists.
Few years down the line, we can imagine computers bring able to draw like Van Gogh and Picasso!
Subscribe to AVBytes here to get regular data science, machine learning and AI updates in your inbox!
An avid reader and blogger who loves exploring the endless world of data science and artificial intelligence. Fascinated by the limitless applications of ML and AI; eager to learn and discover the depths of data science.
