Google has Released the Deep Learning Model used to Create Pixel 2’s Portrait Mode
Overview
- Google has open sourced the code behind Pixel 2’s portrait mode
- Called DeepLab-V3+, it’s built on top of a convolutional neural network
- Works by assigning semantic labels and then categorizing and classifying objects
Introduction
We have covered Google’s releases quite a lot recently on AVBytes and that’s because they continue to open source their releases. They want the community to benefit from their resources.
Keeping that going, Google has now released the deep learning model it used to create the Portrait Mode in it’s signature Pixel 2 phone. It is based on semantic image segmentation.
The deep learning model assigns semantic labels to each pixel in the image. Then, categorization classifies the objects in the image, like animal, human, sky, road, etc. It basically recognises the foreground and background in that particular image.
The latter feature was used to develop the Pixel 2’s portrait mode for “shallow depth of field effects with only one physical lens”. It’s main use is in recognizing the outline of objects in the image, or being able to successfully determine where an object ends and the background begins.
The code open sourced by Google is named DeepLab -V3+. It’s an image segmentation tool built on top of a convolutional neural network (CNN).
Check out the underlying structure used by Google’s team to build the model:
This has been implemented in TensorFlow (of course, it’s Google!) and the release includes the training and evaluation code.
You can read Google’s official blog post about it here.
Our take on this
Modern image segmentation models built on top of CNNs are scaling never before seen accuracy levels. This latest release will again help the deep learning community understand and replicate (if not improve) what Google have done in the photography field.
Subscribe to AVBytes here to get regular data science, machine learning and AI updates in your inbox!
Also, go ahead and participate in our Hackathons, including the DataHack Premier League and Lord of the Machines!
Senior Editor at Analytics Vidhya. Data visualization practitioner who loves reading and delving deeper into the data science and machine learning arts. Always looking for new ways to improve processes using ML and AI.
