Google’s Latest AI Experiment is Fun, Free and Combines TensorFlow.js and Pose Estimation

Overview

• Google has launched a new experiment (free for everyone) that matches your pose with it’s catalog of 80,000 images
• Called ‘Move Mirror’, the experiment uses your computer’s webcam to estimate your pose; it uses TensorFlow.js and PoseNet to do this
• No privacy concerns, your image(s) will not be stored on any server as the experiment is purely browser based and doesn’t require you to upload anything

Introduction

Pose estimation is a hot research topic in machine learning these days. A few weeks back I covered how a Japanese firm is using it to detect shoplifters, and that’s just one example of how useful this technique can be.

Of course, Google has been heavily involved in pioneering this technique. They released an open-source version of PoseNet recently and have now used it, along with TensorFlow.js, to run a fun experiment, called ‘Move Mirror‘. It tracks your movements via your computer’s webcam and looks for images that match the poses you make.

Google has used over 80,000 images to build it’s catalog and it delves into each of these images to find the ones that match your pose. The result, especially if you’re constantly changing your pose, can be dizzying. The experiment shows your image on side of the screen, and the matching pose (from it’s collection of images) on the adjacent side. The above photos illustrate this point.

Pose estimation is, as the name suggests, the machine trying to predict what pose the human is in. It’s far more tricky than it sounds. In a three-dimensional space, people have all sorts of features that can be difficult to track – body shape, joints, etc. The task is made doubly difficult when there are other objects in the frame. The algorithm has to be able to distinguish between that and the human, in addition to doing the tracking task.

The experiment is done solely through the browser, thanks to TensorFlow.js. If you’re worried about privacy, Google claims it does not store your photos on it’s servers since everything is done in the browser itself.

Our take on this

A fun experiment! It makes a nice change from all the serious research going on that is being conducted globally. Trust Google to run funky and weird experiments like these to not only test out their research, but also keep people interested in machine learning.

TensorFlow.js and pose estimation are fields that I have been wanting to try out since they came into mainstream popularity and this has certainly piqued my interest. If you have used this technology before, or want to now, get in touch with me and we can definitely work on this together.

Subscribe to AVBytes here to get regular data science, machine learning and AI updates in your inbox!

Pranav Dar 07 May 2019

Senior Editor at Analytics Vidhya. Data visualization practitioner who loves reading and delving deeper into the data science and machine learning arts. Always looking for new ways to improve processes using ML and AI.