TensorFlow, GANs and AWS were Combined to Make a Movie and the Results are Stunning
Overview
- An AI was created that handles the entire movie production process – from generating dialogues to face swapping
- The AI was built using LSTM (Long Short Term Memory) recurrent neural network; this was built on the AWS platform
- The creators also used 11 different GANs for the face-swapping feature
Introduction
What happens when you apply artificial intelligence in the movie making business? The AI develops a creative streak of its own and delivers a virtuoso movie.
The latest effort to generate a movie using AI is by Oscar Sharp (director) and Ross Goodwin (AI researcher) for the ‘Sci-Fi London 48-Hour Challenge’. As the name suggests, the movie had to be produced inside 48 hours and adhere to specific guidelines. But what makes this effort unique is that the ENTIRE production effort was done by the AI.
Given the short turnaround time, there are awkward scenes (like a mustache appearing on the woman’s face, blurry facial expressions) and clunky and gibberish dialogue at times. But the overall effect is sublime. Without further ado, below is the movie in full:
The AI put the film together using thousands of hours of old movie footage. The film stars Silicon Valley actor Thomas Middleditch, who the AI portrayed in the movie. Using facial recognition, image processing and voice-generating technologies, the AI handled the entire movie making process.
How the technology works
As described in this article, the AI is built using a LSTM recurrent neural network on Amazon Web Services (AWS). The movies that were fed to the neural network were mostly from the 1980s and 90s. The AI, called Benjamin (it named itself!), learned to predict which letters and words followed each other. After a period of time, the AI managed to structure together a screenplay including the dialogues.
Face swapping is also prevalent throughout the movie. This technology has really come into the spotlight ever since a flurry of fake videos were published online last year. 11 GANs (Generative Adversarial Networks) were used to assist with the face-swapping feature. TensorFlow was also leveraged throughout the AI making process.
Our take on this
Ah, the beauty of combining creativity with artificial intelligence. While the end results are not perfect, they are a microcosm of what can be achieved if this technology was applied on a more diverse dataset and with far more time than just 48 hours.
The flip side of this is whether AI will force unemployment in the movie making business. Will we even need actors anymore if the AI can just cobble together a movie from old footage? Time will tell, but this is definitely something to ponder upon.
Subscribe to AVBytes here to get regular data science, machine learning and AI updates in your inbox!
Senior Editor at Analytics Vidhya. Data visualization practitioner who loves reading and delving deeper into the data science and machine learning arts. Always looking for new ways to improve processes using ML and AI.
