Netflix has presented a new model that demonstrates how AI can be used to remove objects from videos and edit footage accordingly.
As outlined in a new paper, Netflix’s Video Object and Interaction Deletion (VOID) is a vision-language model (VLM) designed to edit out specific elements from a video and update the footage to carry on as if no changes were made. As an example of this, Netflix showed images of two cars racing towards one another. In the original footage, the two naturally crash, but using VOID, the car on the right can be removed, while the left vehicle continues driving as if nothing happened.
While similar AI-powered tools exist, like Runway, Generative Omnimatte, and ProPainter, Netflix is trying to show how VOID is better. In the paper, it showed images comparing VOID to the aforementioned platforms. It also surveyed 25 people across five different scenarios and found that VOID was preferred 64.8 per cent of the time, compared to Runway in second with 18.4 per cent. (It’s unclear why only 25 people were surveyed.)
Other examples include removing a person from a video while preserving the form of an object they’re interacting with, like a guitar or a ball. Often, the object can become deformed once the human’s touch is removed, so Netflix is positioning VOID as a solution for that.
Of course, questions remain about this technology. For one, the images that Netflix shared are from videos in decidedly empty areas, like the aforementioned car crash sequence. It’s unclear how well VOID would work should such a scene be set in a more densely populated area, like a city.
And on a broader level, it begs the question of who would even want such a feature. Producers would no doubt see this as a cost-saving measure when it comes to reshoots and editing, but would filmmakers?
In any event, tech like VOID certainly isn’t going away anytime soon. At Netflix, specifically, the company recently spent a reported US$600 million to acquire InterPositive, Ben Affleck’s AI startup that offers proprietary tools to creatives to help them in the filmmaking process. Affleck has previously talked about the importance of using AI “responsibly” and putting filmmakers first, but it’s unclear how Netflix might use InterPositive going forward.
Source: Netflix Via: The Register
MobileSyrup may earn a commission from purchases made via our links, which helps fund the journalism we provide free on our website. These links do not influence our editorial content. Support us here.
