Generative AI filmmaker Dave Clark has tried “every tool under the sun” for making his artificially intelligent movies, most famously for his viral short “Battalion.” But even in the year since he released that video, the technology around Gen-AI has advanced considerably, and he believes that what was considered innovative back then can now be pushed even further.
On Tuesday, Google at its I/O event unveiled Flow, a generative-AI video generation tool powered by Google’s latest AI model, Veo 3. Clark was one of the AI filmmakers who collaborated with Google Labs on the functionality and interface of the new tool such that it was designed with filmmakers and creators in mind. Suffice it to say, he’s impressed by the results.
“My mind is still blown by the level of control,” Clark told IndieWire ahead of the launch of Flow. “This is for me as a filmmaker, this is what I’ve been waiting for. You’ve put in the 10,000 hours, you struggle through these prompts, and you use these different systems. But all we really hope for is that level of control that I feel like we finally got some sense of it, and it’s a long time coming for being so early in it.”
Flow has camera controls that allow filmmakers to specifically articulate the movement and visual angle they want to create for a shot as though they’re directing a cinematographer. It has a feature called Scenebuilder that allows filmmakers to generate a scene, and then extend it while keeping all of their character designs and locations consistent and intact. Creators can upload and combine multiple reference images that will match things like faces, clothing, and locations, doing so with hyper-specific detail.
Flow also allows for more natural language in prompt writing, such that it understands visual language of screenwriting and granular technical details like lens choices, lighting conditions, focal lengths, or film grain. It’s even capable of generating sound in real time with the video generation, including sound effects, background music, and dialogue.
Clark and his AI production banner Promise are using Flow to create a new AI-generated short called “Freelancers” about two adopted brothers who each, through separate paths, become international spies and hitmen. He described one scene he generated in which his two protagonists are sitting in a diner, complete with an establishing shot with a slow dolly in. He then prompted the model to use a dynamic, handheld camera motion, followed by a tracking shot of the two characters running.
“And it did it! To see the way that the camera was shaking, it’s this exact same way I would probably film it if I was in that restaurant. It was pretty cool, and I haven’t seen that level of [control],” Clark said.
It wasn’t that long ago that OpenAI unveiled its Sora video model; AI filmmakers we spoke to then were stunned that it had “prompt adherence,” where you can articulate one thing to happen and then have something else happen to that figure within the same sequence. Flow and Veo 3 demonstrated an even more advanced capability, in which a scene creators have generated can then be dragged and extended on a timeline to effectively continue the action and even lengthen the duration of an individual shot beyond the 6 – 8 seconds the model would normally generate.
Clark likened it to virtually being able to create his own “1917” style “oner” should he want to, with the action extending from location to location even as the character remains the same.
“Let’s say it’s a guy driving a car around the Amalfi Coast. With Scenebuilder inside Flow, the director in me was able to block out the rest of the scene,” Clark explained. “So maybe it starts off inside the car with the guy driving, then it’s POV, then I could switch to a wide shot, and the way Flow works is it actually knows all of the information that was put in, all the prompts, the way the character looks, the way the Amalfi Coast and the road looks, the color of the car. It’s able to translate that information into the next cut or scene or edit, if you will, and continue on that narrative.”
Clark said he got as specific as articulating the scene to look as though it was shot with a 50mm lens all the way up to a 135mm long lens, and Flow not only understands, but can then marry that work with specific blocking.
Matthieu Kim Lorrain, Creative Lead at Google DeepMind, and Thomas Iljic, Product Lead at Google Labs, spoke with IndieWire about understanding someone like Clark’s creative process in order to inform how Flow should operate, and they were stunned by what he and others were able to create with what they thought was, at the time, bare-bones functionality.
“Show and tell is important. They don’t want just to type text,” Iljic said. “They want to bring in these elements, these ingredients. They want consistency across shots. They want to have projects to start managing the structure, because it’s so many files, wondering how am I going to put this thing together?”
Lorrain added that the key word for AI filmmakers is “customization,” and five different AI filmmakers might have five different workflows, or even one filmmaker might have a different method for a different project, and that’s all built into Flow.
“Sometimes you need image to video because actually you want to start from a composition. You want to really curate the perfect image and then expand,” Lorrain said. “Sometimes it’s world-building. And in the case of world-building, that’s why we have the ingredients so you can define your characters, find your environment, and they want that. That’s the type of control. It’s not just control on how they can control the video, but also control on how they can design the creative process for each project.”
Clark’s Promise earlier today announced additional funding from Google’s AI Future Fund, as well as Crossbeam Venture Partners, Kivu Ventures, and Saga Ventures. But Google is also partnering with two other filmmakers on the launch of Flow, including Henry Daubrez and Junie Lau. Flow is also available now to subscribers of Google AI Pro and Google AI Ultra plans in the U.S., with more countries coming soon.
Clark isn’t sure what the specific next frontier is with the rapidly evolving technology, but he does believe that as the tools become more accessible, it’s incumbent on creators to at least understand it.
“I always reference the James Camerons and the George Lucases, some of the top filmmakers were always technologists, and I feel like we’re going to see that next level, that next school of George Lucas, that are going to be inherently technologists,” Clark said. “You’re going to have to understand technology, whether or not you use it.”