IBL News | New York
Stability AI released in research preview mode this week Stable Virtual Camera, a model that transforms 2D images into immersive 3D scenes with realistic depth and perspective.
With its tool, Stability is adding generative AI to virtual cameras, often used in digital filmmaking and 3D animation to capture and navigate scenes in real-time.
The model is available for research use under a noncommercial license. It can be downloaded on Hugging Face, and the code is accessible on GitHub. The full research paper is here.
“We invite the research community to explore its capabilities and contribute to its development,” said the company.
The model can generate videos that travel along “dynamic” camera paths or presets, including “Spiral,” “Dolly Zoom,” “Move,” and “Pan.”
In its initial version, Stable Virtual Camera may produce lower-quality results in certain scenarios, admitted Stability AI.
The current version generates videos in square (1:1), portrait (9:16), and landscape (16:9) aspect ratios up to 1,000 frames in length.
“Input images featuring humans, animals, or dynamic textures like water often lead to degraded outputs.”
“Additionally, highly ambiguous scenes, complex camera paths that intersect objects or surfaces, and irregularly shaped objects can cause flickering artifacts, especially when target viewpoints differ significantly from the input images.”
Stability, the firm behind the popular image-generation model Stable Diffusion, raised new cash last year as investors, including Eric Schmidt and Napster founder Sean Parker, sought to turn the business around.
Techcrunch states, “Emad Mostaque, Stability’s co-founder and ex-CEO, reportedly mismanaged Stability into financial ruin, leading staff to resign, a partnership with Canva to fall through, and investors to grow concerned about the company’s prospects.”
“In the last few months, Stability has hired a new CEO, appointed “Titanic” director James Cameron to its board of directors, and released several new image-generation models. In March, the company partnered with chipmaker Arm to bring an AI model to generate audio, including sound effects, to mobile devices running Arm chips.”