Lightricks shakes up AI video creation with powerful open-source model

Lightricks Ltd. is throwing down the gauntlet to artificial intelligence powerhouses OpenAI, Google LLC and others with the release of its latest open-source video generation model, LTX Video-13B.

The new version is said to be a significant upgrade to Lightricks’ original LTXV model, boosting the number of parameters and enhancing its features to “dramatically” increase the quality of its video outputs, while maintaining its impressive speed. Available as part of Lightrick’s flagship tool LTX Studio, Lightricks says LTXV-13B can generate videos with “stunning detail, coherence and control,” even when running on consumer-grade hardware.

The original LTXV model debuted in November, and it got a lot of attention as one of the most advanced video generation models around. With its lightweight architecture, the 2 billion-parameter model ran efficiently on laptops and personal computers powered by a single consumer-grade graphics processor, and rapidly generating five seconds of slick-looking video with smooth and consistent motion.

However, it was the highly accessible nature of LTXV that really helped it stand out from the crowd. In a world where the most advanced models are typically “black boxes” locked behind pay-to-play application programming interfaces, LTXV was a breath of fresh air. The open-source model, its codebase and its weights were made freely available to the AI community, giving researchers and enthusiasts a rare opportunity to understand how it works and make it even better.

Lightricks made LTXV open-source because it wants to encourage further innovation in the AI industry, and the only way to do that is by making the latest advances available to everyone, so anyone can build on them. It was a calculated move by the startup, which hoped that by getting its foundational model into the hands of as many developers as possible, it could entice more of them to use its paid platforms.

With LTXV-13B, the company is following the same approach, making it available to download on Hugging Face and GitHub, where it can be licensed freely by any organization with less than $10 million in annual revenue. That means users are free to tinker with it any way they want, fine-tune it, add new features and integrate it into third-party applications.

Fine-grained controls

Users will also be able to get their hands on some compelling new features that have been designed to enhance video quality without affecting the model’s efficiency.

One of the biggest updates is a new multiscale rendering capability that enables creators to slowly add more detail and colour to their videos in a step-by-step process. Think of an artist who starts off with a rough pencil sketch before pulling out his paintbrush and adding more intricate details and colors. Creators can employ the same “layered” approach and progressively enhance the individual elements within their videos, similar to the staged scene construction techniques used by professional filmmakers.

The advantage of doing this is twofold. On the one hand, it results in better-quality videos with more refined visual details, Lightricks said. It’s also much faster, enabling the model to render high-resolution video as much as 30 times faster than competing models with a similar number of parameters.

Lightricks also revealed enhancements to existing features for camera motion control, keyframe editing, multishot sequencing and character and scene-level motion adjustment. In addition, the release integrates several contributions from the open source community that improve the model’s scene coherence and motion consistency and preserve its efficiency.

For instance, Lightricks said it worked with researchers to integrate more advanced reference-to-video generation and video-to-video editing tools with LTXV-13B. And there are new upsampling controls, which help to eliminate the effects of background noise.

The open-source community also helped the company optimize LTXV-13B to ensure it can still run efficiently on consumer-grade GPUs, despite being much bulkier than the original model. This is enabled by the UEfficient Q8 kernel, which helps to scale the model’s performance on devices with minimal compute resources. So developers can run the model locally on any machine.

LTXV-13B also stands out as an “ethical” model, since it was trained on a curated dataset of visual assets provided by Getty Images Holdings Inc. and Shutterstock Inc. The high quality of its licensed training data ensures that the model’s outputs are both visually compelling and safe to use commercially, without any risk of copyright infringement issues.

LTXV-13B is available now through LTX Studio, an premium platform which allows creators to outline their ideas using text-based prompts and slowly refine them to generate professional videos. With LTX Studio, creators can access advanced editing tools, enabling them to change the camera angles, refine the appearance of individual characters, edit buildings and objects in the background, adapt the environment and more.

Co-founder and Chief Executive Zeev Farbman said the release is a “pivotal moment” for anyone interested in AI video generation.

“Our users can now create content with more consistency, better quality, and tighter control,” he promised. “This new version of LTX Video runs on consumer hardware, while staying true to what makes all our products different — speed, creativity and usability.”

Image: SiliconANGLE/Dreamina