A group of researchers working for AI chipmaker NVIDIA, the third-biggest company in the world by market capitalization, in collaboration with graduate students from Stanford, UC San Diego, UC Berkeley, and UT Austin, trained an AI system on 81 classic Tom & Jerry theatrical shorts to see if they could create long-form animated sequences of up to a minute in length.
Ostensibly, the point of the exercise was to generate longer and more consistent AI video while overcoming the challenge of “self-attention,” a technical principle that currently makes generating long-form generated AI video computationally prohibitive. All of the current public APIs for AI video generation are still limited by length: as of last month, OpenAI’s Sora has a maximum length of 20 seconds, Meta’s Moviegen is 16 seconds, Luma’s Ray 2 is 10 seconds, and Google’s Veo 2 is eight seconds.
The research team (which includes NVIDIA’s Jiarui Xu, Shihao Han, Ka Chun Cheung, Jan Kautz, Yejin Choi, Yu Sun, and Xialong Wang) explained how they did it HERE as well as in a paper (download PDF) that was published yesterday. They wrote, “The videos tell complex stories with coherent scenes composed of dynamic motion. Every video is produced directly by the model in a single shot, without editing, stitching, or post-processing. Every story is newly created.” (In its paper, NVIDIA does not acknowledge receiving permission from Warner Bros. Discovery to use its characters and copyrighted films for such an experiment.)
Below is one of the videos created by NVIDIA et al. that shows Tom working as an office employee in the World Trade Center:
At first glance, it looks impressive, but watch for a few seconds and it quickly falls apart with illogical performance, weird objects, inconsistent hook-ups, and only the faintest suggestion of coherent storytelling. While this is an experiment and should be viewed in the context of the goals it set out to achieve, it only serves to show that AI is still many years away from generating an entire animated film on its own. Also notable is the amount of text prompting it took to even achieve this. Here is the full prompt that was required to generate the above video:
The World Trade Center towers stand tall against a clear, bright blue morning sky. Streets bustle with pedestrians in suits, and yellow taxis move slowly through heavy traffic. Sunlight reflects sharply from glass windows of nearby buildings. Tom, the blue-gray cat, walks briskly along the wide gray sidewalk with a single black briefcase in his hand.
Inside the World Trade Center lobby, expansive marble floors reflect warm golden recessed lighting. Gray marble pillars and brass fixtures highlight the elegant entryway, along with a brass elevator door. A uniformed doorman wearing a dark navy-blue suit stands behind a polished wooden counter. Tom, the blue-gray cat, with a single black briefcase in his hand calmly walks in from the right and approaches the elevator doors. Tom’s left hand is empty.
Inside the World Trade Center lobby, expansive marble floors reflect warm golden recessed lighting. Gray marble pillars and brass fixtures highlight the elegant entryway, along with a brass elevator door. A uniformed doorman wearing a dark navy-blue suit stands behind a polished wooden counter. Tom, the blue-gray cat, has a single black briefcase in his hand slowly presses the elevator button on the left side of the elevator and waits. Tom’s left hand is empty.
Inside a crowded elevator, mirrored walls reflect brass handrails and subtle ceiling lights. Professionals dressed in dark suits stand quietly facing forward. Tom, the blue-gray cat with a single black briefcase in his hand, enters calmly and blends quietly into the crowd. Tom’s other hand is empty
A bright office wall with cream-colored plain walls complement an oak door. Tom, the blue-gray cat, with a single black briefcase in his hand is in scene. Tom gently places his single briefcase on the floor. Tom, with his hands now empty, starts walking away.
A bright corner office features floor-to-ceiling windows overlooking the vast New York skyline beneath sunny blue skies. Cream-colored plain walls complement a large glossy oak executive desk. On the center of the desk, in front of Tom, is a silver computer monitor and silver keyboard, a silver penholder, and a black leather office chair. On the computer screen, there are various stock merket charts and graphs with color. Tom, the blue-gray cat, sits comfortably in a high-backed black leather chair, and begins quietly typing on his keyboard. Tom has a focused expression on his face.
Under the large oak desk, a single black, intact cable runs neatly along beige carpeting toward a silver desktop computer tower. A clean white wall socket securely holds a single plug. Jerry, the brown mouse, quietly approaches the intact cable unnoticed.
Under the large oak desk, a single intact, black cable runs neatly along beige carpeting toward a silver desktop computer tower. A clean white wall socket securely holds a single plug. Jerry, the brown mouse, carefully chews the black cable. Tiny and hardly noticeable sparks fly as he bites the cable.
A bright corner office features floor-to-ceiling windows overlooking the vast New York skyline beneath sunny blue skies. Cream-colored plain walls complement a large glossy oak executive desk. On the center of the desk, in front of Tom, is a silver computer monitor and silver keyboard, a silver penholder, and a black leather office chair. Tom, the blue-gray cat, is sitting at the desk typing on the keyboard. On top of the oak desk, the silver monitor flickers briefly, displaying static before turning completely black. Tom, the blue-gray cat, pauses typing in confusion.
A bright corner office features floor-to-ceiling windows overlooking the vast New York skyline beneath sunny blue skies. Cream-colored plain walls complement a large glossy oak executive desk. On the center of the desk, in front of Tom, is a silver computer monitor and silver keyboard, a silver penholder, and a black leather office chair. Tom, the blue-gray cat, now with an upset face, leans to the floor to check under the desk.
Under the large oak desk, a single black cable that has exposed copper wire in one spot runs neatly along beige carpeting toward a silver desktop computer tower. A clean white wall socket securely holds a single plug. Jerry, the brown mouse, stands near a chewed threw black cable. Tom, the blue-gray cat, moves his head into the scene and sees Jerry. Tom looks surprised and angry at Jerry, while Jerry looks scared and surprised at Tom.
Under the large oak desk, a single black cable that has exposed copper wire in one spot runs neatly along beige carpeting toward a silver desktop computer tower. A clean white wall socket securely holds a single plug. Jerry, the brown mouse, stands near the chewed threw black cable while Tom, the blue-gray cat, has his head in the scene angry and surprised. Jerry hurries away from Tom out of the scene, while Tom removes his head from the scene.
A bright corner office features floor-to-ceiling windows overlooking the vast New York skyline beneath sunny blue skies. Cream-colored walls complement a large glossy oak executive desk, with a silver computer monitor and silver keyboard, a silver penholder, and a black leather office chair. Tom, the blue-gray cat, chases Jerry, the brown mouse, across the room in front of the table. Jerry exits the scene to the right.
A bright corner office featuring cream-colored plain walls and a brown, oak door is to the right of the scene. There is a small mousehold slightly to the left of the door at the baseboard. Jerry, the brown mouse, moderately runs to the mousehole and disappears through the mousehole.
A bright corner office featuring cream-colored plain walls and a brown, oak door is to the right of the scene. There is a small mousehold slightly to the left of the door at the baseboard. Tom, the blue-gray cat, runs toward the mousehole and slams his body into the wall. Tom sits there stunned with small cartoonish stars circling around his head which indicate dizziness.
A bright corner office featuring cream-colored plain walls and a brown, oak door is to the right of the scene. There is a small mousehold slightly to the left of the door at the baseboard. Tom, the blue-gray cat, sits there stunned. Tom, the blue-gray cat, checks the a small watch on his right wrist. Tom, the blue-gray cat, reads the time on his watch and notices that he is late for a meeting. In a rush, Tom, the blue-gray cat, stands back up, no longer stunned, opens the oak door on the right of the scene and runs through the doorway.
In the large conference room, plain cream-colored walls surround a long wooden table positioned at the center. A small black wall clock is positioned in the top right of the screen. There are several executives who are sitting in black office chairs are neatly arranged around the table. The executives are quietly focused on Spike, the gray bulldog, who stands at a whiteboard at the front of the room. There are several people dressed in suits occupying the seats in the room. Tom, the blue-gray cat, hurriedly enters through the door on the left. Spike stops mid-sentence, and both Spike and Tom have surprised looks on their faces. Spike lowers his marker and glares sharply at Tom’s intrusion.
In the large conference room, plain cream-colored walls surround a long wooden table positioned at the center. A small black wall clock is positioned in the top right of the screen. There are several executives who are sitting in black office chairs are neatly arranged around the table. The executives are quietly focused on Spike, the gray bulldog, who stands at a whiteboard at the front of the room, while Tom, the blue-gray cat, stands left of Spike. Spike, the gray bulldog, angrily points at the small black wall clock at the top right of the screen while maintaining aggressive eye contact with Tom, the blue-gray cat. Spike, the gray bulldog, loudly reprimands Tom. Tom, now with a scared expression, covers his ears due to the yelling and makes an apologetic gesture.
In the large conference room, plain cream-colored walls surround a long wooden table positioned at the center. A small black wall clock is positioned in the top right of the screen. There are several executives who are sitting in black office chairs are neatly arranged around the table. The executives are quietly focused on Spike, the gray bulldog, who stands at a whiteboard at the front of the room, while Tom, the blue-gray cat, stands left of Spike. Spike, the gray bulldog, angrily points and hold his pointing to the left while looking at Tom. Tom sulks and then sadly walks to the left out of scene.
In a cozy mousehole interior with rounded walls, wooden furniture, and warm, soft lighting, Jerry, the brown mouse, stands comfortably in the mousehole. He laughs happily, leaning back in delight, eyes closed and a wide smile on his face.
In the cozy mousehole interior with rounded walls, wooden furniture, and warm, soft lighting, Jerry, the brown mouse, continues laughing cheerfully as the scene gradually fades to black.
10 Comments
whoah this weblog is great i like studying your posts. Stay up the great work! You already know, a lot of individuals are hunting round for this information, you could aid them greatly.
gemini
Je suis totalement seduit par Casino Action, ca ressemble a une sensation de casino unique. La selection de jeux est impressionnante avec plus de 1000 titres, avec des machines a sous modernes comme Mega Moolah. Le service d’assistance est irreprochable, offrant des reponses claires et precises. Les paiements sont fluides et securises par un cryptage SSL 128 bits, cependant le bonus de bienvenue jusqu’a 1250 € pourrait etre plus frequent. En resume, Casino Action est une plateforme d’exception pour ceux qui aiment parier ! Ajoutons que le site est concu avec modernite et ergonomie, ajoute une touche de sophistication a l’experience.
go to casino action|
Je suis fan de le casino TonyBet, ca ressemble a une experience de jeu incroyable. Il y a une tonne de jeux differents, avec des machines a sous modernes. Le service d’assistance est top, repondant rapidement. On recupere ses gains vite, par contre les offres pourraient etre plus genereuses. Pour tout dire, TonyBet vaut vraiment le coup pour les joueurs passionnes ! De plus, le design est attractif, ce qui rend l’experience encore meilleure.
tonybet sportsbook bonus|
J’adore a fond le casino AllySpin, ca offre une sensation de casino unique. Les options de jeu sont incroyablement variees, offrant des jeux de table authentiques. Le support est ultra-reactif, avec des reponses precises et utiles. Les transactions sont bien protegees, cependant plus de tours gratuits seraient top. Pour faire court, AllySpin vaut vraiment le detour pour ceux qui aiment les defis ! De plus l’interface est super intuitive, facilitant chaque session.
allyspin hracГ automaty|
J’adore sans reserve Banzai Casino, ca ressemble a une plongee dans le divertissement intense. Les options de jeu sont epoustouflantes, avec des machines a sous ultra-modernes. Le personnel est d’un professionnalisme hors pair, avec un suivi de qualite. Les gains arrivent en un rien de temps, cependant plus de tours gratuits seraient apprecies. Dans l’ensemble, Banzai Casino vaut largement le detour pour les amateurs de jeux en ligne ! De plus la navigation est intuitive et rapide, facilitant chaque session de jeu.
banzai slots casino en ligne|
Je suis enthousiaste a propos de Azur Casino, on dirait une sensation de casino unique. Il y a une multitude de jeux varies, offrant des jeux de table elegants. Les agents sont d’une efficacite remarquable, avec un suivi irreprochable. Le processus de retrait est simple et efficace, cependant davantage de recompenses seraient bienvenues. Globalement, Azur Casino est une pepite pour ceux qui cherchent l’adrenaline ! En bonus la navigation est intuitive et agreable, ce qui rend chaque session memorable.
azur lane casino|
Why visitors still make use of to read news papers when in this technological globe the whole thing is available on web?
сайт кракен
Je suis fou de JackpotStar Casino, on dirait une supernova de fun. La selection du casino est une explosion cosmique de plaisir, avec des machines a sous de casino modernes et envoutantes. Les agents du casino sont rapides comme une comete, assurant un support de casino immediat et eclatant. Les retraits au casino sont rapides comme un meteore, quand meme plus de tours gratuits au casino ce serait galactique. Au final, JackpotStar Casino offre une experience de casino celeste pour les joueurs qui aiment parier avec panache au casino ! En plus la plateforme du casino brille par son style etoile, amplifie l’immersion totale dans le casino.
iwatskate jackpotstar|
Ich bin suchtig nach Platin Casino, es ist ein Online-Casino, das wie ein Edelstein strahlt. Die Auswahl im Casino ist ein echtes Meisterwerk, mit Casino-Spielen, die fur Kryptowahrungen optimiert sind. Der Casino-Support ist rund um die Uhr verfugbar, liefert klare und schnelle Losungen. Casino-Transaktionen sind simpel wie ein Edelstein, ab und zu wurde ich mir mehr Casino-Promos wunschen, die wie Gold glanzen. Insgesamt ist Platin Casino ein Muss fur Casino-Fans fur Abenteurer im Casino! Nebenbei die Casino-Oberflache ist flussig und glitzert wie ein Juwel, den Spielspa? im Casino in den Himmel hebt.
platin casino kostenlos|
Hey I am so delighted I found your weblog, I really found you by mistake, while I was searching on Yahoo for something else, Regardless I am here now and would just like to say many thanks for a marvelous post and a all round exciting blog (I also love the theme/design), I don’t have time to browse it all at the moment but I have saved it and also added your RSS feeds, so when I have time I will be back to read a lot more, Please do keep up the awesome work.
kra39 at