Text-to-video just took a leap forward

Text-to-image AI has been a pain in artists’ necks, now it’s time for filmmakers to feel the sting. A company called Runway Research introduced us to Gen-2 (via Tom’s Guide). This is an AI-powered text-to-video bot that looks pretty good… scary good.

OpenAI is a rockstar in the AI industry, churning out hits like DALL-E and ChatGPT. However, the company hasn’t given us a taste of text-to-video capabilities. With GPT-4, we thought we’d see something like that; however, it’s proven to be ChatGPT-3 on steroids.

Runway Research pushes Gen-2 down the “runway”

So, text-to-image technology is already labor-intensive. Generating that picture of a warrior princess with messed up hands takes a metric ton of information and processing power. So, bringing that over to full-motion video seemed like something out of a Sci-fi film. However, it’s happening.

Named Gen-2, this is the second generation of Runway Research’s brainchild, and the company showcased how it improved over Gen-1. You can view some examples of what Gen-2 can do here.

Now, there are a few things to know before you get excited. For starters, this tool is not available just yet. We don’t know when it will hit the public, but the company says “Soon”.

Also, don’t start planning your next short film. So far, Gen-2 can only produce 3-second video loops. So, it will basically make GIFs for the time being. In any case, it’s still impressive.

You’ll be able to generate short video clips from text prompts, but you’ll also be able to use video input as well. There’s one clip with a man turning his head toward the camera. That’s the input video. The output shows a creature (based on source images) turning its head. There’s another example where Gen-2 took a source video of a golden retriever and added dalmatian spots.

This is something that’s going to bring Runway further into the spotlight. Right now, OpenAI is leading the AI race, as far as the public is concerned, with Google trailing behind. Another company further adding fuel to the fire will make things interesting in the AI race.