Generative AI has quickly managed to come a long way. Over the past couple of days, I have had several conversations with my relatives and even my tattoo artist, discussing how AI is slowly seeping its way into every industry and the only way to move forward is by properly understanding how it works. Now, Google has just introduced Lumiere, a new generative AI tool that is going to help you create more realistic images and videos via text-based commands.
Google’s Lumiere is another proof that AI is the future and there is nothing we can do to change it
One of the highlights of Lumiere is the fact that it is able to create motion that looks realistic and does not have choppiness in it. This is possible thanks to the software creating all video frames at the same time instead of actually using the keyframes and then training them to learn how the movement should be. This might not sound impressive, but with the help of the latest generative AI tool, we can go really far and create something amazing.
Lumiere also uses diffusion probabilistic models that help it generate images. This is coupled with a Space-Time U-Net. For those who don’t know, U-Net serves as an architecture that brings temporal up and downscaling, as well as attention blocks that are added to the images. The best thing is that this new-generative AI tool can be used with a lot of other models, and this is going ot help create images and videos that will look a lot more realistic than anything that we have seen before.
Lumiere can be used for creating the following,
- Cinemagraphs: Where a single section of an image is animated
- Inpainting: An object in a video is replaced by another
- Stylized generation: The art style of any creation can be changed into something else
- Image-to-video: Helps with animating any desired image
- Video-to-video: Allows the users to create videos in a different art style
At the time of writing, the video length is only 5 seconds long, whereas the ability ot create video transitions and multiple camera angles is not there. It is also worth noting that if you want to try out Lumiere, you are not going to be able to do it with just a standard GPU. There needs to be something powerful driving the graphical power on your PC, or else the tool is simply not going to work, which makes sense.
The introduction of Lumiere does sound interesting from various perspectives. Only yesterday, I was talking to my tattoo artist about how he managed to create a number of tattoos for his client using AI because the client was looking for something that had not been created before. I can only imagine how powerful these tools are going to be in the near future, and I hope that these tools cannot be used by the wrong hands who are looking to cause nothing but chaos.
News Source: Lumiere
