World Labs, a startup founded by AI pioneer Fei-Fei Li, has introduced an AI system capable of generating 3D scenes from a single image.
We’ve been busy building an AI system to generate 3D worlds from a single image. Check out some early results on our site, where you can interact with our scenes directly in the browser!https://t.co/ASD6ZHMwxI
1/n pic.twitter.com/tuvGXHmepP
— World Labs (@theworldlabs) December 2, 2024
The company’s tool can assess the three-dimensional geometry from an input image, fill in unseen parts of the scene, and create new content.
World Labs aims to address the challenges many creators face with existing genAI models: a lack of control and consistency. Given an input image, our system estimates 3D geometry, fills in unseen parts of the scene, invents new content so you can turn around, and generalizes to a… pic.twitter.com/3SaTugmGRX
— World Labs (@theworldlabs) December 2, 2024
Video creators can navigate and explore 3D scenes using a freely moving camera, controlled like in a video game. The scenes remain consistent when the perspective changes and adhere to the laws of physics.
Our output 3D scenes can be rendered in real-time in the browser with full camera control. This means you can explore them with a freely moving camera like in a videogame, or even simulate 3D camera effects like shallow depth of field or dolly zoom.
3/n pic.twitter.com/HqzYaKhtAw
— World Labs (@theworldlabs) December 2, 2024
World Labs’ product is compatible with other well-known AI tools, enhancing the workflow.
Several developers have tested the scene generator. An animator known as enigmatic_e appreciated the simplified process of staging characters and camera movements.
@8bit_e shows how our models fill a gap in his creative workflow, making it easy to stage characters within scenes and direct precise camera movements.
8/n pic.twitter.com/AIbhFtXU1B
— World Labs (@theworldlabs) December 2, 2024
World Labs noted that most generative AI tools create 2D content, while generating 3D scenes will “change how we make films, games, simulators, and other digital manifestations of our physical world.”
The World Labs blog features interactive projects that allow users to navigate directly in the browser using arrow keys or WASD. Movements are limited to a small area.
World Labs’ solution is not the first of its kind. The AI model MaGRITTE generates virtual 3D worlds with a 360° view using prompts from a combination of image, layout, and text input.
In March, Nvidia unveiled an AI model for generating 3D objects from prompts. It can create high-quality three-dimensional images almost instantly.