To overcome this limitation, the OpenAI team trained an additional AI system to convert POINT-E point clouds into meshes.<\/p>\n
POINT-E itself consists of two parts:<\/p>\n
- \n
- text-to-image models;<\/li>\n
- image-to-3D models.<\/li>\n<\/ul>\n
The text-to-image model works similarly to DALL-E 2. It was trained on labelled images so that the algorithm understands associations between words and visual concepts.<\/p>\n
The image-to-3D model was trained on pairs of image and three-dimensional object.<\/p>\n
For example, if you enter the text prompt ‘A cat eats a burrito’, POINT-E will first generate a synthetic image consistent with the prompt. The second model will then synthesize a rough ‘cloud’ with 1024 points, and then refine the 3D object to 4096 points.<\/p>\n
![\\\"Screenshot-17-1\\\"](\\\"https:\/\/forklog.com\/wp-content\/uploads\/Screenshot-17-1-750x1024.png\\\")
Turning a 2D image into 3D. Data: OpenAI.<\/figcaption><\/figure>\n According to the researchers, after training the models on a dataset of ‘several million’ 3D objects and associated metadata, POINT-E can generate coloured point clouds that correspond to textual prompts. They acknowledged the model’s imperfect performance, but noted the speed of generation.<\/p>\n
\n
“Although our method yields worse results in this evaluation than the most advanced methods, it provides samples in a small fraction of the time. This could make it more practical for certain applications or enable the discovery of higher-quality 3D objects,” the developers said.<\/p>\n<\/blockquote>\n
OpenAI released the open-source code for the projects on GitHub<\/a>.<\/p>\n
In December, the company introduced the ChatGPT chatbot<\/a>, built on a large language model.<\/p>\n
In April, OpenAI released the second version of its image generator<\/a> for the text-to-image model DALL-E.<\/p>\n