Populate 3D Worlds Quickly With AI Blueprints

3D artists are constantly prototyping.

In traditional workflows, modelers must build placeholder, low-fidelity assets to populate 3D scenes, tinkering and adjusting the core elements until they’re in place. From there, visuals can be refined, detailed and finalized.

Prototyping is time consuming and often comprises throwaway work, forcing artists to spend time on tedious modeling rather than on creative exploration.

Generative AI assists artists by expediting intermediate tasks like suggesting and generating draft objects to prototype a scene. But it can be technically challenging to connect multiple AI models to unleash these accelerated workflows.

NVIDIA AI Blueprints help address this challenge by providing sample workflows so users can skip the hypertechnical stages and quickly benefit from advanced generative AI techniques. Plus, AI Blueprints can be tailored for each user’s particular needs.

NVIDIA today released the AI Blueprint for 3D object generation — a workflow that lets 3D artists create up to 20 3D objects to prototype a scene from a simple text prompt.

In addition, the new Microsoft TRELLIS NVIDIA NIM microservice works within the AI Blueprint for 3D object generation to generate high-quality 3D assets 20% faster than with the native model.

Generate, Prototype, Repeat

3D projects start with an idea’s genesis, followed by the careful consideration of visual details such as themes, locations, decors, colors, textures and more. Even after populating a scene with assets, artists must review and edit individual and collective visuals multiple times.

The new NVIDIA AI Blueprint for 3D object generation gives users a pipeline to automate the prototyping process. Start by providing an artistic idea through a prompt, and the blueprint’s built-in large language model (LLM) will brainstorm 20 possible objects to include in the scene. The Llama 3.1 8B NVIDIA NIM microservice accelerates the results.

NVIDIA SANA — a text-to-image framework that quickly synthesizes high-resolution images — generates previews to showcase the potential objects. Each object can be regenerated, modified or discarded, giving the artist complete creative control.

Then, artists can convert each object from a high-quality preview into a ready-to-use 3D model with the new Microsoft TRELLIS NVIDIA NIM microservice, which simplifies deploying this state-of-the-art model and accelerates it by 20%.

In addition, the collection of up to 20 3D assets are primed for instant use or further refinement in Blender, an open-source 3D platform, which the AI Blueprint for 3D object generation automatically exports to. Artists can also export to other popular 3D applications.

Setting up this series of workflows would normally take time and require technical knowledge, as well as experimentation with different models to select a suitable pipeline. NVIDIA AI Blueprints ease the process of getting started by preselecting proven workflows, simplifying deployment on NVIDIA GeForce RTX and RTX PRO GPUs by packaging the required components together.

Microsoft TRELLIS

Microsoft TRELLIS, a cutting-edge 3D asset generation model developed by Microsoft Research, is capable of generating richly detailed 3D assets, complete with complex shapes and textures, from text or image prompts.

It’s especially useful for game design, architecture and digital media production, as it can produce preview-ready materials and product-level objects.

High-quality text-to-image 3D models available for consumer use tend to be difficult to set up. The Microsoft TRELLIS NIM microservice seamlessly integrates within the NVIDIA AI Blueprint for 3D object generation, saving considerable time for artists.

When used in the blueprint, the Microsoft TRELLIS NIM microservice is 20% faster with PyTorch optimizations that unlock even quicker iteration. The average time saved is approximately 6 seconds per object generated on an NVIDIA GeForce RTX 5090 GPU. For 3D freelancers generating hundreds of assets across dozens of scenes, these are extraordinary savings.

The Microsoft TRELLIS NIM microservice is supported on all NVIDIA GeForce RTX 50 Series and 40 Series GPUs — desktop and laptop — with 16GB or more.

Ready, Aim, Deploy

Get started with the NVIDIA AI Blueprint for 3D object generation by following these instructions:

1. Load up the blueprint, including models.
2. Type in a simple scene idea (e.g. “night market” or “sunny day at the beach”).
3. Twenty image previews will generate automatically.
4. Replace any unwanted objects by clicking the refresh icon located below the image.
   • Tip: Gain more control over the replacement image by clicking on the pencil icon below the image to edit the original prompt.
5. Delete unwanted objects by clicking the trash can icon under the image.
6. Convert individual images using the “->3D” icon under the image.
7. Convert all previews to 3D using the “Convert all to 3D” button at the bottom.
   • Tip: Click “Start over with a new scene prompt” to start a new scene. This erases current progress, so be sure to save first.
8. Export files to Blender using the built-in button at the bottom of the page to continue refining the 3D models.

Detailed instructions and necessary files can be found on NVIDIA Build and GitHub.

Explore a variety of expertly curated AI Blueprints, including the AI Blueprint for 3D-guided generative AI, and NIM microservices today.

Plus, attendees of the IFA consumer electronics and home appliances trade show can attend a session on “How AI Is Changing Content Creation” on Saturday, Sept. 6, at 2pm CET to hear Sean Kilbride, director of professional visualization technical marketing at NVIDIA, and other panelists discuss how creators are integrating AI into their workflows to unlock new creative approaches, speed up production and push the boundaries of content creation.

Each week, the RTX AI Garage blog series features community-driven AI innovations and content for those looking to learn more about NVIDIA NIM microservices and AI Blueprints, as well as building AI agents, creative workflows, productivity apps and more on AI PCs and workstations. 

Plug in to NVIDIA AI PC on Facebook, Instagram, TikTok and X — and stay informed by subscribing to the RTX AI PC newsletter. Join NVIDIA’s Discord server to connect with community developers and AI enthusiasts for discussions on what’s possible with RTX AI.

Follow NVIDIA Workstation on LinkedIn and X. 

See notice regarding software product information.