RTX AI PCs and workstations offer a powerful way to run Generative AI. They bring extreme performance, allow you to run the latest models, and even come with unique features that improve your applications, like DLSS 4, RTX Video, or the NVIDIA Broadcast effects.
However, if you are an artist interested in starting to use Generative AI, the learning curve is real. You need to find the right AI model to do the job, figure out how to integrate it into your workflow, and ideally run it efficiently on your hardware. It can be a lot for even skilled creators, and with the rapid pace of innovation in this area, there’s a risk of being left behind.
To help artists jumpstart their journey into AI Art, NVIDIA 3D Guided Generative AI Blueprint aims to provide you with a working workflow that you can start using right away, and a way to modify it and adapt it to your needs.
In this AI Blueprint – the first of many to come – NVIDIA showcases how you can control the object placement and camera angle while generating images by using a 3D scene to guide your generations. This is a typical problem in image generation, where you can get close to what you envision through prompts, but composition and layout are very hard to get right, and fine edits are almost impossible without advanced ControlNets.
AI Blueprints come pre-optimized on NVIDIA and GeForce RTX GPUs. They are built on NVIDIA NIM microservices, which include everything needed to run an AI model at maximum performance.
What can a 3D AI Blueprint do for you?
Right now, if you want to generate an image, you can go to your favorite image generation model and start typing in prompt after prompt. With the latest generation of models, you can typically get quite close to what you wanted to generate. But where you wanted it generated within the picture is a whole different story. And attempting to get closer to it while refining your prompt is painstakingly slow, and you rarely get to what you wanted.
NVIDIA’s 3D-Guided Generative AI blueprint gives you precise control into where objects are placed in an image. Here’s how it works. The AI Blueprint guides image generation by using a draft 3D scene in Blender to determine where objects will be placed. The 3D viewport is converted to a depth map, which is used by the image generator — in this case, FLUX.1-dev, from Black Forest Labs — which together with a user’s prompt generates the desired images.
For the non 3D experts, the depth map helps the image model understand where things should go. This process doesn’t require highly detailed objects or high-quality textures, since the depth map is in grayscale and won’t capture fine details. So the user can quickly assemble a scene with stock objects or by using a 3D model generator to create them. And since the scenes are in 3D, artists can move objects around and change camera angles easily.
To put together this workflow, NVIDIA leveraged ComfyUI to run the generative AI pipeline, and the ComfyUI Blender plug-in to connect Blender to ComfyUI. The models themselves are the FLUX.1-dev and FLUX.1-depth-dev models from Black Forest Labs, optimized for RTX GPUs with TensorRT and FP4 quantization. And they are running in NIM microservices to operate at full performance.
What makes this AI Blueprint that much better is that, once it’s set up, it’s easy to run and allows you to make as many edits as you want. And it can be used as a starting point to add Generative AI to other workflows.
This is only the beginning. With 10 NIM Microservices now published and more NIMs and AI Blueprints on the way, now is the time to start your journey with Generative AI. And the latest NVIDIA Blackwell GPUs are here to help you get started.
And there are new tools coming all the time. To find out more about how you can integrate NIM microservices and download AI Blueprints like this one, NVIDIA has introduced the RTX AI Garage Campaign. You can discover digital humans and AI agents, get the scoop on new productivity apps and creative workflows that leverage AI, or start toying with AI Blueprints yourself. You can also find NVIDIA AI PC news on TikTok, Facebook, Instagram and X to stay up to speed on the latest AI developments.
https://cdn.mos.cms.futurecdn.net/5jBAbvX6gJ64yJb563i8tA.jpg
Source link
jacob.york@futurenet.com (Jacob York)
