Gen AI As Smart Render Engine
The Ultimate Goal - Simplicity with Creative Control
3D is complex.
The jargon alone is intimidating. There are point clouds, subsurface scattering calculations, raytracing settings, UV Maps, photon maps, bump maps, normal maps, UDIMs, IK rigs, PBR materials, LOD settings, and the ultimate acronym SSSBRD (Subsurface Scattering Bidirectional Reflectance Distribution Function).
Mastering 3D tools takes months, even years. But the payoff is enormous. Once you harness the power of this workflow, you can create stunning worlds and characters you envision. Despite the learning curve, the reward of total control over every aspect of such a creation is worth the journey.
GenAI: Simplicity, but with Limitations
GenAI offers a stark contrast. Type a few words, and you get an image. It's incredibly simple, and understanding the basics can take mere minutes. However, unlike in 3D, you sacrifice granular control. You can't precisely position objects, fine-tune the camera, or meticulously craft a color palette.
In short:
3D: Complex, but complete creative control.
GenAI: Simple, but frustratingly limited.
Merging the Best of Both Worlds
What if we could harness the simplicity of GenAI and apply it to the complex world of 3D rendering? This has been my obsession lately.
The first step is streamlining isolated tasks. Automating elements of your workflow, like UV unwrapping, would free artists from tedious processes. Text-to-material and text-to-model have the potential to speed up content creation, as well.
But we can go even further. Imagine my newest, most amazing ultimate dream...
AI-assisted render engines.
What is Rendering?
3D rendering is the process of creating two-dimensional images from a three-dimensional scene. Imagine a virtual world with objects, lights, and a camera. Rendering is like photographing or video recording that world from the camera's perspective.
To create a 3D render, countless calculations need to take place. For just the lighting, the software calculates how the lights interact with objects in the scene, considering factors like reflections, shadows, secondary bounces, material settings, and more.
It then translates this information into a 2D image, which you can view on a screen. This process allows designers, animators, architects, and others to visualize and realistically bring their ideas to life.
Today's Renderers: Precise, but Inflexible
Current renderers, while powerful, are inflexible. They do exactly what you tell them, no more, no less. Every tiny detail must be specified with craft. One mistake can break the entire rendering process.
Modern renderers are so literal that if you accidentally put a question mark in the script, they will read it during the local news.
Let’s look at a single light source. You need to tell the renderer the light’s location, radius, color, focal point, shadow casting options, shadow casting samples, specular samples, diffuse samples, cone angle, edge angle, intensity, falloff calculation, roundness, and more.
And that is just one light. I lit shots on animated films with over 500 lights, and any abnormality in any of them would cause the entire render to go haywire.
Again, that is just the lighting. Factoring in the materials, fur, cloth sim, and everything else that goes into these renders is a recipe for disaster. So many inputs could each be a single point of failure, causing lost hours and extreme headaches.
The Future: AI-Infused Smart Renderers
What if we could infuse render engines with GenAI intelligence? Artists could focus on the big picture, letting AI handle the rest. Imagine an animator who cares about the performance but wants the AI to create realistic lighting and materials.
You'd animate the shot, add simple colors for guidance, position a fundamental light source, and hit render. The AI-powered engine would analyze your work and fill in the blanks, creating a polished render that fits your style. You could refine and lock down elements until you're happy, then save those settings for a consistent look throughout your project.
Think about your own situation.
Imagine the number of times you needed to scale back the scope of an image or animated short because of the technical limitations and time crunch of a particular scene element. By allowing the AI-infused renderer take a stab at that component, you have the opportunity always to bring your complete vision to life.
The Promise of AI
Ultimately, whatever AI tools are coming should free us to focus on what we love, automating the tedious parts of the creative process. While a team of specialists handling every micro detail will always achieve the most control, AI-infused renderers could empower small studios and indie creators. Storytelling would become accessible, enabling artists to achieve a higher quality of their work.
Until then... stay classy, San Diego.
I'm... Michael Tanzillo?
The 3D Artist Community Updates!
Big Week in The 3D Artist Community!
The group is up to 14 people!
We have a VFX Supervisor, a 3D Fashion Designer, a Leader of Digital Content Creation for apparel, a Photographer and Art Director, and a true unicorn of the industry…an artist who has worked at the same VFX house for 17 years!
There is still time to join the community before this week’s AMA session with Brett Morris, former Design Director for The Coca-Cola Company.
We also have regular online meetups where community members can connect, ask questions, and make connections around the world (we now have 5 countries represented in the group!)
3D News of the Week
Hundreds More Layoffs Incoming At VFX Firm DNEG - Deadline
CLO now supports USD files - LinkedIn
Isotropix has discontinued Clarisse - CG Channel
Breaking 3D Models to Achieve the Perfect Animation Shot - 80.lv
Unity 6 Preview: five key features for CG artists - CG Channel
3D Merch is here!
3D Tutorials
3D Job Spreadsheet
Link to Google Doc With A TON of Jobs in Animation (not operated by me)
Hello! Michael Tanzillo here. I am the Head of Technical Artists with the Substance 3D Growth team at Adobe. Previously, I was a Senior Artist on animated films at Blue Sky Studios/Disney with credits including three Ice Age movies, two Rios, Peanuts, Ferdinand, Spies in Disguise, and Epic.
In addition to his work as an artist, I am the Co-Author of the book Lighting for Animation: The Visual Art of Storytelling and the Co-Founder of The Academy of Animated Art, an online school that has helped hundreds of artists around the world begin careers in Animation, Visual Effects, and Digital Imaging. I also created The 3D Artist Community on Skool and this newsletter.
www.michaeltanzillo.com
Free 3D Tutorials on the Michael Tanzillo YouTube Channel
Thanks for reading The 3D Artist! Subscribe for free to receive new posts and support my work. All views and opinions are my own!