3D Materials & Lighting in the Apple Vision Pro
Some limitations but overall....WOW!
When I began delving into 3D assets for the Apple Vision Pro, one question kept coming up: How realistic are the 3D models? Over the past week, I've thoroughly explored this area, focusing on creating 3D materials using a Physically-Based Rendering (PBR) system and examining how these materials enhance the spatial experience.
Here's what I discovered…
Lighting In the Apple Vision Pro
My initial inquiry centered on the lighting capabilities of the Apple Vision Pro. I assumed that, given its default preview mode, it might simply apply a standard HDRI (High Dynamic Range Imaging) to objects, MAYBE adjusting brightness based on the ambient lighting of the room, and not much more.
I clearly set the bar too low. The Apple Vision Pro goes far beyond this by integrating real-world lighting conditions into the HDRI map and updating these conditions in real-time. This means if the lighting changes in your environment, the device's display will adjust accordingly.
To put this to the test, I experimented with changing the lighting using a light bulb in my dining room. The outcome was astonishing.
PBR Materials in the Apple Vision Pro
After exploring lighting, I turned my attention to the materials aspect, which introduces a layer of complexity. The quality of a 3D object's surface is determined by its shader, a complex mix of components that together achieve the desired appearance.
PBR Materials, standing for Physically-Based Rendering, are the go-to method for crafting these shaders. This approach layers fundamental elements such as Base Color, Metalness, Roughness, and Height to form the final look of a 3D asset. However, the PBR system also includes more computationally demanding features like Translucency, Emission, and Subsurface Scattering. While these features can add amazing realism, they might require more processing power than the Apple Vision Pro can provide efficiently.
If you want a full breakdown of the PBR system, check out the full-tutorial video below.
What worked?
Essentially, the foundational PBR values like Base Color, Metalness, Roughness, and Height/Normals performed excellently. The detail richness displayed on the high-resolution screens was stunning, allowing for the observation of numerous subtle nuances in the work. The representation of various materials, from metals to plastics to fabrics, appeared outstanding, all managed with simple controls.
What didn’t?
However, the more advanced settings like Emission, Translucency, and Subsurface Scattering didn't quite hit the mark. These features, crucial for elevating the realism of 3D models, were noticeably absent. But this is understandable. Even contemporary real-time gaming systems often omit these complex effects due to their intensive computational demands. Expecting them in the Apple Vision Pro might seem a bit over the top. Given the current state of technology, incorporating such detailed calculations would likely compromise the system's performance and slow down the overall experience.
Conclusion
It might seem straightforward to critique the default Metal renderer's capabilities, suggesting they're insufficient for crafting engaging narratives. However, it's essential to remember that the pioneers of 3D animation faced similar constraints. They lacked the ability to incorporate effects like translucency, subsurface scattering, or emissiveness. Despite these limitations, they brilliantly used their creativity to construct a narrative and setting around plastic toys in a bedroom, leading to the creation of "Toy Story" — one of the most iconic films in animation history. These innovators embraced the medium's restrictions and managed to produce something extraordinary.
This is the crossroads where we find ourselves now. We're at the brink of exploring new technologies, confronted with their technical limitations. The challenge and opportunity lie in our ability to creatively navigate these boundaries. Those who can ingeniously work within these constraints to weave compelling stories are the ones who will truly stand out and make a significant impact.
Join The Spatial Explores Community
Join The Spatial Explorers, a dynamic community for artists and enthusiasts passionate about immersive storytelling. We specialize in creating 3D content for headsets like Meta Quest 3 and Apple Vision Pro. This is a space for collaboration, learning, and growth in the realm of augmented reality. Dive into the future of immersive storytelling with us!!
3D News of the Week
Paramount Hit By Mass Layoffs: 800 People To Lose Jobs - Cartoon Brew
Unreal Engine-Powered 2D in 3D Animation Tool Is Out Now - 80.lv
See how Big Picture Studio de-aged Chow Yun-Fat for One More Chance - CG Channel
The 7 Most-Streamed Films Last Year Were All Animated - Cartoon Brew
Watch ILM’s simulants and robots breakdown for ‘The Creator’ - Before and Afters
3D Merch is here!
Click here to Get Your 3D Artist Swag!
3D Tutorials
3D Job Spreadsheet
Link to Google Doc With A TON of Jobs in Animation (not operated by me)
Check Out The New Wednesday Artist Spotlight Email!
Want to be featured!?!?! Submit your work here
Hello! Michael Tanzillo here. I am the Head of Technical Artists with the Substance 3D Growth team at Adobe. Previously, I was a Senior Artist on animated films at Blue Sky Studios/Disney with credits including three Ice Age movies, two Rios, Peanuts, Ferdinand, Spies in Disguise, and Epic.
In addition to his work as an artist, I am the Co-Author of the book Lighting for Animation: The Visual Art of Storytelling and the Co-Founder of The Academy of Animated Art, an online school that has helped hundreds of artists around the world begin careers in Animation, Visual Effects, and Digital Imaging. I also created The3DArtist.Community and this newsletter.
www.michaeltanzillo.com
Free 3D Tutorials on the Michael Tanzillo YouTube Channel
Thanks for reading The 3D Artist! Subscribe for free to receive new posts and support my work. All views and opinions are my own!
Wow thank you for the article and videos ! Very interesting !