WTF is a Gaussian Splat!?!?
WTF is a Gaussian Splat!?!?
I begrudgingly learn something new
Within the last few weeks, my feeds have been inundated with flashy videos with a new buzz phrase…GAUSSIAN SPLATTING!!! And when I saw it, I believe I reacted in an entirely rational way…I was furious. WTF is this, now!?!?!?
I have spent the last couple of years being thoroughly curious about NERFs and GenAI and Neural Rendering and Deep Learning Upsampling and SDK Modeling and GLBs and GTLFs and USD and USDa and USDc and USDz…..I have no more room in my brain to learn a new!!!
But I eventually calmed down, glued all the pieces of my computer back together, and squeezed out a bit more curiosity to dive into this topic.
So let’s go!
What is 3D Gaussian Splatting?
3D Gaussian splatting, also known as GSplats, is a system of creating a complex 3D scene with only a series of photographs or videos. It begins with inputting images or video that represent that location from multiple angles and software interprets that data and generates a 3D representation of the scene.
Isn’t that just the same as Photogrammetry?
Not exactly. The main difference is Photogrammetry converts a series of images into a more traditional, polygonal 3D mesh with materials that other 3D applications can use. so you can easily bring your photogrammetry mesh into Maya, Blender, Unreal, C4D, or anywhere else. Gaussian splatting, on the other hand, uses that information to generate data in a point cloud or radiance field that stores all the information for generating the image.
So Gaussian Splatting is using a radiance field to generate an image. Isn’t that just the same thing as Neural Radiance Fields (NeRFs)?
Fair question. They both appear very similar at first glance but with one big difference. Gaussian splats are static. NeRFs can use machine learning to calculate more about the space than what is given in the source material. So, Gaussian splats are based on existing information and NeRFs can hallucinate things not seen in the original images.
So NeRFs are better?
Not always. Gaussian splatting comes with a handful of advantages over NeRFs.
They are generally easier and faster to compute since they don’t involve a machine learning component.
They are often smaller files and more accessible to store than NeRFs
Since they have been researched for around 20 years and don’t require machine learning, the final results are much more predictable.
What are the advantages of NeRFs?
NeRFs can use machine learning to understand and create more information about the space than was given in the source images.
They can create camera moves and capture images of landscapes even professional drone flyers cannot navigate.
But they both are coming to replace Photogrammetry?
Not at this moment.
Remember, NeRFs and Gaussian Splats don’t actually make polygonal 3D models so they won’t work if you are creating a film or video game and need to mix this data with elements created in other software. Also, it’s easier to adjust the texture maps of a photogrammetry scan to create a modified version of the object captured than changing the radiance cloud data.
BUT they are great at capturing transparent or reflective objects that photogrammetry often struggles with. And while NeRFs and Gaussian Splats aren’t great for 3D production pipelines, they do work great for displayed backgrounds as part of a virtual production.
Conclusion
Even though I was furious to learn new things, having a better understanding of Gaussian Splats, NeRFs, and Photogrammetry does help me cut through the noise of all the recent clickbait trends. They each play their role and it’s important to not fall for the clickbait headlines and sit back and watch how the technology will develop. I should have listened to my own advice:
**General Disclaimer Whenever I write about new technologies….
Ladies and Gentlement of the jury…I am just a simple caveman. Your advanced computer system frightens and confuses me. I was unfrozen from a block of ice and went to a Big Ten school to study art, therefore, this article was meant to be a beginners guide and in no way a complete coverage over the topics covered. So if I missed anything, please let me know and I will gladly learn and update. I rest my case.
Join The 3D Artist Community
Join The 3D Artist Community!
This community is tailored for artists seeking paths outside traditional fields like VFX and Gaming. Whether you're interested in apparel, footwear, package design, motion graphics, or any other medium, we're here to expand your options and opportunities.
Why This Community?
Value-Driven Resources: A minimal fee accesses workshops and events to refine your skills.
Member Ownership: Influence how we use our resources and shape our future.
Premium Hosting: We use Skool to ensure meaningful exchanges and provide necessary tools.
Our Vision: We're redefining the role of the 3D artist, keeping you at the forefront of industry shifts. Join us to explore rewarding careers that leverage your creativity. For details or to sign up, contact us. Let's innovate in 3D artistry together!
3D Merch is here!
3D News of the Week
ZBrush is coming to the iPad in 2024 - Twitter
Meta Introduces New AI Experiences Across Apps and Devices - fb.com
Animate Anything uses AI to rig your 3D characters - CGChannel
Photorealistic Nokia 3310 Model Made in C4D & Substance 3D - 80.lv
Unity dev group dissolves after 13 years over “completely eroded” company trust - ArsTechnica
Artist of the Week
3D Tutorials
3D Job Spreadsheet
Link to Google Doc With A TON of Jobs in Animation (not operated by me)
Michael Tanzillo is the Head of Technical Artists with the Substance 3D Growth team at Adobe. Previously, I was a Senior Artist on animated films at Blue Sky Studios/Disney with credits including three Ice Age movies, two Rios, Peanuts, Ferdinand, Spies in Disguise, and Epic.
In addition to his work as an artist, Michael is the Co-Author of the book Lighting for Animation: The Visual Art of Storytelling and the Co-Founder of The Academy of Animated Art, an online school that has helped hundreds of artists around the world begin careers in Animation, Visual Effects, and Digital Imaging.
www.michaeltanzillo.com
Free 3D Tutorials on the Michael Tanzillo YouTube Channel
Thanks for reading The 3D Artist! Subscribe for free to receive new posts and support my work. All views and opinions are my own!
Hey Tanzillo. THANK YOU for writing this article. My brain hurts with all the fancy new acronyms, it's so hard to keep up. This is a perfect summation and I appreciate your hours of research so I don't have to!
Please keep the new tech articles coming so I have an avenue to keep up :)
Cheers!