WTF is a 4D Gaussian Splat!?!?!?
I continue to begrudgingly learn new things
A while back I wrote an article about a technology I was begrudgingly learning, and one of the running themes was that I’m getting worse at wanting to learn new things, not better. Every new piece of software, every new buzzword, and I catch myself getting frustrated before I’ve even given it a fair shot.
And here we are two years later, and it’s time for me to write about 4D Gaussian Splats, and I haven’t gotten better. I’m even uping my game.
A few weeks ago my niece played me a Tate McRae song in the car. I shook my head (like actually shook it, side to side) and grumbled, “Can you believe this music today?…just awful.”
So in the spirit of full honesty, Tate McRae is the intro music to this one. Let her play us in.
What is a 4D Gaussian Splat?
I was at SIGGRAPH a couple of weeks ago, and Gaussian splats (or gsplats) kept popping up everywhere. At the booths, in the Immersive Pavilion, and in presentations. The technology seemed to be advancing significantly and starting to make its way into pipelines.
If you haven’t read the original gsplat article, here’s the fast version, and I picked up a better analogy for it while researching this one.
GSplats are like pointillism, those paintings made of thousands of tiny colored dots that resolve into a photorealistic image from a distance but dissolve into dots up close. A Gaussian splat is the 3D version of that. Instead of dots on a canvas, you have millions of small ellipsoidal blobs floating in space, each one carrying a position, a shape, a color, and an opacity. Stack enough of them together and you get a photorealistic 3D scene.
Before splats, the comparable technology was NeRFs, neural radiance fields, which could learn a 3D scene from a set of photos but were painfully slow to render, sometimes seconds per frame, because every single pixel required querying a neural network. In 2023, a research team at Inria essentially asked what happens if you skip the neural network, throw millions of these explicit Gaussians at a scene, and let the GPU rasterize them directly. The jump was from seconds per frame to over 100 frames per second at HD resolution, in real time, on a single GPU. That’s the breakthrough that got everyone’s attention.
But a traditional splat is still frozen, one snapshot in time. A 4D Gaussian splat is a Gaussian splat that moves.
Let’s pause for a second here and recognize that 4D Gaussian Splat is an eye-rolly name for this, right? Couldn’t we go with Dynamic GSplats or Video GSplats? 4D makes it seem like we are about to time-travel, but I digress.
The naive way to add motion would be to store a full 3D snapshot for every single frame, like a flipbook, which is an absurd amount of data. The actual approach borrows an idea from video compression: instead of storing every frame from scratch, you store what changes between frames. Start with one base set of Gaussians, then train a small neural network to predict how each blob shifts, rotates, and scales at every timestamp. You’re not storing millions of new blobs per frame. You’re storing one set plus the instructions for how they move.
If you want the actual technical breakdown instead of my caveman version, Bilawal Sidhu put out a video called “3D & 4D Gaussian Splatting - Explained!” that does it better than I can here. Worth noting he’s not some rando talking head either, he worked on VR camera systems and YouTube VR at Google, live streaming things like Coachella and Elton John concerts back in 2018, so he’s watched this exact category of tech evolve up close for the better part of a decade. He calls what’s happening now “entering the holodeck,” and I think that framing is doing real work, not just marketing copy.
Why I actually cared
Basically, viewing those 4D Gaussian Splats just felt unique. It felt different. It felt cool.
Like holding an iPhone for the first time or the first time someone combined chocolate and peanut butter, it just felt right.
I don’t think that feeling is a substitute for actual analysis of how much people would adopt this, but I also don’t think it should be dismissed as just a feeling. The technologies that eventually matter usually announce themselves that way first, as a sensation before they become a strategy.
Where this is actually being used
Two recent examples got passed around a lot. The first is James Gunn’s Superman, which became the first feature film to use 4D Gaussian splatting in production. VFX studio Framestore worked with a UK volumetric capture company called Infinite Realities, using a spherical rig called the Deis, 192 genlocked cameras firing at 24 frames per second. Bradley Cooper and Angela Sarafyan performed continuous two-minute takes inside the rig, reportedly the first time two performers were captured side by side in a volume like that, while James Gunn directed the session remotely.
That data was fed through a machine learning pipeline and came out the other side as what people in the field call “living photography,” across roughly 40 shots. It’s not a digital double, and it’s not AI-generated. It’s the actual performance, captured volumetrically, that can be reframed from any angle after the fact. In the film it shows up as the holographic message from Superman’s Kryptonian parents, and because the data is fully 3D, the effects team could offset individual Gaussians, rotate sections, and jitter them to sell that glitching hologram look, the kind of thing that would be tedious or outright impossible with traditional compositing.
The second is A$AP Rocky’s music video for “Helicopter.” A volumetric capture company called Evercoast deployed a 56-camera rig in Los Angeles to record nearly every human performance in the video, including performers suspended from wires and doing stunts inside the capture volume. That shoot alone generated over 10 terabytes of raw data. The pipeline ran through Houdini to manipulate the splat streams, with Octane’s newly added splat support handling relighting, shadows, and depth of field in the final render.
The reaction to the Superman moment specifically has included a lot of “yeah, cool, but it’s just a glitchy hologram.” Fair. It is glitchy right now. But that’s how every version of this technology starts. CG in the 80s was a gimmick. Motion capture was a gimmick before it was a pipeline. The gimmick phase isn’t a verdict, it’s a stage.
What makes 4D splats different from many previous 3D-for-film gimmicks is that film is ultimately a flattened, 2D output, no matter how the shot was built. A 4D splat doesn’t have to be. It can live online, in a browser, on Apple Vision Pro, as an actual volumetric object you walk around instead of a camera move someone else already chose for you.
There’s real production use beyond entertainment too. On Jurassic World Rebirth, the VFX team shot 360 video of remote locations in Thailand with an Insta360, converted the footage into Gaussian splats, and loaded them into Unreal Engine so director Gareth Edwards could scout locations from a studio in London using a virtual camera, walking through jungles he’d never physically visited. And a company called Arcturus is doing something similar for live sports, surrounding a ring or field with cameras and fusing the feed into free viewpoint replays you can watch from any angle. I’ll take that description with a grain of salt since it comes from the same person promoting the tech, but the idea alone reframes what “replay” even means once the capture data is fully volumetric.
The problems
The rigs themselves are absurd. Big, expensive, clumsy arrays of cameras that need to fully surround a performer. That’s the obvious problem and it’s not close to solved, even as the rigs get somewhat cheaper.
The less obvious problem is file size, and it’s worse than I first realized. A single static Gaussian splat can run 250 megabytes as a raw PLY file, sometimes over a gigabyte. Add time into the mix and you’re suddenly talking terabytes. The A$AP Rocky shoot generated over 10 terabytes of raw data for one music video. That is not practical for most productions, and even well-funded Hollywood shoots would struggle at that scale for anything longer than a few shots.
That’s created a compression race. Niantic, the company behind Pokémon Go, open-sourced a format called SPZ that’s roughly 90 percent smaller than a raw PLY file with almost no visible quality loss, functionally what JPEG did for photographs. Khronos, the group behind OpenGL and glTF, has since folded SPZ into the official glTF standard through new extensions built with Niantic, Cesium, and Esri, which means there’s now an actual industry standard for shipping this data instead of everyone inventing their own format. On the research side, papers pushing 100x compression gains on 4D scenes are already circulating. The storage problem is real, but it’s being attacked from multiple directions at once, which is usually a good sign for how fast a rough technology smooths out.
And on the render side, Octane’s 2026 release became the first commercial path tracer that can natively load, shade, and relight Gaussian splats using the same physically based lighting as mesh geometry, including reflections, refractions, and cast shadows. That’s a meaningfully different claim than “you can view a splat.” It means splats can now sit inside a full production lighting pipeline as a first-class object instead of something you composite around.
So what is it, really
A 4D Gaussian splat is a gimmicky name for a moving Gaussian splat, built from footage rather than a single image, still too heavy for its own ambitions, and getting lighter by the month thanks to a public compression race. That’s the gist.
But the part I keep sitting with isn’t the technology; it’s my own reaction to it. I wrote an article two years ago admitting I was getting worse at wanting to learn. I have not gotten better since. And yet here I am, at a display booth, feeling something click the same way it clicked the first time I picked up a phone that felt right in my hand.
I am going to keep my eye on this new tech…
Want to See Gaussian Splats in Action?
Here are some websites with cool 3D and 4D Gaussian Splat experiences. If you know of any more, send them over! Enjoying playing around in this space.
3D Gaussian Splatting with Three.js
Who Am I?
New 3D Artist Merch!
New 3D Artist swag is here!
We ran a design contest inside the community, and Randy Mills took it home with a CMYK sweatshirt design that’s honestly too good not to wear.
Grab yours today.
3D News
SIGGRAPH 2026 Opens in LA: First Games Summit, Neural Rendering Bets, and a Chinese AI Keynote - Tech Times
Google open-sources GNM Head, its parametric human head model - CG Channel
Rapid Pipeline Launches AI tool based on CAD - LinkedIn
Substance 3D unveils new innovations that deliver faster workflows, OpenPBR everywhere, and digital twins at scale - Adobe Blog
Physical Atmosphere² generates insanely detailed Blender skies - CG Channel
3D Job Spreadsheet
Link to Google Doc With A TON of Jobs in Animation (not operated by me)
Hello! Michael Tanzillo here. I am the Head of Technical Artists with the Substance 3D team at Adobe. Previously, I was a Senior Artist on animated films at Blue Sky Studios/Disney with credits including three Ice Age movies, two Rios, Peanuts, Ferdinand, Spies in Disguise, and Epic.
In addition to his work as an artist, I am the Co-Author of the book Lighting for Animation: The Visual Art of Storytelling and the Co-Founder of The Academy of Animated Art, an online school that has helped hundreds of artists around the world begin careers in Animation, Visual Effects, and Digital Imaging. I also created The 3D Artist Community on Skool and this newsletter.
www.michaeltanzillo.com
Free 3D Tutorials on the Michael Tanzillo YouTube Channel
Thanks for reading The 3D Artist! Subscribe for free to receive new posts and support my work. All views and opinions are my own!