The Apple Spatial Toolkit: What You Need to Know
Unpacking Apple's Ecosystem for Mixed Reality Development
Over the last few weeks, I’ve dedicated my free time to diving into content creation the Apple Vision Pro. When I started, I assumed there would be a clear-cut guide to what I needed to purchase, download, and learn to make this type of content. Some super clean Apple demo with overly friendly, attractive people that manage to speak to each other without a single filler word.
I was wrong.
Through a process of trial and error, I encountered several mistakes but eventually identified the essential applications and tools for producing content on the Apple Vision Pro. Today, I'm compiling a list of these tools within Apple's Spatial Ecosystem to ease your journey into this field. As I continue exploring these applications, I'll share more detailed insights, but this serves as a starting point.
Hardware
You will need a Mac Computer for the majority of the Apps discussed below and an iPad, iPhone, or Apple Vision Pro to test the spatial experiences. Please correct me if I am wrong, but I do not believe there is a way of using Linux or PC in this process.
Apple Developer Program
To access most of the applications mentioned and to create apps for the App Store, enrollment in the Apple Developer Program is required. This program costs $99 USD annually and offers resources like software development kits (SDKs), beta versions of Apple software, and other crucial app-building tools. While free options are preferable, this program offers good value.
(Note: Some tools are available without the Apple Developer Program, which I'll indicate in their respective sections.)
Swift: The Programming Language
Swift is Apple’s programming language for app development in the iOS world. It’s reasonably intuitive and designed for robustness and speed, Swift is the backbone for most iOS applications.
If you are new to Swift, I recommend checking out the Swift Playgrounds app for handy little tutorials and walkthroughs. I found them very useful when I was getting up to speed.
Xcode: The Development Environment
Xcode is Apple's integrated development environment (IDE) for macOS, iOS, watchOS, and tvOS. Basically, this is the software where you write, compile, and test your Swift code before release. It’s a super simple UI and allows you to plug into other tools for testing like device simulators.
Metal: Rendering 3D Content
Metal is Apple’s renderer. It is an advanced technology designed to optimize the performance of the graphics processor (GPU) on Apple devices. It is instrumental in rendering high-quality 3D content, ensuring smooth and efficient graphics performance in AR applications.
Reality Composer and Reality Composer Pro
Reality Composer is a user-friendly application for adding 3D models and content for an interactive, spatial scene. There are minimal controls, but if you want to add a 3D model to the space around you with some predetermined animation and interactions, Reality Composer is your jam.
Its counterpart, Reality Composer Pro, looks similar but is more robust. It is accessible through Xcode and offers advanced features for building more complex AR scenes based on your customized Swift code.
The basic Reality Composer can be used without purchasing the Apple Developer Kit. It’s free to download through the App Store and you can go crazies.
Reality Converter and USDZ Tools
In order to place a 3D model into any apple experience, it must be in the USDz file format. What is a USDz file?
USD stands for Universal Scene Descriptor. Apple, Pixar, and others developed this file type to serve as a universal file type for all 3D applications. In the past, software applications all had their unique file types that did not play well in other ecosystems. Sure, there were FBX and OBJ files, but they would often lack key attributes like animation or material properties. USD set out to change that by being a file type that would store all the necessary data and be used by any 3D application.
USDz is a USD file that is zipped or compressed (hence the z). These contain all the necessary data but are zipped to be a smaller file size for online or spatial experiences.
The Reality Converter app allows developers to convert, view, and customize other 3D file types into a USDz on Mac. Additionally, USDz Tools provides functionalities for converting, inspecting, and validating USD files, ensuring they are ready for Spatial deployment.
Simulator: Testing Your Creations
Testing is a critical part of development. Apple's Simulator app allows developers to test their creations on different Apple devices, including the Apple Vision Pro, to ensure compatibility and optimize performance.
Conclusion
This guide should provide a solid foundation for starting in Apple's Spatial Ecosystem. Hopefully, this is enough to make each of you dangerous :)
I'll delve deeper into these workflows and share updates on my social channels. Stay tuned for more insights on this journey and click below to follow along:
Join The Spatial Explores Community
Join The Spatial Explorers, a dynamic community for artists and enthusiasts passionate about immersive storytelling. We specialize in creating 3D content for headsets like Meta Quest 3 and Apple Vision Pro. This is a space for collaboration, learning, and growth in the realm of augmented reality. Dive into the future of immersive storytelling with us!!
3D News of the Week
Microsoft lays off 1,900 Activision Blizzard and Xbox employees - The Verge
An Important Update about Riot’s Future - Riot Games
Teenage Mutant Ninja Turtles: Mutant Mayhem Breakdown Reel - YouTube
Dungeons & Dragons Is Officially Coming to VR - 80.lv
This Mixed Reality Concept Makes Formula 1 More Immersive Than Ever - 80.lv
3D Merch is here!
Click here to Get Your 3D Artist Swag!
3D Tutorials
3D Job Spreadsheet
Link to Google Doc With A TON of Jobs in Animation (not operated by me)
Check Out The New Wednesday Artist Spotlight Email!
Want to be featured!?!?! Submit your work here
Hello! Michael Tanzillo here. I am the Head of Technical Artists with the Substance 3D Growth team at Adobe. Previously, I was a Senior Artist on animated films at Blue Sky Studios/Disney with credits including three Ice Age movies, two Rios, Peanuts, Ferdinand, Spies in Disguise, and Epic.
In addition to his work as an artist, I am the Co-Author of the book Lighting for Animation: The Visual Art of Storytelling and the Co-Founder of The Academy of Animated Art, an online school that has helped hundreds of artists around the world begin careers in Animation, Visual Effects, and Digital Imaging. I also created The3DArtist.Community and this newsletter.
www.michaeltanzillo.com
Free 3D Tutorials on the Michael Tanzillo YouTube Channel
Thanks for reading The 3D Artist! Subscribe for free to receive new posts and support my work. All views and opinions are my own!