#1522: A Survey of Spatial Audio Formats with 1618 Digital’s Oliver Kadel

The SXSW XR Experience Selection runs from March 9-11 and contains 15 pieces in competition and 16 pieces in the spotlight, and I had a chance to chat with SXSW's XR programmer Blake Kammerdiener about each of these pieces. He provides some highlights from the XR Track Sessions at SXSW, and also mentions that there is an immersive piece called NOWISWHENWEARE (the stars) that is showing on March 11th & 12th. See down below for some more links to different SXSW events. Here is a list of the XR experiences being shown at SXSW, and I will update this post as I post my Voices of VR podcast interviews as I've featured a dozen projects in my previous coverage. SXSW XR Experience Competition 1906 Atlanta Race Massacre Any War Any Enemy - #1490: “Any War Any Enemy” is a Beautiful Anti-War, Immersive Poem About Nuclear Annihilation Cosmos in Focus Currents Face Jumping Honey Fungus In the Current of Being Proof As If Proof Were Needed Reflections of Little Red Dot Shelter The Choice - Chapters 2 and 3 - [See my coverage of Chapter 1: #1075: Stereoscopic Compositing Innovations & “The Choice” Interactive Documentary Exploring Reproductive Rights in Texas] The Last Practice The Secret Life of Monsters - The Gateway Experience Traces: The Grief Processor Ways of Knowing: A Navajo Nuclear History SXSW XR Experience Spotlight Address Unknown: Fukushima Now - #1438: Honoring the Memory of Abandoned Ghost Towns with “Address Unknown: Fukushima Now” All I Know About Teacher Li - #1452: Immersive Documentary about Chinese Censorship “All I Know About Teacher Li” Tops Venice Immersive Audience Favorites Survey Ancestors - #1507: From Selfies to Virtual Offspring, “Ancestors” Turns Strangers into Family and Intergenerational Speculative Futures Anouschka - #1334: “Anouschka” Wins IDFA DocLab Digital Storytelling Award with AR Narrative Game about Cultural Heritage A Number From the Ghost EchoVision Fly To You Future Botanica - #1514: Creating GenAI Flower Ecosystem with “Future Botanica” AR App Just for You Trilogy All That Remains - #1126: Re-creating Experimental Theatre 1:1 Encounters via 360 video with “All That Remains” Over the Rainbow - #1250: Translating 1:1 Experimental Theatre Encounters into 360 Video to Transcend Expectations with Craig Quintero A Simple Silence - #1422: “A Simple Silence” Wraps Up Craig Quintero’s Trilogy Exploring Spatial Transformations & Immersive Encounters Origins - Life's Epic Journey Oto's Planet Resolution: A Cinephonic Rhapsody for the Soul Sweet! Symphoni The Art of Change - #1446: Studio Syro’s “The Art Of Change” is the Best Immersive Music Video Experience I’ve Seen So Far Pushing at the Edges of Spatial Grammar Uncanny Alley: A New Day - #1442: Combining VRChat Worldbuilding with Immersive Theatre Storytelling in “Uncanny Alley: A New Day” SXSW XR Events Spreadsheet of Unofficial List of Parties & Events @ SXSW 2025 for the XR Community SXSW 2025 Film & TV Opening Night Party (Platinum and Film & TV Badges only) - Mar 7, 2025 10:00pm CT Creators & Cocktails: Happy Hour with Kwame Appiah - Mar 8, 2025 5:00pm CT ASU Happy Hour At SXSW - Mar 9, 2025 5:00pm CT VR/AR/XR Dev Meet Up - Mar 11, 2025 2:30pm CT Television Academy Emerging Media Peer Group Meet Up - Mar 11, 2025 4:00pm CT Future X Music Showcase Presented by Ristband - Mar 11, 2025 8:40pm CT NOWISWHENWEARE (the stars) is showing on afternoon & evening slots on March 11th & 12th SXSW 2025 Film & TV Intermission Party (Platinum and Film & TV Badges only) - Mar 12, 2025 10:00pm CT This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

The Theater application for the Apple Vision Pro by Sandwich Vision launched in June 2024 with the spatial livestreaming of John Gruber's live interview of Apple Executives at WWDC. They have been collaborating with independent producers of spatial video to distribute rentals and purchases of immersive content spanning across stereoscopic 180-videos to educational planetarium content to more experimental dome content as well as stereoscopic captures of live music performances (I can highly recommend checking out Fungi: Web of Life by K2 Studios). Users can also connect their private collections of movies via Plex Media Servers to enjoy their own private movie collection in a virtual home theater experience with a number of different theater options. They're also currently implementing social features, and may start having live screenings of classic movies with potential collaborations with independent movie theaters. I had a chance to catch up with Sandwich Vision co-founder Adam Lisagor to hear a bit more about his journey into the XR space, how he's blended storytelling and technology throughout his career, as well as his hopes for what might be a new renaissance of spatial video content given Apple's emphasis of on 180-degree spatial video rather than the VR filmmaking era of 360-degree videos. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

Lovesick is a narrative game featuring a character-driven story of a bank on the brink of collapse that faces a series of supernatural events that they must escape from. It features a lot of escape room and puzzle mechanics that are mixed with embodied gameplay, but at the heart it's about the relationship between these four different band members in what I found to be a real gem of a story with a payoff ending that made it worth the journey. I had a chance to sit down with director and co-writer Corey Warning from Rose-City Games to get a bit more context for how Lovesick came about catalyzed by some funding from Meta, their iterative design process along the way that was centered and driven by the story they wanted to tell, as well as their hope to bring some narrative-driven content and indie spirit to the VR ecosystem. https://www.youtube.com/watch?v=mAb_7fewd6c This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

In a series of posts from February 8th to 11th, open source developer Matthieu Bucchianeri started making allegations that Meta's OVRPlugin is blocking non-Meta OpenXR runtimes, thereby undermining the open and interoperable spirit of OpenXR. He claims that "the OVRPlugin takes intentional precautions to exclude non-Meta platforms. This means that XR content developed with OVRPlugin will only work with Quest Link, and it will not work with any other runtime." Bucchianeri's allegation is that if a PCVR game includes Meta's OVRPlugin as their OpenXR middleware and doesn't implement the counter-blocking measures that he details, then PCVR users will only be able to use a Quest headset with Quest Link while other non-Meta headsets like "Pimax, Pico, Varjo, Vive" will be blocked, even if they have conformant OpenXR runtimes. Bucchianeri has validated these blocking counter-measures, and he says, "as proven with many applications using OVRPlugin with counter-measures enabled, these applications can run on a conformant OpenXR implementation." Not very many XR developers are willing to speak about these issues on the record, but I did manage to record a Voices of VR podcast interview with Virtual Desktop's Guy Godin who was able to independently corroborate many of the core allegations from Bucchianeri. Godin collaborated with Bucchianeri on the Virtual Desktop OpenXR (VDXR) runtime, but has also been receiving many complaints from PCVR users around Meta's OpenXR non-compliance issues, especially with games that launch both on Quest and PCVR and use Meta's OVRPlugin. Godin told me that games will work fine natively on the Quest, but any non-Quest headsets or if they aren't using Quest Link will be blocked on the PCVR version, unless the developer specifically implements anti-blocking counter-measures detailed by Bucchianeri in his technical write-up. It appears as though OpenXR conformance from the Khronos Group only pertains to the actual OpenXR runtimes on the hardware, but headset manufacturers are able to create their own SDK plug-in middleware that interfaces with OpenXR that doesn't have the same conformance requirements. It appears as though Meta is able to technically maintain their OpenXR runtime conformant status because it does not apply to their OVRPlugin middleware SDK solution, and Bucchianeri is claiming that Meta is undermining the normative standards of interoperability by not following the best practices from the Khronos Group. He says, "For the past several years, Khronos has come up with best practices and solutions to develop OpenXR applications and maximize cross-vendor and cross-platform interoperability. Khronos has asked XR developers all over the world to follow these best practices, however - Meta - the largest vendor in Khronos is refusing to follow these best practices." Bucchianeri is also claiming to have had private communications with Meta confirming that these were deliberate and intentional changes. He says, "This is not an accident: this concern was reported to Meta early in 2024 via official means in the Khronos group. Meta acknowledged purposedly blocking other platforms from running OpenXR content at that time." I was able to confirm in my discussion with Guy Godwin in this Voices of VR podcast episode that he also believes that these were deliberate and intentional changes to undermine the spirit of OpenXR. Part of why Bucchianeri was blowing the whistle is because the Khronos Group had not been taking any action against Meta. He describes Meta's actions as "reverting many of the improvements to the developers and users ecosystem that Khronos has spent time, money, energy into solving for the past 7 years." He says, "Unfortunately, since 2024, Khronos has refused to take actions to stop Meta’s OVRPlugin destructive initiative towards the PCVR ecosystem. By not taking any actions to resolve the issues created by Meta’s OVRPlugin, Khronos is sending the message that OpenXR is no lo...

Niantic launched their Into the Scaniverse application on Quest 3 on February 26th, 2025 that features over 50,000 Gaussian Spats from 120 different countries. They originally launched the WebXR version on December 10th, 2024 at IntoTheScaniverse.com, which was built using Niantic Studio (be sure to check out their comprehensive history of Gaussian Splats by Kirsten M. Johnson released at the same time). Users can use the Scaniverse mobile app on Android or iOS to capture, render, geotag, and upload their own Gaussian Splats onto the Into the Scaniverse mapps that can be viewed on either mobile phone or XR devices. I had a chance to speak more about Into the Scaniverse with Joel Udwin, who is Niantic's Director of Product for Niantic's AR, Research, Developer Platforms, and Scaniverse. Gaussian Splats are only about 1 year and a half old as the original "3D Gaussian Splatting for Real-Time Radiance Field Rendering" paper was presented at SIGGRAPH in August 2023, but it represents a new rendering pipeline for volumetrically captured content. Niantic's Into the Scaniverse apps are able to process and render these splats locally on the phone or Quest devices, and they have a lot of plans for how they will continue to utilize and develop this as a core part of their technology infrastructure and enabling new mixed reality applications. https://www.youtube.com/watch?v=NR51MrAtUM4 This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

HTC has launched Viverse platform that is aiming to become the YouTube of 3D Content. It's a platform that hosts 3D content and worlds built on PlayCanvas as well as many other open source technologies like WebXR, WebGPU, and VRM. It is sort of like a mix between VRChat with social worlds, 3D web crypto-based metaverse worlds, but also with some more enterprise embedding features as well. It's exciting to see this shift towards building out the open and interoperable metaverse, but in the context of this hybrid walled garden context that is built using open web technology stacks. It is also a hybrid in another way in that it is mostly consumer-facing but also has enterprise use cases like privately embedding of 3D content. I interviewed HTC's Andranik Aslanyan about the new VIVERSE platform, how they recruited over a hundred XR and WebXR developers to seed the content, and how VIVERSE fits into their overall strategy. I get some clarifications on the Android XR non-exclusive IP acquisition, and how VIVERSE fits into where HTC will be heading now that we're coming up on 10 years the HTC Vive was announced on March 1, 2014. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

The latest Polys Ombudsperson of the Year is Ingrid Kopp, and previous winners Brittan Heller, Michaela Montagna, and myself gathered again on December 8 2024 to reflect on the current state of the XR industry as well as the specific ethical concerns about the intersection between XR and AI. We also speak about some of the power of storytelling to unpack some of these moral dilemmas, as well as tech policy in the context of the existing political climate, but also some of the broader and general trends of the XR industry. In my takeaways at the end, I also elaborate on the leaked memo from Meta CTO Andrew "Boz" Bosworth saying "We need to drive sales, retention, and engagement across the board but especially in MR. And Horizon Worlds on mobile absolutely has to break out for our long term plans to have a chance. If you don't feel the weight of history on you then you aren't paying attention. This year likely determines whether this entire effort will go down as the work of visionaries or a legendary misadventure." A lot of their Meta's commitment to XR seems to be directly tethered to the success or failure of their own first-party app. Zuckerberg's AR/VR Strategy memo from 2014 leaked to TechCrunch by Blake Harris now seems particularly prescient: "for now keep in mind that we need to succeed in building both a major platform and key apps to improve our strategic position on the next platform. If we only build key apps but not the platform, we will remain in our current position. If we only build the platform but not the key apps, we may be in a worse position." Zuckerberg later elaborates that apps and experiences are their top priority in XR: "I think you can divide the ecosystem into three major parts: apps / experiences, platform services and hardware / systems. In my vision of ubiquitous VR / AR, these are listed in order of importance." As time has passed, it's become more and more clear that the success of Meta's Horizon worlds may be directly tethered to Meta's overall commitment to the XR industry. Just yesterday, Meta announced a $50 million dollar fund for Meta Horizon World creators, with special awards for mobile: "By investing in mobile content, we can reach a lot of new people who don’t yet own a Quest headset and ultimately grow the pie for everyone. That’s why, as part of our $50 million Creator Fund for creators of mobile and MR worlds, we’re also announcing our first creator competition of the year with a focus on mobile." UploadVR did a really great follow-up report about "From Quest To Horizon: How Meta's Shifting Priorities Are Affecting Developers" that includes many anonymous statements from XR developers talking about the impact of Meta's shifting priorities. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

Oliver Kadel's 1618 Digital specializes in spatial audio sound design and engineering across linear immersive storytelling and interactive immersive experiences. Kadel also co-hosts The Immersive Audio Podcast with Monica Bolles, and I was recently featured in episode #109. I sat down with Kadel to get a survey of some of the spatial audio formats and production pipelines ranging from 3rd order ambisonics, object-oriented audio in game engines, working with the audio formats for Apple Immersive Video, Dolby Atmos, MPEG-H Audio, and emerging open standards like the Immersive Audio Model & Format (IAMF), which is "an audio container specification designed to revolutionize immersive audio experiences across a wide range of applications, including streaming, gaming, augmented reality (AR), virtual reality (VR), and traditional broadcasting." We talk about some of his favorite immersive audio productions, his process of getting a Ph.D. studying the impact of spatial audio on cognitive load, and some of the future trends of spatial audio.

The Khronos Group is working on a new open standard tentatively called the glTF External Reference Format (or glTFx for short), which will allow users to add multiple glTF assets to a single scene. This used to be called the "glTF Experience Format (glXF)," and the final name is still being finalized. This will enable new capabilities that are not possible by using glTF itself to contain other glTF assets, which includes is level-of-detail (LoD) management, asset-based lighting Management, and billboarding. The Khronos Group continues to translate the core functionality of game engines into various open standards, and are also working towards finalizing glTF extensions for interactivity as well as for physics. I did a deep dive with Khronos 3D Formats Working Group Chairman Alexey Medvedev (AR Tech Lead at Meta/Facebook), and 3D Formats Working Group Vice Chairman Dan Frith (Chief Creative Officer at London Dynamics) to unpack all of the latest news about glTF and glTF External Reference Format. . This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

I dig into my archives to unpack the one-on-one, immersive theatre piece that I saw online back in 2021 called TM that used a Socratic dialectic to interrogate you as you're going through a process of joining an imaginary cult. I spoke with Ontroerend Goed's artistic director Alexander Devriendt to get a lot more context for how he designs immersive theatre productions for folks who typically don't like immersive theatre. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality