68: How to Build a Mixed Reality Headset

Introducing a new Android UI Framework like Jetpack Compose into an existing app is easy right? Import some AARs and code away. But what if your app has specific performance goals to meet, has existing design components, integrations with navigation and logging frameworks? That is where Summer and her team come in who handle large-scale migrations for Instagram. They aim to provide developers with the best possible experience when working on our code bases, even if that requires some temporary pain on the side of infrastructure teams that have to maintain multiple implementations at once. Why Summer thinks it is worth it, how they approach the rollout of a new framework and so much more is all discussed in episode 70. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Jetpack Compose: https://developer.android.com/compose Litho: https://fblitho.com/ Google Showcase: Meta built threads in only 5 months using Jetpack Compose: https://android-developers.googleblog.com/2023/10/meta-built-threads-in-only-5-months-using-jetpack-compose.html Flipper: https://fbflipper.com/ Timestamps Intro 0:06 Intro Summer 1:29 Notable differences moving from FB to IG 2:26 The Instagram Data & UI Architecture team 2:58 Why modernise? 3:44 Where has the risk paid off? 6:08 What does Compose look like? 7:49 Compose v Litho 11:15 Where does Litho still have the upper hand? 14:53 Meta contributions to Compose 16:38 Compose pitfalls 19:10 Rolling Compose out across the company 20:13 Design systems 22:12 Downsides of establishing another UI framework? 24:22 Rollout stages 28:43 Experimentation stage 32:32 Closed enrollment phase 38:15 Graduation criteria 39:38 Outro 42:20 Bants 44:04

Do types actually make you more productive or is it just more typing for you to do on the keyboard? That's just one of the questions we managed to answer at least on a small scale with Diff Authoring Time or DAT, here at Meta. Want to know how we leverage metrics to run experiments on productivity in our internal codebase? Tune in to episode 69. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host @passy (https://mastodon.social/@passy, and https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Hack language: https://hacklang.org/  Timestamps Intro 0:06 Henri Intro 1:45 Ian Intro 3:13 Moritz Re-Intro 3:28 DAT Recap 3:48 What is Hack? 4:20 Inner and outer loop 14:13 Experimenting with language features 17:47 Code sharing frameworks at Meta 27:43 Measuring framework productivity 29:01 Will we see more experiments? 34:23 Time savings from code sharing 37:28 Outro 39:03 Blooper 39:52

How do you build your own mixed reality headset from sketch to scale? That's exactly what Alfred Jones, VP of hardware engineering at Meta Reality Labs, discussed with host Pascal. From choosing the right display technology, battery, thermal budget and of course hitting the right price point. How he manages to not fall victim to choice paralysis and so much more in episode 68. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod) or Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Caddy: https://engineering.fb.com/2024/07/18/virtual-reality/caddy-cad-mixed-reality-mr-meta/ Timestamps Intro 0:06 Alfred Introduction 1:40 Who do you work with? 3:23 Decision making frameworks 5:20 Is MR the final destination? 7:19 What makes good passthrough such a challenge? 10:18 How to build your own MR headset 13:51 Hardware design constraints 19:00 Prototype phases 22:34 Durability testing 26:23 Dogfooding at Meta 28:55 Magic wand for technical limitations 31:56 Outro 34:26

At Meta, engineers are our biggest asset which is why we have an entire org tasked with making them as productive as possible. But how do you know if your projects for improving developer experience are actually successful? For any other product, you would run an A/B test but that requires metrics and how do you measure developer productivity? Sarita and Moritz have been working on exactly that with Diff Authoring Time which measures how long it took to submit a change to our codebase. Host Pascal talks to them about the way this is implemented, the challenges and abilities this unlocks. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Twitter (https://twitter.com/metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host @passy (https://twitter.com/passy, https://mastodon.social/@passy, and https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. You can follow our guest Moritz on X (https://x.com/Inventitech) or check out his website on inventitech.com. Links Meta Connect 2024: https://www.meta.com/en-gb/connect/ Timestamps Episode intro 0:05 Sarita Intro 2:33 Moritz Intro 3:44 DevInfra as an Engineer 4:25 DevInfra as a Data Scientist 5:12 Why DevEx Metrics? 6:04 Average Diff Authoring Time at Meta 9:55 Events for calculating DAT 10:55 Edge cases 13:15 DAT for Performance Evaluation? 20:29 Analyses on DAT data 22:29 Onboarding to DAT 23:23 Stat-sig data 25:06 Validating the metric 26:34 Versioning metrics 28:09 Detecting and handling biases 29:19 Diff coverage 30:30 Do we need DevX metrics in an AI software engineering world? 31:23 Measuring the impact of AI tools 32:23 What's next for DAT? 33:40 Outtakes 36:22

Bento is Meta’s internal distribution of Jupyter Notebooks, an open-source web-based computing platform. Host Pascal is joined by Steve who worked with his team on building many features on top of Jupyter, including scheduled notebooks, sharing with colleagues and running notebooks without a remote server component by leveraging Webassembly in the browser. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Twitter (https://twitter.com/metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host @passy (https://twitter.com/passy, https://mastodon.social/@passy, and https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Scheduling Jupyter Notebooks at Meta: https://engineering.fb.com/2023/08/29/security/scheduling-jupyter-notebooks-meta/ Serverless Jupyter Notebooks at Meta: https://engineering.fb.com/2024/06/10/data-infrastructure/serverless-jupyter-notebooks-bento-meta/ Jupyter Notebooks: https://jupyter.org/  Timestamps Intro 0:06 Who is Steve? 1:49 What are Jupyter and Bento? 2:48 Who is Bento for? 3:40 Internal-only Bento features 4:42 Scheduled notebooks 11:39 Integrating with existing batch jobs 17:10 The case for serverless notebooks 20:59 Enter wasm 24:29 Upgrade paths from serverless to server 26:29 Bringing more Python libraries to the browser 30:21 Adding magick(s) 31:52 DataFrame magic and AI 36:41 What's next? 38:29 Outro 43:17

We don’t know when but at some point in the future we will face what researchers call a "Quantum Apocalypse". This is when quantum computers will be able to break many of our existing encryption algorithms. To keep Meta’a users safe even from attacks that don’t even exist today, Sheran and Rafael are working on post-quantum-ready encryption. Tune in to learn about the various challenges and trade offs that this work brings with it.   Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Twitter (https://twitter.com/metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host @passy (https://twitter.com/passy, https://mastodon.social/@passy, and https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Post-quantum readiness for TLS at Meta: https://engineering.fb.com/2024/05/22/security/post-quantum-readiness-tls-pqr-meta/  Fizz TLS implementation: https://github.com/facebookincubator/fizz  liboqs: https://github.com/open-quantum-safe/liboqs  NIST Post-Quantum Cryptography Submissions: https://csrc.nist.gov/Projects/post-quantum-cryptography/post-quantum-cryptography-standardization/round-3-submissions    Timestamps Intro 0:06 Meta Open Source 101 1:10 Intros 1:49 Sheran Intro 2:31 Rafael Intro 3:37 Then Quantum Apocalypse 5:24 Why symmetric and asymmetric algos behave differently 8:10 Why invest in tomorrow's problems? 9:21 First deployment target 14:17 Choosing an algorithm 18:06 Choosing the right parameters 19:51 Performance costs and wins 21:28 Stack 23:33 Challenges 25:26 What's next for PQC? 30:38 Working with NIST 32:59 Outro 34:30 Outtakes 35:43

After sitting in one too many Zoom meetings looking at flat images of 3D models, mechanical engineers Ed, Jason, Fan, and Raghavan decided that they could do better, taught themselves how to code and started to build Caddy - a CAD app for mixed reality. Tune in to episode 64 to hear their story. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Twitter (https://twitter.com/metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host @passy (https://twitter.com/passy, https://mastodon.social/@passy, and https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Caddy video: https://heycaddy.net/ Caddy on the Quest Store: https://www.meta.com/en-gb/experiences/24212682218375897/  @Scale conference on YouTube: https://www.youtube.com/channel/UCd9I8ZkgoR1d7GeSj_wi_LQ  MLow @Scale talk: https://www.youtube.com/watch?v=3ypsZUNRjI4 MLow blog post: https://engineering.fb.com/2024/06/13/web/mlow-metas-low-bitrate-audio-codec/  Faster Incident Response with GenAI @Scale talk: https://www.youtube.com/watch?v=rpe7eAR90Ko Llama 3: https://llama.meta.com/llama3/ Meta Unity SDKs: https://developer.oculus.com/documentation/unity/ Prisms VR: https://www.prismsvr.com/  Timestamps Intro 0:06 Ed Intro 2:12 Raghavan Intro 3:15 Fan Intro 3:44 Jason Intro 4:16 What is Caddy? 4:49 Why build Caddy? 6:52 Discovery of hand-based interactions 11:46 Supported import formats 14:09 Learning to code 18:09 Time to Caddy MVP 27:48 Off-the-shelf components 29:04 Outgrowing the initial vision 32:48 AI in Caddy 43:25 Challenges building Caddy 52:38 What's next? 55:40 How to get in touch? 56:56 Excitement in MR 57:38 Outro 1:03:35

Aida was part of one of the first Rust teams here at Meta. One of the biggest challenges was interacting with the large amount of existing C++. With the release of cxx, safe interop between C++ and even async Rust has become a lot easier. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Twitter (https://twitter.com/metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host @passy (https://twitter.com/passy, https://mastodon.social/@passy, and https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Aida’s talk at Rust Nation: https://youtu.be/tsEuA9S5q9Q cxx: https://crates.io/crates/cxx Sapling: https://sapling-scm.com/   Timestamps Intro 0:06 Aida Intro 2:06 Rust in Meta Source Control 2:50 State of Rust at Meta 10:11 bindgen 13:25 cxx vs bindgen 17:49 async Rust and C++ 19:04 Dealing with Lifetimes 28:19 Fixing Memory Leaks 31:25 Thread safety with Send and Sync 33:48 A Magic Wand for Async Rust 39:52 Outro 43:04 Outtakes 43:50  

The basic version of Threads for web was built in just under three months by two engineers, mirroring the nimble engineering practices we talked about on this podcast before when it came to launching Threads for Android and iOS. In this episode, Pascal is joined by Ally and Kevin, two engineers on the Threads Web team. They talk about how shared infrastructure with other Meta web properties allows them to move fast and how they manage to balance the need to ship new features with the desire to craft delightful experiences for their users.  Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Twitter (https://twitter.com/metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host @passy (https://twitter.com/passy, https://mastodon.social/@passy, and https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Threads: https://threads.net StyleX: https://stylexjs.com/  FlowJS: https://flow.org/  Introducing Meta Llama 3: https://ai.meta.com/blog/meta-llama-3/ Building custom silicon for the future of AI: https://www.metacareers.com/life/building-custom-silicon-for-the-future-of-ai Building Meta’s GenAI Infrastructure: https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/  Timestamps Intro 0:06 Intro Ally and Kevin 1:44 Why focus on Web? 2:48 Kevin's contributions 4:42 Focus on craft 6:18 Editing Threads 7:34 Ally's contributions 10:40 Prioritising delight and shipping features 12:02 Launching Threads Web 13:30 Shared Infra 16:13 Tech Stack 19:15 The DevX of Meta www 23:51 Challenges 30:57 Favourite bit of polish 34:32 Outtakes 39:18

Every day, trillions of image download requests are made from Meta’s family of apps. Zuzanna works on the Media Platform Team that owns the entire flow from serving images from the CDN to displaying the pixels on your phone. One of the project she and her team recently worked on was rolling out HDR images to Instagram and Threads and in this episode’s interview, Zuzanna tells show host Pascal how they partnered with large phone manufacturers to develop and roll out the new feature. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Twitter (https://twitter.com/metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host @passy (https://twitter.com/passy, https://mastodon.social/@passy, and https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Fresco: https://frescolib.org/  Timestamps Intro 0:06 Intro Zuza 1:44 Image Infra Team 3:23 The scale of images at Meta 5:10 Measuring quality 10:01 HDR photos 12:13 HDR file formats 19:02 HDR photo availability today 22:34 Partnering with phone vendors 30:40 Why photos after videos? 32:27 Outro 36:22