77: How to build a generic neuromotor interface

Join Pascal as he explores the groundbreaking world of generic neuromotor interfaces with Jesse, Lauren, and Sean. Discover how these technologies enable control of devices with just a flick of the wrist or even a simple intention to move. We'll discuss the role of AI in eliminating the need for personalised training, the differences between non-invasive interfaces and their predecessors, and the exciting implications for accessibility. Don't miss this deep dive into the future of human-computer interaction. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Timestamps Intro 0:06 Jesse introduction 1:29 Lauren introduction 2:42 Sean introduction 3:29 Team's mission statement 3:49 What's a neuromotor interface? 4:24 Paper overview 5:29 Non-invasive interfaces 7:50 How to make it generic 9:42 Design tradeoffs 11:29 Real-world model performance 14:21 Feedback cycle 16:22 LLMs and EMG 17:22 Handwriting vision 18:39 Working with product 20:55 EMG for accessibility 22:25 How Meta helps 25:53 Open-source repos 28:02 What's next? 28:45 Outro 30:51 Links A generic non-invasive neuromotor interface for human-computer interaction - Nature - https://www.nature.com/articles/s41586-025-09255-w  How the low-vision community embraced AI smart glasses - The Verge - https://www.theverge.com/the-vergecast/701018/ray-ban-meta-smart-glasses-be-my-eyes-ceo-accessibility-tech MKBHD on Orion - https://www.youtube.com/watch?v=G0eKzU_fV00 

What happens when decades-old C code, powering billions of daily messages, starts to slow down innovation? In this episode, we talk to Meta engineers Elaine and Buping, who are in the midst of a bold, incremental rewrite of one of our core messaging libraries—in Rust. Neither came into the project as Rust experts, but both saw a chance to improve not just performance, but developer experience across the board. We dig into the technical and human sides of the project: why they took it on, how they’re approaching it without a guaranteed finish line, and what it means to optimise for something as intangible (yet vital) as developer happiness. If you’ve ever wrestled with legacy code or wondered what it takes to modernise systems at massive scale, this one’s for you. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Timestamps Intro 0:06 Introduction Elaine 1:54 Introduction Buping 2:49 Team mission 3:15 Scale of messaging at Meta 3:40 State of native code on Mobile 4:40 Why C, not C++? 7:13 Challenges of working with C 10:09 State of Rust on Mobile 18:10 Why choose Rust? 23:36 Prior Rust experience 28:55 Learning Rust at Meta 34:14 Challenges of the migration 37:47 Measuring success 42:09 Hobbies 45:15 Outro 46:41

Pyrefly is a faster, open-source Python type checker written in Rust, succeeding Pyre. But what prompted the rewrite and what besides the language choice ended up making it faster? Host Pascal talks to Maggie, Rebecca and returning guest Neil about the unexpected complexities of building an incremental type checker that scales to mono repositories in episode 75. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Pyrefly: https://pyrefly.org/ Pyre: https://pyre-check.org/  Ruff: https://github.com/astral-sh/ruff  PEP 484: https://peps.python.org/pep-0484/  Timestamps Intro    0:06 Rebecca Introduction    1:45 Maggie Introduction    2:45 Neil (Re-)Introduction    3:12 Team Mission    3:56 History of Typing in Python    4:29 The State of Typed Python at Meta    5:32 fbcode    6:02 Original Motivation for building Pyre    6:19 Justifying the Rewrite    7:48 Pyrefly vs the Rest    9:41 Why Rust?    10:45 Fearless Concurrency    12:02 Why is it faster?    12:37 Python community and Rust    14:57 Pyrefly wasm crate    15:46 Upgrade experience    17:34 Type checking differences    19:12 IDE experience    21:31 State of Pyrefly at Meta    22:27 Being open-source-first    23:36 Open-source challenges    25:06 Unexpected challenges    26:39 Outro    31:05  

To ensure that everyone has access to resilient, high-speed and low-latency connections to Meta services, no matter where in the world they are, Meta makes large-scale investments into subsea cable infrastructure. The recently announced Project Water worth will, Once complete, reach five major continents and span over 50,000 km (longer than the Earth’s circumference), making it the world’s longest subsea cable project using the highest-capacity technology available.  In this episode, host Pascal talks with another Pascal and his colleague Andy who are involved at every stage of these projects and share the surprising challenges one has to deal with when working on the largest subsea cable project in the world. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Project Waterworth: https://engineering.fb.com/2025/02/14/connectivity/project-waterworth-ai-subsea-infrastructure/  Timestamps Intro 0:06 Introduction Andy 2:14 Introduction Pascal 3:21 Why do we build our own subsea cable infrastructure? 4:15 Current state of Meta-owned subsea cables 6:20 Project Waterworth 7:40 Why invest in more subsea cables? 9:00 What does a cable look like? 11:14 The process of laying subsea cable 16:39 Unexpected findings on the ocean floor 19:25 Shallow vs deep ocean 21:12 Merging different cable types 24:00 What happens when a cable breaks? 25:04 Memorable challenges 27:42 Cable capacity 29:51 The long history of subsea cables 33:07 What's next? 36:27 Outro 39:02

Join Pascal and Sabrina on the latest Meta Tech Podcast episode as they discuss the evolution and future of GraphQL. From client-side consistency to innovative APIs, learn how GraphQL is making developers' lives easier and enhancing user experiences. Discover surprising insights into the challenges of building a mobile GraphQL platform and how it's transforming product development at Meta.  Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links GraphQL: https://graphql.org/  Relay: https://relay.dev/  Sabrina at GraphQL Conf 2024: https://www.youtube.com/watch?v=PGBC-0E-kco  Timestamps Intro 0:06 Introduction Sabrina 1:42 Sabrina's team 2:47 What's GraphQL? 3:18 Relay and Mobile GraphQL Clients 4:01 GraphQL Consistency Engine 4:54 Pando Mobile GraphQL Client 7:16 Interfacing with Pando 8:03 Code generation 9:14 Inventing new features 10:43 The hidden complexity behind pagination 11:52 Working inside the GraphQL spec 16:00 Complexity tradeoffs 18:30 State of GraphQL at Meta 21:16 Measuring success 24:58 Optimistic Mutations 27:31 Collaboration model 31:42 Preventing early adoption 34:43 The challenge of migrating FBApp 37:10 What's next for mobile GraphQL? 40:22 Outro 41:54  

In this episode of the Meta Tech Podcast, host Pascal sits down with Shane, a research scientist at Meta, to explore the cutting-edge research behind Ray-Ban Meta glasses. Shane shares insights from his seven-year journey at Meta, where he focuses on computer vision and multimodal AI within the Wearables AI organization. Tune in to learn how Shane's team is pioneering foundational models for Ray-Ban Meta glasses, tackling unique challenges, and pushing the boundaries of AI-driven innovation. Discover how multimodal AI is transforming user experiences and get a glimpse into the future of wearable technology. Whether you're an engineer, a tech enthusiast, or simply curious about the latest advancements, there is something for everyone in this episode.  Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model - https://arxiv.org/abs/2309.16058  Be My Eyes Programme: https://www.forbes.com/sites/stevenaquino/2024/10/11/inside-the-be-my-eyes-meta-collaboration-and-the-allure-to--impact-humanity/  Meta Open Source on Threads: https://www.threads.net/@metaopensource  CacheLib: https://cachelib.org/  Meta’s AI-Powered Ray-Bans Are Life-Enhancing for the Blind - Wall Street Journal: https://www.wsj.com/tech/ai/metas-ai-powered-ray-bans-are-life-enhancing-for-the-blind-3ae38026  Timestamps Intro 0:06 OSS News 0:56 Introduction Shane 1:30 The role of research scientist over time 3:03 What's Multimodal AI? 5:45 Applying Multimodal AI in Meta's products 7:21 Acoustic modalities beyond speech 9:17 AnyMAL 12:23 Encoder zoos 13:53 0-shot performance 16:25 Iterating on models 17:28 LLM parameter size 19:29 How do we process a request from the glasses? 21:53 Processing moving images 23:44 Scaling to billions of users 26:01 Where lies the optimisation potential? 28:12 Incorporating feedback 29:08 Open-source influence 31:30 Be My Eyes Programme 33:57 Working with industry experts at Meta 36:18 Outro 38:55

How do you translate roughly ten million lines of Java code to Kotlin? Clicking in your the IDE gets pretty repetitive after a while and doesn’t work if you have custom APIs and requirements for null safety. Eve and Jocelyn, two software engineers on the Mobile Infra Codebases Team have taken on this challenge and talk host Pascal through the unexpected difficulties when embarking on the journey to (close to) 100% Kotlin in our Android codebase. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Meta Engineering Blog - Translating Java to Kotlin at Scale: https://engineering.fb.com/2024/12/18/android/translating-java-to-kotlin-at-scale/  Open-source transformations: https://github.com/fbsamples/kotlin_ast_tools  Mobile @Scale Conference recordings: https://www.youtube.com/watch?v=L7xSnbrk4CI Timestamps Intro 0:06 Introduction Eve 1:11 Introduction Jocelyn 2:15 Team mission 2:44 The scale of Meta's codebase 3:40 Why is there so much code? 4:34 Why migrate to Kotlin? 5:45 Isn't Kotlin slow to compile? 7:51 Why not use Android Studio's converter? 8:28 Nullability differences 10:04 Meta Codemod Service 14:50 Kotlin codemod stages 17:07 Headless J2K 20:14 Open-source transformations 23:14 Java Nullsafe 24:47 Leveraging Linters 26:01 Fixing build errors 27:24 Unexpected challenges 29:33 State of the union 33:44 Outro 36:10 Outtakes 37:08

Introducing a new Android UI Framework like Jetpack Compose into an existing app is easy right? Import some AARs and code away. But what if your app has specific performance goals to meet, has existing design components, integrations with navigation and logging frameworks? That is where Summer and her team come in who handle large-scale migrations for Instagram. They aim to provide developers with the best possible experience when working on our code bases, even if that requires some temporary pain on the side of infrastructure teams that have to maintain multiple implementations at once. Why Summer thinks it is worth it, how they approach the rollout of a new framework and so much more is all discussed in episode 70. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Jetpack Compose: https://developer.android.com/compose Litho: https://fblitho.com/ Google Showcase: Meta built threads in only 5 months using Jetpack Compose: https://android-developers.googleblog.com/2023/10/meta-built-threads-in-only-5-months-using-jetpack-compose.html Flipper: https://fbflipper.com/ Timestamps Intro 0:06 Intro Summer 1:29 Notable differences moving from FB to IG 2:26 The Instagram Data & UI Architecture team 2:58 Why modernise? 3:44 Where has the risk paid off? 6:08 What does Compose look like? 7:49 Compose v Litho 11:15 Where does Litho still have the upper hand? 14:53 Meta contributions to Compose 16:38 Compose pitfalls 19:10 Rolling Compose out across the company 20:13 Design systems 22:12 Downsides of establishing another UI framework? 24:22 Rollout stages 28:43 Experimentation stage 32:32 Closed enrollment phase 38:15 Graduation criteria 39:38 Outro 42:20 Bants 44:04

Do types actually make you more productive or is it just more typing for you to do on the keyboard? That's just one of the questions we managed to answer at least on a small scale with Diff Authoring Time or DAT, here at Meta. Want to know how we leverage metrics to run experiments on productivity in our internal codebase? Tune in to episode 69. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host @passy (https://mastodon.social/@passy, and https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Hack language: https://hacklang.org/  Timestamps Intro 0:06 Henri Intro 1:45 Ian Intro 3:13 Moritz Re-Intro 3:28 DAT Recap 3:48 What is Hack? 4:20 Inner and outer loop 14:13 Experimenting with language features 17:47 Code sharing frameworks at Meta 27:43 Measuring framework productivity 29:01 Will we see more experiments? 34:23 Time savings from code sharing 37:28 Outro 39:03 Blooper 39:52

How do you build your own mixed reality headset from sketch to scale? That's exactly what Alfred Jones, VP of hardware engineering at Meta Reality Labs, discussed with host Pascal. From choosing the right display technology, battery, thermal budget and of course hitting the right price point. How he manages to not fall victim to choice paralysis and so much more in episode 68. Got feedback? Send it to us on Threads (https://threads.net/@metatechpod) or Instagram (https://instagram.com/metatechpod) and don’t forget to follow our host Pascal (https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links Caddy: https://engineering.fb.com/2024/07/18/virtual-reality/caddy-cad-mixed-reality-mr-meta/ Timestamps Intro 0:06 Alfred Introduction 1:40 Who do you work with? 3:23 Decision making frameworks 5:20 Is MR the final destination? 7:19 What makes good passthrough such a challenge? 10:18 How to build your own MR headset 13:51 Hardware design constraints 19:00 Prototype phases 22:34 Durability testing 26:23 Dogfooding at Meta 28:55 Magic wand for technical limitations 31:56 Outro 34:26