From Cloud Costs To Tiny Chips: Building Reliable Voice AI On The Edge - with femto.AI

What if your phone could outpace bigger cloud models on the tasks you care about—and still keep quality high? We walk through how we built LFM2, our family of “Liquid Foundation Models,” by starting from the hardware up. Phones face tight memory and bandwidth, laptops have more RAM but similar bottlenecks, and cloud GPUs invite scale but at a cost. So instead of forcing one transformer to fit all, we engineered inference‑aware architectures that adapt to each device class.We explain how our STAR evolutionary search explores a modular design space that mixes attention with state space models, linear attention, and convolutional components. Every candidate is judged in the real world: hardware‑in‑the‑loop measurements capture KV cache, activation memory, prefill throughput, and decode latency on a Galaxy S24, then models are trained and checked for perplexity and benchmark quality. Over generations, weak designs are pruned and strong ones recombine, moving toward an architecture that is not only smaller and faster but also competitive on MMLU, multilingual MMLU, and math.The results are tangible. LFM2 outperforms larger transformer baselines like Llama variants on edge decode speed, and the gains transfer to laptops with AMD Ryzen CPUs. We’ve released open weights at multiple sizes (350M to 1.2B, plus 2.6B and an MoE variant) with integrations in llama.cpp and NPU paths for Qualcomm, with AMD support coming soon. Beyond text, we extend the same efficient backbone to vision‑language models for on‑device video understanding and to a native audio stack that interleaves text and audio tokens—reducing latency and simplifying the classic STT→LLM→TTS chain.If you care about low latency AI, local privacy, and real‑world deployment on phones and PCs, this conversation is for you. Try the open‑weight models, fine‑tune on your data, and let us know how they run on your hardware. Subscribe, share with a teammate who ships on-device, and leave a review to help more builders find the show.Send us Fan MailSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org

Voice should feel instant, private, and reliable—and today’s cloud-only pipelines rarely deliver all three. We walk through a builder’s playbook for voice AI that actually works on the edge, showing how to keep accuracy high while slashing latency, power, and cost. The conversation starts with the real trade-offs of cloud speech systems—network round trips, privacy exposure, and compute bills—then moves into a pragmatic hybrid model where the device handles fast, predictable steps and the cloud adds breadth when needed.We break down the techniques that make this possible. Sparsity lets us prune large speech models to a fraction of their size while preserving the capabilities products rely on, backed by production thresholds for true positive rates and minimal false activations. Data augmentation and confusable-phrase training harden models against room acoustics, accents, and near-sound commands like play, plain, and pay, cutting false wakes and improving user trust. We also show how synthetic data from big cloud models can teach compact on-device models to be robust without endless data collection.The other unlock comes from merging AI with classic DSP on the same edge processor. By integrating steps like noise suppression, beamforming, and VAD, we feed cleaner signals into recognizers and even boost cloud performance when offloading, halving character error rates for models like Whisper in noisy scenes. Finally, we share how to scale this approach using sparsity-driven Pareto curves, selecting the smallest viable model for each product’s power and memory budget. If you care about instant responses, privacy-first design, and shipping voice interfaces that hold up under real-world noise, this deep dive offers a roadmap you can use today. Subscribe, share with a teammate who builds audio products, and leave a review with the toughest edge constraint you want solved next.Send us Fan MailSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org

What does it really take to ship AR glasses you can wear all day without a tether to the cloud? We take you behind the scenes of edge AI design, where every gram on the frame and every milliwatt in the battery shapes what’s possible. From comfort thresholds to compute budgets, we map the constraints that make or break a usable product.We start with the human limits: sub‑40‑gram weight, eight to ten hours of battery life, high‑resolution displays, and a hands‑free UI that leans on eye‑gaze and a few reliable voice actions. Then we dig into the technical spine of modern wearables—an Arm‑based MCU paired with a next‑gen NPU that runs transformer operators and small language models locally. You’ll hear how native generative support on an MCU changes the experience, cutting latency and boosting privacy while keeping assistants, translation, and visual guidance on‑device. We also explain why an integrated ISP matters for clean, low‑light imaging and how quantization, operator fusion, and memory layout turn peak TOPS into real‑world performance.Next, we break down the memory story. Tiny models aren’t tiny enough without faster external memory, so we upgraded from Octal SPI to Hex SPI to move data at much higher rates. That bandwidth feeds vision pipelines and token streams without burning through battery budgets. Along the way, we make the case for open ecosystems: transparent kernels, flexible toolchains, and community momentum beat proprietary walls when model architectures evolve monthly. Our roadmap from early cores to the U85 shows steady gains in perf‑per‑watt, transformer coverage, and concurrent workloads across voice and vision.If you’re curious about building wearables that feel instant, private, and truly useful, this conversation lays out the path—from silicon choices and NPU design to UX trade‑offs and developer workflows. Come see the live generative AI demo running on our MCU platform, and let’s talk about what you want to build next. Subscribe, share with a teammate, and leave a review with the one feature you think AR glasses must nail.Send us Fan MailSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org

What if real AI at the edge didn’t drain your battery, overheat your device, or bloat your bill of materials? We pull back the curtain on EMASS, our edge AI system-on-chip built from the ground up to squeeze maximum intelligence out of milliwatts, not watts. The journey starts “from atoms to apps,” where device physics, circuits, architecture, and algorithms are tuned together—so every layer contributes to performance instead of fighting it.We walk through the architecture: a RISC-V core that manages dataflow and OS tasks, plus two specialized AI accelerators that the compiler selects automatically based on workload. With SRAM and non-volatile memory on board, the chip preserves system state, then powers down aggressively and wakes in microseconds. Add model quantization, pruning, knowledge distillation, and coding, and you can shrink networks by up to 100x; a hardware weight decompressor restores precision just in time, preserving accuracy while slashing memory and bandwidth.Performance isn’t just on paper. Fabricated at 22 nm and moving to 16 nm, XDOT hits up to 12 TOPS/W and around 30 GOPS at roughly 2 mW. Against public edge benchmarks, we match or beat latency while cutting energy by 20x to 200x. Developers can compile from TensorFlow or PyTorch and even remote-run on our boards—hosted in Singapore—so you see real hardware results from anywhere. Partnerships with Literal Labs yield dramatic speed and energy gains, while our Arrow Electronics modular boards let you stack sensors for predictive maintenance, biometrics, cold-chain tracking, storage, and wireless in minutes.The standout use case: drones. Instead of piling on compute-heavy features, we target the battery subsystem with AI-driven control, extending flight endurance by up to 60% and, in some trials, even higher. That means longer missions on the same charge or more payload without sacrificing airtime. It’s a taste of what becomes possible when the entire stack is co-designed—wearables that last, IoT nodes that think locally, AR devices that stay cool, and fleets that fly farther.If you’re building edge AI and want real results on real silicon, subscribe, share this episode with a teammate, and leave a review. Ready to test your model on hardware today? Reach out and try the remote run.Send us Fan MailSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org

The fastest AI you’ll use today might already be in your pocket. We dive into how modern smartphones ship with dedicated MPUs that outperform CPUs and GPUs for inference while slashing power use, unlocking private, real‑time AI without a GPU cloud bill. Our roadmap shows how to turn that potential into products: optimize your model, compile for each vendor target, benchmark on 200+ devices, then drop in a generated runtime for iOS, Android, or mobile Linux.We walk through a developer‑first workflow that starts with a simple upload and ends with a copy‑paste app. Scan a QR code to run your model on your phone and see real latency, accuracy, and power. Meanwhile, automated harnesses test across Apple, Qualcomm, and MediaTek devices to produce a credible performance report. For language models and agents, we compare quantization options like Q2, Q4, and Q8, measure tokens per second, and surface trade‑offs between footprint and quality so you can pick a configuration that feels instant, not sluggish.Beyond the how, we explore why on‑device AI changes the game: zero recurring cloud compute, lower maintenance overhead, and stronger user trust through data locality. Real‑world applications include medical imaging in the operating room, offline audio features, and rugged field tools that work in tunnels, aircraft, or rural areas with poor connectivity. We also address vendor fragmentation and share how a unified SDK bridges iOS and Android, with Flutter and React Native support on the way. If you care about speed, privacy, and cost, this is your blueprint for running AI at the edge—reliably and at scale.Subscribe for new deep dives, share this with a teammate wrestling with GPU bills, and leave a review to tell us which device you want benchmarked next.Send us Fan MailSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org

What if the next great leap in AI isn’t bigger models in bigger clouds, but millions of small, smart systems making decisions where the data is born? Guy Tamir of Intel connects the dots from steam and electricity to today’s language-driven AI, then map the pattern that keeps repeating: parallelism, mobility, autonomy, and finally distributed swarms. Once machines learned our language with tokenization, embeddings, and attention, text unlocked human knowledge—and capability exploded so quickly that benchmarks now show models outscoring average humans on IQ-like tests. The shocking part isn’t just that they’re smart; it’s how fast it happened.We break down why the future belongs to Edge AI. Factories don’t need round trips to the cloud to reject a faulty part. City cameras can detect crashes, alert responders, and create a green wave for ambulances in real time. Laptops can run personal copilots tuned to your style and goals, drawing on your emails and documents without exposing everything to a data center. That shift demands hardware built for tight power and heat budgets—heterogeneous chips mixing CPUs for control, GPUs for vision, NPUs for efficient inference, and media accelerators. On the software side, we cut through complexity with the open-edge platform: OpenVINO for cross-accelerator inference, Deep Learning Streamer for video pipelines, Scenescape for multi-camera 3D understanding, and Getty for fast, GUI-driven dataset creation and custom training.Looking ahead, we outline four trends: models get smaller and more targeted, retrieval replaces stale memorization with live knowledge, continual fine-tuning adapts systems to local contexts, and cognition becomes personal and explanatory. Think a tutor that teaches the way you learn, or a car that becomes safer each night as it absorbs local edge data. Autonomy will grow, but trust must catch up through reliability, guardrails, and transparency before agents get the keys to your finances or schedule. If you’re building for the real world, this is your roadmap to put intelligence where it matters most.If this conversation sparks ideas, follow and share it with a teammate who’s wrestling with edge deployments. Subscribe for more deep dives, and leave a review telling us your top edge use case—you’ll shape what we explore next.Send us Fan MailSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org

What if the next big jump in productivity isn’t a single breakthrough but a flywheel—AI that runs on the edge, energy systems that adapt in real time, and autonomy that safely acts in the physical world? Professor Winston Hsu of NPU in Taiwan brings academic perspective and industry experience together to map where the value actually lands over the next few years.We start by reframing AI’s progress through four waves—perception, language, scaling, and now longer-context reasoning—and explain why the most useful gains are arriving from the opposite direction: small, capable models that run locally. With tokens getting cheaper and inference efficiency rising, billion-parameter LLMs and VLMs can power phones, PCs, and embedded devices where latency and privacy rule. That shift changes what’s possible for autonomy. Instead of waiting for a full robotaxi world, we dig into the realistic trajectory: L2+ features at massive scale, software-defined vehicles, and the safety and cybersecurity discipline that keeps humans in the loop while reliability climbs.Then we cross the bridge from bits to atoms. Robots must perceive, plan, and act under uncertainty, so we explore how foundation models bring common sense, how cross-embodiment learning transfers skills from a few videos, and why dense mapping and closed-loop control belong at the edge. Capability without caution is a trap, so we draw on automotive standards—functional safety, compliance, lifecycle quality—to show how to bound risk and reduce hallucination-induced failures when systems touch the real world.Finally, we turn to energy—the constraint and the opportunity. AI workloads demand power, but AI also strengthens the grid: battery storage coordination, renewable integration, and site-level optimization with reinforcement learning and predictive control. Operators already manage gigawatts and billions of data points with hybrid edge–cloud stacks, cutting curtailment and boosting resilience. If you’re building for the near term, this is your playbook: pick edge-first use cases, favor compact models with strong reasoning, embed safety from day one, and treat AI, energy, and autonomy as one system. Subscribe, share with a teammate who ships products, and leave a quick review with the one edge use case you want launched next.Send us Fan MailSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org

AI’s center of gravity is moving—from monolithic cloud models to a living fabric that connects devices, senses the world, and infers on the spot. We walk through how agentic systems coordinate models across earbuds, phones, gateways, cars, and the cloud to deliver speed, privacy, and lower costs without sacrificing capability.We unpack the practical stack behind this shift: connect, sense, and infer. That means the right protocol for the job—Bluetooth, Wi‑Fi, UWB, Thread, Matter, or 5G—paired with robust sensing for audio, vision, motion, and biometrics, and inference engines sized for each device’s power and compute envelope. From driver monitoring to spatial audio and affordable glucose monitors, we explore how physical AI turns everyday objects into intelligent collaborators and why on-device models often create better user experiences than cloud round trips.You’ll hear how investment is tilting toward inference, why hybrid edge-cloud is the realistic future, and how toolchains that target multiple devices with one software stack will separate winners from the rest. We share a real-time translation walkthrough—noise reduction and wake words on earbuds, ASR and translation on the phone, and long-context insight in the cloud—to illustrate how an AI fabric routes tasks for the best latency, privacy, and cost profile. Along the way, we highlight Siva’s role in connectivity IP, AI DSPs, and scalable inference that already power billions of shipped devices.If you’re building AI products—or deciding where to run your next model—this is your roadmap to the edge. Subscribe, share with a colleague, and leave a review telling us which device you’d empower with on‑device intelligence next.Send us Fan MailSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org

The hype fades fast when your cloud bill spikes and your devices still lag. We sat down with leaders from Qualcomm, Renesas, Enerzai, Kudrat AI, and Advantech to trace how edge AI finally delivers real-world value—by uniting silicon, software, and developer experience into one coherent pipeline. From cost predictability to privacy and power, the economic and technical case for running models on-device has never been stronger.You’ll hear how chipmakers are acting like platform companies, pairing scalable TOPS with compilers, model hubs, and ready-to-run containers that collapse months of integration. Renesas explains how consistent APIs and multi-frontend support (TensorFlow Lite, PyTorch, ExecuTorch, ONNX) tame hardware sprawl from MCUs to MPUs. Qualcomm outlines a developer-first approach—optimizations, reference apps, and ecosystems—that smooth the last mile from prototype to fleet deployment. And when it’s time to ship, Advantech shows how prebuilt containers and integrated OTA cut through toolchain sprawl so teams can focus on product, not plumbing.We also dig into two market-proven stories. Enerzai walks through a nationwide rollout of sub-100 MB voice-control LLMs on two million set-top boxes, matching cloud quality while stabilizing costs and slashing latency. Kudrat AI brings edge AI to the forest frontier, using on-device detection to expand battery life from weeks to six months, protect privacy by filtering human images, and scale monitoring across thousands of kilometers with limited connectivity. Along the way we grapple with the hard parts: missing operator support, talent gaps in embedded ML, and the need for pragmatic standards that keep models off slow software fallbacks.The next five years won’t be a cloud-or-edge debate. Smarter endpoints will decide what stays local, what calls the cloud, and when to sync—especially as robotics stacks multiple models per device. If you care about cutting costs without cutting capability, building reliable fleets, and turning idle data into action, this conversation lays out the edge AI playbook. Enjoy the episode, share it with a teammate, and if it sparked ideas for your roadmap, subscribe and leave a quick review to help others find the show.Send us Fan MailSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org

Join Iri Trashanski, Chief Strategy Officer of Ceva, as we explore how intelligence is shifting into a distributed fabric where devices connect, sense, and infer in concert with the cloud. From phones and PCs to earbuds, cars, and medical wearables, we break down why latency, privacy, cost, and power are driving a new architecture—and how agentic AI coordinates models across endpoints to deliver seamless outcomes.We unpack the connect, sense, infer framework as a practical blueprint for product teams. On connectivity, we compare Bluetooth, Wi‑Fi, 5G, Thread, Matter, UWB, and Zigbee, and explain how each fits different throughput, power, and range needs. On sensing, we look at audio, vision, and motion pipelines, plus sensor fusion running on AI‑tuned DSPs. On inference, we show how NPUs and AI DSPs scale from tiny wearables to automotive ADAS and why monetization increasingly happens at inference, not training. Along the way, we trace real evolutions: headsets moving from simple streaming to AI noise reduction and biometrics, vehicles layering ADAS on top of connectivity, and low‑cost glucose monitors gaining always‑on analysis.To make this tangible, we walk through a real‑time translation chain that spans earbuds, a phone, and the cloud. The earbud enhances and detects wake words, the phone runs ASR and translation for low latency, and the cloud summarizes meetings for long‑form insights. The result is faster response, stronger privacy, and lower OPEX, with heavy compute where it belongs and lightweight models at the edge. If you’re building for the next wave—physical AI across billions of devices—this conversation offers a clear roadmap and concrete choices you can act on today.If you enjoyed this, follow the show, share it with a teammate who builds edge products, and leave a quick review to help others find us.Send us Fan MailSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org