Hugging Face with Sayak Paul

Data retrieval is a fundamental challenge in AI systems, and the approaches for solving it are still evolving. Vector search was an early answer to the retrieval problem, but the rise of agentic systems has raised the stakes considerably. Agents issue queries at machine speed, decompose complex questions into parallel searches, and require retrieval infrastructure that can keep pace without becoming prohibitively expensive. Chroma is a company building open source infrastructure for AI applications, best known for its widely used database of the same name.  The company also published the influential Context Rot paper, which documented how model performance degrades as context window utilization increases, and recently released Context One, a 20 billion parameter retrieval sub-agent trained to do agentic search at frontier model quality but at an order of magnitude lower cost and higher speed. Hammad Bashir is the CTO of Chroma, with a background spanning machine learning, computer vision, and data systems. In this episode, Hammad joins Gregor Vand to discuss the origins of ChromaDB, our current understanding of context rot, why a purpose-built small model can match frontier models on search tasks, the philosophy behind Chroma’s open source approach, and where the company sees AI data infrastructure heading.Sponsorship inquiries:[email protected] The post Chroma and Agentic Retrieval appeared first on Software Engineering Daily.

Many real-world processes produce data as a continuous stream rather than as isolated records. Sensor readings, financial markets, and application telemetry all generate data this way. This kind of time-series data has a distinctive shape. It’s written far more often than it is updated, it accumulates continuously, and it is usually queried across ranges of time. Time-series databases are built specifically for this kind of workload. TimescaleDB is an open source database from Tiger Data that adds time-series capabilities to PostgreSQL. It’s implemented as a Postgres extension, so it introduces new functionality while preserving standard Postgres behavior and SQL. This lets a single Postgres-based system handle both transactional and analytical workloads without splitting data across multiple tools. Brandon Purcell is the Director of Product Management at Tiger Data. In this episode, Brandon joins Kevin Ball to discuss why time-series data breaks conventional databases, how hypertables and Hypercore scale Postgres, zero-copy database forking for agent-based workflows, and much more.Full Disclosure: This episode is sponsored by Tiger DataSponsorship inquiries:[email protected] The post Scaling Time-Series Workloads on Postgres appeared first on Software Engineering Daily.

Most AI agent setups today are built around a single session, where one user interacts with one agent at a time. However, that model breaks down when an agent has to serve a business, where thousands of requests can arrive at once and each session needs to be isolated, durable, and recoverable. Getting agents to run reliably at that scale has meant a lot of hand-rolled infrastructure beneath the agent itself. eve is an open source, cloud-native agent framework from Vercel that removes much of the agent scaling burden. In the eve framework, an agent is defined declaratively through configuration files and these files compile into infrastructure as code so the platform provisions only what the agent actually uses. Andrew Barba is a Member of Technical Staff at Vercel, and Shar Dara is the Product Lead for eve at Vercel. In this episode, they join Kevin Ball to discuss what it means for an agent framework to be cloud-native, why they chose to express agents in plain English, and their view that company building is becoming agent building.Sponsorship inquiries:[email protected] The post Scaling Agent Workloads at Vercel appeared first on Software Engineering Daily.

Historically, databases were responsible for storing data and returning exact results in response to queries. However, AI is now bending that contract in a new direction. Applications increasingly expect structured and unstructured data to come together. This is pushing databases into territory that looks more like search, where relevance and ranking matter and results are no longer strictly exact. Agents are also beginning to write their own queries and even propose their own schemas, which raises new questions about how data should be structured, governed, and trusted. Sailesh Krishnamurthy is a VP of Engineering at Google, and in this episode he joins Matt Merrill to discuss his background, how databases have evolved over the past fifty years, and where the field is heading as AI reshapes how data is queried, structured, and trusted.Sponsorship inquiries:[email protected] The post Inside Google’s Database Infrastructure for the AI Era appeared first on Software Engineering Daily.

A Rust Framework to Simplify Distributed Systems Building software that runs across many machines is notoriously difficult. Developers have to grapple with problems such as race conditions, partial failures, and message ordering. Notably, one category of distributed software has largely escaped these burdens. A distributed database can spread a single query across thousands of machines, handling the coordination, failure recovery, and ordering internally. This raises a natural question of why general-purpose distributed programming can’t feel the same way. This is a highly practical problem at AWS, because the reliability of cloud infrastructure depends on getting distributed systems right at massive scale. Joe Hellerstein spent thirty years as a database and distributed systems researcher at UC Berkeley, where he pioneered much of the foundational thinking on applying database ideas to distributed programming. He is now at AWS, where he works to bring his research into production through Hydro, which is a Rust framework to bring declarative queries to general-purpose distributed programming. In this episode, Joe joins Sean Falconer to discuss how ideas from the database world could make distributed programming dramatically simpler and safer.Sponsorship inquiries:[email protected] The post A Rust Framework to Simplify Distributed Systems appeared first on Software Engineering Daily.

SED News is a monthly podcast from Software Engineering Daily where hosts Gregor Vand and Sean Falconer break down the biggest stories shaping software engineering, Silicon Valley, and the broader tech industry. In this end-of-summer episode, Gregor and Sean turn to a busy season of mergers and acquisitions, including NVIDIA‘s reported $12.9 billion acquisition of Hugging Face, Dynatrace‘s near-billion-dollar deal for AI observability platform Arize, and Temporal‘s rumored raise at a $12 billion valuation. They also cover a run of large funding rounds, from AI security startup HiddenLayer to AI personal assistant Instinct and restaurant software platform Owner.com, all against the backdrop of a deluge of new model releases. The main topic digs into China’s “transfer station” economy, the sprawling proxy market that gives developers cheap access to frontier models officially banned in the country. Drawing on a China Talk report, they walk through the tactics at play, from “one fish, three meals” credit farming to silent model swapping, and unpack how this pipeline of captured outputs and human traces may be fueling the recent surge in high-quality open weight models. Gregor and Sean also examine where Meta now sits in the landscape after shelving Llama and pivoting toward its Muse family. As always, the episode wraps up with a few standout Hacker News threads, including a critique of how log-scale charts obscure the real cost gap between open weight and frontier models, and a look at “invisible companies” as an under-the-radar investment strategy. Sponsorship inquiries:[email protected] The post SED News: The NVIDIA-Hugging Face Deal, China’s Proxy Economy, the Open Weight Surge appeared first on Software Engineering Daily.

Retrieval has become one of the central problems in building useful AI systems. The standard approach to grounding a model in one’s own data has been retrieval augmented generation, or RAG, where an agent searches a vector database for relevant information at query time. That pattern works, but it has limitations, such as retrieving information that’s not truly relevant, repeating the same lookup work on every query, and producing inconsistent answers to the same question. Pinecone is a vector database that’s widely used to power semantic search and RAG at scale. The team recently developed Nexus, which is a knowledge engine that reframes context as a first-class, precomputed asset rather than something reassembled on the fly. The approach borrows the database concept of a materialized view, and curates context once into a versioned artifact that carries its own schema, metadata, permissions, and lineage. Jörg Schad is the VP of Engineering at Pinecone. In this episode, he joins Kevin Ball for an in-depth conversation about the frontier of retrieval technology. They discuss precompiled context, how context artifacts are curated and versioned much like code, how metadata and semantic layers help agents choose the right information, and much more.Sponsorship inquiries:[email protected] The post Moving Beyond RAG with Precomputed Context appeared first on Software Engineering Daily.

Age verification is reshaping how people access the internet. An ever-growing patchwork of laws can now require government IDs, facial age estimation, or behavioral inference before you can enter digital spaces. Discord, app stores, social media platforms, and AI chatbots are all being pulled into these requirements. Almost always, these measures are enacted under the banner of protecting children. However, beneath that goal lies a set of hard questions about privacy, anonymity, security, and the line between what a private platform may choose to do and what a government may compel. John Coleman is a legislative counsel at the Foundation for Individual Rights and Expression, or FIRE, where he focuses on artificial intelligence and free speech policy. In this episode, John joins Kevin Ball to discuss the different methods of age verification and their privacy implications, the long constitutional history of anonymous speech, how recent court decisions are reshaping what governments can require, the data breach and surveillance risks these mandates create, and what engineers and companies should keep in mind when they are asked to build these systems.Sponsorship inquiries:[email protected] The post The Death of Online Anonymity appeared first on Software Engineering Daily.

TypeScript is a programming language that builds on JavaScript by adding a system of types. Those types let developers describe the shape of their data and catch mistakes before code ever runs, while also powering the autocompletion and editor tooling that many developers now rely on every day. It was first released in 2012, and has since become one of the most widely used tools in web development. TypeScript recently underwent one of the most significant changes in its history with the release of version 7. Daniel Rosenwasser is the Principal Product Manager of TypeScript at Microsoft, where he began as an engineer on the team just weeks after the TypeScript 1.0 release. In this episode, Daniel joins Josh Goldberg to talk about the features of TypeScript 7. They discuss the TypeScript team’s approach to tooling, TypeScript’s relationship with the TC39 standards process behind JavaScript, the new API and IPC boundary, how LLMs could reshape type checking and linting, and more.Sponsorship inquiries:[email protected] The post TypeScript 7 and What Comes Next appeared first on Software Engineering Daily.

It is widely reported that a gap has emerged between enterprise spending on AI and the durable value captured from that spend. Individual employees have enthusiastically adopted coding assistants and chatbots, yet those gains do not seem to be transforming businesses at an organizational level. One of the most important questions in the tech industry today is understanding why AI is not yet delivering returns that match the investment, and what separates the small number of enterprises succeeding from the many that are not. Scale AI is known for supplying the human-labeled data behind many frontier models. It now also builds AI applications and agents for large enterprises. That combination of working alongside frontier labs and inside enterprise deployments gives the company a rare view of why enterprise AI may be stalling. Emily Xue is the Head of Enterprise AI at Scale AI, and previously spent over a decade at Google. In this episode, Emily joins Kevin Ball to discuss the three layers where enterprise AI breaks down, why frontier model benchmarks miss what enterprises actually need, the data foundation problem, how the most successful companies combine internal domain expertise with outside AI specialists, and more.Sponsorship inquiries:[email protected] The post The Gap Between AI Spending and AI Value appeared first on Software Engineering Daily.