Why AI Reliability Requires More Than Uptime: Lessons from Microsoft and Atlassian

What makes an AI system reliable when traditional uptime metrics can't tell you whether its answers are correct?In this episode of Future Proof, Xano CEO Prakash Chandran sits down with Priyanshu Jha, AI Engineering Lead at Loom, to explore why building reliable AI systems requires a fundamentally different approach to software engineering.Drawing on his experience at Microsoft, Amazon, Atlassian and Loom, Priyanshu shares how AI is changing the definition of reliability. An AI system can have 100% uptime and still produce incorrect results, creating new challenges for engineering teams moving AI applications into production.In this episode, they discuss:Why uptime isn't enough: How nondeterministic AI systems introduce reliability challenges that traditional monitoring can't detect.The importance of AI evals: Why engineering teams should establish quality benchmarks early and continuously evaluate AI outputs.Production-ready infrastructure: Why reliable data pipelines, security, and traditional engineering fundamentals matter more than ever.The evolution of AI agents: How Loom is enabling new forms of collaboration between humans and AI agents.Balancing cost, latency, and quality: How engineering leaders make tradeoffs when building AI-powered products.The future of software engineering: Why systems thinking, product judgment, and customer focus are becoming essential skills.The takeaway: Building with AI has never been easier. Building AI systems that customers can trust requires much more than a successful demo or a healthy uptime dashboard.About Future ProofFuture Proof, hosted by Xano CEO Prakash Chandran, explores how technology leaders are navigating the changing world of AI, software development, and enterprise technology.Learn more about Xano: https://www.xano.com/Subscribe to Futureproof wherever you get your podcasts.From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.

What happens when AI makes code faster to write than it is to govern?In this episode of Futureproof, Xano CEO Prakash Chandran sits down with Alex Pape, EVP and Chief Technology & Product Officer at AssetMark, a wealth management technology platform serving independent financial advisors. Alex has spent his career at the intersection of investing, product, and technology, from building a direct to consumer investing platform at The Motley Fool to leading product for BlackRock's Aladdin Wealth Tech before joining AssetMark to lead its technology and product organization.Together, they explore why the biggest technology shifts often happen below the surface, and what it takes to modernize a 30 year technology estate without putting the business on pause. Alex breaks down AssetMark's three part AI strategy, including Nitro, the company's internally built AI development harness that encodes its architectural principles directly into the software development process. They dig into how AssetMark says it increased development velocity roughly 6x, why traditional measures like story points break down in an AI powered development environment, and how to decide when to optimize for speed versus experimentation. They close with Alex's framework for hiring adaptable "athletes" in a world where the boundaries between engineering and product are changing fast, and the question he uses to bring every technology decision back to its purpose: are our clients flourishing because of us?Topics covered include:The real transformation happens underneath: Why fragmented data and disconnected systems can matter more than flashy new features, and why AssetMark is rebuilding the foundation that allows advisors to spend more time with clients.When less product usage is a good thing: Why Alex does not see logins and engagement as the ultimate measure of success, and how making technology disappear can create a better advisor experience.Three jobs for AI: How AssetMark separates its AI strategy into AI for advisors, AI for software development, and AI for employees, with each tied back to a specific business outcome.The machine that builds the machine: Why AssetMark built Nitro instead of relying entirely on third party coding tools, and how encoding architectural principles into AI agents can give developers more freedom without giving up governance.How do you measure 6x?: Why story points stopped working as AI changed the amount of software an engineer could produce, and how AssetMark uses lines of code, pull requests, and cycle time as a proxy for development velocity.Hire athletes, not job descriptions: Why Alex looks for people who could become exceptional in a completely different role within months, and why adaptability, curiosity, and humility matter more as AI reshapes traditional disciplines.Subscribe to Futureproof wherever you get your podcasts.From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.

What does digital transformation actually look like when your product is a physical home—and a leaky faucet can't be fixed with a line of code?In this episode of Futureproof, Xano CEO Prakash Chandran sits down with Nikolaj Brammer, Chief Digital Officer at Heimstaden, one of Europe's largest residential real estate companies, managing tens of thousands of homes across multiple countries. Nikolaj's background isn't in technology — he came up through Bain & Company, Goldman Sachs, and Maersk before joining Heimstaden, where he's held roles across business development and commercial leadership before taking on the company's entire digital transformation. Together, they explore what it actually looks like to digitize the operations of a physical business, why culture and mindset matter more than technology in making transformation stick, and how Heimstaden is building toward a future where a junior AI analyst can answer complex data questions in seconds. They also dig into the hardest part of any AI rollout — measuring real impact — and close with a candid conversation about the risk of outsourcing critical thinking to AI at the exact moment judgment matters most.Topics covered include:The product is physical. The operations aren't.: Why the home itself doesn't need to change for a real estate company to undergo a meaningful digital transformation — and what that looks like in practice at scale.Culture before technology: Why the mindset and habits of an organization determine whether digital transformation actually sticks — and why Heimstaden's approach to even the smallest decisions creates the conditions for it.Three levels of AI value: From personal productivity to team-wide tools to full end-to-end workflow redesign — and how to find and prioritize the right use cases at each level.The measurement problem nobody has solved: Why proving that AI created a specific business outcome is harder than it sounds — and why judgment and storytelling end up being just as important as data.Outsource the work, not the thinking: Why both Nikolaj and Prakash have had to consciously pull back from delegating too much to AI — and why the risk of losing critical thinking is one of the most important conversations leaders should be having right now.Subscribe to Futureproof wherever you get your podcasts.From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.

If AI can produce the deck, write the PRD, and draft the battle card—what's left for you to own?In this episode of Futureproof, Xano CEO Prakash Chandran sits down with Paul Traficanti, Director of Product Marketing at Nextworld and writer behind the Substack newsletter Somebody Manage Me. Paul is a founder turned operator who has worked across gaming, direct-to-consumer, and enterprise SaaS—and he's been writing about something most AI conversations skip over: as AI takes over the visible work, the invisible human layer underneath becomes even more invisible. Together, he and Prakash explore what taste, judgment, and critical thinking actually look like in daily work, why AI-generated documents are creating a new kind of organizational noise, and how leaders can start making the invisible contributions of their teams visible again. They also dig into how hiring is changing when the floor has been raised, why meetings are shifting from document reviews to alignment conversations, and what it means that an LLM-written document is now being summarized by another LLM so no one has to read it.Topics covered include:Taste as the new competitive advantage: Why the human decisions behind a prompt—what to say, to whom, and why—matter more than the polished output AI produces.The document graveyard problem: How AI-generated PRDs, battle cards, and decks are creating more content but less understanding, and why less is about to become more again."Show me the prompt": A practical framework for cutting through AI slop by asking teammates what context they gave the model and why—and the water cooler test for whether they truly understand their own output.Hiring when the floor is raised: Why interview homework assignments no longer reveal much, and how companies are rethinking evaluation to test for critical thinking, not production skill.AI literacy gaps across functions: Why developers adopted AI fastest but sales and marketing teams are still catching up, and the emerging need for internal AI enablement roles.Subscribe to Futureproof wherever you get your podcasts.From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.

If everyone in your organization wants AI, but half of them can't explain what for—where do you start?In this episode of Futureproof, Xano CEO Prakash Chandran sits down with Fred Hennige, Director of Enterprise Architecture at Jack in the Box, to explore what enterprise architecture actually looks like inside a fast-growing restaurant company navigating AI adoption. Fred shares lessons from building EA practices at three very different organizations (Jack in the Box, Starbucks, and Alaska Airlines) and explains why the discipline must start with business outcomes, not technology inventories. Together, they unpack how AI is showing up across the organization today, why cross-functional AI value is harder to unlock than personal productivity, and how to govern AI adoption without over-indexing on hype. Topics covered include:Business-first EA over technology-first EA: Why starting from business outcomes and process alignment yields better results than cataloging application inventories.EA across three industries: How enterprise architecture looks radically different at a growing brand, a mature global operation, and a safety-critical airline—and what each taught Fred about the discipline.AI adoption at different maturity levels: Why some teams are already creating value with AI while others are still learning to spell it—and how to stack-rank where to invest.Cross-functional AI is the hard part: Why personal productivity gains come first, but the real challenge is unlocking AI value across departments and business functions.The AI uncanny valley: Why AI output still requires human synthesis, and why using your own voice matters more than copying and pasting what a model returns.Episode ID: 19469167-enterprise-architecture-in-the-age-of-ai-with-fred-hennige-jack-in-the-boxSubscribe to Futureproof wherever you get your podcasts.From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.

If you plug an AI agent into your data, how do you know it's giving you the right answer—and not just a confident one?In this episode of Futureproof, Prakash Chandran sits down with Heather Lutz, Director of Engineering at Datasite, the provider of AI-powered solutions that enable private market investment, including virtual data rooms for mergers and acquisitions. Together, they unpack what happens when you point an agent at a massive data set without doing the foundational work first, why data readiness and governance are non-negotiable prerequisites for any AI initiative, and how Datasite is layering semantic views, verified queries, skills, and a "data doorman" to make agents actually useful. About DatasiteDatasite provides the infrastructure that enables information flow for private market transactions, with purpose-built tools to optimize outcomes. Datasite’s innovative product portfolio, spanning sell-side virtual data rooms, buy-side intelligence, agentic AI applications, and an open data infrastructure layer, drives execution across the full investment lifecycle while generating unique data insights to empower investors, advisors, and deal professionals worldwide. Trusted by top private equity firms, investment banks, and consultancies, Datasite is built on 26 years of enterprise-grade security, compliance, and reliability. For more information, visit www.datasite.com Topics covered include:Agents are confident interns, not seasoned analysts: Why an AI agent querying your data won't know about data quality issues, duplicate revenue tables, or missing filters—and why confidence without context is worse than no answer at all.Data readiness as CI/CD: Why testing data should follow the same discipline as testing software—with checks at every stage of the pipeline—and why continuous data quality monitoring barely exists as a standard practice yet.Data governance makes agents work: How domain ownership, shared metric definitions, and semantic layers turn an unnavigable ocean of tables into a surface an agent can actually be expert on.Don't work with the ocean: Why starting with your top ten metrics, your most important structures, and a bounded consumable layer is the only practical path to making AI-over-data work at scale.Episode ID: 19329740-your-agent-doesn-t-know-what-it-doesn-t-know-with-heather-lutz-datasiteSubscribe to Futureproof wherever you get your podcasts.From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.

If an AI agent gives you the correct answer but took the wrong path to get there, can you actually trust it?In this episode of Futureproof, Prakash Chandran sits down with Karthik Narayan, Director of Product Management at Komodo Health, where he leads Marmot, an enterprise AI product for life sciences. Marmot's promise is that life sciences companies no longer need to send their data to McKinsey and wait months for an answer—they can ask complex healthcare questions and get answers directly. The catch? The answers aren't binary. Together, Karthik and Prakash unpack why grading an agent on whether it got the right answer is only half the story, how Komodo uses parallel critique agents and friction detection to close the gap between AI confidence and analyst rigor, and what changes when AI makes product leaders more powerful than they've ever been.Topics covered include:Why the path matters more than the answer: How an agent can arrive at the correct number through the wrong query, pass traditional evals, and then fail catastrophically on the next question—and why trajectory evals are the real measure of trustworthiness.Steering, not just answering: How Marmot uses research plans, follow-up questions, and full code transparency to give analysts maximum control over subjective healthcare methodology decisions.Friction detection over thumbs up/down: Why users rarely use explicit feedback mechanisms, how Komodo infers dissatisfaction from behavioral patterns, and how that drove a complete platform rewrite at the six-month mark.Build vs. buy when AI makes prototypes easy: Why a junior engineer's weekend demo isn't the same as a production system with fallback models, context compaction, token optimization, and continuous evaluation—and how to think about total cost of ownership.Episode ID: 19252007-the-right-answer-isn-t-enough-with-karthik-narayan-komodo-healthSubscribe to Futureproof wherever you get your podcasts.From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.

If AI is making us faster, why does it feel like we're understanding less?In this episode of Futureproof, Prakash Chandran sits down with Doug Merritt, CEO of Aviatrix. Doug is one of the most accomplished enterprise technology leaders of the last two decades—after serving as CEO of Splunk for years, he's now leading Aviatrix to tackle cloud-native security. Together, they unpack why the speed of AI adoption is outrunning foundational understanding, how a recent supply chain attack on the popular LiteLLM framework exposed a massive blind spot in cloud security, and why the leadership principles that matter most right now—curiosity, empathy, and purpose before action—are the same ones our attention-starved culture makes hardest to practice.Topics covered include:As agents become more human, humans become more binary: Why the speed and abstraction of AI is making our thinking shallower at the exact moment we need it to be deeper—and how to fight back.The LiteLLM supply chain attack, explained: A breakdown of how attackers injected malware into LiteLLM, harvesting credentials from cloud environments—and why basic egress filtering would have stopped the damage cold.The three fundamental runtime controls: Why identity, endpoint, and network security are the only controls that actually stop attacks in progress—and why most cloud workloads are missing at least one.Cloud providers sold speed without brakes: How permissive outbound defaults became the norm, why cloud providers made firewalls an aftermarket add-on, and what that means for every organization deploying AI agents today.Five leadership principles for the AI age: Doug's hard-won framework—relentless curiosity, leading with empathy, purpose before action, radical accountability, and celebrating success—and why daily mastery beats chasing the next shiny thing.Episode ID: 19139581-foundational-thinking-in-the-age-of-ai-with-doug-merritt-aviatrixSubscribe to Futureproof wherever you get your podcasts.From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.

If AI agents are writing your code, how are you making sure it's secure?In this episode of Futureproof, Prakash Chandran sits down with Tim Olshansky, CTO and co-founder of Fencer, to explore what application security really looks like in a world where AI writes most of the code and open source software underpins everything. Tim shares his journey from engineer—navigating bureaucratic security processes at larger organizations—to building a platform that makes security accessible for companies under 200 employees. Together, they unpack why compliance certifications often create a false sense of security, how the open source supply chain has become a prime target for attackers, and what "trust but verify" means when Claude is opening your pull requests. They also discuss practical steps any builder can take today—from package manager hygiene to cooldown periods—and why hiring for engineering talent has never been harder to figure out.Topics covered include:Security as hygiene, not a project: Why treating security like brushing your teeth—small, consistent habits—prevents catastrophic outcomes, and why most small companies still skip it.The open source supply chain is under attack: How threat actors exploit volunteer-maintained libraries like Axios to gain access to thousands of commercial products at once—and why it's only getting worse.AI-generated code and the false sense of security: Why LLMs trained on publicly available code don't encode the highest corporate security standards, and why the code itself may not be what gets you hacked.Trust but verify in an AI-first workflow: How Tim's team moved to nearly 100% AI-driven development while still requiring human review.Episode ID: 19061977-ai-makes-security-everyone-s-problem-with-tim-olshansky-fencerSubscribe to Futureproof wherever you get your podcasts.From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.

What does it really take to build a regulated fintech product from scratch—with a small team and a bootstrapped budget?In this episode of Futureproof, Prakash Chandran sits down with Michael Konrath, co-founder and Chief Product Officer of Choice Digital, a fintech company that ensures customers—including the unbanked—actually receive the payments they're owed. Michael shares the full arc of building Choice Digital: from prototyping on Bubble in a co-working space to processing nearly $1 billion in payments today. Along the way, they dig into the realities of compliance in a regulated industry, the trade-offs of building fast versus building right, and how AI is starting to reshape the way his team ships software.Topics covered include:Starting small with no-code: Why Choice Digital's first product was built on no-code tools in 30 days, processed $20 million in payments, and lasted far longer than expected—plus how that scrappy mindset still matters in the age of AI.Compliance as a foundation, not an afterthought: The case for investing in SOC 1 and SOC 2 frameworks early, how PCI compliance shapes product architecture, and why segmenting systems can simplify your regulatory journey.AI adoption in a regulated space: Why Choice Digital is taking a deliberate, human-in-the-loop approach to AI, focusing on deterministic processes and clear policies before letting models touch customer data.Episode ID: 18983940-from-prototype-to-enterprise-grade-compliance-with-michael-konrath-choice-digitalSubscribe to Futureproof wherever you get your podcasts.From Xano - The fastest way to create a production-ready backend for any app or agent. Xano unifies AI speed, code control, and visual clarity, so you never trade reliability for velocity. Sign up for free today.