AI Insights - Ep.3: Rethinking AI Performance Metrics

In the latest episode of the Cisco AI Insights podcast, hosts Rafael Herrera and Sonia Marques are joined by Dr. Catarina Carvalho, a Cisco leader in machine learning engineering. Together, they unpack the complex academic paper " Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Following," developed by researchers from the University of Maryland and the University of Waterloo. As the industry moves toward more reliable multimodal models, traditional pass-or-fail evaluation is no longer sufficient. This paper introduces a hierarchical framework that uses "LLM-as-a-judge" to evaluate outputs across five distinct criteria: visual grounding, logical coherence, factuality, reflection, and conciseness. Dr. Carvalho guides the discussion through the nuances of this "judge of judges" approach, exploring why human alignment remains the gold standard even as we automate evaluation processes. A special thank you to the teams at both The University of Waterloo and The University of Maryland, College Park, for developing this month's paper. If you are interested in reading the paper yourself, please visit this link: https://arxiv.org/pdf/2511.21662.

30 episodes… already?! 🎙️ And this one hits different. Jeff sits down with Stacy Betts—VP leading Splunk’s Americas Partner Sales—to unpack a career that spans from the early days of the internet to the frontlines of AI transformation. But this isn’t just about strategy or a $28B acquisition… it’s about leadership that lasts. From not knowing what the “internet” was in 1994 to helping shape one of Cisco’s most important moves, Stacy brings a rare perspective—and even rarer humility. Along the way, he shares unforgettable stories, powerful frameworks, and the kind of quotes you’ll carry with you long after the episode ends. And one idea you won’t forget: Who’s packing your parachute? Listen if you want to: 🔍 Rethink leadership through the lens of purpose, gratitude, and action 🤖 Understand where Cisco + Splunk + partners are headed in the AI era 💟 Walk away with stories and frameworks you’ll actually use (and repeat) 30 episodes in—and this is what it’s all about. Let’s make shift happen #CiscoShiftHappensPodcast

AB sits down with Tom Gillis, Cisco’s SVP and GM, Infrastructure & Security Group, to explore the crucial role security plays in the ever-changing AI landscape.

Ian and Kat are back with a "yin and yang" week of personal chaos and professional growth. Kat recounts the harrowing story of her recent hit-and-run—including a very embarrassing realization about her side-view mirror—and shares how she managed to keep her cool (and her work schedule) on track. Beyond the road drama, Kat dives into her experience at a women’s-only C-suite event, discussing the power of intimate networking, the future of AI in the workplace, and the importance of Cisco showing up in non-traditional spaces. Ian and Kat also get real about the "triangle of life"—balancing fitness, money, and sanity—and why sometimes, you just have to give yourself a break. Tune in for an episode full of personal anecdotes, leadership insights, and a reminder that even when things (or cars) get hit, the show must go on. Since we didn't talk (much) Cisco this week, PLEASE give us a click here: https://id.cisco.com/app/ciscoid_aemcloudprodsaml_1/exk14lk1xf6SKDANc5d7/sso/saml

What does it really take to redesign one of the most influential partner programs in the tech industry? In this episode of Shift Happens, Jeff Edwards sits down with Brian Overmeyer, Director of Partner Strategy and Programs at Cisco, to go behind the scenes of The Cisco 360 Partner Program. With a background in industrial-organizational psychology and more than two decades shaping Cisco’s partner ecosystem, Brian shares how programs are more than incentives—they’re roadmaps that guide behavior, reward value, and help partners grow alongside Cisco. From the legacy of Cisco’s Gold program to a new model built around customer outcomes, platform value, and partner co-design, this conversation explores the strategy, complexity, and trust required to evolve a program that supports tens of thousands of partners around the world. Why listen: • 🔧 The inside story behind The Cisco 360 Partner Program • 🧠 How behavioral science shapes partner strategy and incentives • 🤝 Why partner co-design was critical to getting this transformation right And as always… let’s make shift happen. 🚀 #CiscoShiftHappensPodcast

Unified Networking Architecture: Find out how X-Haul consolidates fronthaul, midhaul, and backhaul into a single, software-defined transport platform, and enables seamless 5G deployment across diverse topologies. AI-Drive Automation and Optimization: Learn to reduce operational expenses by up to 40% while improving service reliability. Future-Ready Scalability: Dive deep into how this technology supports emerging applications requiring ultra-low latency and massive bandwidth.

Visibility from diverse networks can help uncover blind spots to detect regional issues and invisible failures impacting end users. In this episode, you’ll hear first-hand insights on the operational challenges and technical strategies involved in building, validating, and expanding a global monitoring infrastructure. ——— CHAPTERS 00:00 Intro 03:21 Why Multi-location Monitoring Matters 06:48 Agent Management and Operational Challenges 14:16 Debugging Methodologies and Validation Processes 19:07 Getting the Most from Vendor Collaboration 24:16 Internet Misconceptions and Considerations 25:22 Notable Operational Outcomes ——— For additional insights, check out the links below: Cloud Agents: The Big Picture To Getting the Best Picture: https://www.thousandeyes.com/blog/cloud-agents-big-picture-best-picture?utm_source=soundcloud&utm_medium=referral&utm_campaign=fy26q3_internetreport_q3fy26ep132_podcast The Internet Outage Survival Kit: https://www.thousandeyes.com/resources/the-internet-outage-survival-kit?utm_source=soundcloud&utm_medium=referral&utm_campaign=fy26q3_internetreport_q3fy26ep132_podcast ——— Want to get in touch? If you have questions, feedback, or guests you would like to see featured on the show, send us a note at [email protected]. Or follow us on LinkedIn or X. ——— ABOUT THE INTERNET REPORT This is The Internet Report, a podcast uncovering what’s working and what’s breaking on the Internet—and why. Tune in to hear ThousandEyes’ Internet experts dig into some of the most interesting outage events from the past couple weeks, discussing what went awry—was it the Internet, or an application issue? Plus, learn about the latest trends in ISP outages, cloud network outages, collaboration network outages, and more. Catch all the episodes on your favorite podcast platform: - Apple Podcasts: https://podcasts.apple.com/us/podcast/the-internet-report/id1506984526 - Spotify: https://open.spotify.com/show/5ADFvqAtgsbYwk4JiZFqHQ?si=00e9c4b53aff4d08&nd=1&dlsi=eab65c9ea39d4773 - SoundCloud: https://soundcloud.com/ciscopodcastnetwork/sets/the-internet-report - YouTube: https://www.youtube.com/@theinternetreport_official

Happy spring cleaning time! We're cleaning house with this episode. Enjoy some behind-the-scenes recordings from 404 over the last few months! If you haven’t already, and you want to keep these BTS episodes coming, check out Cisco’s solutions for small and medium businesses: https://www.cisco.com/site/us/en/solutions/small-business/index.html

Jeff Edwards sits down with Rebecca Leach—former Cisco and AppDynamics leader turned executive coach and founder of Growth Road, an executive coaching practice focused on helping leaders navigate change with clarity, confidence, and connection. After nearly two decades in tech leadership, Rebecca reached a pivotal moment: stay in the familiar rhythm of corporate success, or step into ambiguity and redefine what leadership—and identity—really meant. What followed wasn’t just a career transition. It became the foundation for Growth Road, where Rebecca now helps leaders grow through the same kinds of shifts she experienced herself. In this candid conversation, Rebecca shares the emotions behind leaving a long, successful corporate career, the courage it takes to embrace uncertainty, and the lessons leaders can carry forward when the path ahead isn’t clear. Why listen: • ⚡ The hidden risk of tying your identity to your title • 🔄 Why career growth isn’t always “up and to the right” • 🧭 How leaders navigate ambiguity and reinvent themselves If you’ve ever faced a career crossroads—or wondered who you are beyond the role you hold—this conversation will resonate.

AB sits down with Tim Coogan, Cisco’s SVP, Global Partner Sales, for a fun chat about the important role partners play to drive positive business outcomes. From partnering in the AI era to the link between sales and partnerships, this informative conversation highlights why it’s critical for business to cultivate a powerful partner ecosystem.