What's This Podcast About?

Dr. Lia DiBello is the Chief Science Officer of ACSILabs, Inc, which makes the FutureView Platform — a virtual reality training platform used by the US military and by certain large businesses to accelerate expertise. I last talked to Lia about her groundbreaking work explicating the mental model of business expertise, and in fact created this podcast to interview her. She’s back today to talk about her use of AI to accelerate expertise. Neil Sahota is the CEO of ACSILabs. Neil was part of the original IBM Watson team that won the Jeopardy challenge. After that challenge, he was, amongst things, the person responsible for IBM’s Watson ecosystem strategy, and an IBM Master Inventor (which is a designation for IBM employees who have made outstanding contributions to the IP creation process at IBM). Since leaving Big Blue, Neil has been Artificial Intelligence Advisor to the United Nations, published a book about AI in 2019, and started the UN’s AI For Good Initiative, which is currently hosted under the International Telecommunications Union.In this episode, we take a human-first perspective on using and deploying AI in real world business and military contexts. We discuss Lia and Neil’s history with AI, talk about good and bad implementations of AI they’ve seen in real-world environments, and discuss what it means to get more folks to an expert-level at human-AI symbiosis. ShownotesASCI Labs — https://acsilabs.org/ASCI Labs in the Navy STP marketplace — https://acsilabs.org/acsi-labs-has-met-the-requirements-to-be-listed-in-the-navy-stp-virtual-transition-marketplace/AIQ (Artificial Intelligence Quotient): Helping People Get Smart about the Smart Machines They Are Using — https://medium.com/about-work/helping-people-get-smart-about-smart-machines-they-are-using-f9e0095846feAIQ: Artificial Intelligence Quotient — https://www.psychologytoday.com/us/blog/seeing-what-others-dont/202007/aiq-artificial-intelligence-quotientHoffman, R.R., Mueller, S.T., Klein, G., & Litman, J. (2018). "Metrics for Explainable AI: Challenges and Prospects." Technical Report, Explainable AI Program, DARPA, Washington, DC. — https://arxiv.org/abs/1812.04608Timestamps(00:00) - Lia DeBello and Neil Sahota on Human-AI Symbiosis (00:07) - Introduction (02:08) - Using AI for Expertise Acceleration (17:42) - Good Applications of AI Historically (23:20) - Mistakes With Applying AI (28:09) - Watching Experts Use AI (30:45) - AI Systems as Extensions of Human Expertise (35:04) - Why Won't AI Replace Humans? (41:55) - How To Train Junior Folks For AI Symbiosis (45:42) - Teaching Creative Thinking With AI (50:52) - Moving Beyond Chatbots (57:55) - Adapting Company Processes to New Tech (01:00:00) - How Should AI Systems Designers Think About Affordances (01:03:51) - Possible to Design Normal Business Processes for Expertise Acceleration? (01:08:09) - What Should Expertise Researchers Learn from AI Researchers? (01:09:49) - What Should AI Researchers Learn From Expertise Researchers (01:14:43) - ACSI Labs in the Navy STP Marketplace (01:20:00) - Where Folks Can Find Neil and Lia

Stan Slap is the founder and president of SLAP, an international consulting firm that helps organisations get maximum commitment from their manager, employee, and customer cultures. SLAP has worked with the who's-who of the Fortune 500, companies that you've heard of like Costco, Oracle, HP, Caterpillar, Microsoft, Amazon, General Electric, Intel, Viacom, Tesla, Glassdoor, Warner Music Group — and some companies that you haven't; all told, SLAP has worked with companies doing business in 44 countries. Stan is also the author of two books: Under The Hood and Bury My Heart in Conference Room B.This podcast dives into Slap's ideas about understanding, managing and shaping effective employee cultures. His model for culture is the first coherent model that I've found that is useful and tested at scale; it is therefore more powerful than most other formulations of employee culture that you'll find.SLAP Company — https://slapcompany.com/Under The Hood — https://www.goodreads.com/book/show/22571574-under-the-hoodCommoncog summary of Under The Hood — https://commoncog.com/under-the-hood/Bury My Heart at Conference Room B — https://www.goodreads.com/book/show/7810760-bury-my-heart-at-conference-room-b(00:00) - Introduction (02:32) - Employee Culture is About Business Performance (06:56) - What a Culture Is (10:21) - Thinking About Culture as an Organism (19:40) - What Management Gets Wrong (22:56) - What To Do When You've Lost The Culture's Trust (26:36) - Tension Between Managing Individuals vs Managing Culture (30:09) - Giving a Culture Energy (35:16) - Introducing Change to a Culture (42:28) - Doing Cultural Understanding at Scale (50:48) - Four Step Process for Cultural Commitment (01:00:04) - Danger of Corporate Values (01:04:29) - Managing Culture in a Rapidly Scaling Company (01:10:51) - Dealing with Bad Subcultures (01:18:14) - Stan's Closing Message (01:25:49) - Where Folks Can Find Stan

Colin Bryar joined Amazon really early in its life and spent twelve years as part of Amazon's senior leadership team.For two of those years he was 'Technical Assistant' to Jeff Bezos, as known as 'Jeff's shadow', during which he spent each day attending meetings, traveling with, and discussing business and life with Jeff. After Amazon, he and his family relocated to Singapore for two years where Colin served as Chief Operating Officer of e-commerce company RedMart, which was subsequently sold to Alibaba. Along with his ex-Amazonian colleague Bill Carr, Colin is co-author of Working Backwards, a book on an insider's look at how Amazon works. Bill and Colin are co-founders of Working Backwards LLC, where they coach executives at both large and early-stage companies on how to implement the management practices developed at Amazon. This podcast is a really deep dive into the practice of the Amazon Weekly Business Review, which remains to this day one of Amazon's secret operating weapons, and a big part of what makes for a great operator.Working Backwards, book: https://www.goodreads.com/en/book/show/53138083Working Backwards https://www.workingbackwards.com/Commoncog summary of Working Backwards: https://commoncog.com/working-backwards/(00:00) - Introduction (01:24) - Colin's Background (04:00) - Joining Amazon (06:31) - The Data Situation in Early Amazon (08:22) - Being Jeff Bezos's Shadow (10:37) - Living in Singapore (12:07) - Writing Working Backwards with Bill Carr (15:17) - The History of the Weekly Business Review (17:50) - How the Amazon WBR is different (20:45) - Customer Experience Metrics vs Business Metrics (22:33) - Controllable Input Metrics vs Output Metrics (30:39) - What a Typical WBR Looks Like (34:22) - Why Glancing at Metrics is Important (35:43) - What kinds of discussions should you have in the WBR? (37:30) - Understanding Variation (41:28) - Stories About Figuring Out Controllable Input Metrics (48:19) - Applying the WBR to internal business functions (49:49) - Introducing the WBR to a New Company (55:55) - Applying the WBR to New Products (01:01:04) - Not Using Surveys as Primary Research on Customers (01:04:17) - What Makes for a Good Operator? (01:05:23) - Operating Cadence (01:07:05) - What Colin Wishes All Operators Knew Tomorrow (01:07:57) - What You'd Wish You'd Known (01:09:50) - Would Many of These Lessons Apply to Early Stage Startups?

Eric Nehrlich is an executive coach, and was formerly the chief of staff on the Google Search Ads team. Before becoming chief of staff, Eric was part of the team that got Google's revenue forecasting down from an error rate of 10-20% to an error rate of around 0.5%. We open up with some wild stories of Google's early attempts at revenue forecasting, and then dig into how that forecasting success happened. Along the way, Eric explains how he, his boss, and his team developed a fingertip feel for variation in data. We then switch gears to talk about Eric's coaching practice. We discuss the differences between mentorship and coaching, and talk about the tricky art of helping leaders grow their impact. Eric has a wealth of knowledge on coming up with small experiments in order to make personal growth easier. We talk about how he comes up with those experiments, and what some of those experiments look like in practice. Finally, we close with a chat about Eric's upcoming book. - Eric's Executive Coaching practice: https://www.toomanytrees.com/ - Eric's upcoming book is named You Have a Choice, and you may sign up for updates here: https://www.toomanytrees.com/book - Eric's LinkedIn (where he posts insights every week): https://www.linkedin.com/in/nehrlich - Eric's blog: https://www.nehrlich.com/blog/ - Eric's newsletter: https://www.nehrlich.com/blog/newsletter/ (which I highly recommend) - Full list of coaching and leadership development resources that Eric recommends: https://www.toomanytrees.com/resources - Surely You’re Joking Mr Feynman!: https://www.goodreads.com/book/show/35167685-surely-you-re-joking-mr-feynman - Becoming Data Driven in Business Series: https://commoncog.com/becoming-data-driven-in-business/ - Be Good To Your Mentors: https://commoncog.com/be-good-to-your-mentors/ - Eric Nehrlich — Commitment, Competence, Structure: https://www.nehrlich.com/blog/2017/10/20/why-dont-we-change/ - James Clear — Atomic Habits (on Structure): https://www.goodreads.com/en/book/show/40121378 - Robert Kegan and Lisa Lahey — Immunity to Change: https://www.gse.harvard.edu/hgse100/story/changing-better - Eric Nehrlich's post on Immunity to Change https://www.nehrlich.com/blog/2018/06/18/immunity-to-change-methodology/ - Claire Hughes Johnson — Scaling People: https://www.goodreads.com/book/show/63063173-scaling-people - Eric's thoughts on Scaling People: https://www.linkedin.com/posts/nehrlich_google-stripe-leadership-activity-7044441138623127552-jhau - Dan Martell — Buy Back Your Time: https://www.goodreads.com/en/book/show/60880804 - Terrence Real — Us Getting Past You and Me: https://www.goodreads.com/book/show/58502657-us - Eric's post on Us Getting Past You and Me: https://www.nehrlich.com/blog/2023/02/21/us-getting-past-you-and-me-to-build-a-more-loving-relationship-by-terrence-real/(00:00) - Introduction (01:21) - Eric's Background (09:00) - Google Revenue Forecasting Stories (13:16) - Driving the Forecasting Error Rate Down (18:00) - Developing an Intuition for Variation (29:45) - What a Forecasting Novice Would Get Wrong (36:46) - What Does an Executive Coach Do? (45:34) - The Difference Between Mentorship and Coaching (53:42) - Changing Behaviour: Will, Skill, Structure (01:02:14) - Changing When Your Identity Holds You Back (01:05:50) - Coming Up With Small Experiments to Change (01:08:48) - Getting Organisational Identities to Change (01:11:55) - More Examples of Change Experiments (01:15:25) - Books Eric Recommends for Experiments (01:20:25) - Finding Eric Online (01:21:10) - Eric's Book

Lesley Sim coached the Singaporean Ultimate Women's World Championship team in 2020. We open with an introduction to the sport of Ultimate (sometimes known as frisbee), her experience coaching the women's team in late 2019, and then move on to her remarkable approach to pedagogical development and skill acceleration in the game of Ultimate.Along the way, we talk about desirable and undesirable problems in training, playing to play vs playing to win, and how she used a training method originally designed for dolphins and dogs and adapted it to humans — with great success!Lesley's Twitter — https://twitter.com/lesley_pizzaLesley's Personal Site — https://lesley.pizza/Newsletter Glue — https://newsletterglue.com/Karen Pryor's Book Reading the Animal Mind: Clicker Training and What It Teaches Us About All Animals (on TAG Teach) — https://www.goodreads.com/en/book/show/2412884How to Get Rich by Felix Dennis — https://www.goodreads.com/el/book/show/1837402.How_to_Get_RichSticky.fm, Lesley's Podcast on Building Sticky Newsletters — https://sticky.fm/(00:00) - Introduction (03:25) - The Sport of Ultimate (10:29) - Defining The Metagame for Ultimate (15:14) - How Lesley Got Into Ultimate (17:27) - Different Styles of Play in Ultimate (20:42) - Coaching Singapore's Women's Worlds Team (31:01) - Using TAG Teach as a Teaching Tool (37:31) - Why Positive Reinforcement (44:16) - Failing Forwards as a Training Philosophy (56:27) - Desirable and Undersirable Problems (01:08:37) - Drills and Simulations But Nothing In Between (01:13:23) - What Makes for a Good Drill? (01:16:23) - Playing to Play vs Playing to Win (01:23:07) - On Newsletter Glue

David MacIver is most known for pushing the adoption and ergonomics of property testing in software with his testing library Hypothesis. Hypothesis is well regarded and widely used in the Python programming language community, and it introduced a handful of innovations that are now quite widespread in the practice of property testing. You’ll hear more about Hypothesis during the podcast, as we talk about what he’s learnt pushing the boundaries of a domain. Then, we shift gears to talk about his coaching practice. David specialises in helping programmers with self improvement, more effective learning, and developing soft skills, which many computer programmers are likely to struggle with, in ways that may limit their careers or their personal development.David’s Substack — https://drmaciver.substack.com/David’s Twitter — https://twitter.com/DRMacIverHypothesis — https://hypothesis.works/, docs: https://hypothesis.readthedocs.io/en/latest/Ethics of False Negatives in Interviewing — https://www.drmaciver.com/2019/06/the-ethics-of-false-negatives-in-interviewing/Life as an Anytime Algorithm — https://notebook.drmaciver.com/posts/2020-03-23-15:52.htmlIf You’re Stuck, Try Something Different (on chopsticks) — https://drmaciver.substack.com/p/lateral-movesHow To Do Hard Things — https://www.drmaciver.com/2019/05/how-to-do-hard-things/Stargate Physics 101 (fanfiction) — https://archiveofourown.org/works/3673335David on why people struggle with mathematics — https://twitter.com/drmaciver/status/1422208261349052420How to Explain Anything to Anyone — https://www.drmaciver.com/2018/10/how-to-explain-anything-to-anyone/0:00 Introduction1:09 What Hypothesis Is3:47 The Story of Hypothesis6:43 Hypothesis’s Contribution to Property Testing12:51 Exploring the Design Space for Hypothesis17:24 When David Knew He Was On To Something with Hypothesis20:35 From Hypothesis to Coaching25:21 Emotional Reactions as Legacy Code29:08 Why David’s Approach to Self Improvement Works for Programmers31:15 Ethical Problems with Optimising False Positive in Hiring37:44 Ways that Programmers Harm Themselves in Their Careers43:28 What Non-Technical People Get Wrong when Dealing with Programmers48:00 Applying Lessons Learnt from Hypothesis to Coaching50:03 Rigour in Self Improvement Writing56:30 Explaining Computers to Non Technical People01:02:55 The Nature of Mathematical Expertise01:11:32  David’s Practice with Teams and Organisations01:14:23 Getting Better at Sprint Planning

Dr Lia DiBello is the CEO, President, and Director of Research of WTRI (Workplace Technology Research Inc), and Senior Scientist at Applied Cognitive Sciences Labs Inc. She is a cognitive scientist as well as a businessperson. In the late 2000s Dr DiBello discovered in an NSF-funded study that all great businesspeople share a common mental model of business, and that mental model can be used for all sorts of interesting things, including the assessment of business expertise, which she did — she was the principal inventor of something called the FutureView Profiler. In more recent years, Dr DiBello is more well known for her work on accelerated expertise — she published a book with a few other researchers in 2016 with that very title. She and her team have created something they call the Strategic Rehearsal, and this actually stemmed from her PhD work, where it was called the OpSim. What the Strategic Rehearsal allows WTRI to do is to accelerate the acquisition of business expertise in the businesses that they consult for, and her training interventions have been used in industries as diverse as biotech, pharma, manufacturing, financial services, and others.- The Triad Mental Model of Business, paper: https://wtri.com/wp-content/uploads/2015/06/Informed-By-Knowledge-Chapter-12.pdf - A summary blog post of the triad mental model: https://commoncog.com/blog/business-mental-model/- FutureView Platform — http://futureviewplatform.com or https://acsilabs.org/- WTRI — https://wtri.com/- List of publications by WTRI, which, if read chronologically, includes a full history of the Strategic Rehearsal — https://wtri.com/research/publications-by-wtri/- Lia's LinkedIn — https://www.linkedin.com/in/liadibello/- Lia's Twitter — https://twitter.com/LiaDiBello4- Gary Klein on WTRI's training with Rio Tinto, on miner safety — https://www.psychologytoday.com/sg/blog/seeing-what-others-dont/201803/training-if-your-life-depended-it- The Oxford Handbook of Expertise — https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780198795872.001.0001/oxfordhb-9780198795872- Accelerated Expertise — https://www.amazon.com/Accelerated-Expertise-Research-Applications/dp/184872652X00:00 Introduction01:32 Lia’s Story04:49 The Triad Mental Model of Business10:38 Transfer of the Triad Between Business Domains16:18 The FutureView Profiler Explained20:42 How She Gets Her Clients To Accept Profiler Recommendations25:21 The Midwest Foundry Story40:14 Cognitive Agility47:32 How Great Businesspeople Learn in the Real World51:08 How This Has Affected Her Practice as a Businessperson54:42 What Lia is Currently Working On1:01:10 The Cognitive Science Behind the Strategic Rehearsal1:09:47 Piaget and Vygotsky’s Theories of Expertise Development1:17:50 The General Form of the Strategic Rehearsal1:21:56 Non-Business Applications of the Strategic RehearsalThe coffee company Lia mentioned: https://amorperfecto.com/The Hollywood Bowl performance: https://www.hollywoodbowl.com/events/performances/1258/2021-08-27/carlos-vives-with-the-la-philSome words from Amor Perfecto to describe what they do:This is a coffee movement launch. For almost two centuries coffee has been at the service of the intermediaries and not of the coffee growing countries. Amor Perfecto, in collaboration with amazing people like Carlos Vives, his wife, Claudia Elena Vasquez and their Tras La Perla Foundation, is going to push to rebalance the unfairly tilted value chain of coffee. Every one in the coffee world knows it exists. Equally important, the product of Roasted at Origin coffee by Amor Perfecto, harvested, roasted and set to you from the mountains of Colombia is a uniquely amazing coffee. It might be the best tasting coffee available today, because producing coffee where it is grown makes a much fresher, better tasting coffee. As Luis Fernando Velez says “have you ever seen the French exporting containers full of grapes to make champagne in Brooklyn?” Coffee should be no different.

A short trailer for Commonplace Expertise, the new podcast from Commonplace https://commoncog.com/blog/