How Meta Built Llama 3

This is a free preview of a paid episode. To hear more, visit www.bigtechnology.comFor years, Siri co-founder Norman Winarsky asked his Stanford classes whether he and his colleagues made the right decision selling their voice bot to Apple. The investor, entrepreneur, and lecturer told me he leaned toward staying independent and IPO. But there was risk operating alone without a deep integration into a mobile operating system like iOS. So after much persistence from Steve Jobs, the deal went through.Today, fifteen years after Apple bought Siri and one year after the company launched Apple Intelligence, Winarsky’s question is more thought provoking than ever. As Apple gets ready to announce its latest vision for Siri and Apple Intelligence at next week’s WWDC, the company is, in many ways, returning toward the plan that Winarsky and his colleagues laid out in Siri’s early days.“This personal assistant would be able to take actions for you,” Winarsky told me of the original Siri vision in our latest Big Tech War Stories podcast episode. “How do you find the nearest gas station? Get me a ticket to the local theater? Buy me a hotel room near San Francisco? All these capabilities were things that you wanted it to do rather than just search.”The full interview — covering Siri’s founding and what’s happened since — is available in full for Big Technology paid subscribers, with preview available for everyone.As Winarsky tells it, the Siri project began in 2003 with DARPA’s ‘Cognitive Assistant that Learns and Organizes’ project.

This is a free preview of a paid episode. To hear more, visit www.bigtechnology.comWalt Mossberg is a legendary tech journalist and longtime columnist for the Wall Street Journal. Mossberg joins Big Tech War Stories to discuss how he formed connections with industry titans like Steve Jobs, Bill Gates, and Jeff Bezos, and what drove their thinking and strategies. Tune in to hear revealing stories of how these CEOs handled criticism, approached product development, and viewed their role in shaping the tech landscape. We also cover Mossberg's own evolution as a journalist focusing on products rather than business, and how that unique lens gave him rare insights. Hit play for an engaging conversation that reveals the story behind the figures who shaped the technology world. To listen to the full episode, go to bigtechnology.com if you're a paid subscriber or upgrade here.

This is a free preview of a paid episode. To hear more, visit www.bigtechnology.comAhmad Al-Dahle is Meta's VP of generative AI. He joins Big Tech War Stories to disucss how the company built its Llama 3 model, releasing today. Training Llama 3 took ten times more data, one hundred times more computing resources, and personality tweaks to make it more willing to answer questions it otherwise would’ve refused. You can get the full show as a paid subscriber on bigtechnology.com

This is a free preview of a paid episode. To hear more, visit www.bigtechnology.comGoogle made $31 billion running ads across the web last year, capitalizing on a tech footprint that touches nearly every web page on the internet. The company’s ad server and ad exchange are ubiquitous, behind-the-scenes technology that helps websites make money, and it all dates back to its 2007 acquisition of DoubleClick. In this month’s edition of Bi…

This is a free preview of a paid episode. To hear more, visit www.bigtechnology.comBy now you know about Facebook’s role in politics but how did it get there. And why is it now backing away. Today, on Big Tech War Stories, we take you deep inside the company, on the ground with someone who spent 10 years on Facebook’s policy team — including running its elections unit — to discuss how Facebook handled politics, why it wanted it, and whether it can actually take a step back as major elections come its way.Our guest is Katie Harbath, she’s the founder and CEO of anchor change and someone with firsthand knowledge of how Facebook helped political candidates operate on the platform, where things went south, how influence campaigns impacted the company, and the social network’s calculations moving forward. 

Google had an equal — and perhaps better — version of ChatGPT working inside its company before OpenAI released its bot to the world. But it never shipped. So why didn’t Google ship it?Gaurav Nemade, the bot’s first product manager, shares what happened on our new Big Tech War Stories podcast. This is the show’s first-ever episode —available to premium Big Technology subscribers — and we’re debuting with a free preview today.The key issues holding the bot back, according to Nemade, were public relations concerns and the dark cloud cast by Microsoft’s misadventure with Tay, a teenage character bot that turned Nazi overnight.“PR always top of mind for leaders,” says Nemade. “On the other side, OpenAI — they don't give a s**t about PR. For the most part, they don't. They are like, ‘Okay, this is what we think is right. This is a reasonable way of putting it out.’ They become vulnerable, they put it out, and then they work with the community.”Nemade also shares a list of potential products his team thought LLM technology could be used for, including imbuing non-playable characters in video games with complex personalities and bots that would get you out of awkward situations by calling your phone and talking you to safety. Google eventually did ship Bard, an LLM-powered chatbot, but Nemade is convinced it would’ve sat on the technology if OpenAI hadn’t forced its hand. More in the interview!Launch SpecialBig Technology’s launch special is running for the next few days. Sign up and you’ll get Big Tech War Stories, our panel of experts reacting to major news events, Kristi Coulter’s Amazon column, and my weekly Friday stories. This deal is $90 for the year, or 50% off the monthly price. Check it out →Big Technology AMATomorrow, Tuesday, October 31, at 11 a.m. pacific time / 2 p.m. eastern, I am hosting an ‘ask me anything’ session here on Substack. You can join, ask anything, and register here: https://lu.ma/Alex-Kantrowitz-amaThanks!AlexPS: Here’s our Big Tech War Stories Podcast art, courtesy of Dall-3 This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.bigtechnology.com/subscribe

We're live this week! Subscribe to get this monthly show in your podcast player of choice. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.bigtechnology.com/subscribe