Start tilbuddet

Lenny's Podcast: Product | Growth | Career

Lenny's Podcast: Product | Growth | Career

Af
- Lenny Rachitsky

0 Anmeldelser: 0
Afsnit: 299
Gns. varighed: 1T 25M
Sprog: Engelsk
Format
Kategori: Økonomi & Business

Interviews with world-class product leaders and growth experts to uncover concrete, actionable, and tactical advice to help you build, launch, and grow your own product.

Episodes

Inside Google's AI turnaround: The rise of AI Mode, strategy behind AI Overviews, and their vision for AI-powered search | Robby Stein (VP of Product, Google Search)Robby Stein is VP of Product at Google, where he oversees the core products of Google Search—including the new AI Overviews, AI Mode, search ranking, Google Lens, and more. Previously, he led consumer products at Instagram, where he and his teams built Stories, Reels, Close Friends, and other key features now used by billions. What you’ll learn: 1. Why Google’s AI products are suddenly taking off after years of perceived stagnation 2. How AI is expanding Search rather than replacing it, contrary to what many predicted 3. The three core product principles that have helped Robby build multiple billion-user products 4. Inside Instagram’s decision to build its own version of Snapchat Stories 5. His mantra of “relentless improvement” 6. How Google developed AI Mode from concept to launch in just one year 7. Why most teams give up too early on potentially transformative products — Brought to you by: • Vanta—Automate compliance. Simplify security: https://vanta.com/lenny • Jira Product Discovery—Confidence to build the right thing: https://atlassian.com/lenny/?utm_source=lennypodcast&utm_medium=paid-audio&utm_campaign=fy24q1-jpd-imc • Orkes—The enterprise platform for reliable applications and agentic workflows: https://www.orkes.io/ — Transcript: ⁠https://www.lennysnewsletter.com/p/how-google-built-ai-mode-in-under-a-year⁠ — My biggest takeaways (for paid newsletter subscribers): ⁠https://www.lennysnewsletter.com/i/175041217/my-biggest-takeaways-from-this-conversation⁠ — Where to find Robby Stein: • X: https://x.com/rmstein • LinkedIn: https://www.linkedin.com/in/robbystein/ — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — In this episode, we cover: (00:00) Introduction to Robby Stein (04:46) Google’s recent success with AI (06:08) The evolution of Google Search (09:41) AI Mode and its impact (15:30) The rise of AEO (18:50) Building successful AI products (21:31) Embodying relentless improvement (30:10) Lessons from Instagram Stories (35:20) Driving growth in established products (40:08) Balancing optimization and innovation (43:39) The journey of AI Mode: From launch to expansion (48:05) Organizational changes and urgency (49:51) AI Mode vs. competitors (51:35) Core product principles (57:07) Instagram’s Close Friends feature (01:03:01) The importance of resources in development (01:06:39) AI corner (01:11:19) Curiosity and learning (01:15:01) Lightning round and final thoughts — Referenced: • Google Gemini: https://gemini.google.com/app • Nano Banana: https://aistudio.google.com/models/gemini-2-5-flash-image • Chat GPT: https://chatgpt.com/ • Perplexity: https://www.perplexity.ai/ • Google Lens: https://lens.google/ • AI Google search: https://www.google.com/ai • Why ChatGPT will be the next big growth channel (and how to capitalize on it) | Brian Balfour (Reforge): https://www.lennysnewsletter.com/p/why-chatgpt-will-be-the-next-big-growth-channel-brian-balfour • Alex Rampell on X: https://x.com/arampell • A 4-step framework for building delightful products | Nesrine Changuel (Spotify, Google, Skype): https://www.lennysnewsletter.com/p/a-4-step-framework-for-building-delightful-products • Look broader, look closer, think younger: Tony Fadell speaks at TED2015: https://blog.ted.com/look-broader-look-closer-think-younger-tony-fadell-speaks-at-ted2015/ • Jobs to Be Done: https://www.christenseninstitute.org/theory/jobs-to-be-done/ • The ultimate guide to JTBD | Bob Moesta (co-creator of the framework): https://www.lennysnewsletter.com/p/the-ultimate-guide-to-jtbd-bob-moesta • Rinstagram or Finstagram? The curious duality of the modern Instagram user: https://www.theguardian.com/technology/2016/sep/26/rinstagram-finstagram-instagram-accounts • V03: https://v03ai.com/ • Pirate GPT: https://www.kickstarter.com/projects/silentmeditation/pirate-gpt/ • The Bear on Hulu: https://www.hulu.com/series/the-bear-05eb6a8e-90ed-4947-8c0b-e6536cbddd5f • Dune on HBO Max: https://www.hbomax.com/movies/dune/e7dc7b3a-a494-4ef1-8107-f4308aa6bbf7 • Top Gun: Maverick: https://www.imdb.com/title/tt1745960/ • Purple pillows: https://purple.com/pillows • Avocado pillow: https://www.avocadogreenmattress.com/products/green-pillow • Justin Bieber’s website: https://www.justinbiebermusic.com/ • Scooter Braun’s website: https://scooterbraun.com/ — Recommended books: • Competing Against Luck: The Story of Innovation and Customer Choice: https://www.amazon.com/Competing-Against-Luck-Innovation-Customer/dp/0062435612 • The Design of Everyday Things: https://www.amazon.com/Design-Everyday-Things-Revised-Expanded/dp/0465050654 • Aurora: https://www.amazon.com/Aurora-High-Stakes-Survival-Navigate-Darkness/dp/0062916475 • Project Hail Mary: https://www.amazon.com/Project-Hail-Mary-Andy-Weir/dp/0593135202 — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. — Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com
299
|
1T 21M
First interview with Scale AI’s CEO: $14B Meta deal, what’s working in enterprise AI, and what frontier labs are building next | Jason DroegeJason Droege is the CEO of Scale AI, a company that provides foundational training data to every major AI lab. He previously co-founded Scour with Travis Kalanick and built Uber Eats from idea to $20 billion in revenue. In this conversation, Jason shares lessons from getting sued for $250 billion, discovering restaurant economics by weighing sandwich ingredients, and over 25 years of launching transformative technology businesses. What you’ll learn: What actually happened with Meta’s $14 billion investment in Scale AI Why AI models still need human experts to improve, and how that relationship is evolving How AI models learn from experts building websites and debugging code The business lessons from building Uber Eats from zero to $20 billion Why most enterprise data is useless for AI models today Why urgent daily problems beat super-valuable occasional problems when building products How to think independently when building new products and businesses — Brought to you by: Merge—The fastest way to ship 220+ integrations: http://merge.dev/lenny Figma Make—A prompt-to-code tool for making ideas real: https://www.figma.com/lenny/ Mercury—The art of simplified finances: https://mercury.com/ — Transcript: https://www.lennysnewsletter.com/p/first-interview-with-scale-ais-ceo-jason-droege — My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/174979621/my-biggest-takeaways-from-this-conversation — Where to find Jason Droege: • X: https://x.com/jdroege • LinkedIn: https://www.linkedin.com/in/jasondroege/ — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — In this episode, we cover: (00:00) Introduction to Jason Droege (06:01) Jason’s early career and lessons learned (10:27) The current state of Scale AI (12:37) The shift to expert data labeling (17:02) Challenges and strategies in finding experts (18:48) Reinforcement learning and AI environments (28:18) The future of AI and human involvement (31:21) The role of evals (35:25) What AI models will look like in the next few years (41:43) Building Uber Eats and understanding customer needs (48:19) The importance of independent thinking (50:45) Setting high standards for new businesses (53:03) Exploring and selecting business ideas (57:07) The McDonald’s story (01:00:13) The role of gross margins in business feasibility (01:04:49) Why Jason says, “Not losing is a precursor to winning” (01:09:12) Hiring and building teams (01:12:11) AI corner (01:14:47) Lightning round and final thoughts — Referenced: • Travis Kalanick on X: https://x.com/travisk • Scour: https://en.wikipedia.org/wiki/Scour_Inc. • Scale: https://scale.com/ • Alexandr Wang on X: https://x.com/alexandr_wang • Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor): https://www.lennysnewsletter.com/p/experts-writing-ai-evals-brendan-foody • Brendan Foody’s post on X about knowledge work changing: https://x.com/BrendanFoody/status/1970163503702188048 • MIT Finds 95% of GenAI Pilots Fail Because Companies Avoid Friction: https://www.forbes.com/sites/jasonsnyder/2025/08/26/mit-finds-95-of-genai-pilots-fail-because-companies-avoid-friction/ • Uber Eats: https://www.ubereats.com/ • Stephen Chau on X: https://x.com/thestephenchau • a16z Podcast: https://a16z.com/podcasts/a16z-podcast/ • F1: The Movie: https://www.imdb.com/title/tt16311594/ • V03: https://v03ai.com/ • Careers at Scale: https://scale.com/careers — Recommended books: • The Selfish Gene: https://www.amazon.com/Selfish-Gene-Anniversary-Introduction/dp/0199291152 • The Road Less Traveled: A New Psychology of Love, Traditional Values, and Spiritual Growth: https://www.amazon.com/Road-Less-Traveled-Timeless-Traditional/dp/0743243153/ • Good to Great: Why Some Companies Make the Leap . . . And Others Don’t: https://www.amazon.com/Good-Great-Some-Companies-Others/dp/0066620996 • Thinking, Fast and Slow: https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555/ — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. — Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com
298
|
1T 24M
How to find hidden growth opportunities in your product | Albert Cheng (Duolingo, Grammarly, Chess.com)Albert Cheng has led growth at three of the world’s most successful consumer subscription companies: Duolingo, Grammarly, and Chess.com. A former Google product manager (and serious pianist!), Albert developed a unique approach to finding and scaling growth opportunities through rapid experimentation and deep user psychology. His teams run 1,000 experiments a year, discovering counterintuitive insights that have driven tens of millions in revenue. What you’ll learn: 1. How to use the explore-exploit framework to find new growth opportunities 2. How showing premium features to free users doubled Grammarly’s upgrades to paid plans 3. What good retention looks like for a consumer subscription app 4. Why resurrected users drive 80% of mature product growth 5. Why “reverse trials” work better than time-based trials 6. The three pillars of successful gamification: core loop, metagame, and profile — Brought to you by: Vanta—Automate compliance. Simplify security. Jira Product Discovery—Confidence to build the right thing Miro—A collaborative visual platform where your best work comes to life — Where to find Albert Cheng: • X: https://x.com/albertc248 • LinkedIn: https://www.linkedin.com/in/albertcheng1/ • Chess.com: https://www.chess.com/member/Goniners — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — Referenced: • How Duolingo reignited user growth: https://www.lennysnewsletter.com/p/how-duolingo-reignited-user-growth • Inside ChatGPT: The fastest-growing product in history | Nick Turley (Head of ChatGPT at OpenAI): https://www.lennysnewsletter.com/p/inside-chatgpt-nick-turley • Explore vs. Exploit: https://brianbalfour.com/quick-takes/explore-vs-exploit • Grammarly: https://www.grammarly.com/ • Reforge: https://www.reforge.com/ • Chess.com: https://www.chess.com/ • Everyone’s an engineer now: Inside v0’s mission to create a hundred million builders | Guillermo Rauch (founder & CEO of Vercel, creators of v0 and Next.js): https://www.lennysnewsletter.com/p/everyones-an-engineer-now-guillermo-rauch • Building Lovable: $10M ARR in 60 days with 15 people | Anton Osika (CEO and co-founder): https://www.lennysnewsletter.com/p/building-lovable-anton-osika • Figma: https://www.figma.com/ • Cursor: https://cursor.com/ • The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell • Claude Code: https://www.anthropic.com/claude-code • GitHub Copilot: https://github.com/features/copilot • Noam Lovinsky on LinkedIn: https://www.linkedin.com/in/noaml/ • The happiness and pain of product management | Noam Lovinsky (Grammarly, Facebook, YouTube, Thumbtack): https://www.lennysnewsletter.com/p/the-happiness-and-pain-of-product • Kyla Siedband on LinkedIn: https://www.linkedin.com/in/kylasiedband/ • The Duolingo handbook: https://blog.duolingo.com/handbook/ • Lenny’s post on X about the Duolingo handbook: https://x.com/lennysan/status/1889008405584683091 • The rituals of great teams | Shishir Mehrotra of Coda, YouTube, Microsoft: https://www.lennysnewsletter.com/p/the-rituals-of-great-teams-shishir • Duolingo on TikTok: https://www.tiktok.com/@duolingo • Kasparov vs. Deep Blue | The Match That Changed History: https://www.chess.com/article/view/deep-blue-kasparov-chess • Magnus Carlsen: https://en.wikipedia.org/wiki/Magnus_Carlsen • Elo rating system: https://www.chess.com/terms/elo-rating-chess • Stockfish: https://en.wikipedia.org/wiki/Stockfish_(chess) • AlphaGo on Prime Video: https://www.primevideo.com/detail/AlphaGo/0KNQHKKDAOE8OCYKQS9WSSDYN0 • Statsig: https://www.statsig.com/ • The State of Product in 2026: Navigating Change, Challenge, and Opportunity: https://www.atlassian.com/blog/announcements/state-of-product-2026 • Erik Allebest on LinkedIn: https://www.linkedin.com/in/erikallebest/ • Daniel Rensch on X: https://x.com/danielrensch • Chariot: https://en.wikipedia.org/wiki/Chariot_(company) • San Francisco 49ers: https://www.49ers.com/ • Breville Barista Express: https://www.breville.com/en-us/product/bes870 — Recommended books: • Snuggle Puppy!: A Little Love Song: https://www.amazon.com/Snuggle-Puppy-Little-Boynton-Board/dp/1665924985 • Ogilvy on Advertising: https://www.amazon.com/Ogilvy-Advertising-David/dp/039472903X • Dark Squares: How Chess Saved My Life: https://www.amazon.com/Dark-Squares-Chess-Saved-Life/dp/1541703286 — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com
297
|
1T 25M
A 4-step framework for building delightful products | Nesrine Changuel (Spotify, Google, Skype)Nesrine Changuel helped build Spotify, Google Chrome, and Google Meet. Her work has helped her discover the importance of emotional connection in building successful products. At Google, she served as a dedicated “delight PM,” a role specifically focused on making products more delightful. She recently published Product Delight, a book that provides a practical framework for creating products that serve both functional and emotional needs. Based in Paris, she now coaches founders and CPOs on implementing delight strategies in their organizations. What you’ll learn: 1. Why delight is a business strategy, not just “sprinkling confetti” on top of functionality 2. How to identify emotional motivators that drive product retention 3. The 50-40-10 rule for balancing delight in your roadmap 4. The 4-step delight model 5. The origin story of Spotify’s Discover Weekly 6. Why B2B products need delight just as much as B2C products 7. How to get buy-in from skeptical leaders who think delight is a luxury — Brought to you by: DX—The developer intelligence platform designed by leading researchers: https://getdx.com/lenny Jira Product Discovery—Confidence to build the right thing: https://atlassian.com/lenny LucidLink—Real-time cloud storage for teams: https://www.lucidlink.com/lenny — Transcript: https://www.lennysnewsletter.com/p/a-4-step-framework-for-building-delightful-products — My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/174199489/my-biggest-takeaways-from-this-conversation — Where to find Nesrine Changuel: • LinkedIn: https://www.linkedin.com/in/nesrinechanguel/ • Newsletter: https://nesrinechanguel.substack.com/ • Website: https://nesrine-changuel.com/ — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — In this episode, we cover: (00:00) Introduction to Nesrine and product delight (04:56) Why delight matters (09:17) What makes a feature “delightful” (12:29) The three pillars of delight (13:03) Pillar 1: Removing friction (Uber refund example) (15:07) Pillar 2: Anticipating needs (Revolut eSIM example) (17:21) Pillar 3: Exceeding expectations (Edge coupon example) (18:35) The “confetti effect” and when it actually works (22:02) B2B vs. B2C: Why all products need emotional connection (29:52) The Delight Model: A 4-step framework (30:57) Step 1: Identifying user motivators (functional and emotional) (33:55) Step 2: Converting motivators into product opportunities (34:46) Step 3: Identifying solutions with the delight grid (36:46) Step 4: Validating ideas with the delight checklist (40:22) The Delight Model summarized (42:18) The importance of familiarity (Spotify Discover Weekly story) (45:21) Real examples: Chrome’s tab management solution (51:32) Google Meet’s solution for “Zoom fatigue” (55:02) Getting buy-in from skeptical leaders (59:39) Prioritizing delight: The 50-40-10 rule (1:02:41) Creating a culture of delight in your organization (1:06:45) The habituation effect (1:08:15) When delight goes wrong: Apple reactions example (1:10:21) How delight motivates product teams (1:12:24) Lightning round and final thoughts — Referenced: • Spotify: https://open.spotify.com/ • Linear: https://linear.app/ • How Linear builds product: https://www.lennysnewsletter.com/p/how-linear-builds-product • Jira: https://www.atlassian.com/software/jira • Asana: https://asana.com/ • Monday: https://monday.com/ • The Product Delight Model: https://nesrinechanguel.substack.com/p/the-product-delight-model • Revolut: https://www.revolut.com/ • How Revolut trains world-class product managers: The “local CEO” model, raw intellect over experience, and a cultural obsession with building wow products | Dmitry Zlokazov (Head of Product): https://www.lennysnewsletter.com/p/how-revolut-trains-world-class-product-managers • Microsoft Cashback: https://www.microsoft.com/en-us/edge/features/shopping-cashback • Superhuman’s secret to success: Ignoring most customer feedback, manually onboarding every new user, obsessing over every detail, and positioning around a single attribute: speed | Rahul Vohra (CEO): https://www.lennysnewsletter.com/p/superhumans-secret-to-success-rahul-vohra • Brian Chesky’s secret mentor who died 9 times, started the Burning Man board, and built the world’s first midlife wisdom school | Chip Conley (founder of MEA): https://www.lennysnewsletter.com/p/chip-conley • Workday: https://www.workday.com/ • SAP: https://www.sap.com/ • ServiceNow: https://www.servicenow.com/ • Salesforce: https://www.salesforce.com/ • GitHub: https://github.com/ • Atlassian: https://www.atlassian.com/ • Snowflake: https://www.snowflake.com/ • Data Superheroes: https://www.snowflake.com/en/data-superheroes/ • Google Meet: https://meet.google.com/ • Andy Nesling on LinkedIn: https://www.linkedin.com/in/andynesling/ • Matic: https://maticrobots.com/ • Diego Sanchez’s (Senior Product Manager at Buffer) post on LinkedIn: https://www.linkedin.com/feed/update/urn:li:activity:7365014292091346945/ • Miro: https://miro.com/ • Arc browser: https://arc.net/ • Competing with giants: An inside look at how The Browser Company builds product | Josh Miller (CEO): https://www.lennysnewsletter.com/p/competing-with-giants-an-inside-look • Migros Supermarket: https://www.migros.ch/ • The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell • Building Lovable: $10M ARR in 60 days with 15 people | Anton Osika (CEO and co-founder): https://www.lennysnewsletter.com/p/building-lovable-anton-osika • Linear’s secret to building beloved B2B products | Nan Yu (Head of Product): https://www.lennysnewsletter.com/p/linears-secret-to-building-beloved-b2b-products-nan-yu • Suno: https://suno.com • Snapchat: https://www.snapchat.com/ • Use Reactions, Presenter Overlay, and other effects when videoconferencing on Mac: https://support.apple.com/en-us/105117 • Dr. Lipp: https://drlipp.com/ • How to be the best coach to product people | Petra Wille (Strong Product People): https://www.lennysnewsletter.com/p/how-to-be-the-best-coach-to-product • The Great American Baking Show: https://www.imdb.com/title/tt21822674/ • Le Meilleur Pâtissier: https://en.wikipedia.org/wiki/Le_Meilleur_P%C3%A2tissier • The Upside on Amazon Prime: https://www.amazon.com/gp/video/detail/amzn1.dv.gti.3cb8500f-31af-9f4f-5dec-701e086d58e8 • The Intouchables: https://www.imdb.com/title/tt1675434/ • Yoyo stroller: https://www.stokke.com/USA/en-us/category/strollers/yoyo-strollers • UppaBaby strollers: https://uppababy.com/strollers/ — Recommended books: • Product Delight: How to Make Your Product Stand Out with Emotional Connection: https://www.amazon.com/Product-Delight-Stand-Emotional-Connection-ebook/dp/B0FGZ93D9Y/ • Factfulness: Ten Reasons We’re Wrong About the World—and Why Things Are Better Than You Think: https://www.amazon.com/Factfulness-Reasons-World-Things-Better/dp/1250107814 • STRONG Product Communities: The Essential Guide to Product Communities of Practice: https://www.amazon.com/STRONG-Product-Communities-Essential-Practice/dp/3982235189/r — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com
296
|
1T 24M
Why AI evals are the hottest new skill for product builders | Hamel Husain & Shreya Shankar (creators of the #1 eval course)Hamel Husain and Shreya Shankar teach the world’s most popular course on AI evals and have trained over 2,000 PMs and engineers (including many teams at OpenAI and Anthropic). In this conversation, they demystify the process of developing effective evals, walk through real examples, and share practical techniques that’ll help you improve your AI product. What you’ll learn: 1. WTF evals are 2. Why they’ve become the most important new skill for AI product builders 3. A step-by-step walkthrough of how to create an effective eval 4. A deep dive into error analysis, open coding, and axial coding 5. Code-based evals vs. LLM-as-judge 6. The most common pitfalls and how to avoid them 7. Practical tips for implementing evals with minimal time investment (30 minutes per week after initial setup) 8. Insight into the debate between “vibes” and systematic evals — Brought to you by: Fin—The #1 AI agent for customer service Dscout—The UX platform to capture insights at every stage: from ideation to production Mercury—The art of simplified finances — Where to find Shreya Shankar • X: https://x.com/sh_reya • LinkedIn: https://www.linkedin.com/in/shrshnk/ • Website: https://www.sh-reya.com/ • Maven course: https://bit.ly/4myp27m — Where to find Hamel Husain • X: https://x.com/HamelHusain • LinkedIn: https://www.linkedin.com/in/hamelhusain/ • Website: https://hamel.dev/ • Maven course: https://bit.ly/4myp27m — In this episode, we cover: (00:00) Introduction to Hamel and Shreya (04:57) What are evals? (09:56) Demo: Examining real traces from a property management AI assistant (16:51) Writing notes on errors (23:54) Why LLMs can’t replace humans in the initial error analysis (25:16) The concept of a “benevolent dictator” in the eval process (28:07) Theoretical saturation: when to stop (31:39) Using axial codes to help categorize and synthesize error notes (44:39) The results (46:06) Building an LLM-as-judge to evaluate specific failure modes (48:31) The difference between code-based evals and LLM-as-judge (52:10) Example: LLM-as-judge (54:45) Testing your LLM judge against human judgment (01:00:51) Why evals are the new PRDs for AI products (01:05:09) How many evals you actually need (01:07:41) What comes after evals (01:09:57) The great evals debate (1:15:15) Why dogfooding isn’t enough for most AI products (01:18:23) OpenAI’s Statsig acquisition (1:23:02) The Claude Code controversy and the importance of context (01:24:13) Common misconceptions around evals (1:22:28) Tips and tricks for implementing evals effectively (1:30:37) The time investment (1:33:38) Overview of their comprehensive evals course (1:37:57) Lightning round and final thoughts — LLM Log Open Codes Analysis Prompt: Please analyze the following CSV file. There is a metadata field which has an nested field called z_note that contains open codes for analysis of LLM logs that we are conducting. Please extract all of the different open codes. From the _note field, propose 5-6 categories that we can create axial codes from. — Referenced: • Building eval systems that improve your AI product: https://www.lennysnewsletter.com/p/building-eval-systems-that-improve • Mercor: https://mercor.com/ • Brendan Foody on LinkedIn: https://www.linkedin.com/in/brendan-foody-2995ab10b • Nurture Boss: https://nurtureboss.io/ • Braintrust: https://www.braintrust.dev/ • Andrew Ng on X: https://x.com/andrewyng • Carrying Out Error Analysis: https://www.youtube.com/watch?v=JoAxZsdw_3w • Julius AI: https://julius.ai/ • Brendan Foody on X—“evals are the new PRDs”: https://x.com/BrendanFoody/status/1939764763485171948 • Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences: https://dl.acm.org/doi/abs/10.1145/3654777.3676450 • Lenny’s post on X about evals: https://x.com/lennysan/status/1909636749103599729 • Statsig: https://statsig.com/ • Claude Code: https://www.anthropic.com/claude-code • Cursor: https://cursor.com/ • Occam’s razor: https://en.wikipedia.org/wiki/Occam%27s_razor • Frozen: https://www.imdb.com/title/tt2294629/ • The Wire on HBO: https://en.wikipedia.org/wiki/The_Wire — Recommended books: • Pachinko: https://www.amazon.com/Pachinko-National-Book-Award-Finalist/dp/1455563935 • Apple in China: The Capture of the World’s Greatest Company: https://www.amazon.com/Apple-China-Capture-Greatest-Company/dp/1668053373/ • Machine Learning: https://www.amazon.com/Machine-Learning-Tom-M-Mitchell/dp/1259096955 • Artificial Intelligence: A Modern Approach: https://www.amazon.com/Artificial-Intelligence-Modern-Approach-Global/dp/1292401133/ Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. — Lenny may be an investor in the companies discussed. My biggest takeaways from this conversation: To hear more, visit www.lennysnewsletter.com
295
|
1T 46M
From managing people to managing AI: The leadership skills everyone needs now | Julie Zhuo (Facebook VP, Sundial CEO, The Making of a Manager author)Julie Zhuo is the former VP and Head of Design at Facebook (now Meta), author of the bestselling book The Making of a Manager, and co-founder of Sundial, an AI-powered data analysis company. Also, my first-ever podcast guest over 3 years ago! In our conversation, we discuss: 1. The three core manager skills that translate directly to managing AI agents 2. How her team uses AI to learn new skills 10x faster 3. The “diagnose with data, treat with design” framework for balancing gut and data 4. Why hypergrowth AI companies have terrible data infrastructure (and why it doesn’t matter) 5. How to give feedback that actually lands—including Julie’s exact script for difficult conversations 6. What Julie’s teaching her kids about an AI future (hint: it’s not coding or STEM) — Brought to you by: Mercury — The art of simplified finances DX — The developer intelligence platform designed by leading researchers PostHog—How developers build successful products — Transcript: https://www.lennysnewsletter.com/p/from-managing-people-to-managing-ai-julie-zhuo — My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/172723725/my-biggest-takeaways-from-this-conversation — Where to find Julie Zhuo: • X: https://x.com/joulee • LinkedIn: https://www.linkedin.com/in/julie-zhuo/ • Website: https://www.juliezhuo.com/ • Newsletter: https://lg.substack.com/ • Sundial: https://sundial.so/ — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — In this episode, we cover: (00:00) Welcome back, Julie! (05:18) The success of The Making of a Manager (08:41) Why AI will make everyone a manager (11:38) The future of management roles (14:00) Empowering teams with AI (21:30) Specific roles being accelerated by AI (26:53) Data analysis in AI companies (32:02) The role of data in design (37:21) The evolving role of managers in the AI era (40:22) Embracing change and uncertainty (42:14) Timeless lessons for managers (49:03) Balancing strengths and weaknesses (57:49) Building a feedback culture (01:05:33) Creating win-win situations (01:09:27) Being aware of your own energy and conviction (01:12:12) Navigating disagreements with higher-ups (01:15:57) AI corner (01:20:08) Contrarian corner (01:23:14) Lightning round and final thoughts — Referenced: • Julie Zhuo on accelerating your career, impostor syndrome, writing, building product sense, using intuition vs. data, hiring designers, and moving into management: https://www.lennysnewsletter.com/p/episode-2-julie-zhuo • Waymo: https://waymo.com/ • How we restructured Airtable’s entire org for AI | Howie Liu (co-founder and CEO): https://www.lennysnewsletter.com/p/how-we-restructured-airtables-entire-org-for-ai • Cursor: https://cursor.com/ • The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell • Inside ChatGPT: The fastest growing product in history | Nick Turley (Head of ChatGPT at OpenAI): https://www.lennysnewsletter.com/p/inside-chatgpt-nick-turley • Behind the founder: Marc Benioff: https://www.lennysnewsletter.com/p/behind-the-founder-marc-benioff • OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter): https://www.lennysnewsletter.com/p/kevin-weil-open-ai • Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram): https://www.lennysnewsletter.com/p/anthropics-cpo-heres-what-comes-next • The Magic Loop: https://www.lennysnewsletter.com/p/the-magic-loop • Dunning-Kruger effect: https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect • Eric Antonow on LinkedIn: https://www.linkedin.com/in/antonow/ • Methaphone: https://methaphone.com/ • Replit: https://replit.com/ • “Baby” by Justin Bieber on Spotify: https://open.spotify.com/track/6epn3r7S14KUqlReYr77hA • Kingdom Rush: https://www.kingdomrush.com/ • Dr. Becky on TikTok: https://www.tiktok.com/@drbeckyatgoodinside • Emily Oster on TikTok: https://www.tiktok.com/@profemilyoster • La La Land on Netflix: https://www.netflix.com/title/80095365 • Granola: https://www.granola.ai/ • Matic robots: https://maticrobots.com/ • Limitless pendant: https://www.limitless.ai/ • How I AI: https://www.youtube.com/@howiaipodcast — Recommended books: • The Making of a Manager: What to Do when Everyone Looks to You: https://www.amazon.com/Making-Manager-What-Everyone-Looks/dp/0525540423 • High Output Management: https://www.amazon.com/High-Output-Management-Andrew-Grove/dp/0679762884/ • Zen and the Art of Motorcycle Maintenance: An Inquiry into Values: https://www.amazon.com/Zen-Art-Motorcycle-Maintenance-Inquiry/dp/0061673730 • Conscious Business: How to Build Value Through Values: https://www.amazon.com/Conscious-Business-Build-through-Values/dp/1622032020 • Good Inside: A Practical Guide to Resilient Parenting Prioritizing Connection Over Correction: https://www.amazon.com/Good-Inside-Guide-Becoming-Parent/dp/0063159481/ — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com
294
|
1T 36M
Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor)Brendan Foody is the CEO and co-founder of Mercor, the fastest-growing company in history to go from $1M to $500M in revenue (in just 17 months!). At 22, he is also the youngest American unicorn founder ever. Mercor works with 6 of the Magnificent 7 and all top 5 AI labs to help them hire experts to create evaluations and training data that improve their models. In this conversation, Brendan explains why evals have become the critical bottleneck for AI progress, how he discovered this massive opportunity, and what the future of work might look like in an AI-driven economy. What you’ll learn: 1. Why evals are becoming the primary bottleneck for AI progress and what this means for AI startups 2. How Mercor grew to $500M revenue in 17 months (fastest in history) 3. Brendan’s meeting with xAI that changed his company’s trajectory 4. Which skills and jobs will remain most valuable as AI continues to advance (hint: jobs with “elastic” demand) 5. Why Brendan believes AGI and superintelligence are not happening anytime soon 6. The three unique core values that drove Mercor’s success 7. How Harvard Lampoon writers are making Claude funnier — Brought to you by: WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs Jira Product Discovery—Atlassian’s new prioritization and roadmapping tool built for product teams Enterpret—Transform customer feedback into product growth — Transcript: https://www.lennysnewsletter.com/p/experts-writing-ai-evals-brendan-foody — My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/173303790/my-biggest-takeaways-from-this-conversation — Where to find Brendan Foody: • X: https://x.com/BrendanFoody • LinkedIn: https://www.linkedin.com/in/brendan-foody-2995ab10b/ — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — In this episode, we cover: (00:00) Introduction to Brendan Foody and Mercor (05:38) The “era of evals” (09:26) Understanding the AI training landscape (17:10) The future of work and AI (25:54) The evolution of labor markets (29:55) Understanding how AI models are trained (38:58) Building Mercor (53:27) Lessons from past ventures (56:55) The future of AI and model improvement (01:00:41) His personal use of AI and final thoughts — References: https://www.lennysnewsletter.com/p/experts-writing-ai-evals-brendan-foody — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com
293
|
1T 6M
The ultimate guide to AEO: How to get ChatGPT to recommend your product | Ethan Smith (Graphite)Ethan Smith is the CEO of Graphite—the leading SEO growth agency—and my go-to expert on SEO. After 18 years of mastering traditional SEO, Ethan has been at the forefront of what is called AEO: answer engine optimization, or, more simply, getting your product to show up in ChatGPT/Claude/Gemini/Perplexity answers. He’s discovered that ChatGPT traffic converts six times better than Google search—and most companies are completely missing this opportunity. In our conversation, we discuss: 1. His 7-step playbook to rank #1 in ChatGPT 2. Why ChatGPT traffic converts 6x better than Google 3. How early-stage startups can win at AEO immediately (unlike with SEO, which takes years) 4. The three tactics that actually work: landing pages, YouTube videos, and Reddit comments 5. Why help-center content can suddenly be your highest-ROI investment 6. The specific Reddit strategy that works (spoiler: be authentic) 7. Why AI-generated content doesn’t work — Brought to you by: Orkes—The enterprise platform for reliable applications and agentic workflows Vanta—Automate compliance. Simplify security. Great Question—Empower everyone to run great research — Where to find Ethan Smith: • Twitter: https://twitter.com/ethan_l_s • LinkedIn: https://bit.ly/ethans-linkedin • Graphite: https://graphite.io/ • Graphite Research Papers: https://bit.ly/graphite-five-percent — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — In this episode, we cover: (00:00) Welcome back, Ethan (04:34) The changing landscape of SEO (06:19) AEO (answer engine optimization) vs. GEO (generative engine optimization) (08:13) The impact of AEO (11:51) How early-stage startups can win at AEO (14:34) The quality of AEO leads (15:35) On-site vs. off-site traffic (16:32) Reddit’s role in AEO and avoiding spam (20:11) How AI models use citations (RAG) (21:41) Key principles for winning at AEO (25:00) Avoiding hyper-SEOed content, and the importance of originality (28:55) Actionable AEO playbook: steps and experiments (33:35) Tracking, measuring, and share of voice (38:34) Adapting AEO for B2B, commerce, and early-stage companies (41:11) Is letting AI index your content good? (43:06) Experimentation, control groups, and measuring results (46:15) The future of AEO, SEO, and search channels (51:35) AI-generated content: what works and what doesn’t (55:25) The dangers of infinite AI derivatives (58:44) The future: convergence of LLMs and search (01:00:40) Help-center optimization and the long tail (01:03:18) Lightning round and final thoughts — Resources and episode mentions: https://www.lennysnewsletter.com/p/the-ultimate-guide-to-aeo-ethan-smith — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. — Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com
292
|
1T 11M
$46B of hard truths from Ben Horowitz: Why founders fail and why you need to run toward fear (a16z co-founder)Ben Horowitz is the co-founder of Andreessen Horowitz, Silicon Valley’s largest and most influential venture capital firm, with over $46B in committed capital across multiple funds. He took Loudcloud public with just $2 million in revenue (dubbed “the IPO from hell”), sold it for $1.6 billion, and has backed companies from Facebook to Stripe to Airbnb to OpenAI to Databricks (now worth more than $100 billion). His management philosophy—forged through near-death experiences and refined through coaching hundreds of CEOs—contradicts most conventional startup wisdom. In our conversation, Ben shares: 1. Why “founder mode” is half right and half dangerously wrong 2. The story behind “Good Product Manager/Bad Product Manager” and why it went viral despite being written in anger 3. Where the biggest AI startup opportunities remain 4. Why you need to run toward fear, never away 5. The one trait that predicts that a founder will fail as CEO 6. Inside Paid in Full, Ben’s nonprofit awarding pensions to pioneering hip-hop artists — Brought to you by: DX—The developer intelligence platform designed by leading researchers: http://getdx.com/lenny Basecamp—The famously straightforward project management system from 37signals: https://www.basecamp.com/lenny Miro—A collaborative visual platform where your best work comes to life: https://miro.com/lenny — Transcript: https://www.lennysnewsletter.com/p/46b-of-hard-truths-from-ben-horowitz — My biggest takeaways (for paid newsletter subscribers): ⁠https://www.lennysnewsletter.com/i/172439345/my-biggest-takeaways-from-this-conversation — Where to find Ben Horowitz: • X: https://x.com/bhorowitz • LinkedIn: https://www.linkedin.com/in/behorowitz/ • Website: https://benhorowitz.com/ • Andreessen Horowitz’s website: https://a16z.com/ — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — In this episode, we cover: (00:00) Introduction to Ben Horowitz (04:09) Important leadership lessons from Shaka Senghor (10:15) Running toward fear and why hesitation kills companies (19:35) Who shouldn’t start a company (22:36) The Databricks story: thinking bigger (24:54) Managerial leverage and CEO psychology (28:06) When founders should be replaced as CEOs (31:20) Normalizing failure for CEOs (37:57) Counterintuitive lessons about building companies (42:31) “Good Product Manager/Bad Product Manager” (48:21) Product managers as leaders (51:16) Why a16z invested in Adam Neumann after WeWork (56:23) Is AI in a bubble? (01:02:43) The biggest opportunities in AI (01:12:51) Why U.S. leadership in AI matters (01:18:53) The Paid in Full Foundation for hip-hop pioneers (01:23:18) Lightning round: book recommendations, products, and life mottos — References: https://www.lennysnewsletter.com/p/46b-of-hard-truths-from-ben-horowitz — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com
291
|
1T 37M
How AI is reshaping the product role | Oji and Ezinne UdezueEzinne and Oji Udezue have over 50 years of combined product leadership experience at Microsoft, Twitter, Atlassian, WP Engine, Typeform, and Calendly. They’ve witnessed every major shift in product management, and, despite their seniority, they’re taking beginner AI courses and learning from engineers half their age, and Oji is coding more now than in the past decade—from Waterfall to Agile to AI. They are also the authors of Building Rocketships, a guide to building great products. In this conversation, the couple shares hard-won lessons they’ve learned from companies successfully adapting to AI, including their “shipyard” framework and their “sharp problem” methodology. What you’ll learn: 1. The “shipyard” framework: why the best AI teams embrace controlled chaos 2. Why Oji writes more code now than in the past 10 years—despite being a PM for more than 25 years 3. The three skills that matter most for PMs in 2025: curiosity, humility, and agency 4. How to identify “sharp problems” 5. AI at the core vs. AI at the edge: why companies that are building entirely new AI-centric codebases will beat those just “sprinkling AI” on existing products 6. The counterintuitive truth: engineers are moving so fast with AI that PMs are now the bottleneck 7. Their biggest product lesson from 50 combined years — Brought to you by: Mercury—The art of simplified finances Vanta—Automate compliance. Simplify security. Coda—The all-in-one collaborative workspace — Where to find Oji and Ezinne: • ProductMind on Substack: https://substack.com/@ojiudezue • ProductMind on LinkedIn: https://www.linkedin.com/company/productmindco • ProductMind on YouTube: https://www.youtube.com/@ProductMindX/videos • ProductMind on Spotify: https://open.spotify.com/show/07OVh5pdSv0szHPwWktzQQ • ProductMind website: https://www.productmind.co/ • Oji on LinkedIn: https://www.linkedin.com/in/ojiudezue/ • Ezinne on LinkedIn: https://www.linkedin.com/in/ezinne/ — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — In this episode, we cover: (00:00) Introduction to Oji and Ezinne (04:14) The evolving role of product managers (08:01) Challenges and opportunities in product management (10:34) Sharp problems (12:37) The shipyard model for product development (17:02) Hiring PMs in the AI era (24:55) The importance of staying humble (27:16) Hands-on learning and personal projects (39:10) Companies succeeding with AI adoption (46:25) Lessons from 50 years in product (49:22) Simplicity in design (51:24) The role of communication in strategy (55:17) Career intentions and personal growth (01:00:00) Ethics and responsibility in product management (01:03:09) Introducing Building Rocketships (01:06:42) Lightning round and final thoughts — Referenced: • How 80,000 companies build with AI: products as organisms, the death of org charts, and why agents will outnumber employees by 2026 | Asha Sharma (CVP of AI Platform at Microsoft): https://www.lennysnewsletter.com/p/how-80000-companies-build-with-ai-asha-sharma • Picking sharp problems, increasing virality, and unique product frameworks | Oji Udezue (Typeform, Twitter, Calendly, Atlassian): https://www.lennysnewsletter.com/p/picking-sharp-problems-increasing • Atlassian: https://www.atlassian.com/ • Joff Redfern on LinkedIn: https://www.linkedin.com/in/mejoff/ • Brownian motion: https://en.wikipedia.org/wiki/Brownian_motion • Calendly: https://calendly.com/ • Women in Product: https://womenpm.org/ • Brian Chesky’s secret mentor who died 9 times, started the Burning Man board, and built the world’s first midlife wisdom school | Chip Conley (founder of MEA): https://www.lennysnewsletter.com/p/chip-conley • Home Assistant: https://www.home-assistant.io/ • What people are vibe coding (and actually using): https://www.lennysnewsletter.com/p/what-people-are-vibe-coding-and-actually • How many layers should I wear today?: https://layers.today/ • Typeform: https://www.typeform.com/ • David Okuniev on X: https://x.com/okuiux • Clay: https://www.clay.com/ • Martin Eriksson on LinkedIn: https://www.linkedin.com/in/martineriksson/ • Geoffrey Moore on finding your beachhead, crossing the chasm, and dominating a market: https://www.lennysnewsletter.com/p/geoffrey-moore-on-finding-your-beachhead • Dave Mendlen on LinkedIn: https://www.linkedin.com/in/davemendlen/ • Deepfake: https://en.wikipedia.org/wiki/Deepfake • How to kickstart and scale a marketplace business: https://www.lennysnewsletter.com/p/how-to-kickstart-and-scale-a-marketplace • Forever on Netflix: https://www.netflix.com/title/81418639 • Paradise on Hulu: https://www.hulu.com/series/paradise-2b4b8988-50c9-4097-bf93-bc34a99a5b4f • Sinners: https://www.imdb.com/title/tt31193180/ • Claude: https://claude.ai/ • Nespresso Vertuo: https://www.nespresso.com/us/en/vertuo-coffee-machines • Gamma: https://gamma.app/ • Framer: https://www.framer.com/ • Lovable: https://lovable.dev/ • Building Lovable: $10M ARR in 60 days with 15 people | Anton Osika (CEO and co-founder): https://www.lennysnewsletter.com/p/building-lovable-anton-osika • Llama: https://www.llama.com/ — Recommended books: • Building Rocketships: Product Management for High-Growth Companies: https://www.amazon.com/Building-Rocketships-Management-High-Growth-Companies/dp/1962339068 • Coda version of Building Rocketships: https://www.productmind.co/brpro • Build: An Unorthodox Guide to Making Things Worth Making: https://www.amazon.com/Build-Unorthodox-Guide-Making-Things/dp/0063046067 • The Let Them Theory: A Life-Changing Tool That Millions of People Can’t Stop Talking About: https://www.amazon.com/Let-Them-Theory-Life-Changing-Millions/dp/1401971369/ Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. — Lenny may be an investor in the companies discussed. My biggest takeaways from this conversation: To hear more, visit www.lennysnewsletter.com
290
|
1T 18M

Lyt når som helst, hvor som helst

Nyd den ubegrænsede adgang til tusindvis af spændende e- og lydbøger - helt gratis

Lyt og læs så meget du har lyst til
Opdag et kæmpe bibliotek fyldt med fortællinger
Eksklusive titler + Mofibo Originals
Opsig når som helst

DK - Details page - Device banner - 894x1036

Cover for Inside Google's AI turnaround: The rise of AI Mode, strategy behind AI Overviews, and their vision for AI-powered search | Robby Stein (VP of Product, Google Search)

Læs mere

Investor Relations

Tilgængelighedserklæring

Udforsk

Mofibo Originals

Brugbare links

Indløs gavekort

Indløs kampagnekode

Sprog og region

Dansk
Danmark

Følg os

Privatlivspolitik