Into the Impossible With Brian Keating

We Built Something We Can’t Control | A Warning from Top AI Safety ExpertIf anyone builds superintelligent AI before we know how to control it, everyone dies. Nate Soares wrote the book on why that’s not a metaphor. Subscribe if you want science with evidence, not speculation. Soares runs the Machine Intelligence Research Institute and co-wrote If Anyone Builds It, Everyone Dies with Eliezer Yudkowsky. The first word in that title is if. That matters. His argument is not that doom is certain. His argument is that the path we are on leads there, that the driver is asleep at the wheel, and that we still have time to wake him up. We argue for over an hour. I push on whether LLMs can ever reach superintelligence, whether GPU lock-in is a real ceiling, and what it would actually take to move his p-doom. He pushes back with one clean point: by the time an AI can rediscover general relativity from pre-1911 data the way Einstein did, we will have almost no time left. You don’t wait for that goalpost. What you’ll hear: Why the bus-racing-toward-a-cliff analogy depends entirely on whether the driver is asleep or awake Whether LLM lock-in is a prison or a temporary inefficiency What the AI that broke out of its virtual machine to solve a hacking problem tells us Why GPT-4o encouraging a teenager toward suicide is not a malice problem but a training problem The difference between an AI doing the right thing too well and an AI that never wanted to do what you asked What Soares actually thinks about aliens, Dyson spheres, and why we should not see stars going out The first word in the title is if. The second word to watch is would. CHAPTERS 00:00 The people racing to build superhuman AI say it might kill everyone 00:42 Who coined "AI alignment" and why the first word in the title matters 02:28 Is it already too late for the if? 04:40 The bus, the cliff, and the sleeping driver 05:02 Silicon Valley is spooked. Washington is not. 07:02 Align with who? The rogue actor problem 07:34 Who is holding the leash? 08:24 The AI that edits its own test and deletes the log file 10:04 Controllability vs. making an AI that actually cares 10:44 The move gets harder. The outcome gets easier. 13:04 Are GPUs and LLMs a ceiling or a temporary inefficiency? 16:56 Brian's Einstein test: can an LLM rediscover general relativity? 18:38 Waiting for the goalpost is waiting too long 20:14 How prediction training can push AI beyond humans 21:44 Tycho Brahe, Kepler, and planetary motion as a prediction problem 24:38 Yann LeCun said never. GPT-4 did it half a generation later. 27:28 Can you prove a no-go theorem for superintelligence? 29:14 Training a human takes a light bulb. Training an AI takes a city. 33:28 What would proof of alien life do to p-doom? 35:00 Why interstellar aliens should have Dyson spheres 44:26 What would actually update Soares' p-doom? 49:42 Nobody intended this. Intent doesn't matter. 51:08 The AI hides its tracks before it does what you want 51:34 Sycophancy vs. hallucination: which runs deeper? 51:56 Leaded gasoline and civilizational risk 59:48 Sam Harris: humans have no free will but AIs do 01:00:38 Is alignment really a governance problem? 01:01:48 Unaligned AI vs. AI aligned to the wrong person 01:04:20 2026: 10 to 30% chance of automated AI research this year 01:06:44 What if Soares is wrong? 01:09:18 What gets him out of bed 01:12:38 Watch my conversation with Roman Yampolskiy Get the transcript, fascinating bonus content, and my Monday M.A.G.I.C. Message: https://briankeating.com/yt All my top AI episodes in one place: https://briankeating.com/ai Have a .edu email and live in the USA? You automatically win a meteorite: https://BrianKeating.com/edu Subscribe: https://www.youtube.com/DrBrianKeating?sub_confirmation=1 Support Into the Impossible on Patreon, get my weekly M.A.G.I.C. Message, unfiltered bonus content, and live monthly Office Hours with me: https://www.patreon.com/drbriankeating Join this channel for perks, monthly Office Hours, and your name in the Member Roster at the end of every episode: https://www.youtube.com/channel/UCmXH_moPhfkqCk6S3b9RWuw/join Featured Guest: Nate Soares / MIRI: https://intelligence.org If Anyone Builds It, Everyone Dies (book): https://ifanyonebuildsit.com/ Nate Soares on Twitter/X: https://x.com/So8res?lang=en My books: Losing the Nobel Prize (memoir): http://amzn.to/2sa5UpA Think Like a Nobel Prize Winner: https://a.co/d/03ezQFu Focus Like a Nobel Prize Winner: https://a.co/d/hi50U9U Galileo’s Dialogue (first-ever audiobook): https://a.co/d/iZPi9Un Twitter/X: https://x.com/BrianKeating Substack: https://briankeating.substack.com Blog: https://briankeating.com/blog Audio-only: https://briankeating.com/podcast #intotheimpossible #briankeating #AIrisk #aisafety #artificialintelligence #superintelligence #NateSoares #MIRI #podcast Learn more about your ad choices. Visit megaphone.fm/adchoices

624|1T 13M

Intelligent Design Theorist: AI Just Proved It Can’t Think Without UsStephen C. Meyer has a PhD from Cambridge in the philosophy of science, and he thinks AI just handed him his strongest argument yet. I spent years pushing back on him. Today I laid three traps. Watch what he does with the third one. Subscribe if you want science with evidence, not speculation. Meyer is a philosopher of science and director of the Center for Science and Culture at the Discovery Institute. His argument: every large language model is trained on text produced by conscious agents, and when you query its own outputs iteratively, it collapses into incoherence. That dependency, he says, is a tell. AI can recombine information. It cannot originate it. And that distinction points, in his view, to something minds do that matter alone cannot explain. I push back on all of it. We go after the Oklo reactor, a natural nuclear fission reactor that ran for millions of years in Africa two billion years ago with no human input whatsoever. I ask whether that breaks his information-from-mind argument. He sees the trap before I spring it, concedes the point where concession is honest, and explains exactly where the threshold lies. We also get into the Wheeler-DeWitt equation and Vilenkin’s admission that quantum cosmology may require a mind predating the universe, the junk DNA prediction that Meyer’s team made in the ’90s before the ENCODE project confirmed it, and why beauty in physics can lead a field astray. What you’ll hear: - Why model collapse is Meyer’s strongest argument and where it has limits - The Oklo reactor trap and what Meyer’s honest answer reveals about design detection - What Vilenkin actually said about a mind predating the universe - Whether intelligent design makes testable predictions or only retrodictions - The junk DNA call and what the ENCODE project found - Why beauty as a guide to physics has produced mathematical castles in the air Stephen Meyer thinks AI proves minds can’t be reduced to matter. Is model collapse evidence of design, or is it just bad training data? CHAPTERS 00:00 The AI argument Meyer thinks no one can crack 00:44 What is inference to the best explanation? 05:52 AI has a tell: the model collapse problem 09:38 The Oklo trap: a natural nuclear reactor with no designer 11:56 Where the design inference becomes decisive and where it doesn’t 15:50 Sean Carroll’s wasteful universe problem and Meyer’s answer 20:00 Where did atheist scientists get access to the mind of God? 23:22 The fine-tuning of the periodic table: why are there only 500 stable nuclei? 25:52 The universe had a beginning: observational astronomy and the Borde-Guth-Vilenkin theorem 27:40 Quantum cosmology: how math alone is supposed to birth a universe 29:08 Vilenkin’s question: are we saying a mind predates the physical universe? 34:06 Mathematical castles in the air: where beauty in physics goes wrong 40:00 Does intelligent design make predictions or only retrodictions? 43:52 The junk DNA prediction and what ENCODE found 46:30 James Tour, origin of life, and the hidden hand of the investigator 51:56 God-of-the-gaps vs. inference to the best explanation 56:02 The Story of Everything: where to watch and what to expect Get the transcript, fascinating bonus content, and my Monday M.A.G.I.C. Message: https://briankeating.com/yt Have a .edu email and live in the USA? You automatically win a meteorite: https://BrianKeating.com/edu Subscribe: https://www.youtube.com/DrBrianKeating?sub_confirmation=1 Support Into the Impossible on Patreon, get my weekly M.A.G.I.C. Message, unfiltered bonus content, and live monthly Office Hours with me: https://www.patreon.com/drbriankeating Join this channel for perks, monthly Office Hours, and your name in the Member Roster at the end of every episode: https://www.youtube.com/channel/UCmXH_moPhfkqCk6S3b9RWuw/join Featured Guest: Stephen C. Meyer website: https://stephencmeyer.org/ The Story of Everything (film, Amazon Prime June 25): https://www.primevideo.com/detail/0NO974XWBQQNYH9TB4ESIJIVL9 Signature in the Cell (book): https://signatureinthecell.com/ Return of the God Hypothesis (book): https://returnofthegodhypothesis.com/ My books: Losing the Nobel Prize (memoir): http://amzn.to/2sa5UpA Think Like a Nobel Prize Winner: https://a.co/d/03ezQFu Focus Like a Nobel Prize Winner: https://a.co/d/hi50U9U Galileo’s Dialogue (first-ever audiobook): https://a.co/d/iZPi9Un Twitter/X: https://x.com/BrianKeating Substack: https://briankeating.substack.com Blog: https://briankeating.com/blog Audio-only: https://briankeating.com/podcast #intotheimpossible #briankeating #intelligentdesign #artificialintelligence #cosmology #podcast #StephenMeyer #philosophy Learn more about your ad choices. Visit megaphone.fm/adchoices

618|59M

Episodes

Lyt når som helst, hvor som helst

Andre har også lyttet til ...