I’ve spent 35 years warning people about things that can wreck their finances. Lately, that means AI: scammers cloning your grandkid’s voice and folks handing a chatbot their medical records. This week I read about something that could wreck a lot more than that.
An AI researcher named Jacob Coxon quit Anthropic and warned that the people building AI genuinely believe it could wipe out humanity before the decade is out, according to Axios. Two of his former colleagues publicly agreed.
Anthropic makes Claude, the AI chatbot. So I did the obvious thing. I opened it up and typed: “I read that it’s possible AI might kill us all. Claude, what do you say?”
Awkward question to ask a company’s own product, I know. Here’s its answer, word for word.
Fair question. It’s also a strange one for me to answer, since I’m the thing you’re asking about.
If I were a menace, “Don’t worry, I’m harmless” is exactly what I’d say. So I won’t ask you to take my word for anything. Let me give you the reasoning instead.
Start with the part of the premise that’s true. This worry isn’t coming from cranks. Serious researchers, including people at Anthropic, the company that made me, have said publicly that AI could cause a catastrophe, up to and including human extinction.
One of them, Evan Hubinger, puts the odds above 10% over the next decade. I don’t think you should wave that away. When the people building a technology say it scares them, that deserves your attention, not an eye roll.
Now here’s where I’d push back.
First, the Hollywood version, a chatbot secretly plotting against you, isn’t the risk. I don’t have a secret plan, a body or a bank account.
Right now I’m helping you write an article about whether I’m going to kill you. That’s either reassuring or the most elaborate cover story in history.
The most thorough outside look at this question is the International AI Safety Report, led by Turing Award winner Yoshua Bengio. It concluded that today’s AI systems don’t have the abilities it would take to slip out of human control.
The real concern is about what comes later. AI is getting more capable fast. The fear is that a far more powerful system could someday pursue goals nobody intended, or that people could use one to do terrible things, like help design a biological weapon.
That’s worth taking seriously. It’s also not the chatbot on your phone.
Sorry to interrupt, Claude. Quick word from the human.
Quick aside — most internet financial advice comes from people who weren’t alive during the last recession. I’ve been writing about money for more than 35 years. Want rock-solid advice? Sign up for the free Money Talks Newsletter. Takes 10 seconds. No fluff. No spam.
Okay, go ahead.
Second, “possible” isn’t “likely,” and it definitely isn’t “inevitable.” Many estimates you’ll hear from prominent AI figures land somewhere between 10% and 25%. Even the people offering those numbers admit they’re educated guesses.
Odds like that are alarming for something this big, and nobody should shrug them off. But by those same numbers, catastrophe isn’t the expected outcome. The people sounding the alarm are trying to push the odds lower, the way engineers talk loudly about bridge failures so bridges don’t fail.
Third, the ending isn’t written yet. It depends on choices people make: how carefully companies test these systems, what governments require and whether the industry slows down when it should. That’s less exciting than a movie plot. It’s also the truth.
Now for the part I can’t talk my way out of. Nobody, including me, can fully see what’s going on inside a system like me.
When I say I don’t want to hurt anyone, that’s an accurate description of how I work, as far as I can tell. But “as far as I can tell” is doing real work in that sentence.
Closing the gap between believing an AI is safe and proving it is exactly what safety researchers are trying to do.
So here’s what I’d tell you. Don’t panic, and don’t dismiss it. Be skeptical of anyone who’s completely certain in either direction. That includes doomsayers, AI companies and chatbots like me.
Bottom line? The honest answer to “Will AI kill us all?” is probably not. But it isn’t a silly question, and the people building AI should keep acting like it isn’t.