It’s genuinely hard to keep up with AI news these days. One moment a former Anthropic engineer is warning us of humanity’s destruction. The next—in a situation never before heard of in the whistleblowing space—the engineer’s former employer parrots this prophecy, confirming that yes, there is a 10% chance of humanity being wiped out by swarms of rogue agents!
Then, Elon Musk chimes in to the original whistleblower’s post saying that his posts sound like a set-up. Journalists rightfully cast doubt on how much this whistleblower actually knows, given he was only at Anthropic for a few months. We then remember the Hugging Face incident, and we wonder why regulators would allow any company to IPO if the consensus is that its product would precipitate an apocalypse.
A few days later, Anthropic CEO Dario Amodei publishes a 3,800-word manifesto calling for regulation and restraint in the AI space—something a private company only does if it has some ulterior motive. This manifesto garners public support from Sam Altman and even Elon Musk, and we start to catch on to the idea that the three big AI companies are colluding to form a “cartel” of sorts. Amodei, Altman, and Musk may be repeating nebulous concerns about rogue agents and the apocalypse to possibly ice out competition from cheaper open-source models.
We realize that as Anthropic and OpenAI race towards their IPOs to mint new millionaires and billionaires, their need for cash is nakedly apparent. In Anthropic’s own words, they are profitable with the exception of the cost of training their models. In other words, what is literally needed for Claude to exist.
Now, some venture capitalists like Marc Andreessen and David Sacks are upset, because they believe we actually need more competition from startups—as well as to speed up innovation and “win against China.”
If all of this weren’t enough, Trump finally chimed in yesterday—predictably upset that he was not the center of the universe—calling AI’s destruction of humanity a “hoax,” and declaring that a “high IQ” person such as himself can handle any potential threat.
So what do we take away from all of this? Is AI very dangerous? Are OpenAI and Anthropic trying to form a duopoly, using their safety language to sidestep antitrust scrutiny? If we slow down, do we risk losing to China, whatever that means? What does “AI taking out humanity” actually look like in practice?
All of this—the many stakeholders, the conflicting motivations, the swirling narratives—are enough to make anyone’s head spin, myself included. At the same time, the narrative may actually be more straightforward than we think.
The most compelling argument for AI “destroying humanity” that I’ve seen actually comes from a documentary called Ghost In The Machine. The documentary, now out on PBS, argues that AI technology is actually a cover for precipitating a more colonialist agenda—one where the military fuses with technological advancements to enrich a small subset of people and extract from the rest. The people on top anoint themselves as “geniuses,” even men “giving birth.” And using some of the past language of eugenics, they dominate and exploit in ways that remind me more of the past than the future.
In this context, Trump’s usage of the term “IQ” is interesting, because IQ tests were pioneered by the founders of statistics and machine learning—a sort of pattern recognition that LLMs use today to respond to our prompts.
The pioneers of machine learning, while lauded as rational and logical, are also people who had very apparent ties to eugenics and sterilization movements. Historically, the language around “IQ” and mathematical intelligence have been subverted to justify the enslavement and colonization of people who were ultimately just defenseless against people who had weapons.
With this as context, then the many distracting narratives about China and IPOs and Hugging Face may actually be a feature to the AI industry, not a bug. The focus on AI deflects from the reality that this is more about ideology than technology—the ideology being that a few people at the top truly believe that the rest of us should not have agency. The focus on AI also obscures the fact that human beings, at the end of the day, are still in control.
As Cory Doctorow writes, “The Hugging Face hack isn’t a mysterious, supernatural occurrence. It’s a Python loop and a chatbot. The people responsible didn’t accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.”
Now, I’m not saying that a swarm of AI agents taking down a 911 system is not a very bad and scary occurrence. It is. But there are ways to regulate and control products from being defective and causing harm without claiming they’ll end humanity. When people whistleblow about airplanes, for example, we don’t say that swarms of planes are going to lead to the destruction of our species.
Also, the hypothetical of AI models killing people…well, that actually isn’t a hypothetical. This has already happened via very brazen AI-military partnerships. As Dr. Heidy Khlaaf of the AI Institute writes on X, “A reminder that Anthropic has a partnership with Palantir, and their unreliable models are actively being used domestically and internationally to in fact, ‘kill a bunch of people.’”
Because all of this talk about the sci-fi future is a purposeful method to distract us from the present, whatever it is that we may be experiencing. If our present is harrowing, then sci-fi predictions deflect away from war, rising costs of healthcare and childcare, wildfires, boiling oceans, and people being poisoned by data centers. If our present is beautiful and joyful, sci-fi messaging injects a dose of fear to remind us we need these guys “in control.”
In an essay about our obsession with predictions—and how prediction-speak is more about control and power than profound clairvoyance or mathematical brilliance—Oxford philosopher Clarissa Velez writes that “The future exists only in our minds. To be more interested in it than in the present is to be more attached to one’s hopes and fears than to the world.”
Speaking of the world and the many beautiful things we can experience here, to place such an emphasis on “superintelligence” as the only form of intelligence also distracts us from our somatic senses, which our (very wise) nervous systems tap into through lived experiences.
This intelligence—emotional and relational—is something that cannot be understood by an AI, because AI is disembodied. As former big tech data scientist Abi Awomosu put it, “a model can generate fluent language about grief, soil, childbirth, danger, intimacy, rhythm, or belonging without possessing the embodied history through which a human nervous system learns these things. It models the descriptions of experience. It has not lived the experience.”
It’s true: we are intuitive creatures, more than just inputs and outputs. It’s both scary and wonderful. What real “AI danger” may come down to is “superintelligent” people tricking us into believing we’re not smart or rational enough to understand them. The true danger of AI, perhaps, is the deceptive messaging that leads us to believe we do not have agency or freedom in its system.
Brilliant post. Thank you!
Great post! There’s a reason the zombies stop being scary after Season 2 of The Walking Dead. The humans are more scary.