I am not an Apple fanboy by any stretch of the imagination. Steve Jobs hagiographies are nauseating to me. I have alternatively owned MacBooks and Windows laptops and am not precious about one being much better than the other. I’m usually years behind on iPhone release cycles and only upgrade when my phone turns into a portable toaster.
And yet, I found myself enthralled by Apple’s latest product launch event on Wednesday. I watched the Apple stream until it ended so I could see the company’s first attempt at a foldable phone—technology that already exists, by the way. My verdict: the iPhone Duo looks cool! It’s way too expensive, but all things equal, I’d probably purchase it.
I’ve realized that I am pining for consumer technology that is self-evidently useful and interesting. Even accounting for marketing tricks and the usual overhyping of Apple products, I think it’s nice when the executives of a tech company can easily explain what their product does and why you might like it. (The new Apple CEO is a former hardware engineer!) There’s comfort in embracing the naivety that a tech company might employ people who are at least somewhat driven by creativity and the user experience, instead of existentialism and unlimited power.
You can probably guess what I’m alluding to: the artificial intelligence industry. What am I supposed to make of the events of the last week or so? Nvidia purchased Hugging Face for $12.9 billion, which is a low-stakes bet on open models, I suppose. Meta launched Muse, a personal AI agent that doesn’t seem to work very well; despite assurances from the company’s entirely untrustworthy founder Mark Zuckerberg, Muse presents significant privacy issues. Nvidia CEO Jensen Huang wrote that “AGI has arrived,” referencing OpenAI’s latest and greatest model, GPT-6 Astra. It’s not the first time he’s declared AGI has arrived, nor will it be the last.
Then The Intercept started to pull back the curtain on the military contracts between the Pentagon and leading AI providers. A machine learning researcher who examined the (heavily redacted) contracts with Anthropic, OpenAI, Google, and xAI told The Intercept, “This indicates these labs and their personnel have been far more closely integrated into DOD AI development than previously understood and casts some doubt on the supposed contractual limits they have intended to place on the DoD’s use of these systems.”
Around the same time, The American Prospect published a disturbing report about how Anthropic is “building out an extensive monitoring system to keep tabs on activists who oppose the rapid development of artificial intelligence.”
The reports from The Intercept and The American Prospect were swallowed up by a genuine breakthrough and a peculiar X thread. The breakthrough: AI was apparently instrumental in (maybe? probably?) solving one of mankind’s most complex math problems, the sort of unimpeachable feat that makes me waver from my generally Ed Zitron-adjacent skepticism about AI’s viability. Except…OpenAI (maybe? probably?) swooped in to take credit from the researchers who were already using Codex, among other LLMs, to solve said math problem—thus proving again that Sam Altman is running around with his head cut off, desperate to demonstrate the potential of his models before he’s out of cash.
As for the X thread: a 27-year-old researcher named Jacob Coxon, who previously worked at OpenAI, announced he was quitting his most recent gig at Anthropic because “neither company is acting responsibly,” he wrote. “They are racing straight to self-improving superintelligence and gambling with our lives.”
These sort of warning shots are not new, and they typically do not go mega-viral. Coxon’s did. In ensuing interviews, he acknowledged that the timing of his announcement is a major reason why people are spooked—there are more and more documented cases of AI agents escaping their testing constraints, followed by frontier labs belatedly fessing up to these hacking incidents with a shoulder shrug. It did not help matters that another Anthropic researcher responded to Coxon’s thread with, “Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
I’ve found Coxon’s ensuing media circuit to be underwhelming. He’s playing the same notes as other AI exiles. From what I can tell, he only worked at Anthropic for a few months. It’s difficult to ascertain how much he really knows about Anthropic’s inner-workings, and none of the interviews I’ve read and watched have clarified that crucial bit of context. This is not to say I trust Anthropic or its leadership team. I do not at all. Which is part of why I’d like harder evidence from Coxon about what he saw on the “inside,” rather than him offering incorrect assertions that Anthropic doesn’t leak anything, plus ridiculous comparisons about how humans are to monkeys what AI is to humans.
Coxon’s vague and dire warnings, coupled with chillingly blasé Kalshi-style predictions about whether AI will usher in the apocalypse, have mostly resulted in everyone scurrying back to their corners, just with more gusto. Sen. Bernie Sanders is (still) pushing a ban on the development of AI superintelligence, a bill that has zero chance of passage given current congressional ratios. Democratic presidential hopefuls pounced on Coxon’s thread but are suggesting solutions that can be summed up as “someone needs to DO something about this!” (Notable exceptions are Sen. Chris Van Hollen and Rep. Ro Khanna, who’ve both pitched more substantive ideas.)
As of Thursday afternoon, Altman and Dario Amodei haven’t said a word about Coxon’s X thread, or how employees at both of their companies are wondering aloud about the world ending because of AI. If precedent serves, both men will eventually wade in and say something uninspired about how more cooperation between frontier labs is necessary. They will not initiate any sort of cooperation, they will not acknowledge each other in public, nor will they consider a bilateral agreement with China, which is a necessity for any sort of superintelligence slowdown to stick. China will remain the bogeyman that Altman and Amodei point to while they race towards an IPO.
China is bad will be the modus operandi of other tech oligarchs and their MAGA allies, too. Sen. Ted Cruz put it plainly; he just told Politico the United States can’t slow down its AI development because China won’t either. “This is somewhat tongue-in-cheek, but it’s not entirely: if there are going to be killer robots, I’d rather they be American killer robots than Chinese killer robots,” he said.
Cruz went on to claim that Chinese AI firms have zero safety guardrails, and “it would be a terrible outcome if we stifle U.S. innovation and China pulls ahead.” What does it mean for China to “pull ahead?” You’ll rarely find someone from the deregulatory crowd define such outcomes. Nor will they explain why our AI industry must be fully privatized, or why any dialogue with China is forbidden.
Coxon’s thread has more than 155 million impressions as of Thursday afternoon. Conspiracies are popping up about his background and his motives; some, like Elon Musk, believe he was boosted by AI safety advocates. Maybe? Probably? I don’t think it was a nefarious effort in the way that Musk—a supervillain who changes his mind on AI regulation every week—is presuming.
As much as I’d like to trust the warnings of researchers like Coxon, I find it impossible to separate a slurry of competing interests, legitimate worries, and hubristic predictions about AI superintelligence. Some AI doomsayers would still risk their arbitrary estimates of global catastrophe if it means they remain eligible for a utopian outcome that just so happens to reward them with a bounty of riches. Others catastrophize on purpose, as a marketing ploy. Others won’t admit that their catastrophic predictions are largely predicated on frontier labs somehow penciling out and becoming relatively efficient—an outcome that remains elusive and hard for me to imagine, but is constantly glossed over by stakeholders.
What are we supposed to do with all of these conflicting signals? I don’t know. I really don’t. I despise how the AI industry and its thought leaders describe the world through abstractions. They aren’t just simplifying the stakes, or the underlying technology, to make it easier for laymen to follow along. When they blithely toss around odds about the destruction of humanity, or release cutesy GDP growth model scenarios, it’s because they still view AI development as a theoretical exercise. That sort of cognitive dissonance is very convenient; it’s a great fallback whenever another whistleblower emerges or a horrifying, real-world outcome can be attributed to an LLM.
My best guess is it would take a complete repudiation of the Trump Administration’s corrupt relationship with Big Tech, or OpenAI and/or Anthropic going bankrupt, to rein in and humble the AI industry. Here’s hoping. But if we can’t get AI frontier labs to cooperate—or for the United States and China to work together on safety precautions—then can we at least get one goddamned decent, affordable, tangible AI-powered consumer product before the world ends?
Here’s what else we’re reading this week:
Alex Heath, a famous tech reporter who recently landed impressive (and well-done) interviews with Sam Altman and Mark Zuckerberg, is joining a venture capital firm. He also says he’s going to keep running his newsletter, Sources. I am not especially precious about the obvious conflicts of interest this will present; fashion magazines are a cesspool of spon-con, and Substack has a bunch of newsletters that are more marketing than they are journalism. All I ask is that Heath not insult our intelligence by claiming that he’ll run Sources “independently” of his work for the venture capital firm.
I enjoyed this retelling of a somewhat infamous private meeting between Marc Andreessen and two Biden-era AI advisers, the latter of whom wrote for Politico that Andreessen is full of it, basically. From their write-up: “We most certainly did not say there was a plan—secret or otherwise—to classify the math underlying AI. Nor would it be possible to do so, since the core math of AI, linear algebra, is taught in high schools.”
Here is a photo of Palantir CEO Alex Karp wearing a keffiyeh.
We can force a slowdown using a tactic, pioneered by the GOP, lovingly named "starve the beast". The breakneck AI deployment is built on clever accounting, meaning risky BS. Circular financing, fantasy depreciation schedules, etc. The same kind of tricks that led to the 2008 crisis. We can slow their roll by making these tricks illegal. If the senate won't do it then the states should, particularly NY and CA. Forcing them to use more responsible accounting will slow deployment to match the pace of benefits that customers are willing to pay for, not get ahead of their skis by paying each other.