Hard Reset hosted an event this past weekend in San Francisco — a tour of the historic North Beach neighborhood’s architecture — where I chatted with an attendee about the major AI CEOs, and speculated what’s driving them ideologically. He pulled up an academic paper on his phone, and introduced me to the rabbit hole I’ve been going down since. TESCREAL is an acronym coined by the paper’s authors, philosopher Émile P. Torres and computer scientist Timnit Gebru.
You may have heard of Gebru from her coauthorship of an AI ethics paper examining and critiquing large language models, their social and ethical implications, and their proclivity for reproducing encoded racial and sexual bias. She was abruptly fired from Google soon after its publication.
TESCREAL stands for Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism, and Longtermism. It’s a term to describe the ideology and worldview of the tech elite.
The best explainer, for those of us without academic journal minds, is this article by Torres — in simple terms, it’s an acronym of the ideologies Torres and Gebru argue have become influential among the executives, funders, and main architects of AI, and the worldview pushing the tech industry towards furthering AI and achieving AGI at any human cost.
If you’ve wondered what belief system could be driving tech billionaires towards the bleak, anti-human, AGI-controlled future we’re getting glimpses of, this is its definition. Gebru and Torres say it’s “eugenics on steroids,” modernizing the (extremely unethical, to say the least) selective “perfection” of genetics to fuse technology and biology and transform humanity into a post-human species.
Musk’s Neuralink, for example.
It’s not just tech billionaires, however — improbably, it’s an ideology shared by AGI accelerationists and doomers. They just disagree whether the picture of the future it illustrates is beneficial or harrowing.
“While it may appear that ‘doomers’ and ‘accelerationists’ have little in common,” Torres writes, “The backdrop to the entire debate is the TESCREAL worldview.”
I’ll share Matt Mikalatos’s summary:
“The term is designed to describe something simple: a movement of billionaires and technologists who push personal scientific projects without regard for the human costs, claiming that there is an “existential” need to advance the technology despite the “short term” fatal risks to human beings.”
If we’re on the verge of AGI, and an unprecedented level of technology and intelligence, the small group of industry leaders on course to define humanity’s future will decide what all of us will sacrifice to achieve it.
This week — after a long month of existential warnings from architects of Frontier AI models that culminated in a suspicious show of acquiescence and OpenAI saying they’ve paused training models that hacked American and Australian government websites — President Trump and most major AI CEOs emerged from a White House meeting with a set of “voluntary safety standards” and a promise from Trump that the industry will police itself. When a reporter asked him if or how the deal was binding, Trump responded that he thinks it’s “morally binding.” There was talk of appointing an “AI czar,” which means nothing, and praised data centers as the new and improved industrial revolution. NVIDIA’s Jensen Huang proposed renaming them “Super Intelligence Factories,” on the heels of Trump’s Executive Order renaming AI.
This all looks like the mitigation of a PR crisis. It’s the likely reason Anthropic and OpenAI endorsed two bills in California (branded as AI safety legislation, but only regulating third-party groups monitoring AI safety risk) that Governor Newsom signed into law this month. Bipartisan public opinion is not in Frontier AI’s favor right now. Politico just polled a wide plurality of individuals, American and beyond, and the majority want AI’s development paused. Midterms are around the corner. Candidates are running on platforms that heavily feature regulation of AI and data centers. There are stakes to public opinion, right now.
The AI industry is moving at a pace the federal government is not capable of matching. Even Jacob Coxon has said Congress should let labs self-regulate, and that the people running them are “completely genuine” in their public mea culpas and promises to pace themselves.
The biggest AI companies in the world, in rare fashion, are publicly owning up to the fact that they are losing control of their models, and that a catastrophic end result is not out of the question. They are also tripping over themselves to add that nobody else should intervene.
I fell into TESCREAL research, the White House summit in mind, just in time for a new Wired interview with Gebru to be published yesterday. She said that focusing AI safety concerns on an amorphous existential threat is a distraction, and a profitable message for executives.
“The same billionaires who founded and funded Anthropic, the ones who stand to benefit from its IPO more than anybody else, also founded and funded the institutions warning about the, quotation marks, ‘existential risk’ of AI,” Gebru told Wired yesterday. “So people should ask why. If I stand to get lots of money from a particular thing, why do I seem to be the person saying this particular company might build something that kills us all? It sounds counterintuitive.”
It does. It’s why I immediately distrusted the neatly self-contained show of unity and restraint from the leaders of a historically competitive and voracious industry. Public relations ahead of the midterms could very well be motivation. Gebru clarified the stakes executives lay out in private, and how intentionally shaped public rhetoric is, even when it’s negative at face value.
“If I’m talking about potentially eradicating all of humanity, things like pollution, data centers, they sound kinda small potatoes, right? You’re talking about copyright protection for artists, actual lawsuits that are going on right now? The government should not waste its time thinking about this stuff.”
Gebru also discusses Anthropic co-founder Jack Clark publicly mocking her groundbreaking 2021 ‘stochastic parrots’ paper as a “cognitive virus” that “blinded many gifted people to the nature of AI progress,” saying the research was quickly outdated and cost critical time when researchers like herself could have thought a little harder about a response to AI’s development.
“I was not surprised, let me just tell you,” she said (this interview is in print, and I wish I could hear her tone here). “And these chatbots still have large language models as a basis. It’s so ridiculous that Jack Clark is talking about overestimating systems, because what I’m seeing is the examples that I just told you. It’s over-trusting these systems, not having checks and balances, and ending up misdiagnosing someone’s breast cancer to the wrong side.…To say that the research is outdated is ludicrous because right now, as we are hyping up superintelligence and all that, there are news stories that are going unnoticed, which are about the complete opposite scenario that is actually happening in the real world.”
‘Stochastic parrots’ is Gebru’s metaphor referring to how large language models are designed to function; repeating back succinct amalgamations of massive amounts of data without conscious understanding of the meaning or implication of its language, like a parrot repeating syllables. Some executives like Clark, in the five years since Gebru’s paper coining the term, ridicule it as outdated for the capabilities of today’s LLMs. These days, Gebru seems beside herself with the invalidity of that criticism, when evidence of imperfect, incorrect, and biased results from the newest LLMs are in plain sight.
It doesn’t help that most of the criticism of her and stochastic parrots comes from the very executive accelerationists TESCREAL describes, and is worded with palpable condescension. Alec Stapp, co-founder of the innovation policy think tank IFP, tweeted a brief quote yesterday from Gebru’s Wired interview where she uses Google AI overviews as one visible example of LLMs continuing to generate false information today, saying, “We need to start a GoFundMe to pay for these people to have access to Opus 5.5 and Astra.”
I find that rebuttal insincere and the wording patronizing, suggesting a globally respected computer scientist and AI ethicist just doesn’t have the time or money to try out the ‘good’ large language models, but my response won’t be better than Gebru’s:
The industry has spent years convincing us AI’s integration into the future is inevitable. As the path to that future becomes politically volatile, its architects are tripping their own alarm wires, and asking us to trust them. It’s not a matter of trust. It’s a matter of which consequences are tangible (labor, surveillance, data centers), and which risks deserve our attention.
Gebru urges the public to think of AI’s pace of development not as inherently dangerous because it may gain superhuman strength and sentience on its own, but to refocus our mistrust on the people defining that danger, promising to self-mitigate it, and the stakes and motives driving them to continue.
TESCREAL: Currently trendy forms of sociopathic insanity.
Because while recognizing that these notions - I hesitate to dignify them with the term "ideologies" - are indeed sociopathic, we should also recognize that they're deranged. These notions aren't going to help even their adherents thrive, let alone the rest of us.
And yes, LLMs are still stochastic parrots, which continually "hallucinate", because all they have is correlations among tokens in their training data. Fundamentally, they're models of documents, not observations or experiences, except in the indirect, murky, and unreliable sense that the documents are such models. This is obvious, and it's a sign of stupidity that some people in the industry still don't get it.