Machine learning expert: “AI hype” is the true product, to provide cover for social agendas

An ex-big tech researcher worked deep in the "information retrieval" space. Now he wants to educate the public on the true capabilities and limitations of AI.

Bhaskar Mitra worked for 19 years in big tech, most recently as an information retrieval researcher focused on machine learning and AI.

In June of this year, Bhaskar’s employment was terminated as part of one of the mass layoffs. But prior to that, he had been speaking vocally about Palestine and also more broadly about the systemic problems with AI.

Hard Reset initially went into the conversation with questions about Gaza—but the talk veered into an analysis of AI, LLMs (large language models), and the centralization of information.

I found it interesting to hear straight from someone who had a senior role working on AI at a big tech company and who has been deep in this technology for a few years now. Here is our conversation below.

Ariella Steinhorn: Bhaskar, what did you do in big tech? And when your employment came to an end, were you surprised? Sad? Relieved?

Bhaskar Mitra: I started as a software development engineer, then moved to applied sciences, and eventually transitioned into research. Over the last three years, I worked within the company’s research team, focused on information retrieval and machine learning / AI. There were large layoffs happening across the company in June, and I was part of those layoffs. 

But it was a surprise for me, and it wasn’t particularly obvious what the reasons were. All I was told was that the decision was not performance-based.

I was thinking about leaving big tech for a while at that point. I’ve been unhappy with the social impact of big tech, especially as it relates to what is happening in Gaza. Sure, people initially could have said that these companies didn’t have a chance to react properly given the complexities. 

But then, people started getting fired for protesting. I couldn’t get behind that, it felt like a deliberate choice to defend big tech’s complicity in what’s happening in Gaza.

AS: What was making you uneasy about the ethics of big tech?

BM: I was personally struggling with some of their ethical decisions. The main reason I didn’t proactively leave, frankly, is because I struggled to think of any new tech employer where I could pursue ethical work.

Initially, I made the choice to not talk a lot about Palestine at work out of fear of reprisal. But it was always on my mind and I was talking about it openly outside of work. Part of my research focus was already on fairness and other societal concerns in the context of AI and information access technologies in general. At the same time when I was struggling morally with what’s happening in Gaza, my research interests were also expanding to include concerns of how these technologies concentrate wealth and power and impede social and climate justice. Outside of work, I was also starting to participate in social movement spaces. Increasingly, the work on responsible / ethical tech and social justice movement building started to feel like two sides of the same coin. Since leaving big tech, I have been talking about these important issues more vocally. The more I am talking about it, the more joyful I have been feeling as well as less despondent about the current state of the world.

AS: This makes sense to me, because you were liberated to speak and did not suppress your true feelings. Anyway, you were in big tech for 19 years. What do you think catalyzed your interest in social issues?

BM: My background is in computer science and not in social science or political science. I have always been attracted to the potential of technology to affect positive social transformations.

Brexit and Trump’s first election in 2016 definitely made me more invested in the political situation. At the same time, the rise of Hindutva fascism in India also moved me to act. And as a tech worker, it was the role of technology in enabling the rise of authoritarianism across the globe that concerned me the most.

But it’s a difficult balance. The narrative bolstering AI is so strong, you have to be careful not to be labeled as just a hater. 

AS: Tell me more about this, the hype. ChatGPT exists as a product, but I suppose what you’re saying is that the story of AI “taking over” most things that human brains can do is pretty far-fetched.

BM: There is a lot of really exciting progress that has been happening in the field of machine learning over the last decade. But a lot of what big tech would like all of us to believe AI will do is actually unsubstantiated and lacking in evidence. The reality is that we are positioning large language models (LLMs) in a way in several application contexts that are incorrect. For example, LLMs will not replace doctors, teachers, or scholars. To suggest so is to fundamentally misrepresent what people in those professions do.

LLMs are also being positioned as information access systems without sufficient considerations for their systemic impact on society and our information ecosystems. Historically, web search engines and large content producers, like Wikipedia and news sites, shared a symbiotic relationship wherein these websites helped make the search result pages much richer and in return the search engine sent traffic back to these websites that contributed to their increased readership, subscriptions, and monetization. 

Now, LLMs gobble up large quantities of content from these websites as part of their training data and later regurgitate the same information without adequate attribution back to the sources. In some systems, LLMs also get access to relevant documents from the web at runtime, as part of a process called retrieval-augmented generation (RAG), and they can cite these documents as references in their responses. 

Even when these systems cite their sources, the fact that these systems generate a summarized answer disincentivizes the user from clicking through these links. There is evidence to suggest that this is already happening at scale and is jeopardizing what is called the “grand bargain at the heart of the web.”

You might often hear about “hallucinations” from these models which refers to the phenomenon of incorrect claims in the LLM’s responses that are unsubstantiated by the documents based on which the model generates the answer. I think that is a very reductive view of what the actual issue is. By inserting an LLM between the actual online sources of information and the user we are shifting the responsibility of inspecting the information in the documents and assessing their relevance, trustworthiness, and surrounding context from the user to the LLM. 

This has huge implications for future information literacy. And while the “hallucination” framing makes it sound like a technical shortcoming that can be eventually overcome as the technology matures, the latter is pointing to a deeply human and societal challenge that our tech does not have an answer for.

Another serious issue at hand is the fact that these systems are centralizing the power of what information is accessible and from whose perspective it is presented. There has been a ton of work recently on how to align these models to societal values. This work has been critical to ensure these systems do not produce harmful responses. 

But who gets to decide which values are desirable and societally beneficial? Model alignment approaches centralize that power in the hands of the platform owners and the same technology that can remove harmful content can also be employed to espouse bigotry and selectively present information in ways to manipulate users’ commercial and political preferences over time. History is riddled with examples of authoritarians trying to control the information flow, and our information access platforms unfortunately are increasingly becoming susceptible to authoritarian capture. That’s why Elon Musk bought Twitter/X, for instance, and we all witnessed something very similar unfolding with their Grok chatbot in recent months. 

Interestingly, even in academic circles, the hype has been so crazy and so much of research funding and future job prospects have been tied to big tech, that it’s difficult to raise these issues and have nuanced conversations about our collective responsibilities to society. Any form of doubt or questioning is quieted. Everyone is already on the train, it’s full-speed ahead without any thoughtfulness or appropriate societal guardrails. 

AS: There is a community of AI ethicists who are thinking about all of this already. What would you add?

BM: As a computer science researcher, it has been fascinating to witness how the AI ethics landscape has evolved over the last several years. You may hear about Fairness, Accountability, and Transparency (FAccT), Responsible AI, and AI Safety. People often use these interchangeably to refer broadly to work on AI ethics and many researchers in the field may identify with multiple of these framings. Which is to say there are large overlaps between these framings and corresponding communities. But what is often implicit is that these different frames correspond to different politics and social values. From what I have seen, the FAccT community is deeply concerned about questions of social and environmental justice. 

In contrast, big tech framing of responsible AI focuses on fairness and representational harms but rarely talk about how some of these technologies concentrate power in the hands of the platform owners. It feels like in corporate responsible AI, the bad actor always has to be “out there” and the critical gaze should never be turned inwards to those building and deploying these technologies. 

AI safety, in my view, on the other extreme largely represents a securitization of the field that distances itself from most concerns of social justice.

On this spectrum, there are incredible tech scholars and activists who are doing critical work to raise awareness about the systemic risks of AI and to challenge the concentration of power in big tech. They give me hope. We have to work together to ensure that any kind of tech ethics work is grounded in ongoing struggles and movement building for social justice, and must challenge war, colonial extractive practices, racial capitalism, gender and sexual injustices, and other forms of oppression.

AS: What concerns you the most about the way in which the AI hype is being propelled? 

BM: It really upsets me to hear talk of LLMs replacing doctors and teachers. The tech is not designed for that, and those claims feel a tad bit nefarious. Globally we are wrestling with underfunding of public healthcare, education, and other social services. The claims that AI can stand in as doctors or teachers, is providing political cover for further proposals to defund these services that are critical for our communities. 

Instead of investing in training more doctors and teachers in our communities, big tech wants you to redirect more investments to… (you guessed it…) big tech. It’s a world where the rich want everyone to work to make them more capital instead of getting socially conscious education and serving their communities. It feels like big tech’s response to the real needs of our communities is simply “let them eat chatbots.”

Proponents of AI would also like to convince you that AI will make knowledge workers radically more productive. But what is left unsaid is who gets to keep any surplus from the productivity boost. It is never the workers, their jobs are being commodified and they will need to work that much harder to now be compensated the same amount. And it is also not the actual productivity boost that matters but what the AI lobby can convince us is possible, because that provides the necessary arguments for bosses to renegotiate and pay you that much less.

And that’s where the hype comes in. It is exactly for these reasons that the “AI hype” is not extraneous to the value proposition of AI, but rather is an integral part of the same package.

AS: You said earlier that it would be hard to find a new employer to work with. What’s next for you? 

BM: I don’t know yet—but that’s largely intentional. I want to take the time to fully detox myself from big tech and Silicon Valley ways of thinking about technology. I am challenging myself to remember why I came into tech in the first place, the joys of building and the belief in technology as a tool for positive social transformations, not to amass obscene amounts of personal wealth. I am also focusing on connecting with other progressive tech workers and learning about the many creative ways they are trying to challenge the status quo.

In the long-term, I want my research to be situated in movement building for justice, sustainability, and our collective wellbeing. In my field of information retrieval, I am interested in reimagining radically different ways in which we individually and collectively can experience information in the future, and how technology can support those experiences. I want us to think about how information access methods can be humanizing and emancipatory, and challenge all forms of human oppression—including colonialism, ableism, casteism, and transphobia.

To push towards systemic change, we need to work on multiple fronts. We need to tell a much more articulate story about the systemic risks of what we call AI today. Not just about how these systems discriminate or contribute to misinformation, but also how they mirror racial capitalism and coloniality, employ global labor exploitation and extractive practices, and reinforce the global north and south divide. 

We need to inspire our peers in the tech community to radically reimagine our possible futures that are emancipatory and sustainable. We need to identify and work on incredibly challenging sociotechnical questions that require us to reach beyond our disciplinary boundaries. And we also need to radically reimagine how tech institutions themselves could be organized differently from the status quo of big tech and silicon valley.

AS: There are so many engineers who are just in it for the math (and maybe the $250 million pay packages). Why do you think you felt interested in the implications of technology beyond math and science?

BM: I was born in Kolkata in India. The city and its people still bear the scars of British colonialism and continue to struggle under the neocolonial globalized economy. But it is also the City of Joy. It is a city where one grows up extremely aware of the fragility of their existence. But it is also the city that teaches you the importance of practicing hope, the ability of bold dreams to challenge power, and the strength we have in community. 

For example, most Bengalis I know would fondly remember watching Satyajit Ray’s film “Goopy Gyne Bagha Byne” as a child. In the film, the two protagonists (Goopy and Bagha) receive three boons (think of them as “superpowers”) from the king of ghosts: (i) The ability to get any kind of food or clothing, (ii) the power to instantly travel anywhere, and (iii) the power to produce flawless music that renders the audience motionless in awe. And to activate the first two boons, Goopy and Bagha need to clap each other’s hands. 

It is only as an adult that I understood the significance of this imagery. What the two struggling protagonists were really granted were: (i) Independence from a capitalist structure, (ii) the power to travel the world and be in community with all its peoples, and (iii) the power of art and expression in liberating us from all forms of oppression. And of course to achieve that they must work collectively, and not just as individuals. That to me is beautiful.

How can such a story not capture our imagination? Shouldn’t we expect the same from our technologies—to liberate us, to connect us, and to support the deepest expressions of our humanity? Shouldn’t we be developing the tools that we must use collectively to realize a better future for all of us, instead of actualizing technologies that reduce us to objects to be surveilled, modeled, and manipulated? I genuinely believe very different futures are possible, but we have to dream it and collectively work on it.

Thank you for chatting, Bhaskar! A quick round-up of a few stories I noticed this week…

• Narrative before reality, as if Bhaskar predicted this story in the WSJ from yesterday: How Palantir Won Over Washington—and Pushed Its Stock Up 600%

• The Gulf’s trillion-dollar AI dream has a water problem (if you read last week’s newsletter about data centers, this story is very related…)

• A humanities professor received a student’s paper submission. He suspects it’s AI, but he can’t for sure prove it. Read more in A Professor’s Dilemma.

• Could Dementia Patients Benefit from an A.I. Companion? The good and the bad of pairing an elderly person with an AI companion.

• Google’s AI hallucinated a new body part.

• Montana is now the place to explore longevity, via an expanded “Right to Try” law.

• SEO for brands wanting to get noticed by Google Search may be a thing of the past. Meet GEO.

• TechEquity has a new paper out on worker voice and workplace tech, which includes examples of models for how workers can be included in the design of technologies.

Have a great week everyone!