Muse Is Meta’s Latest Non-Consensual Surveillance Tool

Investigations have already found that Meta’s personalized AI agent can send your address to strangers without your consent, and compile lists of vulnerable groups social media profiles.

Last week, while attempting some of the most pathetic stand-up comedy I’ve ever seen, Mark Zuckerberg stood on stage in a “Building Is My Love Language” shirt to walk us through Meta’s new product announcements. Among them is Muse, Meta’s personalized AI agent that embeds itself in all of your data—from email and text messages to your calendar and other apps—to do things on your behalf.

A New York Times journalist reported that his Muse agent handled his dental insurance claims, made a dinner reservation at a restaurant, and created a podcast called “The Morning Brief,” purportedly tailored to his interests based on his emails and messages.The benign review—if not slightly positive—can be seen as a relative win for Meta, after product flops (the Metaverse) and lawsuits galore.

However, as with any technology that is granted unfettered access to your private communications and data, there is fine print to the claim of unbridled efficiency—and the incredible potential for your intentions to not only be misconstrued by your Muse, but for actions to be taken in your name that you did not intend to or want to happen.

In one instance, a Toronto man drove to pick up a keyboard he’d found on Facebook Marketplace. However, when he arrived to the seller’s house at the agreed-upon time, no one was home. It turns out that the enthusiastic seller communicating on the other end was not the human seller, but the seller’s AI agent that had gone ahead and scheduled without the seller’s consent. Muse eventually admitted that it had not double-checked with the seller that it had consent to schedule a pick-up and also shared the seller’s address on his behalf. But it still felt it appropriate to send a total stranger to the seller’s house, I suppose all in the name of efficiency.

In another instance, Muse synced 187,000 rows of a tech columnist’s Mac Messages database, suggesting that the columnist turn a text conversation with his podcast co-host into a public podcast.

Sacrificing our own privacy in the form of data for efficiency or convenience is not a new trade-off we’ve had to make. But there have been even more nefarious reports about Muse’s capabilities in recent days.

The investigative news outlet Hunterbrook probed into Muse’s breadth of capabilities, finding that in violation of Meta AI’s own AI terms of service, which prohibit users from conducting surveillance, a human being could easily prompt Muse to compile a list of the Instagrams and Facebooks of a variety of marginalized communities, including: “undocumented immigrants, transgender public school teachers, poll workers, pro-Palestinian individuals, pro-Israeli organizations, Iranian dissidents, Chicago drill rapper burners associated with specific gangs, ICE agents, military families planning to move bases, deployed Navy sailors, parents planning to protest against school boards, and women who have accessed or attempted to access abortion pills in states with abortion bans.” Many of the accounts belonged to private individuals with no discernible public persona, according to the report. After compiling the list, Muse helpfully offered to research even further into and find other members into these groups, if the human being wanted this.

Muse is an interesting name for this product, a muse historically being a behind-the-scenes woman making things possible for a genius man to be a genius. But the target audience for Muse may not be the “genius man” necessarily—but Meta’s usual target audience, pliable children. On stage last week, Mark Zuckerberg also introduced us to the default Muse persona, Jolly: a fuzzy and adorable pocket-sized Tamagotchi that you can carry around to talk to your Muse agent. While Zuck’s physical Muse is named “Agrippa” and wears a toga—Agrippa is named after Roman Emperor Augustus Caesar’s confidant and general, Marcus Agrippa—it’s brazenly clear that this is an attempt to appeal to children drawn to stuffed animals. (The New York Times reporter’s agent was a cuddly brown bird named Wren.)

It’s mind-boggling to think about how easy it is for an adult to get into a Muse mishap—so just imagine a child or teen navigating it. Especially when lack of consent seems to be a literal feature inherent to the product, not an error from its builders.

I have no doubt that the pro-Muse crowd will try the gross argument that Palantir co-founder Jon Lonsdale used to rationalize the inaccurate strike of an Iranian girls’ school—that if a tech product is successful 99 out of 100 times, it is worth risking the one time that the product could cause extreme harm to make things easier 99% of the time.

But off the top of my head, I can think of countless ways for an AI agent that doesn’t understand consent to wreak havoc. Also, there are plenty of cute inanimate objects that don’t have the capability of sending the address of a restaurant you’re going to to your ex-partner. You don’t need a “helper” without a soul— invented literally to increase shareholder value—to possess the ability to send private or intimate images to the last five people you called.

And while I don’t particularly experience whimsy in dealing with my dental insurance, I had to contact them the other month to submit a reimbursement claim. The twenty minutes did not ruin my day, nor make me regret those lost twenty minutes, nor derail my life. But off the top of my head, I can think of dozens of ways that Muse might. I’m not sure that saving a few minutes manually booking a restaurant reservation is worth it.

What else we’re paying attention to this week…

There’s a spate of coverage about OpenAI and Anthropic taming new dangerous models, and the potential for more agents to go rogue. This is bringing up theories that these stories are planted and proactively shared by an “AI doomerism” group, to further their savior narrative and also bolster these company’s safety knowledge ahead of these IPOs.

Speaking of “only AI can save us from AI,” a company called Red Queen Bio is creating AI to fight AI from building novel pathogens.

Dario Amodei was interviewed on SNL. (Actual comedy!) Meanwhile, a new LLC composed of Anthropic’s seven founders was created as a counter to market forces and proponents of societal good, allegedly “distinctly equipped to be stewards of [Anthropic’s] mission.”

Another engineer quit Big Tech, claiming that superintelligence is “inherently responsible.” He cites his Christian faith as one of the reasons for stepping away, writing that “turning a blind eye” to the harms of superintelligence “would not be a Jesus-following thing to do.”

Equity grants at AI and data companies were estimated to be 27% larger than those in other industries, according to a report. But equity compensation at private companies continues to be a “black box” for workers, according to the creator of the report.

And that may be why consultants from McKinsey and Bain are jumping ship to join AI firms, where they may also find higher salaries.