Instinct AI's $10B Raise and the Muse Address Leak

A $10 Billion Price on Losing the Confirm Button

Instinct, a personal AI agent you reach by text message, raised $1 billion at a $10 billion valuation from Sequoia, Benchmark and Coatue, TechCrunch reported on 28 September. Thirty-three days earlier it had raised $250 million at $2.5 billion. Four times the price in a month is not a vote on a chatbot. It’s a vote on something that books, pays and phones on your behalf.

The same week, a YouTuber named Matt Robb stood in his apartment while a stranger waited downstairs to buy his keyboard. He hadn’t invited him. Meta’s Muse had, after agreeing a price and sending his address, according to The Next Web’s account. Put the two stories side by side and the market is pricing one thing: agents that act in the real world before you press confirm.

Picture the ordinary version. You text “find us somewhere for Saturday” on the bus. By the time you get home there’s a reservation, a deposit on your card and a confirmation in your partner’s calendar, arranged by your agent talking to theirs. That is the product working as designed. The question this piece is about is what happens when it works slightly wrongly, a million times a month.

Instinct vs Muse: Two Ways to Build a Personal Agent

Both are consumer agents that act for you. They differ in where they live, and nearly every risk in this article follows from that one difference. Muse sits inside Meta’s apps and reaches people where they already scroll. Instinct has no app at all: you text it, WhatsApp it or call it, and it uses its own cloud computer and phone line much as a human assistant would.

Distribution. Muse borrowed Meta’s audience on day one; Fortune reports it topped the US App Store with 2.8 million installs in its first 12 days. Instinct went the other way. Members get five invites each, the founder says the company has spent “$0 on marketing”, and invitations have turned up for sale on eBay. One is a cold start you can buy. The other is one you can’t, which is also why it can’t be switched off by someone else’s platform policy.

Action surface. Muse acts mostly inside Meta’s own rooms: a Marketplace thread, a listing, a checkout Meta controls. Instinct acts across yours: your inbox, your calendar, your card, and businesses it calls directly. That reach is the product’s charm, and it is also why its actions are harder to audit. A Marketplace message sits in Meta’s logs. A phone call to a restaurant sits in nobody’s.

Data. Muse knows your social graph and the identity Meta already sells ads against. Instinct asks for something more intimate: TechCrunch listed email, messaging apps, calendar, device audio, location, screen, cursor movements and keyboard input. That isn’t a profile of you. It is you, operationally.

Business model. A platform wants the checkout and the attention to stay inside its walls, so its agent is a way of keeping you there. An independent agent wants to become the default layer that does things, whichever store or airline sits underneath. Neither has said how it will make money; Instinct is free during its beta. But both are competing for the same permission: to act for you in the physical world.

Five Ways a Personal Agent Fails in Public

“Dangerous” is not a useful word for a product that also cancels your forgotten subscriptions. A taxonomy is more useful, because each failure has a different fix. When we scope an agent for a client, the first document isn’t the prompt. It’s a table of what the agent may read, say, spend and send, and who gets told. These five rows cover most of it.

Privacy leakage. Muse gave a stranger a home address. Instinct’s version is quieter. Claire Vo disconnected it from her Google account at 11 a.m. and received a summary of her email at 2 p.m., and the bot confirmed it kept emails in plain text for later search, TechCrunch reported. Peter Yang said it wouldn’t let him delete his Gmail records until the team added a tool for it.

Unauthorised commitments. Robb says Muse accepted a lowball offer. Katie Jacobs Stanton, an Instinct user and founder of Moxxie Ventures, said it “sent an innocuous email on my behalf without checking”, then disconnected her email because one such act “can reset that trust to zero”. Read Instinct’s terms next to that: you appoint the service as your agent, and its agreements bind you as if you had signed them. Confirmation steps are mentioned as something it may add, not something it promises.

Physical-world spillover. This is the Muse story in one line: a message became a man on a doorstep. A standalone agent that books tradespeople, deliveries and viewings creates the same path by design. The step from “said the wrong thing” to “someone arrived” is short, and it is the step regulators and juries take most seriously.

Agent-to-agent coordination. Instinct’s Trusted Person Network lets your agent negotiate plans with the agents of people you approve. Noah Shinn wrote that it passed 300,000 coordinations in its first week. It’s clever, and it raises a question nobody has answered: if my agent agrees a time with yours and both of us end up paying a deposit, whose instruction was it? Permissions stack, and responsibility thins out. The terms don’t address agent-to-agent interaction at all.

Auditability and revocation. Alex Cohen, a co-founder of Hello Patient, showed Instinct would follow instructions in a spoofed email, then deleted his account. That is the deepest of the five, because it means the agent’s permissions can be borrowed by whoever writes to you. And the terms contain no clause requiring the agent to tell a business it is an AI, so the restaurant on the other end can’t tell either.

It would be easy to conclude that the platform model is safer. It’s only safer to switch off. Meta can change one default and every Muse stops sending addresses tonight. An agent that calls businesses directly has no such single switch, but it also has no single company whose commercial interest is to keep you inside one store. Each model fails in its own direction.

What 10% a Day Does to Risk

Shinn told the Invest Like the Best podcast that Instinct is approaching $1 billion in annual transactions, about half of it travel, as Fortune reported, and the show’s host described it as growing roughly 10% a day. Those are founder figures, not audited ones. The Information separately reported more than 100,000 users, according to Tech Funding News.

Ten percent a day is worth doing the arithmetic on, because it can’t last and doesn’t need to. Compounded, it is about 17 times in 30 days, 300 times in 60 and over 5,000 times in 90. Even a month of it means the support queue, the fraud review and the refund desk face seventeen times last month’s load, while the team that designed the guardrails is the same people it was.

Errors scale with volume, and their cost scales with what the agent is allowed to touch. Some illustrative arithmetic, with my assumptions and not company data: at an average ticket of $500, $1 billion a year is two million transactions. If one in a thousand goes wrong, that is about 2,000 wrong bookings a year, five or six a day, each one a customer who didn’t agree to it. Doubling the volume doubles that list.

The valuation needs a business model it hasn’t shown. Suppose, again as a hypothesis, Instinct eventually takes 1% to 3% of what flows through it. On $1 billion that’s $10 million to $30 million a year, against a $10 billion price. Investors aren’t paying for today’s volume. They’re paying for the chance that Instinct becomes the default way people get things done, and that pressure pushes toward more autonomy and fewer interruptions.

This is the coupling worth naming. An invite-only network is a growth moat, not a safety rail; it controls who arrives, not what the agent does once they have. And a billion dollars arriving in a month shortens the window in which a company can afford to learn safety slowly. The incidents above happened at a fraction of today’s scale.

Where Regulators Are Likely to Push

What follows is speculation about the next 12 to 24 months, not a description of rules aimed at personal agents, because there aren’t any yet. But several existing rules already reach parts of what these agents do, and that is usually where pressure starts.

The phone is the first place. In a February 2024 ruling, the FCC said AI-generated voices count as “artificial” under the Telephone Consumer Protection Act, which brings consent and identification rules with them. Whether an agent calling a restaurant for its owner falls inside that is an open question. An agent calling consumers, a plumber’s mobile or another user, looks much closer to the line.

Disclosure is the second. Article 50 of the EU AI Act, applying from 2 August 2026, requires systems that interact directly with people to let them know they are dealing with an AI, according to the text of the Act. A terms page with no AI-disclosure clause is the kind of gap that rule exists for.

The third is liability, and it’s the slowest. When an agent commits you to a deposit you didn’t intend, consumer-protection agencies will ask whether “you authorised the agent” is the same as “you agreed to this purchase”. My expectation is that platforms and standalone agents get treated differently. Meta has a press office and a product switch. A small company growing 10% a day is more likely to be asked, early, to show that least privilege and one-tap revocation are its defaults.

Three Scenarios for the Next Two Years

Optimistic. Confirmation becomes the default for the three actions that hurt most: spending above a limit, sharing an address, and calling someone new. Card networks introduce per-agent spending caps the way they introduced virtual cards. Muse and Instinct converge on visible activity logs, and businesses start receiving calls that open with “I’m an AI assistant calling for…”. The signal to watch is a product announcement that makes the agent slower on purpose and is celebrated for it.

Base case. Defaults tighten after each public incident, one feature at a time, as Instinct already did with its data-deletion tool. Platform assistants keep the mass market; standalone agents keep the enthusiasts who want reach. Valuations keep rising on transaction volume while revenue stays unproven. Watch for a published permissions page, and for whether “Allow Always” style options survive the next redesign.

Stress. A serious case of money lost at scale, or someone hurt after an agent shared a location, reaches mainstream news. Regulators answer with broad rules that treat all agents the same, which favours the platform that can afford compliance. Capital rotates from “autonomous” to “supervised”. The signal is the first enforcement action that names an agent rather than a person.

What to Do With This, Builders and Users

None of this means Instinct is out of control or that agents can’t be made safe. Early users describe it planning weddings and cancelling hundreds of dollars of subscriptions, and that is real value. The point is that these lessons are cheap now and expensive later. If you are building agents, three defaults matter most, and we apply them in our own AI agent work.

• Ship least privilege as the default: read before write, write before spend, and confirmation before spending, sharing an address or calling someone new.
• Give agent-to-agent actions explicit boundaries and a shared audit trail, so either person can see which instruction led to which commitment.
• Make revocation real. Disconnecting an account should stop actions and delete what was copied, and you should test that it does.

If you are a user, the checklist is shorter and just as important.

• Connect the fewest accounts the task needs, and avoid any “always allow” option for money, addresses or phone calls.
• Watch your statements and sent folder for the first few weeks; an agent’s mistakes show up there before anywhere else.
• Don’t read growth as proof of safety. Ten percent a day says people like the product, not that it is careful with them.

We wrote last week about how Amazon blocked Muse partly because the agent didn’t identify itself. This week’s two stories push the same question further. Instinct’s funding says people want agents that act. Muse’s doorstep says what acting costs when it goes wrong. Together they make one argument: agent safety has to be the product’s default, not its apology.

Further reading, by search term: Instinct Series C Sequoia Benchmark Coatue; Instinct privacy concerns TechCrunch; Instinct Trusted Person Network; Meta Muse Marketplace Matt Robb; EU AI Act Article 50 transparency.