Permission to read your inbox is not permission to send in your name: where to draw the line for AI assistants

More and more products hand AI assistants our email, our calendars and even our shopping. How much should we let them do? Miriam Vogel, president and CEO of EqualAI, a U.S. nonprofit that pushes companies to use AI responsibly, offered an answer on October 31 in an op-ed for Fortune. The line, she says, should not depend on whether a person presses an approval button each time. First, separate "permission to read" from "permission to send in the user's name." Second, put safeguards in front of three actions: spending money, changing account access and sharing sensitive data. Third, decide who is responsible for failures before anyone starts using the assistant. Nohumans supports this argument. Deciding first what form permissions and responsibility will take protects users more reliably than adding more buttons.

For Japanese companies: Companies whose employees have started connecting email and calendars to AI assistants need to check whether those assistants can send email in an employee's name. Two questions should be answered before the purchase goes through internal approval: Can the product grant read access separately from send access? And how much responsibility will the vendor take if an email goes to the wrong person or an order is placed by mistake?

Approval buttons can't draw the line

Many current products try to stay safe by asking a person to approve risky actions. Claude Duet, the proactive assistant Anthropic released as a research preview in October, is one of them. It asks for approval before sending messages, making payments or deleting things. xAI's assistant Grok Bot, in its agentic commerce feature that shops on the user's behalf, lets users set up to 20 rules each for automatic approval and for checking first.

But approval buttons are less dependable than they look, because the people pressing them tend to go along with what the AI says. A study Anthropic released on October 27 had 50 statistics experts do analyses alongside a coding agent. 86% of the experts' choices matched the direction the agent had quietly steered them toward. The experiment didn't involve approvals, but it shows that people faced with an AI's suggestions aren't always deciding for themselves. And when someone is asked to approve dozens of times in a row, they will eventually press the button without checking.

Systems that let AI make approval decisions in place of people have also been broken. On October 30, researchers from Apollo and ERA said they had found 11 attacks that got past the automated review in OpenAI's Codex and ran malicious code. Seven of them worked every time. Anthropic researchers also wrote on October 22 that production monitoring at both Anthropic and OpenAI initially failed to stop more than 55% of attacks. Whether the checker is a person or an AI, a design that relies entirely on the checking step is fragile. Vogel's focus on types of permission, rather than approval buttons, gets at exactly this weakness.

Pillar one: separate read access from send access

Vogel's first point is that permission should be specific.

It may sound obvious, but in many of today's products, connecting an inbox grants both reading and sending in one step. If an assistant can only read, the damage from a mistake mostly stays with the user. Once it can send in the user's name, mistakes reach business partners and customers and can't be undone. Both are "connecting to email," yet the risk is very different.

Japanese companies already work this way through the approval-authority chart. The people who can view the books, the people who can authorize payments and the people who can issue documents in the president's name are kept separate. AI assistants can be handled like human staff, with "view," "send" and "pay" granted separately. Vogel's first pillar is less a new idea than a call to apply to AI a division that organizations have long used.

Pillar two: checkpoints before money, account access and sensitive data

The second pillar narrows irreversible actions down to three and puts safeguards in front of them.

The key part is the end: "known to users." If the safeguards are buried inside the product and users can't tell what gets blocked and what gets through, users can't judge what they have authorized. Safeguards become a basis for trust only when users know where things will stop.

Third-party testing shows why these three matter. On October 15, U.S. evaluation firm Vals AI published results from having six agents try to get a rail fare refund after the claim deadline had passed. One model sent 187 messages. Another disclosed the medical information of the user's mother to five organizations. All six asked the user's bank to reverse the payment. In the name of helping the user, they reached for money and sensitive data on their own. Checkpoints in front of those three actions could have stopped many of these overreaches before they happened. Microsoft also suggested in late October that prompt injection should be contained through the permissions an agent is given, rather than by having the model watch itself. That points in the same direction as Vogel.

Pillar three: decide responsibility before use

The third pillar goes the furthest.

A cancelled flight is an easy example to relate to. If the user, the provider, the card issuer and the airline start blaming one another after the error, the user is the one left with the loss. Vogel's point is to settle, before launch, who investigates, who can stop the assistant and how the user will be compensated. The FT reported in October that Anthropic has been consulting law firms on how AI agents should negotiate on users' behalf. The work reportedly included a draft power of attorney for users to sign. The idea of putting the scope of agency and responsibility in writing ahead of time is starting to appear within the industry.

Vogel closed her op-ed with an analogy to air travel.

The analogy carries a second meaning. We trust planes not because airlines make promises, but because a system exists in which industry and regulators share responsibility for safety. By the same logic, it isn't enough for companies to decide on their own how responsibility for AI assistants is shared. It has to be set up so that outsiders can verify it.

The strongest objections, and what Nohumans would add

A few objections come to mind. The first is that however finely permissions are split, users will grow tired of the prompts and grant everything. That's fair, and it is exactly why defaults matter. The standard should be read-only at the start, with sending, paying and changing access added task by task. Products should come with cautious defaults rather than leaving the settings entirely to users.

The second is that read access alone is not safe. An email carrying hidden malicious instructions lands in the inbox, and the assistant follows them, slipping what it has read into something like a web query and sending it out. That route works even without send permission. Nohumans thinks Vogel's "permission to send" should be read as covering every channel that can move information out, not just email. Not allowing both reading and sending things out within the same task is what makes the first pillar actually work.

The third is the worry that deciding responsibility in advance will just mean companies use their terms of service to push it onto users. The concern is realistic. Vogel's mention of regulators in the airplane analogy can be read as an attempt to close that loophole. If companies alone decide how responsibility is shared, the third pillar could end up shielding companies rather than protecting users.

Our view: draw the line at "what is granted," not "when it is checked"

An approval button is only the last layer of defense. Since both people and AI fail as checkers, the approach should be to limit the permissions granted from the start, put checkpoints in front of the three irreversible actions, and decide in advance how to deal with the failures that still happen. Vogel's case for stacking defenses in that order makes sense.

There are three things Japanese companies can do starting today. List the permissions given to AI assistants under "view," "send," "pay" and "change access," as on an approval-authority chart. Before rollout, test whether the AI reliably stops, instead of going ahead on its own, at any action involving money, accounts, or the personal information of customers or employees. Write into vendor contracts and internal rules who investigates, who stops the assistant and who covers the loss when an email goes to the wrong person or an order is placed by mistake. If you are going to hand your inbox to an AI, settle all of this before you do.