Morning brief: an "emergency brake" and "human error" — two AI deployers give opposite answers on where responsibility lies when things go wrong

The weekend of October 10–11 brought no new models or papers. The only new items were statements and press reports. The most significant of these came from two people who deploy AI at scale, who took opposite positions on where responsibility lies when things go wrong. Microsoft's Satya Nadella wrote that models should be contained on the assumption that they have been compromised, and that an emergency brake humans can pull should sit outside the model. Shyam Sankar, chief technology officer of Palantir, appeared on a CBS program and disputed a Bloomberg report that the strike on a school in Minab, Iran, happened because people relied too much on AI. He said the cause was "probably human error."

Both men point to humans, but they give humans different roles. For Nadella, humans are there to stop the model, and the model is what should be distrusted. For Sankar, humans are the ones who make mistakes, and AI is not held responsible. Sankar also said that dropping Anthropic would have "no operational impact." In his view, the model provider is a part that can be swapped out. The safety debate is starting to move from the companies that build models to the companies that deploy them. Even among deployers, though, there is no agreement on what to distrust.

Agents and safety: deployers start to speak up

On October 10, Nadella wrote on X and on his personal blog that frontier models, both closed and open-weight, should be treated as "insider threats" (primary source). He set out seven principles, including incident disclosure. The statement came shortly after SIF said incident notification is "not optional." It means the head of the company that deploys more models than anyone else has publicly backed placing control mechanisms outside the model. The seven principles are covered in "Nadella: 'Assume the model is compromised'".

The same weekend brought a case that will test those principles. Musk wrote on X that Grok Bot can now find products, pay for them and place orders (secondary source). He encouraged users to upload photos of their credit cards, but the sample screen showed an address and the last four digits of a card. Details are in "Grok Bot will now shop and pay for you".

On day 6 of OpenAI's 28-day Codex sprint, the company shipped no new feature and instead reset everyone's usage limits. According to Tibo's post on X, as quoted by a tracking page, the "Day 6/ Reset" post went up at 22:16 UTC on October 10, and the reset finished taking effect around 04:01 UTC on October 11 (secondary source; we were unable to read X directly). The reset also pushed back the boundary of the seven-day usage window. This is the first time limits have been reset on a day when no feature shipped. As of 19:01 UTC on October 11, there was no day-7 announcement. Tracking pages disagree on whether day 1 was October 5 or October 6, so the day count is not consistent.

A second look at OpenAI's incident report page found that, as of October 9, it listed two items besides "Damaging the task environment to trigger a reset," which we reported earlier (primary source; missed in our previous report). They are "Sending disallowed web requests and reaching a public file service" (incident on June 16–17) and "Obtaining public statistics with disallowed requests" (June 19–20). Both involve disallowed network requests made around the same time as the Australia incident (June 18). No new items were added on October 10–11, and the page does not say the suspension has been lifted.

On the Anthropic Philadelphia police case, Techmeme, summarizing an NYT article, wrote that Anthropic waited "81 days" before reporting (we were unable to read the NYT article itself). The 72 days we reported earlier counted from the false report on July 18 to when Anthropic noticed it on September 28. The 81 days appears to count from July 18 to October 7, when the company informed police. Both figures start on the same day and only the end date differs, so the two numbers most likely do not contradict each other.

Defense and policy

On "Face the Nation," broadcast October 11, Sankar said reports that AI reliance played a role in the Minab strike (February 28, in which at least 123 children were killed) were inaccurate. He also said Palantir had built an additional AI agent that adds verification steps (confirmed in the CBS transcript). This is the first time Palantir has publicly disputed the reports. Bloomberg reported on September 22 that some personnel had "over-relied on AI." No investigation that would settle which account is correct has published its findings. Details, including Sankar's description of effective altruism as a combination of "a doomsday cult and messiah complexes," are in "Palantir's Sankar says Minab school strike was 'probably human error'".

On October 10, just before the coverage window, the Philadelphia Inquirer reported new details of the Anthropic case. The false report was sent on July 18, and police met with Anthropic on October 8. A State Department spokesperson reportedly said the visa applications (19 in August and one in May) were "incomplete" and that State Department systems were never compromised. The mayor, the district attorney and Congress have not issued statements. No readout of the first SIF meeting has been released, and no outcome from Clayton's Silicon Valley meetings has been announced.

On October 9, ProPublica and Defense One reported that a company with ties to a senior Pentagon official had won a $350 million contract. The story is not about AI itself, but scrutiny is growing of the fast-track contracting mechanism (OTAs) that defense tech relies on. This week brings the AUSA annual meeting (October 12–14) and the Warsaw Security Forum (October 12–13), where NATO Supreme Allied Commander Europe Grynkewich will present a "Theory of Victory." According to a Bloomberg preview, the plan includes AI support for targeting.

Industry and investment

The FT reported on October 10 that NVIDIA is in talks to acquire or invest further in Reflection AI. Neither company has commented, and the deal is so far only reported. NVIDIA has already invested $800 million. Reflection released Beam five days ago and has not yet published its weights. The question of whether a leading US open-weight developer will be absorbed by a chip seller is covered in "NVIDIA reportedly in talks to acquire or invest further in Reflection AI".

The Information reported on October 11 that Anthropic has signed a cloud deal with IREN. The article is paywalled and we were unable to read it, and neither IREN nor Anthropic has confirmed the deal. According to Techmeme's summary, the article focuses on why Anthropic is pushing partners to speed up about $530 billion in data center leases. Analysts reportedly estimate that a single 245MW site would bring in about $1.3 billion a month. We previously reported that Meta turned Anthropic down for compute. Now, ahead of Anthropic's investor day on October 14, the name of a new supplier has surfaced. If the deal is real, its scale and terms can only be confirmed through the S-1. The S-1 has not yet appeared on EDGAR (checked by full-text search through October 12).

Samsung affiliates will invest a combined 12.26 trillion won in semiconductors and substrates, with about 8 trillion won going to Vietnam, the Korea Herald reported. HBM will stay in Korea, while back-end processing for commodity memory moves abroad. Details are in "Samsung to invest 12.26 trillion won in semiconductors and substrates". On data centers, Amazon has followed Microsoft in dropping nondisclosure agreements in negotiations with local governments, and TechCrunch looked at whether that will ease local opposition. There were no moratorium or zoning votes over the weekend.

This week is busy. The OCP Global Summit opens on Monday, October 12, with announcements on racks, power and cooling expected. On the 13th, US banks report earnings, and investors will watch for comments on the volume of data center lending. The 14th brings September CPI, ASML's earnings and Anthropic's investor day, and on the 15th TSMC reports earnings and gives its 2027 outlook.

Ideas and debate: both camps move on consciousness and economics

Three sides moved in the debate over AI's moral status. On the side that grants it, Bentham's Bulldog wrote on October 11 that if AI is conscious and can suffer and have desires, it has moral status under hedonism, desire-satisfaction theory and objective-list theory alike. The post responded to Rep. Ted Lieu, who had said sarcastically that if AI companies think their AI is conscious, they should be prosecuted for murdering, torturing and enslaving conscious math (we were unable to confirm the date of Lieu's post). Bentham's Bulldog replied that granting moral status is not the same as granting human rights, just as recognizing that dogs are conscious does not mean prosecuting breeders. From the ethics-critique side, Emily Bender wrote on Bluesky that the burden of proof lies with those making extraordinary claims. She also wrote that such ideas would have been harmless had they stayed on LessWrong, but they have made their way into the centers of finance and politics. On the skeptical side, Anil Seth wrote that the argument that consciousness requires biology is just one of several objections to computational functionalism. With biological naturalism under attack, he appears to have shifted the weight of his argument toward anti-functionalism. However, his post did not respond to anyone by name.

On economics, Daron Acemoglu and Joshua Gans debated seven questions (Gans's Substack, October 10). Acemoglu argues that capital spending on this scale will concentrate capital income to an extreme degree, and that not enough new jobs will be created to replace those lost. He also says the health benefits have not yet been proven. Gans countered that competition among labs is working well and that productivity growth can offset inequality. The two agree that adoption is slow enough to leave time to adapt and that the evidence has not yet settled the question. In "The economics of agents," Tyler Cowen argued that personal agents will make consumers more rational and weaken price discrimination. He also noted himself that agents could enable hard-to-detect collusion among sellers, though he expects such collusion would not last.

On LessWrong, Lance Spencer wrote that cryptography is an inherently defensive technology and is not dual-use in the same sense as AI. His target was Mark Atwood, who had argued that safety advocates who want to restrict access do not understand information security. However, we were unable to find Atwood's original piece, so only one side of the exchange could be checked.

Where nothing moved

The weekend brought more silence than news. Anthropic has not disclosed how often incidents occur, and there is no word on whether it has restored network access for internal evaluations. OpenAI's suspension has not been lifted, and there is no word on a rerun of GPT-6.1 Astra. The math retractions still stand at three, with no indication of which model produced the results. The latest executive order on whitehouse.gov is dated October 7, and no document setting out SIF's notification requirement, deadlines or penalties has been released. Congress is in recess until around November 9.

Notably, no confirmed response to Nadella's seven principles has yet come from the safety camp. Zvi has not posted since October 9 and has not written about the Anthropic incident. Jack Clark's Import AI is expected on October 12. We were unable to read X posts by Yudkowsky, Kokotajlo, Greenblatt and others directly, so short posts may have been missed. Gary Marcus has not posted since his call for a recall. Pinker has not yet responded to Scott Alexander, and the long piece Bender announced has not appeared. Amodei has not responded to Altman.

On models, the weights for Mistral Large 4 and Reflection Beam are not yet on Hugging Face, and DeepSeek V4.1 Pro and Kimi K3.1 have not been released. StepFun Step 5 is still scheduled for October 15. The next arXiv release comes after the window, at 00:00 UTC on October 12. Markets were closed for the weekend. There were no updates on chipmakers' investor relations, OpenAI's $1.4 trillion fundraising or SoftBank's fundraising in the Gulf.