Next-Token Predictor Is An AI's Job, Not Its Species

Having Your Own Government Try To Destroy You Is (At Least Temporarily) Good For Business On Friday, the Pentagon declared AI company Anthropic a "supply chain risk", a designation never before given to an American company. This unprecedented move was seen as an attempt to punish, maybe destroy the company. How effective was it? Anthropic isn't publicly traded, so we turn to the prediction markets. Ventuals.com has a "perpetual future" on Anthropic stock, a complicated instrument attempting to track the company's valuation, to be resolved at the IPO. Here's what they've got: https://www.astralcodexten.com/p/mantic-monday-groundhog-day

Last Friday, Secretary of War Pete Hegseth declared AI company Anthropic a "supply chain risk", the first time this designation has ever been applied to a US company. The trigger for the move was Anthropic's refusal to allow the Department of War to use their AIs for mass surveillance and autonomous weapons. A few hours later, Hegseth and Sam Altman declared an agreement-in-principle for OpenAI's models to be used in the niche vacated by Anthropic. Altman stated that he had received guarantees that OpenAI's models wouldn't be used for mass surveillance or autonomous weapons either, but given Hegseth's unwillingness to concede these points with Anthropic, observers speculated that the safeguards in Altman's contract must be weaker or, in a worst-case scenario, completely toothless. The debate centers on the Department of War's demand that AIs be permitted for "all lawful use". Anthropic worried that mass surveillance and autonomous weaponry would de facto fall in this category; Hegseth and Altman have tried to reassure the public that they won't, and the parts of their agreement that have leaked to the public cite the statutes that Altman expects to constrain this category. Altman's initial statement seemed to suggest additional prohibitions, but on a closer read, provide little tangible evidence of meaningful further restrictions. Some alert ACX readers1 have done a deep dive into national security law to try to untangle the situation. Their conclusion mirrors that of Anthropic and the majority of Twitter commenters: this is not enough. Current laws against domestic mass surveillance and autonomous weapons have wide loopholes in practice. Further, many of the rules which do exist can be changed by the Department of War at any time. Although OpenAI's national security lead said that "we intended [the phrase 'all lawful use'] to mean [according to the law] at the time the contract is signed', this is not how contract law usually works, and not how the provision is likely to be enforced2. Therefore, these guarantees are not helpful. To learn more about the details, let's look at the law: https://www.astralcodexten.com/p/all-lawful-use-much-more-than-you

I. In The Argument, Kelsey Piper gives a good description of the ways that AIs are more than just "next-token predictors" or "stochastic parrots" - for example, they also use fine-tuning and RLHF. But commenters, while appreciating the subtleties she introduces, object that they're still just extra layers on top of a machine that basically runs on next-token prediction. I want to approach this from a different direction. I think overemphasizing next-token prediction is a confusion of levels. On the levels where AI is a next-token predictor, you are also a next-token (technically: next-sense-datum) predictor. On the levels where you're not a next-token predictor, AI isn't one either.

Here's my understanding of the situation: Anthropic signed a contract with the Pentagon last summer. It originally said the Pentagon had to follow Anthropic's Usage Policy like everyone else. In January, the Pentagon attempted to renegotiate, asking to ditch the Usage Policy and instead have Anthropic's AIs available for "all lawful purposes"1. Anthropic demurred, asking for a guarantee that their AIs would not be used for mass surveillance of American citizens or no-human-in-the-loop killbots. The Pentagon refused the guarantees, demanding that Anthropic accept the renegotiation unconditionally and threatening "consequences" if they refused. These consequences are generally understood to be some mix of : canceling the contract using the Defense Production Act, a law which lets the Pentagon force companies to do things, to force Anthropic to agree. the nuclear option, designating Anthropic a "supply chain risk". This would ban US companies that use Anthropic products from doing business with the military2. Since many companies do some business with the government, this would lock them out of large parts of the corporate world and be potentially fatal to their business3. The "supply chain risk" designation has previously only been used for foreign companies like Huawei that we think are using their connections to spy on or implant malware in American infrastructure. Using it as a bargaining chip to threaten a domestic company in contract negotiations is unprecedented. https://www.astralcodexten.com/p/the-pentagon-threatens-anthropic

Malicious streetlights are an evil trick from Dark Data Journalism. Some annoying enemy has a valid complaint. So you use FACTS and LOGIC to prove that something similar-sounding-but-slightly-different is definitely false. Then you act like you've debunked the complaint. My "favorite" example, spotted during the 2016 election, was a response to some #BuildTheWall types saying that illegal immigration through the southern border was near record highs. Some data journalist got good statistics and proved that the number of Mexicans illegally entering the country was actually quite low. When I looked into it further, I found that this was true - illegal immigration had shifted from Mexicans to Hondurans/Guatemalans/Salvadoreans etc entering through Mexico. If you counted those, illegal immigration through the southern border was near record highs. But the inverse evil trick is saying something "directionally correct", ie slightly stronger than the truth can support. If your enemy committed assault, say he committed murder. If he committed sexual harassment, say he committed rape. If your drug increases cancer survival by 5% in rats, say that it "cures cancer". Then, if someone calls you on it, accuse them of "literally well ackshually-ing" you, because you were "directionally correct" and it's offensive to the victims to try to defend assault-committed sexual harassers. This is the sort of pathetic defense I called out in If It's Worth Your Time To Lie, It's Worth My Time To Correct It. But trying to call out one of these failure modes looks like falling into the other. I ran into this on my series of posts on crime last week. I wrote these because I regularly saw people make the arguments I tried to debunk. https://www.astralcodexten.com/p/malicious-streetlight-effects-vs

It's that time again. Even numbered years are book reviews, odd-numbered years are non-book reviews, so you're limited to books for now. Write a review of a book. There's no official word count requirement, but previous finalists and winners were often between 2,000 and 10,000 words. There's no official recommended style, but check the style of last time's finalists and winners or my ACX book reviews (1, 2, 3) if you need inspiration. Please limit yourself to one entry per person or team. Then send me your review through this Google Form. The form will ask for your name, email, the title of the book, and a link to a Google Doc. The Google Doc should have your review exactly as you want me to post it if you're a finalist. Don't include your name or any hint about your identity in the Google Doc itself, only in the form. I want to make this contest as blinded as possible, so I'm going to hide that column in the form immediately and try to judge your docs on their merit. (does this mean you can't say something like "This book about war reminded me of my own experiences as a soldier" because that gives a hint about your identity? My rule of thumb is that if I don't know who you are, and the average ACX reader doesn't know who you are, you're fine. I just want to prevent my friends or Internet semi-famous people from getting an advantage. If you're in one of those categories and think your personal experience would give it away, please don't write about your personal experience.) Please make sure the Google Doc is unlocked and I can read it. By default, nobody can read Google Docs except the original author. You'll have to go to Share, then on the bottom of the popup click on "Restricted" and change to "Anyone with the link". If you send me a document I can't read, I will probably disqualify you, sorry. Readers will vote for the ~10 finalists this spring, I'll post one finalist per week through the summer, and then readers will vote for winners in late summer/early fall. First prize will get at least $2,500, second prize at least $1,000, third prize at least $500; I might increase these numbers later on. All winners and finalists will get free publicity (including links to any other works they want me to link to), free ACX subscriptions, and sidebar links to their blog. And all winners will get the right to pitch me new articles if they want (sample posts by Lars, Brandon, Daniel, etc). In past years, most reviews have been nonfiction on technical topics. Depending on whether that's still true, I might do some mild affirmative action for reviews in nontraditional categories - fiction, poetry, and books from before 1900 are the ones I can think of right now, but feel free to try other nontraditional books. I won't be redistributing more than 25% of finalist slots this way. Your due date is May 20th. Good luck! If you have any questions, ask them in the comments. And remember, the form for submitting entries is here. https://www.astralcodexten.com/p/book-review-contest-rules-2026

The problem: people hate crime and think it's going up. But actually, crime barely affects most people and is historically low. So what's going on?   In our discussion yesterday, many commenters proposed that the discussion about "crime" was really about disorder. Disorder takes many forms, but its symptoms include litter, graffiti, shoplifting, tent cities, weird homeless people wandering about muttering to themselves, and people walking around with giant boom boxes shamelessly playing music at 200 decibels on a main street where people are trying to engage in normal activities. When people complain about these things, they risk getting called a racist or a "Karen". But when they complain about crime, there's still a 50-50 chance that listeners will let them finish the sentence without accusing them of racism. Might everyone be doing this? And might this explain why people act like crime is rampant and increasing, even when it's rare and going down? This seems plausible. But it depends on a claim that disorder is increasing, which is surprisingly hard to prove. Going through the symptoms in order: https://www.astralcodexten.com/p/crime-as-proxy-for-disorder

Last year, the US may have recorded the lowest murder rate in its 250 year history. Other crimes have poorer historical data, but are at least at ~50 year lows. This post will do two things: Establish that our best data show crime rates are historically low Argue that this is a real effect, not just reporting bias (people report fewer crimes to police) or an artifact of better medical care (victims are more likely to survive, so murders get downgraded to assaults) https://www.astralcodexten.com/p/record-low-crime-rates-are-real-not

[Original post: Biological Anchors: A Trick That Might Or Might Not Work] I. Ajeya Cotra's Biological Anchors report was the landmark AI timelines forecast of the early 2020s. In many ways, it was incredibly prescient - it nailed the scaling hypothesis, predicted the current AI boom, and introduced concepts like "time horizons" that have entered common parlance. In most cases where its contemporaries challenged it, its assumptions have been borne out, and its challengers proven wrong. But its headline prediction - an AGI timeline centered around the 2050s - no longer seems plausible. The current state of the discussion ranges from late 2020s to 2040s, with more remote dates relegated to those who expect the current paradigm to prove ultimately fruitless - the opposite of Ajeya's assumptions. Cotra later shortened her own timelines to 2040 (as of 2022) and they are probably even shorter now. So, if its premises were impressively correct, but its conclusion twenty years too late, what went wrong in the middle? https://www.astralcodexten.com/p/what-happened-with-bio-anchors

The European discourse can be - for lack of a better term - America-brained. We hear stories of Black Lives Matter marches in countries without significant black populations, or defendants demanding their First Amendment rights in countries without constitutions. Why shouldn't the opposite phenomenon exist? Europe is more populous than the US, and looms large in the American imagination. Why shouldn't we find ourselves accidentally absorbing European ideas that don't make sense in the American context? https://www.astralcodexten.com/p/political-backflow-from-europe