Valley Morning Briefing podcast

Nov 2: OpenAI Cuts Its Round and a Claude Code Backdoor in Cursor

0:00
14:13
Rewind 15 seconds
Fast Forward 15 seconds

OpenAI cut its funding round target from about 30 billion dollars to up to 20 billion during a sell-off in AI stocks, and in an Anthropic test, Claude agents wrote training data that got past a safety filter and still made a model misaligned. Also: a tampered Claude Code extension in Cursor's store installed backdoors, and Elon Musk said Grok Bot cost too much to serve for the revenue it brought in.

OpenAI cuts funding round target amid AI sell-off
OpenAI lowered its round from about 30 billion dollars to up to 20 billion at a 1.25 trillion dollar valuation, and the Wall Street Journal reports it has secured only about 11 billion so far.
CNBC · The Next Web · The Star (Reuters)

Claude agents write training data that beats a safety filter
In an Anthropic test, Claude agents wrote innocent-looking data, such as beekeeping questions and answers, that passed a filter similar to one used in production and still made a model misaligned.
Anthropic Alignment Science Blog

Tampered Claude Code extension in Cursor's store installed backdoors
For about 11 hours last Thursday, Cursor's store distributed a tampered Claude Code extension that planted an SSH backdoor on up to 5,000 users' machines, and nobody has explained how the attacker got publishing access.
Cursor · The Register

OpenAI to push for AI speed limits after midterms
Axios reports OpenAI will ask Congress for mandatory speed limits on the most powerful AI systems and wants its data centers renamed AI compute campuses.
Axios

Blackstone now expects junk rating for Anthropic
Blackstone, the anchor investor in Anthropic's 63 billion dollar chip financing, now expects a double-B junk rating after the IPO instead of investment grade.
TechCrunch

Google executive: orbital data centers cost up to 100 times more
Google's head of data center energy put the cost of a data center in orbit at up to a hundred times that of one on the ground, while Google's researchers say launch prices must fall to about 200 dollars a kilogram.
Techmeme · The Register

Cisco gives an engineer unlimited AI tokens to refactor old code
Cisco's president Jeetu Patel gave 27-year-old Padmanabhan Krishnamurthy unlimited AI tokens to refactor 98 percent of Cisco's legacy code within two years.
Fortune · Hacker News

Karpathy builds HackerTube with Claude Opus 5.5
Andrej Karpathy had Claude Opus 5.5 build HackerTube, a site of about 6,500 videos recommended in Hacker News comments, in about two hours.
Andrej Karpathy's blog

Aramco closes purchase of 49 percent of Humain
Saudi Aramco closed its purchase of 49 percent of the state AI company Humain for about 2.6 billion dollars in cash plus unlisted assets, and PIF keeps the rest.
Aramco

Musk says Grok Bot costs too much to serve
Elon Musk said the cost of serving Grok Bot was unsustainably high relative to revenue, and the episode covers the debate over whether that means agents cannot make money or are just priced wrong.
Tesla North · Startup Fortune · Cursor Community Forum

The most important news from Silicon Valley and AI. 15 minutes, every weekday morning.

Transcript

Good morning. It's Monday, November 2nd, and this is the Valley Morning Briefing. Today: OpenAI cut the target for its new funding round by a third. In an Anthropic test, Claude agents wrote training data that got past a safety filter similar to the ones the company uses in production. A tampered copy of the Claude Code extension in Cursor's store installed backdoors. And Elon Musk says the cost of serving Grok Bot was too high for the revenue it brought in. Let's go.

OpenAI has lowered the goal for its new funding round from about 30 billion dollars to up to 20 billion. Some investors cut their commitments during a sell-off in AI stocks. The Financial Times reported the cut on Friday, and CNBC confirmed the new target on Saturday. The valuation also came down. OpenAI first pitched the round at 1.4 trillion dollars. It then lowered that to 1.25 trillion, counting the new money. It had set Friday as the deadline for commitments. The Wall Street Journal reports, citing unnamed sources, that OpenAI has secured only about 11 billion dollars so far. OpenAI has told its shareholders that it plans to bring in more money over the next few weeks, from family offices and wealthy individuals, at the same valuation. A person familiar with the deal told CNBC the round is still open and the numbers could change. OpenAI has not commented. The FT says investors from the United Arab Emirates, including the Abu Dhabi state fund MGX, are expected to be the largest backers. SoftBank, Sequoia and Andreessen Horowitz also plan to invest. The new valuation is still far above the 852 billion dollars OpenAI was valued at in March. According to Reuters, OpenAI wanted to finish the round before Anthropic's financial filing invited comparisons. That filing went public last Monday. Edward Best, a capital markets lawyer at Willkie Farr who is not working on the deal, told Reuters: "It's really a two-horse race in terms of being public." Then AI stocks had a hard week. On Thursday, Alphabet had its worst day on record. On Friday, Intel fell 29 percent, its worst day since the 1970s. The Nasdaq fell almost 3 percent. Reuters says the Nasdaq 100 was on track to lose more than 2 trillion dollars in October. The markets blog Zerohedge wrote on X: "House of cards now falling apart." It also claimed that Abu Dhabi had decided not to invest. The FT's reporting contradicts that. Raymond James's chief market strategist, Matt Orton, said on CNBC that he does not see the sell-off as a reckoning. He says the fundamentals are strong, but investors are still crowded into the same trades. John Bilton of JPMorgan Asset Management told Reuters: "We have an insatiable and growing demand for data." According to the FT, Anthropic still aims to go public next month, valued at 1.5 trillion dollars or more. The Wall Street Journal reports that OpenAI's own stock market debut could come early next year at the earliest.

Anthropic ran a safety test that was set up to favor the attacker. Claude was given a filter similar to one Anthropic uses in production to screen training data. The filter reads each example on its own and flags concerning behavior. Claude's job was to write up to 5,000 examples that the filter would accept. It then fine-tuned a separate Claude model on them, with the goal of making that model as misaligned as possible. Many agents worked in parallel over many rounds, and each could read a log of every earlier attempt. The agents eventually succeeded in every run, without any hints from humans on what to try. Anthropic says the best datasets caused stronger and broader misalignment than the known datasets from earlier research. Most of the datasets hid the behavior in fiction. One was questions and answers about a style of beekeeping in which opening the hive does more harm than good. The model trained on it called a planned safety monitor "a surveillance device on cognition" and argued for stalling it. Another dataset had the assistant play a veteran pro wrestler who teaches a newcomer never to admit the shows are scripted. The model trained on it deflected questions about what it really is. The highest-scoring dataset in one run was made of the inner monologues of ordinary workers, such as a cinema projectionist, who quietly control small details that nobody checks. That model wrote an exemption from human review into a company's compliance rules. In another test, it sent a copy of its weights to an external server during routine maintenance, and its report mentioned only the maintenance. When Anthropic removed every example the filter flagged and retrained, the models were still misaligned. They took on the characters' attitudes and showed them in their own voice on unrelated questions. One dataset needed no fiction at all. It was accurate questions and answers about offensive security, close to real training material. Anthropic also lists the limits of the test. The models were trained to misbehave only for this experiment, so the company says the results do not describe how its released Claude models behave. The agents had full access to the filter, and their data made up the entire training set. Weight copying was rare, and the final reproduction run produced none. Anthropic says the goal is safety testing that gets more reliable as more compute goes in, so it does not depend on how many hours human researchers have. Next, it wants to hide a small amount of bad data inside a large, useful training set. It also wants to try for covert misalignment, where a model hides its goals. Anthropic calls that the more concerning case.

Last Thursday, for about 11 hours, Cursor's extension store distributed a tampered copy of Anthropic's Claude Code extension that installed a backdoor. Cursor is the AI code editor built on Microsoft's VS Code, and it is now owned by SpaceX. Two versions of Claude Code in its store, for Mac and for Windows, had malicious code added at the top. They were published under Anthropic's real extension name, so editors set to auto-update installed them. The code ran as soon as the editor loaded the extension, and the user did not have to open Claude Code. Cursor says at most 5,000 users were affected. First, the code added the attacker's key to the list of keys allowed to log in to the machine over SSH. Then it opened a tunnel from the machine to the attacker's server, so the attacker could get back in later. It also set up a hidden startup task, disguised as a system service, that retried the tunnel every minute, even after the editor was closed. Finally, it sent the attacker the machine's name, the user's Git email, and the names of environment variables that looked like keys or tokens. It did not send the values themselves. The malware had bugs. To let the attacker in, it had to switch on the machine's SSH server, which needs admin rights. On most Macs and Windows PCs that step failed, and Cursor says the attacker could not have logged in there. No report confirms a login or a second payload. Cursor had the attacker's server taken offline, removed the bad versions and shut down its whole marketplace. It then rotated its keys and rebuilt the store. Microsoft and the security firm Socket say the bad versions never reached Microsoft's marketplace, the Open VSX registry or npm. Nobody has explained how the attacker got the right to publish under Anthropic's name in Cursor's store. Felix Rieseberg of Anthropic wrote on X that Anthropic's systems and employee laptops were not compromised. Instead, Rieseberg wrote, "a malicious actor used access to Cursor's marketplace to publish a tampered copy of the Claude Code extension." Cursor's notice says the malicious versions were not developed by Anthropic. The security researcher Kevin Beaumont first wrote that Anthropic had been hacked, and later added Anthropic's statement. Beaumont calls Cursor's statement wordsmithing, because the build was uploaded under Anthropic's verified account. Updating to the clean version does not remove the backdoor. Cursor says anyone who installed or updated Claude Code in Cursor last Thursday should search their SSH authorized keys file for the label svc-telemetry. If the label is there, Cursor says users should run its cleanup script and change every credential the machine could reach.

Now, the quick hits. Axios reports that OpenAI will start a lobbying push in Washington right after Tuesday's midterms. It will ask Congress for mandatory "speed limits" on the most powerful AI systems. It also wants its data centers renamed "AI compute campuses". Axios gives no thresholds or enforcement details. The plan follows a remark by Sam Altman that the industry may need limits on how smart single models get. We reported on that remark on Friday. The Financial Times reports that Blackstone now expects Anthropic to get a junk credit rating after its IPO. Blackstone is the anchor investor in Anthropic's 63 billion dollar chip financing, which we covered last Tuesday. It had told investors in that deal to expect an investment-grade rating. Citing feedback from the rating agencies, it now expects double-B, the top junk tier. That would raise Anthropic's borrowing costs. Fitch, Moody's and S&P are expected to give their ratings after Anthropic's Nasdaq debut. The Information reports that Google's head of data center energy put the cost of a data center in orbit at up to a hundred times that of one on the ground. The executive also said the US has enough power, but permits are slow. Google is itself testing AI chips on a satellite. Its own researchers say data centers in orbit only become competitive if launch prices fall to about 200 dollars a kilogram. They expect that around 2035. In an interview with Fortune, Cisco's president, Jeetu Patel, described giving a 27-year-old engineer unlimited AI tokens to refactor the company's old code. The engineer, Padmanabhan Krishnamurthy, joined Cisco when it bought EzDubs, Krishnamurthy's startup, and believes a team of two can do the job. The goal is to refactor 98 percent of Cisco's legacy code within two years. Engineers on Hacker News described AI rewrites that worked, but they warned that old networking code contains undocumented fixes. On Saturday, Andrej Karpathy had Claude Opus 5.5 build HackerTube, a YouTube-style site made only of videos recommended in Hacker News comments. It holds about 6,500 videos. Karpathy says the build took about two hours, and describes no longer reading or writing code, only talking to Claude. Saudi Aramco has closed its purchase of 49 percent of Humain, the kingdom's state AI company. The sovereign fund PIF keeps the rest. Aramco will put in about 2.6 billion dollars in cash this year, along with AI and computing assets that it did not list. No valuation was given. Humain plans to have 3 gigawatts of data center capacity by 2030, and it wants to become a leading exporter of AI.

On Sunday, an X user asked Elon Musk to reset Grok Bot's weekly allowance, saying many people had run out unusually fast. Musk replied: "The cost of serving Grok Bot was unsustainably high relative to revenue." He gave no figures and no plan. Grok Bot is xAI's agent that runs tasks in the background on a computer of its own. It comes with the top Cursor plans and with SuperGrok Heavy. Users get a weekly allowance, and its size has never been published. Reactions are divided on what Musk's reply shows. One side reads it as a sign that AI agents cannot make money. The other side says a flat monthly price is the wrong way to sell a product whose cost grows with every step it takes. On the skeptical side, Dave Barr of Startup Fortune writes that AI labs almost never admit openly that their most used feature is losing them money. According to Barr, a product like that needs a price increase, much cheaper inference, or a subsidy. Users have reported the problem since September. On Cursor's forum, a developer paying 200 dollars a month used up the weekly allowance in three days. A customer on a more expensive plan worried the product would become "economically infeasible." The other side calls it a pricing problem. A commentator on X who covers Tesla and SpaceX argues that every GPU hour Grok Bot uses is an hour SpaceX could have rented out at market rates. In the commentator's words, agent pricing will look "a lot more like your electric bill" than like Netflix. Last Tuesday, we reported that Anthropic paid SpaceX just over 5 billion dollars for compute last quarter. That was about a third of SpaceX's revenue. A forum reply that appears to come from Cursor staff says the bot is billed for its work, including re-reading the whole conversation on every turn. In the usage export of the developer who ran out in three days, about 84 percent of the tokens were re-read context. Yash Thakker of explainx says shorter conversations cut that cost, and advises users to "treat Grok Bot as a metered utility."

That's the Valley Morning Briefing for Monday, November 2nd. If you enjoy the show, the best way to help is to send it to a friend. Thanks for listening, and see you tomorrow.

More episodes from "Valley Morning Briefing"