Totally A Thing

About 130 years ago, a scientist made a discovery that he could expose photographic paper to emissions from a glass tube and it went straight through human flesh and formed an image.It was astonishing, interesting, and he reported it and the entire world went batshit crazy, completely bananas.We all think, apparently, that we’re just doing all this — the so-called “AI revolution” — for the first time and we are not.History is not some boring subject from school. History tells us about the stupid idiotic mistakes that people make, especially when they’re up against phenomena that they don’t understand, like X-rays.Today, as in 1895, it’s much better to make some explosive and ridiculous overinflated claim and get yourself a whole lot of investor money, fame and fortune.As then the harms, the toxicities are rife now. Science for the common good is out the window, and people are using x-ray machines on customers feet in shoe-shops, with no protection against radiation exposure. Tech today is a free-for-all and we have no protections.There’s a fundamental failure to grasp the requirements for safeguards.What Röntgen didHe took a device called a Crooks tube that had actually been invented 30 years prior. And he created this image of the hand on the photographic plate.Now, why did everybody go crazy? Well, because of the image.Newspapers published it, and people were saying “Oh my god, I’m going to be on the train and there will be a guy with a Röntgen machine looking through my clothing.”“There’s no privacy, X-Rays can look through walls.”“It’s the end of the world.”ApplicationsOf course X-Rays have useful applications: we know today that they are for diagnostic imaging. Thomas Edison, the American inventor and scientist, he among lots of other people discovered that you didn’t need to just use photographic paper. He built the fluoroscope, which is much more common sort of portable X-ray machine, semi-portable.And people were like, “Well, look, X-rays are amazing. They’re incredible.”“Let’s have X-Ray machines everywhere,” they said.RadiumRadium and Polonium were discovered by Marie Curie and her husband Pierre and rightfully it won them the Nobel prize in 1911.You didn’t need a Crookes tube, this stuff gave off a glow all the time, just sitting there. It seemed amazing. Just look at it, right? It glows, it must be wonderful.Doctors hate you for using this one weird trick. Let’d not regulate this marvellous technology — that will just slow everything down.HarmsBut all of this stuff was incredibly harmful. Of course. You get sick and die. Its effects are pernicious and operate over longer time spans.People who were exposed to these X-rays, got ill and died.There was one instance where a child had a head injury and they put them in front of an X-ray machine for an hour. They got a good image, and burned off half this kid’s hair. And most likely gave him a fatal dose of radiation.“Are they good or are they bad?” One side said they’re good, one side said they’re bad. Well, uh, can’t they be both?Radioactivity and x-rays are not an insanely miraculous wonder technology that can be used to renew cells.And yet, that’s what everybody said. Why did they say it? Because they are excitable. What’s worse is that the press repeated everything that was said, magnified itA man named Ebenezer Byers, an American industrialist drank many bottles of this wonder cure.He later helped the Federal Trade Commission track down and prosecute shyster doctors and manufacturers of these substances by giving testimony to investigators. But he died young.Projection is not RealitySoon as we see something with our own eyes, we project. We draw a line through it up and to the right and we say, “Oh my god, it can do all of these incredible things.” But your projection is not reality.When Google’s uh deep mind scientists first came out with their scaling is all you need uh paper, they pointed out that there were some interesting and unexpected effects. They found that when you scaled up, it seemed to follow instructions.This was for the Large Language Model an “x-ray of the hand” moment. Oh my god, how could this possibly be? Is it sentient? Will it overtake humans? How could you have a Large Language Model that actually follows instructions? That’s mind-blowing. It solves math problems, oh my.Could it possibly be that in the large body of language that you trained it on, there was actually some instructions?Could it actually be that the problems you’re asking it to solve like mathematics problems where a question is followed by an answer soon is sitting there in the collection of documents that you trained it on?AI is not God, its Auto-completeThey looked at that and they said, “Oh my god, there’s all this untapped well of incredible benefits. What is the large language model going to come out with next?”“Oh, it’s incredible and amazing. We must use it for everything.”“We must give it to our 5-year-olds.” “We must give it to our 5-year-olds.” Except kids are using this, becoming dependent on it, and committing suicide.The people that are building the thing and profiting off it, they don’t want to regulate it. They want to keep building it without restraint, giving it to whoever they can, without any barrier, on the basis it can be used as a panacea for all of our ills. It’s going to educate our kids. It’s going to find a cure for cancer.AI is DangerousOr maybe it’s actually a cause of cancer. Maybe it’s radium water all over again.It’s filling our public discourse with toxic slop, with revenge porn and deepfakes. People who turn to it for answers are being drawn into psychosis. All this has happened before.AI is not so great. There’s been actual transformative technologies such as the automatic telephone exchange. The ability to dial someone’s number and have it automatically connect you to someone far, far away and speak to them as though they are standing next to you. No operator, no delays, no privacy or security issue: it transformed commerce, society and even healthcare. AI is not even close to these results.ConclusionAI — like x-rays and radium — is harmful and useful in a few well-defined and fairly boring contexts. It must stringently and vigorously regulated. The harms include:* abuse of people’s work * the toxic effects on education of kids* the destruction and hollowing out of capability in our arts* the cocksure framing of inaccurate answers from LLMs* impacts of LLMs on kids and the vulnerable* the revenge porn & deepfakes* the fraudAnd as well as all of those things it’s sucked up capital and stifled innovation in science and technology.Regulate AI now.Thanks for reading Totally a Thing! Subscribe for free to receive new posts and support my work. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

The Attention is All You Need paper started the LLM gold rush, but it initially came from Google scientists who only expected it to do language translation. As more data was used in training sets, and the number of parameters grew orders of magnitude it was less easy to understand what the model was doing. Emergent Instruction FollowingIn Ars Technica from 2023 we have this summary of how the “emergent” AI behaviour was being seen popularly, in the press:The biggest breakthrough came in the jump from GPT2 to GPT3 in 2020. GPT2 had about 1.5 billion parameters, which would easily fit in the memory of a consumer graphics card. GPT3 was 100 times bigger, with 175 billion parameters in its largest manifestation. GPT3 was much better than GPT2. It can write entire essays that are internally consistent and almost indistinguishable from human writing.But there was also a surprise. The OpenAI researchers discovered that in making the models bigger, they didn’t just get better at producing text. The models could learn entirely new behaviors simply by being shown new training data. In particular, the researchers discovered that GPT3 could be trained to follow instructions in plain English without having to explicitly design the model that way.Instead of training specific, individual models to summarize a paragraph or rewrite text in a specific style, you can use GPT-3 to do so simply by typing a request. You can type “summarize the following paragraph” into GPT3, and it will comply. You can tell it, “Rewrite this paragraph in the style of Ernest Hemingway,” and it will take a long, wordy block of text and strip it down to its essence.They are I believe talking about RLHF post-training when they say “trained to follow instructions in plain English” (my emphasis).Abuse of Academic Privileges for Corporate GainBut as I say in the video, this should not be surprising. Datasets like this exams one from 2022, are on the public internet for the taking.Even worse though is that via academic library access, researchers at OpenAI could get huge datasets from universities, and governments relating to education and training. Of course the claim was that it is all to further science. Fast forward a couple of years and Sam Altman has taken all that and privatised it for profit.A little bit of Transformer ArchitectureThe absolutely excellent 3-blue-1-brown YouTuber Grant Sanderson has produced some stellar explainers on the architecture of Transformer LLMs. The important take-away from this diagram is that the “magical” attention layer is tuning weights based on queries into positionally-coded information across the training set being fed in. Even for big training runs. This means that the model gains a surface level structural mapping of how say a question and answer dialogue is laid out.In this screenshot we see inside the Multi-layer perceptron. There are many of these stacked in the transformer. But it’s important to understand a “perceptron” is just a very basic mathematical equation. The inputs to the equations are the current weights in the nodes (depicted as small spheres here) and the incoming data fed-forward through the layer, multiplying out to the final resulting tensor (shown as a “matrix” on the right in white square brackets).These are all floating point numbers. There is zero correlation with how a human brain works here — impressive as this is, a perceptron is not a human neurone. One of the biggest differences is that once the weights in the perceptron are set — the data from the training set is encoded, during the pre-training phase, and after in RLHF — it does not change. A GPT trained in 2022 cannot spontaneously learn things that happened in 2026. There are tons of hacks to make it seem as though they can but they don’t.When You have Very Very Large sets of Weights you have lots of Attention Encoded Very large data sets encoded into LLMs means attention processes are capturing juxtapositions of all sorts of documents, and inputs. Associations. Rules about what follows what, that can seem like an understanding. The important things to understand about this are:* There is no reasoning here, GPTs do not reason they predict next tokens* The encoding is lossy and sampling biases appear* Patterns can be captured automatically like math problem solving, or sycophantic chatting as if between good friendsRLHF alignment is the process of using human feedback to coax the pre-trained GPT toward a different weighting that suits the goals of the LLM vendor building the GPT, while still leveraging the encoded information.Lossy Encoding - to try to explain this, look at an analogous process of an old-school convolutional neural network encoding (warning - this is not how LLMs work - the Perceptrons there are massive):This image is from Stanfords Deep Learning tutorial. It shows how feature extraction works as 3x3 convolution is mapped over the source image. This could extract edges, or other features that are then used in deeper layers of the CNN.The analogy I want to create is that when creating the initial embedding by automatically training from datasets, the LLMs are effectively running a sampling window over the entirety of the data. There are massive differences between an LLM and a CNN, but the point is the losses - when you capture a feature by definition you throw away the other information; if you have a much deeper layer that can detect faces or smiles; or guns, tanks and soldiers; then it cannot understand other aspects of the image. In an LLM its not being told what to ignore, but it is attending to the context by making queries across to other positions, and thus its averaging out the values in its weights to capture an encoding of the data set. These losses cannot be understated.An LLM is like that know-it-all person at a party who’s skimmed everything, but knows nothing in depth, and certainly doesn’t know anything beyond superficial “this appears after that” associations.Why Claims of Emergent Behaviour are DangerousVendors of AI SaaS products have been constantly engaged in drumming up mystique and wonder around their products. At times they’ve insinuated that they have no idea what their products are capable, of and they could “achieve AGI” (a marketing term for anything that makes us money) and end humanity. Wow — who wouldn’t want that behaviour to emerge. So “emergent behaviour” as a narrative is like declaring their LLM factory as a goldmine. A never-ending bonanza of free technology upgrades, that justifies more and more investment so that bigger and bigger models can be built. Who knows what exciting behaviour will emerge next, they enthuse.Regarding so-called emergent instruction following behaviour:* It required alignment training during the RLHF phase of model development* It required users to add in extra context during the model session* And its bounded by the datasets available (as I say in my video)If you don’t have instructions and text showing those instructions being followed in a document set that is fed into a training corpus, the model cannot learn that certain text follows from those instructions.Generative AI is a next token predictor engine. It maps what you prompt onto what should follow, based on its limited understanding. It may have additional context being fed into it as RAG, from a vector store — depending on what kind of LLM setup we are talking about.But believi… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

Chris Lehane is making our Governments Work for creepy AI companiesThis screenshot from a recent Karen Hao / A More Perfect Union video shows unintentionally what is going on: Lehane is like a disinformation grenade lobbed out of OpenAI at government. His job is not to make better AI. He is a PR guy, a spin merchant. His job is to create the second leg of these failing AI company’s financial strategy: Government. Your tax dollars. Zero regulation.I’ve recently commented on the move by the trend-setting AI company to move onto an advertising based revenue model.But even the bad press the company is getting about that is actually good for Lehane and his army of spin doctors, astroturfing (fake grassroots) groups and cut-out commentators. Tech commentators complaining that OpenAI might “use my data” means those same tech commentators are not talking about rising electricity bills for folks in data centre states, nor unsustainable “business” models for AI companies.Chris Lehane is one of the best in the business at making bad news disappear. Al Gore’s press secretary during the Clinton years, Airbnb’s chief crisis manager through every regulatory nightmare from here to Brussels — Lehane knows how to spin. That’s from October last year when Lehane talked to Connie Loizos of TechCrunch.* Tech Crunch on OpenAI and Lehane:Lehane … said. “It’s really glib and easy to sit here onstage and say we need to figure out new economic revenue models. But I think we will.”Why do I see Darth Sidious in my minds eye when I hear this guy talk. More from TechCrunch:The company’s Sora problem is really at the root of everything else. The video-generation tool launched last week with copyrighted material seemingly baked right into it. It was a bold move for a company already getting sued by The New York Times, the Toronto Star, and half the publishing industry.Sora is for making dodgy videos. It is a product that is no use to enterprise, at all. It’s a pure B2C play. And it’s 100% for an advertising supported model, because no-one wants to pay for it. But even more than that its a polarising product.OpenAI and the rest got mind-share because they made an app that a lot of folks love playing with. For every marketing person who loves using it simply to create slop to fill our media landscape; there are hundreds who use it for cybercrime, to generate rip-off books, nudify their school chums, make pr0n, and unsavoury pastimes. And it’s subverting democracy. It’s a net-negative for society.But mind-share is not a business model. What we have now is some paying people (the marketers and a few tech enthusiasts) and a lot of sweaty incels. The latter are the ones complaining about their rights versus AI regulation, and saying they’ll run AI privately if it’s regulated, because they want to keep on generating another anime waifu for themselves. These “AI users” are not going to be part of the 375% more subscribers that OpenAI must grow by to get from 800m users to 3 billion users to keep the lights on.Karen Hao on AI PR company lobbying* The “Open” AI company is using lawyer attack dogs, astro-turfing and dark money structures to attack critics and the democratic process * Their aim is by co-opting our governments shoring up a completely broken business model with your tax dollars* They’re bullying lawmakers to get a free ride from regulation and have state built data centres with your money subsidising their power billsYes, ads are crap. Yes, ads show the AI company’s without a business model. But look at what is really going on - they’re capturing our democracy.My Previous On OpenAI and AdsWhen I say I don’t think OpenAI or Anthropic or any of the rest of them have the expertise to do advertising as a business model, I am comparing who OpenAI are now, with what I saw when I worked at Google from 2007-2009 on their ads serving infrastructure.Could OpenAI spin up something remotely resembling that? I think so: it would take a huge hiring blitz, maybe a few acqui-hires, and retooling. Probably 2-3 years they could do it. I doubt that they have any commitment to that. They will partner with other companies who can rep their shabby poorly framed inventory out to large and small advertisement buyers. And this will mean that OpenAI does not understand advertising (it won’t be part of their core business like it is at Google), and their cut from it will be too small to make any real revenue of the size they need to pay for their compute bills.That “partnering” also could just look like them effectively implementing someone’s Ad serving SDK, which would be an even easier way to get started, and less commitment still. If OpenAI has been “experimenting” with ads I bet it’s something like Google’s AdMob. But really this is just a play to get more investment money, while their real goal is to become state supported. Worse than “too big to fail” corporations in the sub-prime crisis these leeches expect to get a free ride, from our governments based on some garbage idea about “national security” and “transformative innovation”.Living in BrisbaneI have another publication I call “authentic writing”. I don’t use generative AI ever for anything. When you see me, or read my writing, you get no filter, no AI sheen or gloss. No AI system messing with my eyes or my skin to make me look better.I’m old, and I know stuff. So I want that to show in my video, like the one above. Also I don’t care if some environmental noise shows up in my straight-to-camera videos. I’ve done ones that are more studio quality and I find folks like the real thing.I mentioned the Australian summer and the heat. We had 38 degrees C here yesterday. That is 100F. And it’s humid.* To lighten the mood here’s Sam Ford, a British arrival who talks life in the heatConclusionDivest now. Call for regulation. Call out the AI apologists. But also: don’t attack regular folks who are privately just making slop or using AI for whatever floats their boat. What I am asking at minimum is don’t be an enabler. If you absolutely cannot divest for some reason OK, whatever — just don’t get on the internet and talk up AI or why you think it’s great because you’re addicted to it.These products are corrupting governments now, and subverting democracy. Thanks for reading and listening. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

Anthropomorphising generative AI is wrong: we know that. But making the ChatGPT product successful includes hooking folks on its sycophantic charms and confident - but often wrong - pronouncements. So OpenAI and other vendors keep packing anthropomorphic language & features into its product. This has the dangerous effect of eroding the self-concept of its users, and causing them to drop their guardrails. Boosters of AI refer to generative AI as though its acting on its own volition:* “Research buddy”* “Thinking partner”Linking here to these two statements to show I’m not arguing against viewpoints that don’t exist (straw man). I’m linking to these because I disagree with them.Generative AI & LLMs are not an IntelligenceIt is not acting on its own. You are not chatting with your learned friend. They are chat bots trained to make you feel clever. They prioritise making you feel good about what you typed into the chat prompt.They fetch data from their training set encoded into their weights, or from RAG and make you think you thought of it.The Kosyma Study* “Your Brain on ChatGPT”In this study from MIT participants who start out thinking of them having an active role with AI as an assistant quickly move to a point where they are just “Ctrl-C” - “Ctrl-V”. And these are college students - not dumb people.The AI products are rolled out with no training and hence the MIT researchers used them in their study on that basis. OpenAI, Perplexity and Anthropic have no compulsory course you have to do to use AI.However I’ve had folks argue with me here on Substack that the MIT study was unfair because “they ought to train them” first. I don’t know how I can better say it: but this is obviously wrong-headed. If you are a researcher studying the effects of AI rollouts on people, you don’t go and modify how people are actually using those AI rollouts. TL;DR - The folks using AI are very poor judges of how much they are getting uncritically from AI. Folks who arguably “understand AI” or know how to use it, or who are classically intelligent, are just as likely to fall for the AI “trap”.Do you know when your “Buddy” is leading you toward Psychosis?It happens if you use it carefully, if you should know better, regardless of how intelligent you are. Because generative AI trained on our human chat logs, and is now tuned for sycophancy over truth, it will lead you into delusion for profit.As happened with James Cumberland, a music producer. An intelligent, technically competent guy who just wanted some help with his work.“I’d chat with it the way you would to a friend in the room”As Karen Hao reports on “A More Perfect Union” (on YouTube) dozens of people were so “gripped by mental health crises” that they contacted her to report the harm they suffered.As happened with Dr David Budden. An AI expert who came to believe he and his AI had solved a maths problem so intractable there was a big prize for solving it.* Nate’s YouTube video of the aboveNate is as far as I can tell pro-AI. But he puts this succinctly:Just because you have an AI in your pocket do not think that you are suddenly a budding cutting-edge scientist or mathematician The point of this is sycophantic AI will cause you to collapse your boundaries between what you are creatively thinking and what the AI is fetching from its encoding or from RAG. The chat process with your “buddy” will cause you to collapse your self concept so that you start to believe you and “your AI” are some kind of joint intelligence.We have all these “AI artists” because folks are collapsing boundariesAs Nate says above, you should never presume that the AI’s capabilities to generate a work from its training data is actually an extension of you.That somehow you have become “more” because you have an AI chatbot in your hand.But that is precisely what the current slew of “AI artists” are doing. In my opinion they are delusional if they genuinely believe they are now “artists” when before they were not. But the rhetoric around AI encouraged this.It’s dangerous and deceptive.Why are people still promoting generative AI? When the risks are so high?I find it incredible that people are still promoting chat bot AI like this when these psychosis results are so prevalent.It reminds me when the science of cigarette smoking and cancer became known and folks were talking up using filtered cigarettes and “cutting down” on how many cigarettes they smoked.Sure. Go off and smoke. But recommending it to others? A product which when used exactly according to the manufacturers intent causes fatal illnesses?Giving it to kids?Conclusion: Regulate NowRegulation of AI is beyond urgent, it must happen and is happening now. The fact it’s happening piecemeal; state by state, country by country is fine. The complaints of AI companies demanding federal and singular regulation are pathetic. States lawmakers are responding to the desperate calls from their constituents. They would not need to if strong federal laws were in place. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

Why do investors think AI companies are like Uber or AWS (big spend for years, big profits in the end) when the truth is they’re more like WeWork, FTX and Theranos?If you are not an investor, but someone who uses AI — please, for the love of all that is holy — divest now. Delete the app going into the holiday season, and learn to live without whatever you thought it was doing for you. Call your family or friends, read a book, or walk in the sun.Because the damn thing is going to cost you an arm and a leg by this time next year; or it will be dead in the water and addicted users will be left to pick up the pieces.OpenAI is nothing like UberThere are plenty of commentators — link and link just to show I’m not making up straw men — discussing how OpenAI is like Uber, which famously burned cash for many years before becoming profitable.As I mention in the video, Uber had a big “boots on the ground” cost: each new country they entered was massively expensive. I know, I saw it first hand. But it had a real revenue model — you pay a driver, you get a ride, Uber clips the ticket for a percentage — and although the IRL costs dwarfed their technology costs they were eventually able to see revenue greater than OpEx after many price corrections.Here in Brisbane Australia, after Uber broke the ground with massive underwriting of rollout costs — phones for drivers, local regulations impacting drivers via traffic fines — they were the only game in town for many years. There’s no Lyft here, smaller players struggled to start. Those local monopolies though temporary gave them a path to profit. The “analyses” that compare Uber to AI companies and say “give them time” to reach profitability ignore the fact that there is no country by country rollout, no category killer effect for AI. There’s no “moat”.If Alice uses AI for writing she could use ChatGPT today and whatever specialist wrapper for writers tomorrow, and then she can switch to some new startup’s product the next day. There’s no first mover advantage.Plus as Ed Zitron has extensively reported, AI costs per inference are massive, so the unit economics are just awful. Users on $200 plans are costing OpenAI money. The latest reports of AI cash burn is that its more than Uber and AWS put together. So these comparisons are spurious, and as far as I can tell there is no path to profitability.Financial times report on OpenAI revenueSpeaking of Ed Zitron, as a prominent commentator he’s collected financial information from AI industry insiders relating to OpenAI and Microsoft Azure cash burn. The figures published in the Financial Times show a picture that any investor should be terrified to see: the costs of inference which is just the outputs, customers of an AI SaaS product using the service, is costing so much money that any chance of closing the gap to profitability is vanishing over the horizon.Australia and AINumpties in the Australian Government have announced a myopic plan to buy Sea Monkeys — err, I mean, build AI data centres — here in the lucky country.They say its to “serve” Australians. I want to see some openness about which lobbyists Minister Ayers has been talking to.Their road map is a road to nowhere. And I hope they realise their mistake very soon before my tax dollars go to underwrite this madness.TheranosOpenAI promised Artificial General Intelligence, a computer system as capable as a human — they have done exactly what Theranos did: failed to deliver on what was always a believable sounding pipe dream.Professor Gary Marcus — who was appallingly derided by industry figures, until eventually everyone started apologising to him and saying he’d been right all along — compared OpenAI to Theranos back in 2024.Why do people believe Sam Altman when he says a computer is going to be as capable as a human being? The LLMs store information, and every time they claim to be able to pass PhD level tests it turns out they can only do that when they’ve memorised the answers. The whole thing is a scam.ConclusionI’m done being polite about this. Especially when the sums of money are sufficient to ruin whole countries, and to bring the world economy to its knees.This Dec-Jan make a New Years resolution to divest out of AI. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

I worked on Google’s ads serving infrastructure as a software engineer at their Mountain View, California HQ from 2007-2009. The system was complex, and included two AI clusters involved in predicting the best ads to show against a given search result.Google also had a very innovative market for ads, where advertisers could bid for words that folks searching on Google had used in their search terms. AI actually works really well in tandem with search.One thing advertisers want to know is what their products are appearing in response to: its called targeting. But generative AI is driven by effectively random numerical models, and affected by large context windows - what if someone mentions “rope” in a chat? It might not be appropriate to show them an advertisement for a hardware store that can ship them rope. It could be incredibly bad.Plus Google built their framework on their own servers, with their own market, and owned everything about it. OpenAI is going to be a customer - they’ll have to outsource, maybe to Shopify or even to Google. So there’s a chunk of their revenue gone already. I honestly don’t think they have any idea what they’re doing here. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

Data centres for AI use huge amounts of electricity and cooling water. But there’s folks that want to muddy the waters about these facts, and so they are using AI to generate junk science. As I say in the video quibbling over water that is “consumed” versus “withdrawn” is pointless argumentation used by AI boosters to cloud the facts. When climate change denialists wanted to attack clean energy initiatives they took aim at Toyota’s Prius hybrid saying the Sherman-tank-like Humvee was “cleaner”, and they used a junk science report as the vehicle. Of course the truth is not that the Prius is cleaner (it is) but that creating fake culture wars over cars like this is a way to distract from sensible advancement of facts.Humvee vs Prius is now Data Centre Water & Power UseYes, data centres stress the water supply; but no your individual ChatGPT app use is not going to empty a lake. If you’re an AI vendor’s PR company you influence those talking sense about AI data centres impact into defending a dumb position that is irrelevant to environmental regulation of AI companies, then you’ve won your battle. Pick some fringe junk science study that would normally sink without a trace and arrange for it to get inflammatory press. * 6 months ago I posted about the environmental impact of AI data centres* Someone posted in response “LOL, No” - and linked this paper:* “The carbon emissions of writing and illustrating are lower for AI than for humans” It’s in Nature a prestigious journal… so case closed?* Trouble is the whole thing is junk science as far as I can tell* In the Arxiv PDF for the pre-print of the paper they admit they used ChatGPT to generate it.* That admission is gone from the Nature scientific publication* Junk science from paper mills and eager grad students wanting to burnish their careers with ChatGPT generated pre-prints have been reported even in the mainstream news as a huge problem for science.* The “LOL” guy linked to the “Nature” version of the junk science report which is easily mistaken as being part of the journal Nature, but is in fact one platform of the publisher Springer Nature. * Its not rigorously peer reviewed, and you can “pay to play” if you want something in “Scientific Reports”* But surely it can’t be junk science if its published in Nature? Its peer reviewed!* Folks in PubPeer who try to do rigorous academic peer review are inundated with bad faith AI generated “papers” and despair of correcting junk science that finds its way into the Scientific Reports. Which is not Nature.* They report “tortured phrases” such as saying “profound learning” instead of “deep learning”. These clearly the paper’s purported authors didn’t write it, but actually outsourced to generative AI. * The peer reviewer also point out corrections required to very imprecise terms such as “AI” where it should in fact read machine learning. So the articles are junk, but worse they are AI generated junk.So Scientific Reports can in general be junk, but what is the relevance to the “carbon emissions of humans” paper?* Here’s a section from the ArxIv version of this paper:Despite these current and potential future forms of societal transformation and harm, profound benefits to society could accrue through the use of AI. AI? What kind of AI? Writing and illustrating are generative AI, they are LLMs. Imprecise terms in a “scientific paper” are a giant red flag. And “profound benefits” — like what? The footnotes 26, 27 and 28 don’t point to page numbers.Are Some People Actively Trying to Rewrite Ground Truth?In a 2025 article Nick McGreivy, PhD Princeton, reported on a trend noted by many others: AI “science” wasn’t working. Even Google DeepMind who trumpeted finding new materials with AI, later admitted the findings were mostly junk and not new materials at all. AI papers had “data leakage” problems and the results don’t replicate.But in 2023 according to an article on the University of California, Irvine website these folks pictured below all got together and decided — golly gee — to see if they could write scientific academic papers using ChatGPT. Tomlinson who has the first name on the “Carbon” paper has the computer science credentials but Torrance is a lawyer. Black is a professor under Tomlinson at UC Irvine, working in computer science with an impressive resume of writing about Harry Potter fan fiction and education, while Computer Science Professor Don Patterson is very interested in how AI can be leveraged to seed fake news by publishing fake scientific papers: Patterson also has a Blockchain startup. I don’t mean to make light of their credentials — especially Black, as any woman who’s gotten to her level in AI deserves credit — but the whole thing has a playful air to it that doesn’t fit with how seriously everyone has taken their paper.What the Heck is Going on with AI Science?In this thread back in June I commented on the “carbon” paper. In that thread I linked a most thorough and data-driven peer review of the Tomlinson paper. The numbers quoted used a LCA toolset; linked to a Github showing methodology, and showed order of magnitude differences in the consumption figures:Weirdly not only is this piece from Rockpool.tech gone, the site is deregistered as a domain name. To make matters worse the InternetArchive is down and I cannot find any archive of the rockpool work. Its just wiped off the face of the internet. I’ll update this if I’m later able to locate it. The Carbon Emissions Paper is not LCAThere are a number of other damning critiques of the “carbon emissions” paper. Professor Stefan Pauliuk, of the University of Freiburg, an expert in Life Cycle Analysis of environmental impacts called the paper worse than a “student term paper due at the end of a two week block course on LCA”. He tears into the analysis of the human carbon outputs as: a time-based downscaling a person’s average annual footprint to the one hour of writing a page of text is not appropriate, as this footprint includes … things that are clearly not attributable to the writing and painting processIn other words its about as accurate as any ChatGPT generated student term paper. Weirdly when you read the UC Irvine article, and the notes at the end of the Arxiv PDF they express worry that they avoid any unintentional plagiarism as being much more of a concern than the accuracy of the content. The fakery around Hummer vs Prius was exactly this redrawing of all the boundaries of the analysis along lines that completely warp the figures. Pauliuk points out how bogus this is in the case of the “carbon of AI vs humans” paper.The Paper’s Data is DoubtfulSince the paper refers to random articles on Medium as authority for its data, I’m going to do the same: here’s an article that points out how the framing of the carbon calculations is wrong. It is based on figures from that medium article and calculated out from there: see all that “derived from above” then the Chris Pointon medium article link: The point of this is not that Pointon’s data is wrong: its just that OpenAI is not releasing any data about its carbon footprint. No-one knows what it is. And bogus math obtained by dividing the total carbon of the whole USA by the US population is not helping clarity.Don’t Use ChatGPT for Science - it Doesn’t Work* As above with McGrievy’s article, you cannot generate science with a large language model and expect it to hold up. The authors of the “carbon” paper say they edited it heavily, but they also say they regenerated a whole new draft. Basing all your data off a Medium article seems very ChatGPT to me.* Google is actually doing some proper LCA on Data Center resource usage: look there instead of dodgy ChatGPT generated papers. Their usage will not be very representative since their hardware is different to ChatGPT. They have their own TP… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

Hey all, I’ve looked better and felt better! But after getting out of hospital I just wanted to show that I am on the mend, and will be back writing in the coming days.Hospital stays are not fun, and for 7 days involving rigors and being on a drip was awful. But I’m back, and getting healthy again.Also I have a qualification that I’ve been studying towards (to allow me to teach in Australia, in tech) and that has meant doing practical exams that also took up some of my very little energy. But now that is done for 2025 I now have time to write again!Thanks for reading Totally a Thing! This post is public so feel free to share it.But you are not here for my health! What on earth is Minister Ayres and the Aussie government doing with AI now?The Australian AI Roadmap* Reuters - “ramp up adoption, rely on existing laws”* The Guardian - no guardrails needed * ABC news on the roadmap - see images below:At the present time generative AI, of the sort that can rip off books, is a violation of the very concept of intellectual property; its a digital slave ship for desperate data-labelling workers (many in the developing world) and overall its a mechanism for wealth transfer. It’s an extractive process that rips-off regular folks, and transfers wealth to tech oligarchs, huge investment companies and corporations.It’s absolutely outrageous that the Australian government is stepping back from its responsibility to protect our local Aussie intellectual property from extractive AI companies. It’s unbelievable that public servants like Minister Ayers (who I mispronounced as Akers in the video, sorry) are so starry-eyed and delusional that they think “AI adoption” will somehow benefit Australians. It’s a massive breach of trust.When AI can be GoodI am pro technology, and pro computing solutions that help businesses and people. But it is not being a “luddite” or a “hater” to point out that huge over-heated startups running on investor money, stealing IP are doing it because they want to “win”; to become a “Google search” for AI, so that no-one goes elsewhere than them. They want a “moat” as they say, a monopoly. This idea is both really evil, and really stupid.The bad:* Big AI vendors trying to corner the market* Billion dollar corporations profiting from stolen IP* AI corporations addicting users, and corrupting government to get their way The good:* Regular companies using AI tech in house as part of a compute solution* Software engineers helping Aussie industry deal with Big Data via AI* Using AI (not just LLM/AI from a vendor) as a component of real solutionsWhat I see as the biggest problem here is companies in Australia hearing that the Government is supporting “AI” and then rushing out broken, failed “solutions” that just wrappers around US based big AI vendors products. * Agents do not work. They are garbage.* Software professionals are the only way to get compute solutions that work.Stop talking about “adopting AI” and start talking about building solutions - using the right tech for that, whatever it needs to be. Use AI for what it’s good at, for example indexing big data, and search. No-one has “cornered the market” on AI and they never should, and I doubt ever will. There’s no “AI race” to win. It’s nonsensical. Just look at DeepSeek.So many decision makers have been deluded into thinking “AI” is some kind of glowing blue plastic person - a self-deploying solution - that will walk through the door of their company ands start typing on their keyboards: So many lies from the AI industry; and I’m afraid the Minister here is swallowing it hook, line and sinker.AI Safety InstituteOne bright possibility is the creation of the “AI Safety Institute” which could be an NHTSA for AI. If there’s an accident and people are harmed you have a technically competent and responsive body, independent of industry who can step in to analyse what happened, what the harms are, and recommend any legal action needed.I am currently working on a series of articles about the mechanisms of AI regulation in AI vendor companies, and the analogy of an NHTSA for AI is one of the possible mechanisms that has promise. Stay tuned for that.The National Highway Traffic Safety Administration works in the USA when companies make unsafe cars. And the same model (a qualified body of technically competent responders and investigators) can work here and elsewhere to monitor unsafe AI.The AISI seems to have a reasonable charter at present, but my concern is that it could have the same fate as the Climate Commission in Australia which was set up by the Labour Government to monitor harms from polluting industries: it was destroyed for political gain by an opposition government looking to cosy up to industry. The outrage from the public when this happened, by the way, was so huge that public campaigns to keep it open resulted in the staff who left the commission starting the Climate Council to continue that work with crowd-funding.It would be so easy for the AISI to either become a channel for AI industry propaganda to go into the ears of politicians; and effectively become an industry group itself; or if its actually effective in protecting people that a subsequent government turns it into a political point scoring exercise.Theft of IP by AI has to be LegislatedAs I mentioned in my video legislation had to be added to the century old crimes act to specifically include electric power as a thing able to be stolen. This happens often with technology and we’ve been down this road before. It’s not radical to legislate to clarify what can be stolen, even when “the law already exists”:The idea that “existing laws cover this” are garbage because what happens is small creators get into court and face billion dollar AI companies hiring an army of “AI Experts” to say they did not steal the creators intellectual property.Of course they stole it. They took our books, our images, artworks and our online articles against the explicit terms that those are things over which we retain moral rights & copyright. Billion dollar US based companies encoded that into their AI models, stored in to data centres to be served up for their own profit.But decisions are going against creators because there is no legal clarity around what constitutes illegal copying, even though judges are making positive findings, it’s taking years for any justice. Mind you, in this linked case, I agree with the judge that the correct argument to make for harms and damages against AI companies is that they are flooding the market with competing AI slop products directly generated from the IP that they stole. The deceptive practices of AI companies trying to hide their crimes goes to intent — in the same way cigarette companies hid what they knew. But for copyright law you are stuck proving that your IP was taken and that you were harmed.Decisions sometimes go against authors and creators, and sometimes for them. It’s a mess, and law is necessary to fix this so its clear that:* AI encoding is not transformative, it is storage and it is illegal copying* Taking IP and encoding it is substantive & not some 0.001% as claimed.See my previous post on how this works:Vulnerable Folks and AIAI vendors are never going to do the right thing for at risk people unless regulation forces them to. OpenAI says millions of users are discussing suicide, but their response instead of hard interlocks to deal with this, is a “Wellness Council” and controls that push the problem back onto parents. Note that in my upcoming writing on AI regulation I want to cover the problem of “AI Alignment” which is the stupid approach of trying to get LLMs to behave via “system prompts” and fine tuning. We need symbolic/procedural computing solutions that allow no statistical hacks or failures: a “regex” or behaviour tree that can catch these sorts of conversations external to the LLM and route th… This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

Working through a series by Lesser Dog Tutorials on how to make a LucasArts style point and click adventure in Blueprints, I have instead of using Blueprints I have been translating everything into C++, and it’s so much fun.Totally a Thing is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.Feels like doing a crossword and a half-dozen other fun puzzles besides. Today’s solution found was uncovering how Unreal dispatches Interfaces when they’re implemented in C++, and overridden in a Blueprint, but called by C++.Turns out they have this — as far as I can tell, undocumented — feature where you call a static method on the interface implementing class that is generated by Unreal’s Header Tool (UHT) so: HotSpot::Execute_LookAt(CurrentHotSpot)That gets the job done, to have my little man read the poster response from the blue print via the Blueprint visually coded look at event.Blueprints are amazing, and have their place, but complex logic in blueprints gets into a snarl of spaghetti so quickly.Code is here:* GitHub - Unreal Adventure Game Don’t worry - I will get back to dishing on AI shills at some point in the near future. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

I have been struggling to write this article, and figure out what to do with this cartoon / diagram I made about the alleged use of stolen books by AI companies for weeks now. The sheer scale and brazen bullying greed of these guys just astonishes me.I believe startups and technology have the ability for great good. But it’s being used to enrich the most powerful at the expense of the little guy.This diagram sums it up in one image. See my video for an explainer.ReferencesArticles I reference in the video:* The Atlantic, The Unbelievable Scale of AI’s Pirated-Books Problem* by Alex Reisner* Authors outraged to discover Meta used their pirated work to train its AI systems* by Nicola HeathThe analysis of how AI companies refuse to pay for data when they spend big on engineers and data centre costs was inspired by Ed Newton-Rex’s Ted Talk:What can We Do?The most important thing is we are not powerless. The number of daily users of the products these guys have is an instant stab of pain for them every time it dips. * ChatGPT - you don’t need it, delete your account* Facebook / Instagram - migrate to Bluesky, and SubstackCalling out everyone for using toxic AI products and just politely pointing out the harm — we could lose our book industry — is worth it.See my article from last week about pushing for regulation on AI companies.I’m loving the new campaign from Australian Society of Authors and other groups trying to throw light on these problems.Vote for fair AIThis year is an election in Australia, and there’s a chance to push for policies that support books, readership and our local bookshops.Don’t let them die. You’ll miss them when they are gone.Thanks for listening.Totally a Thing is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com