Showing posts with label llm. Show all posts
Showing posts with label llm. Show all posts

Saturday, May 16, 2026

If an LLM is too expensive it won't be next year

AI can cost more than human workers now

This is an Axios headline but the tldr is that once an LLM can do your job it will undercut you the next year.

This is the article and it focuses on token and infrastructure costs. Not cleaning up messes or other times a person needs to be in the loop. For example "Uber's chief technology officer already blew through his full 2026 AI budget due to token costs"

Example of someone thinking tokens will stay expensive. 
No shade, it is just my belief they are wrong on this one claim. 

There is a 7 year and a 4 year trend of the price drop to 1/10th a year for a set quality of token. As in $1 million become $1 hundred cost in 4 years. This article looks at gpt 3.5 level tokens from late 2022 to late 2024


Example price decline in 2 years
But the deflation keeps going Qwen3.5 1.7B–2B is well above gpt 3.5 level in tests and in usage. This you can run really fast locally on any newish laptop or smartphone. Tokens at this quality level are now just electricity costs. That's less than 4 years data but gpt 2 level has followed the same trend for 7 years. 7 more years of the trend turns $10 million cost into $1. Lindy Effect says a reasonable first guess is if a trend has gone on for X length of time assume it will keep going for another X unless theres a good reason for it to stop. Coding and Video Image generation are still improving so fast there is not great reason to assume they will suddenly plateau yet. And the same for efforts to make smaller models smarter using big models. Teacher->pupil, chain of thought, distillation, pruning, low rank factorisation all keep getting better in a way that suggests that smaller models will keep getting better even if big ones plateau.

This is not an argument that LLMs in general will keep getting better though they will and that will help. Or that GPUs will keep improving though they will and that will help. Or that open weight models will stay about 9 months behind state of the art apis though they will and that will help a lot. Or that methods to condense smart models into smaller less demanding machines will improve though they will. It is that all these and a few other trends combined will work together to make a million tokens with a certain score on metrics and in a users opinion an order of magnitude cheaper every year.
Slop Tsunami comes down to the cost of tokens coming down.

The cost of tokens will come down. And at such a rate that it makes Moore's Law look tame. It won't go on forever but it is on a seven year trend with good reason to see it keeping going for at least 3 more years. At which point todays best current token will be 1/1000th the cost.




Thursday, February 19, 2026

Fake People Everywhere

Soon you will only trust people you know.  



I have been learning a fun programming language called Gleam. And I wanted to read a book on it. 
The one book on it I could find was by someone called Julian Lornfield. Who has written over a dozen books in 2025 each on a different topic in computers. Theres no photo of them on the internet other then this amazon author photo. No linkedin. No youtube talks at conferences etc.



So baring some major mistake by me this is a bot churning out AI written books. It could be a real person but one without the usual trail we leave. A fake author would be odd but not something that unusual in 2025.

But to take a step up from these checks. What happens when fake reviews, linkedin accounts and these things I checked are much easier? If there was a linkedin, lots of reviews, better spacing of the release dates and even videos of talks would I have been fooled? Probably.

Making a fake account involves making photos, passing captchas, sending text back and forth between some other bots and some real people. All this is easily doable now. 

AI LLM arguments focus on the abilities at the extreme. Writing a new math proof or acing some test. But what happens when they get really good at stuff designed for normal people to easily do. Gmail wants you to get an email account. Amazon wants you to leave reviews. Emergency services want you to ring when your house is on fire. 

When LLMs get good enough at these tasks that Social media, reviews, email inboxes get flooded you will not trust anyone in the digital space you do not personally know. They pretty much are already on Twitter and facebook but other digital locations are next. The cost of an llm for the same level of intelligence decreases 10 fold per year. So if it is too expensive to build up 100 bot reviewers today it will be 1/1000th the cost in 3 years. 

For online review sites this is probably fairly obvious. But I do not think the effects on offline services are considered yet. 
999 services or dentists are not designed for 100 bots that sound like people ringing them.
Politicians and newspapers are not expecting 100 physical letters what have all been written by different bots.
If we get 100 fake ads from what looks like our bank we are likely to get one just when we are having a problem and at our most confused moment believe them.

LLMs are smart enough now to create and operate online accounts including making phone calls. The price of this will drop orders of magnitudes in a few years. The online and especially the offline world is not set up for when this happens. 



 

Wednesday, April 30, 2025

We have a 'Can I get a Loan?' Problem

I made a chatbot for a bank a decade ago and one answer was getting terrible ratings from users. I looked up the questions being asked that lead to the answer and they should have gone to that answer. I checked the answer in case it was rude, unclear or missing details and it was not.

The 'bad' answer was not wrong it was just what people did not want to hear. It was telling people they were not qualified for an immediate loan and giving details of how they could try a slower method to get a loan. The answer was not wrong just not what people wanted to hear. 

LLMs have gotten so good at telling us what we want now that they just make stuff up more than they used to. There is a great article here on the increasing misalignment. Hee hallucination growing is an indication the LLM is making stuff up to make us happier.


We should reward correctness (including in the steps getting to the answer) than people liking the answer. But the incentives of the testing mechanism and more seriously the companies making LLMs do not do this. If users prefer LLMs that tell them they are great and give plausible sounding reasons why they should do what they want to do these will be more popular. 

Dirk Gently's Holistic Detective Agency by Douglas Adams has a program called REASON that LLMs are turning into. It which would take any conclusion you gave it and construct a plausible series of logical steps to get there.




They will give us a series of plausible sounding steps that will make us happy. It is us who are misaligned not just the LLMs