kristopolous
Other datacenter prices aren't up 500%. CPUs, tape storage, mainboards, racks, cases, PSU, infiniband hardware, HVACs, all price stable.
But consumer grade RAM? Up 500%. USB disks? They're hundreds of dollars.
Also low density platter is up 2-300% as well. Don't tell me people are building out datacenters with 16tb spinning disks, I don't believe you.
It's companies that have price fixed doing it again.
Their EPS are up like 60x. Their stock is up like 10x and we're supposedly like "well I guess this thing you could sell for profit at $20 is now $960, cool"
And what's micron doing with it? $35 billion stock buy back. Alright. Asshole prices it is.
Prices of cheap 10 year old tech continues going up at crypto asshole speeds. Everyone should be insanely angry.
These prices aren't going to stop until either a non cartel player enters the market or there's a DOJ that's not just a bunch of kalshi and crypto scam bros.
We'll see what happens when everyone hears "black Friday? Nah, bro, prices are up another 20%, lol"
boom, jan 2007: https://www.eff.org/deeplinks/2007/01/effs-sweet-16-party-su...
Apparently there's a mitch kapoor connection between the two
In practice what the "small government" ideology actually does is get rid of these accountability mechanisms and put scammers and corrupt people in charge instead or relabels corruption as the policy through public-private partnerships.
https://jeffwerner.ca/kryptonite_lock/
People swapped out their bike locks en masse, it was a pain.
Been around over 20 years
Maybe the technical stuff is more accessible then it was beforehand but the thousands of hours of the other stuff that leads to good software is still thousands of hours.
Quality has become less work, but only by a little.
But this doesn't mean people value the craftsmanship because they never did before...
It's always been simply a popularity contest and now it's that more than ever.
The question is, why are you spending your time building things? Is it to make things that are finely crafted? Is it the road to wealth? As a tool to get other work done? Just make sure you have an answer.
It's your time after all...
software has always been a popularity contest
It came out of a lack of their being an LLM-native tool that feels like cURL or other unix-friendly things such as awk or sed.
I wanted something stateless, explicit, low-level, and composable without "settings" just like you don't with grep or sort.
Instead, you can index into files with jq or line numbers and save invocations for later for instance
llcat --save invocation.json \
-k "@~/credentials.txt:12" \
-u "@settings.json:.[3].host" \
-s "@system_prompts:8" \
-m "@settings.json:.[3].model" \
"@query.txt:12" > output.txt
llcat --load invocation.json { ... parameters you want to override ... }
I've put a lot of effort into this and use it daily. I really wish people would give it a go and give me feedback and file bugs!
this one is especially annoying ... i can run these on lots of hardware but there's vast differences in many aspects.
Whether I'm getting a good deal isn't what I'm thinking about if I'm getting garbage results. It's more about why am I wasting my time on the wrong tool.
I mean what I really want is a more reliable classic eval benchmark for the decision models that I can easily deploy, like the official ollama ones: https://ollama.com/library?sort=newest
perplexity has a 27b decision then there's a 3b from liquidai and another 600M one. Then there's one called kev and bespoke has nimble-v3. clef from cloudflare is there along with strands-decider from aws. Then there's laya and tev1 ... I mean the AA page seems to be the only real third party doing benchmark and evals on it but most of these are missing from it. It has a lot to be desired.
And you go over to HF and it's even more mysterious https://huggingface.co/models?pipeline_tag=text-classificati...
Maybe an agentic eval framework that tests anything that hits 100 downloads would be a decent weekend project...
Also there's like wildly different tasks. choices versus noul have way different performances and also if you are doing things like text classification such as using it for theme extraction, I mean this does work with nimble-v3 but d1 gives mostly noise and pplx-decider is in between and their performance degradation curves are dramatically different depending on the input-token length.
This is all going to be bitter lessoned eventually but I can't just sit on my hands and wait around 12 months.
So now I have a proxy that looks at the input size and do dynamic routing and sometimes fire off both and do ensemble weightings ... I mean why do I have to be clever here, this is a generic problem.
I had to discover all this myself and I really shouldn't have to do that...I guess I'm just expecting some quant results from AA that they aren't in the business of providing.
They have some math here https://artificialanalysis.ai/methodology but ok how are the prices weighted among the providers... and even if you do that 7:2:1 math it doesn't match the providers. Specifically the model in question: https://artificialanalysis.ai/models/deepseek-v4-1-flash-101... doesn't match the first listed provider https://deepinfra.com/deepseek-ai/DeepSeek-V4.1-Flash-1017
Do the math yourself, it's actually $0.08 not the AA claimed $0.10 (and that's the more expensive of the two pricing tiers). That's a pretty large error bar.
Also many of the providers are now doing "peak" and "offpeak" pricing (deepseek.com itself for instance) so it can differ dramatically. These "peak" times depend on where the service is most popular. You can basically ride "off-peak" pricing by cascading from global provider to global provider.
The majority of the time taken in an inference call is doing the inference so the extra latency of traveling across an ocean is actually a fairly small wall-clock tax here. People still seem to be partial to nearby servers for ai-related workloads but really, try some decent ones 10,000 km away, you probably won't notice.
I know AA thinks highly of the price metric (I've been to their events, even have their hat, they brag about it) but I have learned to ignore it as noise.
It's an especially strange metric with open-weights decision models. I mean I can run those on my laptop ... cost is very abstract there.
But for closed-source models with only a handful of providers (sometimes just 1) great, the price is a very reasonable number.
For the open models a more useful thing would be like "100 tok/s spot price build" and then it's using pcpartpicker or something in some kind of partnership and can query a build for buying a machine right now that can achieve a certain benchmark number and maybe something for less interactive workloads like "5 tok/s". Of course getting some base unit of power cost in there might be important as well since the power bill is the majority of what you pay for when you rent a commodity gpu.
Depending on who you sign up for, that's within the introductory credits (usually like 5-10$) so it's actually $0.
People with Nazi tattoos who make songs called heil hitler and chanting "Jews will not replace us" on the far right passing around protocols of the elders of Zion calling human rights watch "the actual antisemites" doesn't make it so.
Words have meaning and lying criminal hate mongering scam artists who are viscerally and proudly antisemitic slandering human rights organizations... I mean you take scam artists and conmen and all of a sudden there's this shroud labeled political identity that they tie around their hips and now their crimes qualify them to scam the civic government?
No.
But no we're like "oh so we have one candidate that wants to adjust the budget to fund some libraries and this other one that wants to build concentration camps and imprison transgendered people. Alright, let's allow both on the ballot"
Being a violent hateful ignorant bigot isn't a qualifying condition for administering civic life.
that modality is long dead. we still have email but it's not that same ... I however could take mirc from like 1995 and still use everything.