g-mork

It is hilarious, because it is blinded by our own self-imposed optics. It has been our policy to import droves of immigrant workers who have little hope but to take up gig economy jobs often illegally and remain fixed at the same (or worse) levels of economic status as the day they arrived in the country. Yes in Dubai they simply confiscate passports. At least they're honest about it

This is a hilarious comparison given Amsterdam's own history with regard to immigration. Not even historically but contemporarily too.. Just Eat, probably the largest employer of bargain bucket labour across Europe today is headquartered in Amsterdam

It's possible that it's simply paranoia, but moments where Opus starts acting like Haiku seem to correlate with periods of higher latency and HTTP errors. Don't like reporting this because it's so hand-wavy and conspiratorial, but it's difficult not to think they're internally using extraordinary measures of some sort to manage capacity.

But even when Opus is running healthy, it still doesn't address the underlying issue that these models can only do so much. I have had Opus build out a bunch of apps but I'm still finding my time absorbed as soon as it comes to anything genuinely exceeding "CRUD level difficulty". Ask it to fix a subtle visual alignment issue, make a small change to a completely novel algorithm, or just fix a tiny bug without having to watch for "Oh, this means I should rewrite module <X>" is something that simply isn't possible while still being able to stand over the work.

It's not to say I don't get a massive benefit from these tools, I just think it's possible to be asking too much of them, and that's maybe the real problem to solve.

Oops, fixed

Speaking only personally of course, I'm completely over the chat idiom in almost every way. Where is all this future demand coming from? By the time Android lands a God mode ultimate voice assistant it's pretty much guaranteed I will be well beyond the point where I'd want to use it. The whole thing is starting to remind me of 3G video calling where the networks thought it'd change everything, and by the end of it with all the infrastructure in place, the average user has made something like 0.001 3G-native video calls over the lifetime of their usage.

Would really love some path forward where the AI parts only poke out as single fields in traditional user interfaces and we can forget this whole episode

My answer to this is simply rolling back to the pro plan for interactive usage in the coming month, and forcefully cutting myself over to one of the alternative Chinese models to just get over the hump and normalise API pricing at a sensible rate with sensible semantics.

Dealing with Claude going into tardo mode 15 times a day, constant HTTP errors, etc. just isn't really worth it for all it does. I can't see myself justifying $200/mo. on any replacement tool either, the output just doesn't warrant it.

I think we all jumped on the AI mothership with our eyes closed and it's time to dial some nuance back into things. Most of the time I'm just using Opus as a bulk code autocomplete that really doesn't take much smarts comparatively speaking. But when I do lean on it for actual fiddly bug fixing or ideation, I'm regularly left disappointed and working by hand anyway. I'd prefer to set my expectations (and willingness to pay) a little lower just to get a consistent slightly dumb agent rather than an overpriced one that continually lets me down. I don't think that's a problem fixed by trying to swap in another heavily marketed cure-all like Gemini or Codex, it's solved by adjusting expectations.

In terms of pricing, $200 buys an absolute ton of GLM or Minimax, so much that I'd doubt my own usage is going to get anywhere close to $200 going by ccusage output. Minimax generating a single output stream at its max throughput 24/7 only comes to about $90/mo.

250 ms f/4 ISO 512000 in case anyone was wondering. I wonder if they applied any denoise, it looks great for such high ISO

Saw a comment here yesterday referencing the Attention Is All You Need paper title in a tongue in cheek way. Kinda fun to imagine the friend/romance angle is just a bunch of socially awkward folk at OpenAI misinterpreting the original paper

Another instinctual reaction here. This specific formulation pops out of AI all the time, there might as well have been an emdash in the title

That etched 3 colour look to this very day remains the peak of modern aesthetic for me. Thought it was so cool and sophisticated when I was 12

I've read so much trump spam recently that on reading this my first thought was that you misspelled winning hehe

Planet announced last week there will be a 14 day delay on all commercial satellite imagery from the middle east. It shocks me how transparent we are about information war and voluntarily lying to ourselves at particular moments

This is far too simplistic, you can't discuss perf per watt unless you're talking about a job running at any decent level of utilisation. Numbers like that only matter for larger scale high utilisation services, meanwhile Intel boxes mastered the art of power efficient idle modes decades ago while almost any contemporary GPU still isn't even remotely close, and you can pick up 32 core boxes like that for pennies on the dollar.

Even if utilisation weren't a metric, "efficient" can be interpreted in so many ways as to be pointless to try and apply in the general case. I consider any model I can foist into a Lambda function "efficient" because of secondary concerns you simply cannot meaningfully address with GPU hardware at present (elasticity and manageability for example). That it burns more energy per unit output is almost meaningless to consider for any kind of workload where Lambda would be applicable.

It's the same for any edge-deployed software where "does it run on CPU?" translates to "does the general purpose user have a snowball's chance in hell of running it?", having to depend on 4GB of CUDA libraries to run a utility fundamentally changes the nature and applicability of any piece of software

A few years ago we had smaller cuts of Whisper running at something like 0.5x realtime on CPU, people struggled along anyway. Now we have Nvidia's speech model family comfortably exceeding 2x real time on older processors with far improved word error rate. Which would you prefer to deploy to an edge device? Which improves the total number of addressable users? Turns out we never needed GPUs for this problem in in the first place, the model architecture mattered all along, as did the question, "does it run on CPU?".

It's not even clear cut when discussing raw achievable performance. With a CPU-friendly speech model living in a Lambda, no GPU configuration will come close to the achievable peak throughput for the same level of investment. Got a year-long audio recording to process once a year? Slice it up and Lambda will happily chew through it at 500 or 1000x real time

CPU compute is infinity times less expensive and much easier to work with in general

Yep same, I'd sooner starve than cut my Anthropic sub

Don't forget Wine ships a faithful notepad.exe reimplementation. It should run just fine on Windows

edit: just checked the version that ships with Steam on Linux, yep, works great in a VM

How does this compare to Parakeet, which runs wonderfully on CPU?

The massive DC overbuild matches demand, prices normalise somewhat in 3-5 years.

The massive DC overbuild does not match demand, prices tank in 3-5 years.

Third possibility: some approach like Taalas renders the current storyline meaningless. Would put 3 in 10 odds of this happening but I'd looove to see it.

Fourth: entire planet gets profoundly sick of emdashes, we all move back into caves and live in eternal gratitude of the moment humanity woke up to how little all of this really matters.

What would that achieve? Here, have 1.5% discount on your subnet purchase

The way I imagine it in 2-4 years we're going to be hit with a triple glut of better architecture, massive oversupply of hardware and potentially one or two hardware efforts like this really taking off. It's pretty crazy we're already 4 years in and outside of very niche / low availability solutions, it's still either GPU or bust

how do you make CC talk via a proxy? I had a few googles for this and got nowhere

I do like the idea of an aftermarket of ancient LLM chips that still have tons of useful life on text processing tasks etc. They don't talk about their architecture much, I wonder how well power can scale down. 200W for such a small model is not something I see happening in a laptop any time soon. Pretty hilarious implications for moat-building of the big providers too.

One of these things, however old, coupled with robust tool calling is a chip that could remain useful for decades. Baking in incremental updates of world knowledge isn't all that useful. It's kinda horrifying if you think about it, this chip among other things contains knowledge of Donald Trump encoded in silicon. I think this is a way cooler legacy for Melania than the movie haha.

The irony with Zitron is that summarising his astoundingly verbose anti-AI articles is one of the most consistently productive uses I've had for AI

cool story! how does this problem differ from the positioning requirements in e.g. DVD or blu-ray

You're not "rolling your own client." You're using a subscription that prices in a specific usage pattern, the one mediated by their client, and trying to route around it to extract more value than you're paying for. That's not hacking, it's arbitrage, and pretending it's about editor philosophy is cope.

Anthropic sells two products: a consumer subscription with a UI, and an API with metered pricing. You want the API product at the subscription price. That's not a principled stance about interface freedom, it's just wanting something for less than it costs.

The nvim analogy doesn't land either. Nobody's stopping you from writing your own client. You just have to pay API rates for it, because that's the product that matches what you're describing. The subscription subsidises the cost per token by constraining how you use it. Remove the constraint, the economics break. This isn't complicated.

"I don't give a shit about Anthropic's credit liability," right, but they do, because it's their business. You're not entitled to a flat-rate all-you-can-eat API just because you find metered pricing aesthetically displeasing.

Imagine having a finite pool of GPUs worth more than their weight in gold, and an infinite pool of users obsessed with running as many queries against those GPUs in parallel as possible, mostly to review and generate copious amounts of spam content primarily for the purposes of feeling modern, and all in return for which they offer you $20 per month. If you let them, you must incur as much credit liability as OpenAI. If you don't, you get destroyed online.

It almost makes me feel sorry for Dario despite fundamentally disliking him as a person.

Far more usable (and older AFAIK) site: https://european-alternatives.eu/