asjir
asjir on Hacker News
Exactly, they put a lot of money into engineering and it does give results
Just keeping it up to date with competitors is much cheaper, by copying better ones like Qwen did with Claude. Also a bunch of research is trickling into open source / arxiv so catching up should continue becoming cheaper at least as a fraction of training from scratch
maybe you can preselect good ideas, build up guidelines describing most common pitfalls, extrapolate from ideas you already vetted etc and run on autopilot on a safe-ish subset
But what if it's a bubble driven by speculation?
It wouldn't pay off.
Starting a futures exchange on RAM chips, on the other hand...
Nvidia's parakeet dropped recently with better performance and 0.6B params, so the rate of progress here looks good, probably next year (or mby the year after) they'll be running no probs
Not really answering your question, but:
One completely imo unnecessary category of sloppy software is electron apps.
It's totally ridiculous how little resources are put into alternatives like tauri given how most dekstop apps run on electron, and we know how bad it is.
I read this comment first and thought: "oh come on, how bad can it be". So then seeing this really being so bad(-ly overdesigned) was quite amusing.
I'm not disagreeing per se, but in my experience my tests are so disposable they're never worth investing that much engineering effort in past getting them to pass
The author makes an argument that at least looks like people choose YT over Nebula because YT is free. I, for example, already pay for Nebula, so I can watch it for free, but I still go to YT.
IMO it might be just a product problem. I opened nebula and:
* The same video had a better title on YT that was actually less clickbaity and more informative - assumedly because of YT algorithm for optimization
* Nebula auto set quality to 480p compared to 1080p in YT - if I wasn't tech-savvy I'd assume it's just worse quality.
* The loading times when you seek to part that's not loaded yet are 10x longer
The recommendation algorithm is weaker too, I can't tell to what extent this is due to YouTube having simply more data and to what extent it's weaker engineering.
If they do some decarbonization real estate SaaS and need connections with both regulators or whoever issues certificates and REITs etc, then it makes sense. I don't think they need a lot of software for that.
To expand upon the other comment:
Indexing and multiplying with one-hot embeddings are equivalent.
IF N is vocab size and L is sequence length, you'd need to create a NxL matrix, and multiply it with the embedding matrix.
But since your NxL matrix will be sparse with only a single 1 per column, it'd make sense to represent it internally as just one number per column, representing the index at which 1 is. At which point if you defined new multiplication by this matrix, it would basically just index with this number.
And just like you write a special forward pass, you can write a special backward pass so that backpropagation would reach it.
I'd be more concerned about the size used being 70b (deepseek r1 has 671b) which makes catching up with SOTA kinda more difficult to begin with.
In a 20-person office you'll pass by like 2 people on the way with the card, so it's not a problem. If the office was bigger, and it was a problem, you could just designate a spot for the card next to the pantry.
Ahh, I see, thank you (and other commenters)
I was a bit surprised by the change in tone in a way I probably wouldn't've been if I'd read chronologically.
Why not start with Colour of Magic?
For context, I started with it, then Light Fantastic and I'm finishing Sourcery now.
His argument is that it's effectively a legal moat now that protects monopoly. Like we shouldn't accept that you need to break the law to have a chance to compete with them.
I thought about it for a full day, and I have one idea for how to handle copyrighted data training.
It would need to be open / regulated and training till double descent would need to be disallowed, to make sure that the model is not memorizing the data.
I think this was overly self-critical - what would researching fully even mean? They can have however many sheep they want hidden within one-mile distance from Trafalgar square, no-one would expect the author to scour every possible location ensuring that there is no sheep hidden lol just to make a funny post 100% sure to be true
The compound: state hate crime victim numbers
Translated into Polish is:
liczby ofiar państwowych zbrodni nienawisci
Which translates back into:
numbers of victims of governmental crimes of hate
So except for the state turning into governmental makes everything a genitive case. I didn't notice the relationship until translating
Reminds me of Go being released without a package manager because at Google they didn't need it lol
What codebase size do you have?
Actually, only today I noticed that Cursor's Agent Composer now can properly respond to linting errors.
In my typescript codebase case, it solves a lot of problems, it probably helps that I use tRPC type rather aggressively (i.e. using UUID types, separate dates and datetimes etc).
A few weeks ago it wasn't working nearly as well in my experience.
Exactly, especially when the clouds are so low you can't even see where the sun is shining from (common here in late autumn)
Oh, on the reversible webpage it's written that categories are by users so it's like wiki - At first I thought that it's done by any user and not pages authors. But now I'm interpreting it as just authors, just like in Octothorpes.
In Polish we have: a bad ballerina is hindered by the hem of her skirt
Yeah, interesting why there isn't a US version of hybrids yet