News for hackers. Stuff that matters.

eugenhotaj

Re:Maintaining large-scale AI capacity at Meta on Sunday June 16, 2024 @09:29PM (#40701534)
This is because everyone is training with synchronous sgd. all gpus need to synchronize on each gradient step so tail latency will kill you.
Re:C++ GPT-2 inference engine on Saturday July 01, 2023 @10:33PM (#36557615)
This is pretty cool. I had the same idea but in zig: https://github.com/EugenHotaj/zig_gpt2

Not fully finished yet, haven't gotten around to implementing bpe encoding/decoding and only some ops use BLAS.

In my experience processes like these rarely work out as intended and usually add layers of bureaucracy for marginal benefit. It’s usually senior engineers or middle managers looking for “org wide impact” so they can get promoted.
Re:The Python Paradox on Thursday January 26, 2023 @11:47PM (#34542581)
This post would sound so dumb if it didn’t come from the almighty pg.
Re:ChatGPT Can't Kill Anything Worth Preserving on Friday January 13, 2023 @12:05PM (#34370594)
The issue is not that ChatGPT will kill things off, the issue is that ChatGPT 4.0 will kill things off. If you don’t think that’s a real possibility, you’re sleeping.
Re:Small Teams on Wednesday January 04, 2023 @06:24PM (#34253474)
Some of these are so odd. When Kylie Jenner launched her company she could have sold celery and still made $1B+. It wasn’t because of the effectiveness of her small team.
Re:No, this isn't over-complicated, it's just complicated on Wednesday December 21, 2022 @07:52AM (#34080058)
My experience is exactly the opposite in almost all cases. Most software is much more complicated than it needs to be. Reads like the author is just butthurt at feedback they received about their work.
Re:Ask HN: Anyone else suffering from tinnitus? on Thursday December 08, 2022 @07:28AM (#33907137)
I’ve likely had it all my life, but really started noticing about two years ago during the pandemic. Now I can’t unhear it. Went to a doctor a couple of times but nothing they tried really helped. Thankfully mine is not too bad and I’m mostly able to ignore it.
Re:Stable Diffusion 2.0 Release on Thursday November 24, 2022 @12:05AM (#33727967)
The network communication overhead would be way too high to make this useful. At least for current methods of training large models.
Re:George Hotz is leaving Comma.ai on Monday October 31, 2022 @11:59AM (#33407665)
Source: trust me bro
Re:Whisper – open source speech recognition by OpenAI on Wednesday September 21, 2022 @08:58PM (#32933618)
Now someone just needs to pipe the output into stable diffusion.
Re:10x (engineer, context) pairs on Saturday June 11, 2022 @03:23PM (#31707410)
Just take a look at big tech salaries on levels.fyi. Very senior engineers (L8/L9) easily make 10x the salary of entry to mid-level engineers (L3/L4/L5). For a lot of tech companies, L5 is a terminal level meaning most engineers will spend the rest of their careers there. Clearly the market thinks 10x engineers exist.
Re:Tell HN: GitHub is down again on Saturday November 27, 2021 @04:50PM (#29363917)
Let me guess, someone pushed a bad config.
Re:Show HN: Hamilton, a Microframework for Creating Dataframes on Tuesday November 09, 2021 @08:49AM (#29161656)
This is neat for toy problems but I don't see it working well for "real" pipelines. The magical DAG creation is going to be super hard to wrap your head around and even worse to debug.

This reminds me of an internal Google tool for doing async programming in Java (ProducerGraph or something). The idea was that you'd just write annotated functions and the framework would handle all the async stuff. Wasted many thousands of engineering hours while giving an even worse experience than manipulating futures directly.

> Since the bigger network contains the smaller network, it is perfectly capable of achieving the same performance, so the only reason why this does not happen is that SGD cannot find it.

This is maybe true in the limit of infinite data, but not true in any practical sense, and I don't think it has anything to do with SGD. E.g. polynomial basis functions also have this property, but you can't use an arbitrarily large polynomial order or you'll eventually overfit. You can get a closed-form solution for polynomial regression problems, so no SGD involved.

If your pipeline uses only “classic” ml models, then this won’t make too much sense. It’s mostly applicable to NNs.
I also don’t have any data, but I feel like these types of tactics don’t account for higher order effects of pushing people to more extreme platforms, and in general seem extremely short sighted.
For kernel SVMs, one needs to keep around part of the training data (the support vectors) right? With DNNs, after training, all you need are the model parameters. For very large datasets, keeping around even a small part of your training data may not be feasible.

Furthermore, number of parameters do not (necessarily) grow with the size of the training data, can be reused if you get more data, can be quantized/pruned/etc. There's not really an easy way to do these things with SVMs as far as I understand.

True, but it sounds like you’re just shifting computation from training to inference. And I’m not sure that’s a very good trade off to make, you’re likely to predict on much more data than you trained on (e.g. ranking models at google, fb, etc)
Fair enough, but the number of support vectors for non trivial problems is still pretty large (as I understand but could be wrong), e.g. 20-30% of the dataset. Having to iterate over 30% of say imagenet on each batch of predictions seems unfeasible.
Don’t kernel SVMs need a full pass through the data they were trained on to make predictions? How is that faster?
Re:Show HN: A/B test Hacker News titles with AI on Sunday November 29, 2020 @01:08PM (#25246777)
It's also strange because what they've trained is essentially a recommender system. However, at inference time you're not trying to rank 2 different articles, but the same article with 2 different titles, which I guess you could do, but I'm not sure how well this correlates to your actual ranking on HN.

I guess something more interesting would be to take into account the current front page and see how your article would rank against those, or something along these lines.

In any case, neat idea.

Hi HN!

I’ve been working on the pytorch-generative library in my spare time for a couple of months now as a way to dive deeper into recent work on neural autoregressive generative modeling. The goal of the library is to provide clean, high quality implementations of recent SOTA work as the code for a lot of papers is hard to read and understand. I’m also trying to see what parts can be abstracted and easily reused to make future implementations and experiments easier.

Please take a look! I’m happy to answer any questions and would love any feedback.

Older »