atylerrice
Re:ChatGPT Hardware a Look at 8x Nvidia A100 Powering the Tool
on Tuesday February 14, 2023 @10:03PM (#34799522)
You can also use int8 quantization which comes for free with huggingface and isn’t a huge cost to accuracy. Which cuts the memory usage almost in half again. (almost in half because some layers might stay 32 bit if they make a tiny part of the model)
Re:ChatGPT Hardware a Look at 8x Nvidia A100 Powering the Tool
on Tuesday February 14, 2023 @10:01PM (#34799505)
This is incorrect. Thought would be interesting. Pruning usually removes some parameters from the model (setting certain ones to 0 for instance). The smaller models all have less layers and therefore weren’t pruned but were smaller to begin with.
Re:ChatGPT Hardware a Look at 8x Nvidia A100 Powering the Tool
on Tuesday February 14, 2023 @09:59PM (#34799479)
Well the batch size also helps with making training more stable. So a larger batch size computes the gradient across more examples so it ends up being more stable than only using a batch size of day one. A batch size of one is only fixing the error on one example rather than many.
a larger Batch size can help keep the gpu fed but usually that’s not a problem with these larger models.
Re:Show HN: Create your own video clips with Stable Diffusion
on Sunday January 15, 2023 @03:20PM (#34392898)
I don’t usually comment, but aws is actually rather expensive and i’ve hit this annoying quota problem as well. Been back and forth and still hasn’t been raised. I can also recommend coreweave banana.dev and pipeline.ai as great alternatives. This service looks awesome good luck with the launch!
Ask HN: How do you stay focused with competing priorities?
Between work, university, and a side business, I have a lot of priorities in my life currently. What do you do to stay focused and make sure all of your priorities get the attention they need?
Re:People Who Claim to Work 75-Hour Weeks Usually Only Work About 50 H...
on Thursday March 18, 2021 @01:34PM (#26505273)
I’ve seen this a lot. I think sometimes in my experience it comes down to what is work?
As a developer I would classify a lot of the meetings I’ve had at past companies where it takes up 40% of my time as not really working. Sometimes these meetings push over the hours for the week but weren’t really productive in any meaningful way.
Re:Transformers in Medical Computer Vision
on Wednesday March 17, 2021 @01:43PM (#26494516)
I totally agree but I’m excited for some recent initiatives to fix this. https://www.researchallofus.org/ This research project is open even to at home scientists although I think some of the genomic data may reasonably not be as open as other parts.
Re:Ask HN: What's a side project you built to make money that hasn't?
on Thursday December 31, 2020 @01:32PM (#25595497)
I built Hatchedwith.com mostly because I wanted to mess around with language models. I can’t seem to find any customers even though I got two from a fake landing page but realistically it’s not the product I advertised in the beginning I’m going to lower the pricing to extremely cheap and call it done unless google starts picking up on the tens of thousands of pages.