News for hackers. Stuff that matters.

jolieli

jolieli on Hacker News

Re: on Friday June 07, 2024 @12:23PM (#40610139)
I built an open-source project to recommend the best computing resource configuration for ai application, but no reply or feedback at all. So I wonder if AI application developers care about their GPU utilization now? Or I target the wrong audience?
Re: on Friday June 07, 2024 @09:54AM (#40608736)
I built an open-source project to recommend the best computing resource configuration for ai application, but no reply or feedback at all. So I wonder if AI application developers care about their GPU utilization now? Or did I target the wrong audience?

Show HN: I found a way to increase the perforamce of GPU cluster scheduling

Posted by jolieli on Thursday May 30, 2024 @11:55AM
from the show-hn dept.

https://github.com/Emerging-AI/ENOVA

I worked on this project one year ago since I started working on developing new programs by using GPT and GPU could. And I found out these applications typically run on GPU clouds in industrial environments, where the cost of LLM requests may be ten times higher than that of traditional queries. Therefore, previous researchers have focused on optimizing LLM inference to increase throughput...

( Read More... | 1 comment | on HN )
Re:Ask HN: Will AI tech, like LLM, keep needing tons of computing power? on Thursday May 23, 2024 @10:53AM (#40455488)
Valid point. Scaling up seems to be the default approach for many AI companies to enhance performance, albeit with potential environmental concerns. While transitioning to more specialized chip architectures might mitigate power usage to some extent, the overarching trend seems to be towards increased power consumption in pursuit of larger scale. It's a trajectory worth monitoring closely.
Re:Ask HN: Will AI tech, like LLM, keep needing tons of computing power? on Thursday May 23, 2024 @10:50AM (#40455452)
sounds terrific. we can see the model companies are seeking ways to improve the model performance as the priority currently. More computing power is needed now. I do agree with your two ideas, but I am concerning a lot about whether the current GPUs are fully utilized or if they truly need more GPUs to improve the performance...