roanakb
Show HN: Pluto – open-source Experiment Tracker for Neptune users
https://github.com/Trainy-ai/pluto
Hey HN! We're Roanak and Andrew from Trainy (YC S23). We build GPU infrastructure for ML teams (scheduling multi-node training jobs on Kubernetes). When Neptune announced they're shutting down, our customers didn't have a clear path forward. The alternatives exist but none of them matched the UI experience Neptune had, especially at scale. So we decided to build Pluto.Pluto is an open-source...
Re:GPU utilization is a misleading metric
on Friday August 23, 2024 @02:34PM (#41331701)
Unfortunately, SM efficiency is not accessible via nvidia-smi. The best methods to track it would be to:
1. Profile your model with Pytorch Profiler 2. Export metrics with Nvidia DCGM
Re:GPU utilization is a misleading metric
on Friday August 23, 2024 @12:13AM (#41326121)
oh this looks great, thank you for bringing this up! I'll have to give it a try, but seems like the FSDP limitation on torch.compile might carry over?
Re:GPU utilization is a misleading metric
on Friday August 23, 2024 @12:09AM (#41326107)
Yup, you'll see 100% utilization on a kernel over a time period if it's considered active, which includes just having a single thread executing [1]. SM occupancy is great but can be a little difficult to interpret since you're not simply trying to maximize it, unlike SM efficiency.
[1]: https://pytorch.org/blog/pytorch-profiler-1.9-released/#gpu-...
Re:GPU utilization is a misleading metric
on Thursday August 22, 2024 @02:31PM (#41323130)
Nice, seems like ML Productivity Goodput is a pretty well thought-out metric to understand the overall efficiency of your cluster. I'll consider adding this into our cluster management platform. Only potential drawbacks I'd guess are it being somewhat difficult to compute since it relies on metrics like MFUs, and not something we can observe layer-by-layer to understand inefficient kernels, but I'll take a deeper look. Thanks!
Re:GPU utilization is a misleading metric
on Thursday August 22, 2024 @02:26PM (#41323084)
Agreed, roofline plots would be quite powerful in this context. From a quick search, seems like the only way to create a roofline plot for your model would be to use Nsight [1]? Would be interested to know if there are any simpler tools, since one of the big benefits of SM efficiency is how easily the metric is accessed.
[1]: https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s...
Re:GPU utilization is a misleading metric
on Thursday August 22, 2024 @01:53PM (#41322746)
Yup, similar to SM efficiency in that sense too. If you aren't seeing >80%, there is certainly time left on the table. But getting a high SM efficiency value doesn't guarantee you're making good use of the hardware as well. (still a better proxy than GPU util though)
GPU utilization is a misleading metric
Re:Instrumentation checklist for running large GPU clusters
on Tuesday August 13, 2024 @10:20PM (#41241950)
this is a good one for debugging rdma: https://docs.redhat.com/en/documentation/red_hat_enterprise_...
Re:How to Use PromptGuard – Meta's Llama Moderation Model
on Thursday July 25, 2024 @06:24PM (#41074041)
Looks great, you guys made it really easy to integrate!
Show HN: GPU Health Controller
https://trainy.ai/blog/konduktor-controller
docs: https://konduktor.readthedocs.io/en/latest/admin/controller.... github: https://github.com/Trainy-ai/konduktor
GPT-4 can catch edge cases in your code using unit tests
Re:Monitor and Optimize your large-scale model training
on Tuesday August 01, 2023 @07:12PM (#36964575)
Thanks! Let me know if there are any features you'd like to see added.
Monitor and Optimize your large-scale model training
Re:Launch HN: PeerDB (YC S23) – Fast, Native ETL/ELT for Postgres
on Thursday July 27, 2023 @01:25PM (#36897047)
Looks really cool! Nice work.
Re:Ask HN: Who wants to be hired? (October 2020)
on Friday October 02, 2020 @04:02PM (#24666510)
Location: Fremont, CA
Remote: Yes
Willing to relocate: No
Technologies: Python, R, Pytorch, Keras, SciPy, Machine Learning, Deep Learning, Computer Vision, NLP, Javascript, MERN Stack
Résumé/CV: on request
Linkedin: https://www.linkedin.com/in/roanakb/
Email: roanakbaviskar at gmail dot com
I have experience with Machine Learning research and have implemented SoTA architectures. I am comfortable with Python and various ML/DL libraries.