News for hackers. Stuff that matters.
Re:Tokasaurus: An LLM Inference Engine for High-Throughput W... by bobrenjc93 on Thursday June 05, 2025 @11:18PM (#44197554)
Hi! I work on dynamic shapes in pytorch and would love to hear more about the challenges you’ve run into. We’re always looking to improve the experience, so if you’re open to chatting, feel free to DM me on Twitter (@bobrenjc93) or email me at [email protected].
Re:Tokasaurus: An LLM Inference Engine for High-Throughput W... by gricardo99 on Friday June 06, 2025 @01:00AM (#44197936)
since you work on pytorch, what would you say is the best place to ask questions about general usage, trouble shooting? I’ve been struggling with a, what I would consider, a simple torchrun elastic training example, and haven’t found any good resources online. I’ve been spelunking through pytorch but have a feeling a little back and forth with someone familiar with these features would immensely clear things up.
Re:Tokasaurus: An LLM Inference Engine for High-Throughput W... by bobrenjc93 on Friday June 06, 2025 @01:10AM (#44197962)
PyTorch Dev Discuss is a fantastic forum where many core devs actively participate and answer questions: https://dev-discuss.pytorch.org

In addition to Dev Discuss, a number of core contributors are also active on Twitter. Two particularly helpful and prolific voices are @ezyang and @cHHillee.

Finally, don’t overlook GitHub issues—they’re a surprisingly effective way to start conversations. If you’ve found a bug or have ideas on how to improve the APIs, opening an issue is always welcome.

Re:Tokasaurus: An LLM Inference Engine for High-Throughput W... by almostgotcaught on Friday June 06, 2025 @07:26AM (#44199760)
There's also the slack but you gotta know someone to get on that ;)