News for hackers. Stuff that matters.
Re:Tinygrad: Hacked 4090 driver to enable P2P by Emerson1 on Saturday April 27, 2024 @03:42AM (#40178101)
how could you go about testing this on say llama3 70b with two 4090s - vllm supports tensor parallelism, would the expectation be that inference would be faster with p2p? How would you update the nvidia driver tho? Thanks any thought appreciated

There are no comments on this comment yet.