Re:Tinygrad: Hacked 4090 driver to enable P2P
by Emerson1 on Saturday April 27, 2024 @03:42AM
(#40178101)
how could you go about testing this on say llama3 70b with two 4090s - vllm supports tensor parallelism, would the expectation be that inference would be faster with p2p? How would you update the nvidia driver tho? Thanks any thought appreciated
[ Parent
| on HN ]
There are no comments on this comment yet.