News for hackers. Stuff that matters.
Re:Score matching with Langevin Sampling: a new contender to... by fxtentacle on Thursday September 03, 2020 @07:54PM (#24370216)
Except that cloud-ified V100s are significantly less powerful than if you have direct access to the hardware. Last time I checked, in AWS they're actually external devices mapped in over GBit ethernet, which is significantly slower than the 8GB/s that PCIe x4 has.
Re:Score matching with Langevin Sampling: a new contender to... by nl on Friday September 04, 2020 @02:58AM (#24372431)
This isn't true. AWS V100s are standard V100s.

I think you are confusing this with AWS Elastic Inference.

If you use AWS Elastic Inference, then you get networked attached devices. But these are Amazon's own (non-NVidia) devices and only used for inference, so it's not really comparable.

https://aws.amazon.com/machine-learning/elastic-inference/fa...

Re:Score matching with Langevin Sampling: a new contender to... by p1esk on Thursday September 03, 2020 @10:57PM (#24371285)
I routinely switch between AWS 8x V100 instances and on-premise 8x V100 servers and I observe no difference in speed (time per epoch).
Re:Score matching with Langevin Sampling: a new contender to... by Reelin on Thursday September 03, 2020 @11:24PM (#24371426)
Presumably that depends on maximum PCIe bandwidth consumption before your workload bottlenecks elsewhere? A 2018 benchmark (https://www.pugetsystems.com/labs/hpc/PCIe-X16-vs-X8-with-4-...) seems to indicate that x8 isn't generally a bottleneck for common (at the time) workloads. x8 is a far cry from the claimed gigabit ethernet though!
Re:Score matching with Langevin Sampling: a new contender to... by p1esk on Friday September 04, 2020 @01:10AM (#24371878)
AWS is tricky in terms of how storage is provisioned - I don't remember details, but it's easy to put your datasets on storage that is connected to your GPU servers over 1Gb link. That could easily become a bottleneck. Datasets should live on Elastic Block Storage or something like that, over high speed links. Again, it's been a while since I looked into that, so I don't remember the details.
Re:Score matching with Langevin Sampling: a new contender to... by Reelin on Friday September 04, 2020 @02:20AM (#24372236)
The earlier comment claimed that the GPUs (!!!) were located elsewhere on the network; I suspect that the scenario you describe is what they intended to refer to.

(IIRC AWS offers compute optimized instances with a volume that's guaranteed to be backed by blocks on a local NVMe drive.)

Re:Score matching with Langevin Sampling: a new contender to... by nl on Friday September 04, 2020 @03:00AM (#24372444)
I think they are confused with AWS Elastic Inference. That is a different thing which does have network attached accelerators:

Amazon Elastic inference accelerators are GPU-powered hardware devices that are designed to work with any EC2 instance, Sagemaker instance, or ECS task to accelerate deep learning inference workloads at a low cost. When you launch an EC2 instance or an ECS task with Amazon Elastic Inference, an accelerator is provisioned and attached to the instance over the network.

https://aws.amazon.com/machine-learning/elastic-inference/fa...

Re:Score matching with Langevin Sampling: a new contender to... by marcinzm on Thursday September 03, 2020 @11:39PM (#24371490)
Citation? AWS GPU instances are cpu+gpu+memory in fixed ratio packages so I don't see why it'd be over ethernet.