Vast.ai – marketplace for renting out your GPU, or renting someone else's GPU

Re:Vast.ai – marketplace for renting out your GPU, or rentin... by HWR_14 on Sunday March 20, 2022 @10:55PM (#30749627)

Are the PBs of free space necessary? I thought training a ML model on a terabyte of data was computation expensive, but not storage intensive.

[ Parent | on HN ]

Re:Vast.ai – marketplace for renting out your GPU, or rentin... by a_bonobo on Sunday March 20, 2022 @11:37PM (#30749932)

Sometimes! Not for the GPU part, but to make the summary data for the ML data.

For example, genome sequencing data and intermediate results are easily in the TB area of space, but the resulting table of genomic variants (k-mers like n-grams in NLP) is only a few hundred GB.

[ Parent | on HN ]

Re:Vast.ai – marketplace for renting out your GPU, or rentin... by HWR_14 on Monday March 21, 2022 @12:50AM (#30750420)

I'm confused from your post where the petabytes of storage come in.

[ Parent | on HN ]

Re:Vast.ai – marketplace for renting out your GPU, or rentin... by freeqaz on Monday March 21, 2022 @01:39AM (#30750682)

I'm guessing that there are just many data sets, and each one is in the TBs of size.

[ Parent | on HN ]