News for hackers. Stuff that matters.

A comment on Beating OpenAI CLIP with 100x less data and compute.

Re:Beating OpenAI CLIP with 100x less data and compute by swyx on Tuesday February 28, 2023 @10:31AM (#34970386)
> The original CLIP was trained on 500x A100 Nvidia GPUs.

> The latest Open_CLIP trained on 1024x GPUs.

> We trained on the setup of 3x workstations, with 4x RTX 3090 consumer-grade GPUs in each, connected over 200 GBit InfiniBand HDR.

i must be mising something - where does it say it uses 100x less data?

[ Parent | on HN ]
Re:Beating OpenAI CLIP with 100x less data and compute by brookst on Tuesday February 28, 2023 @10:34AM (#34970412)
Look at the “dataset” column: CLIP was trained on 400m images, UForm on 4m.
[ Parent | on HN ]
Re:Beating OpenAI CLIP with 100x less data and compute by vov_or on Tuesday February 28, 2023 @11:40AM (#34971383)
There are also dataset sizes for Albef and ViCHA.
[ Parent | on HN ]