I cannot edit the comment anymore but I should have written MAE ViT-H and not CLIP-H (same size but CLIP models are trained in a different way).
Is MAE ViT-H available freely? My system could certainly handle it locally if it can be obtained and dropped in.
Yup
MAE ViT-H is available freely, in fact the entirety of the SAM pipeline is open source, and they even released the dataset.
Also read the "Efficient & flexible model design" section on the page.
Also in the FAQ: "How big is the model?
The image encoder has 632M parameters. The prompt encoder and mask decoder have 4M parameters."