Normally, this type of comment is Hacker News reductiveness, but yes, image classification via CLIP is that easy, especially with Hugging Face's API for it: https://huggingface.co/docs/transformers/model_doc/clip

I created a Python package to generate image embeddings from CLIP's vision model without requiring a ML framework (https://github.com/minimaxir/imgbeddings ), and a simple linear classifier on those embeddings does the trick, demo here: https://github.com/minimaxir/imgbeddings/blob/main/examples/...