Relatedly someone linked a "all in one ML imaging software" called chaos vision or something, and the first thing on the linked page was "you probably need an Nvidia rtx 3090, or another Nvidia card with 24GB of memory"

I've tested a 'machine vision for image tagging' self-hosted service and it seemed reasonably responsive, CPU only, too - but I ran a pre-trained model for that.