There are a number of on-demand providers of GPU compute out there. It is relatively straightforward to run inference on them.

I've got a box of 8x MI300x sitting around waiting for stuff like this.

$128...

8x MI300X Bare Metal - $15.94/hour (1 available) CPU: Xeon Platinum 8462Y+ (64 cores) • Memory: 2.0 TiB • Disk: 124 TB • Minimum Reservation: 8 hours

ssh admin.hotaisle.app

(i'm ceo)