There are myriad providers competing to offer this, nicely packaged with all the accoutrements (IP rotation, location spoofing, language settings, prebuilt parsers, etc.) behind an easy to use API.

Honestly it is a very healthy competitive market with reasonably low switching costs which drives prices down. These circumstances make rolling your own a tough sell.

there are scraping subreddits.

if you browse them you will see that bot writers are very annoyed if they can't scrape a site with a headless browser.

you can do what you suggested, but with Linux VMs/containers. windows is too heavy, each VM will cost you 4 GB of RAM

I’m in those. xvfb and headless=false still works great

The reason to use windows is that anti bot tech is going to be a lot stricter if Linux is detected...

If you know of a simple way to run a Windows 11 VM with good graphics acceleration (no GPU passthrough), please contact me.

I assume your concern with GPU passthrough is that each VM needs a whole GPU? You can use GPU-PV to split your GPU between VM instances. Then the main bottleneck becomes how thin you split out your VRAM.

More info here:

https://web.archive.org/web/20231107182321/https://mu0.cc/20...

https://youtu.be/XLLcc29EZ_8?t=570

https://github.com/jamesstringer90/Easy-GPU-PV

Wouldn't virtualbox or vmware's paravirtual GPUs be a better fit for this use case? Unfortunately the offerings with qemu/libvirt still lag vmwares by a lot.

I know those offer virtual GPUs, but I am unfamiliar with any paravirtual GPU offerings from VMWare or VirtualBox. The virtual GPUs are much more limited in performance and graphics API support.

They do, but the fact that they have to do this means there are fewer bots because it's less economical to go to such lengths, compared to something much less complex (which is orders of magnitude cheaper).

284 on 296gb of ram with deduplication enabled on a 128c with 32Q vgpu.

In theory you could run hundreds of full-fat Chrome bots if you don't care about the ops mess, but keeping Windows images stable while Cloudflare and friends keep changing the fingerprinting game turns the cheap math into a maintenance job from hell. AWS VM signals are a big red flag, so you still eat CAPTCHAs and blocks even with a full browser stack. The page load number looks cheap.

I am reasonably sure that these kind of fingerprints can detect if the browser is inside a VM.

… yup?

I mean you missed the minigame of preventing Chrome from signaling that it’s being programmatically (webdriver etc) driven and tipping your hand, but … yup?