OP's comparison wasn't a homemade filter versus no filter, it was versus commercial options.

I've done box fan filters before, and they certainly help if they're what you can afford or can find in stores, but after enough years of dealing with wildfire smoke this year I invested in commercial units and I am significantly happier.

After doing a fair amount of research, I ended up with a series of Coway units. They're more expensive than box fan filters, but they're smaller and significantly quieter, especially with a sensor-driven auto mode that adjusts fan speed based on detected pollution levels. This means I'm far more willing to leave them on all the time and get the benefits 24/7. The fact that they're multi-stage is a big plus too, particularly with cat hair around. It means that I have to clean the outer charcoal filters more often but the inner HEPA filters only 1-2 times a year. As an added bonus, power consumption is significantly lower than my box fans.

Which Cowan units do you use? The only concern I have with smaller air filter devices is they sometimes can have significantly less surface area than the furnace filters typically used with box fans.

The Airmega 400 for the main living room and a pair of AW-1512/Airmega 200 (they're the exact same unit with cosmetic differences, get whichever you can find in stock unless you strongly prefer the look of one). Surface area wasn't my main focus. I'd rather change filters more often (not that that's a huge issue outside of wildfire season) and deal with a quieter unit.

These articles made sense 5 years ago when there were only a few high priced air filters widely available online, but now the chinese copies have proliferated amazon and other sites and the market is pretty saturated imo. My filter was $30, about the size and look of a larger bluetooth speaker, it is dead silent on the lowest setting, replacement filters another $15 or so, and its good to filter my bedroom pretty heavily in terms of square footage. Can't beat that. At that price we have one of these in a couple different rooms.

Have you actually measured particulates to see if it is doing its job? Currently running a box fan plus filter but might switch if these are actually good.

I am using a cheap Xiaomi (there's not much choice where I live), and a few PMS5003 sensors scattered around the apartment. It works fine for 25 m²-something room when it's relatively clean outside (around 100 µg/m³ or less), but you really have to push it (and suffer the resulting noise) when pollution outside goes above 300 µg/m³ or so. It's been able to keep indoor air around 20 µg/m³ with outside levels going above 800 µg/m³, but I have to use ear plugs because the noise is impossible to tolerate for more than half an hour (and it's pretty difficult to sleep next to).

If you let the heavily polluted air in (a few hundred µg/m³), then close the window and set the purifier to maximum power, it cleans the room in ~20-25 minutes.

Ditto. Mine is really quiet too on low settings, which is great if you want to leave it on while you sleep.

They also have a carbon layer that's supposed to help with odours and that pollution that can't meaningfully be caught by a simple HEPA filter, although I don't know how effective it is.