One thing they don't get into really is that the IRT model they are using is pretty simple, which is typical of the random/mixed effects model formulation as a way of keeping things mathematically tractable to permit parameter explanation/prediction.
They could add other components to the model, though, such as different dimensions of fit (different aspects of body shape and fit), or how closely different items track those dimensions (as opposed to just how large or how small; e.g., maybe some types of fabric provide more information about fit than others). The models they're fitting are a sort of entry point in that regard.
It wouldn't solve the problem of course, and I agree with you 100% about in-person fitting being the final word, but they also have a lot of ways they could improve these models.
Excellent point. :) What is the point of finding the perfect fit, after all?
It's so that the person (or persons) whose opinions you care about think you dress well and look well put together. That is an important piece of data.
1) Body types that aren't average (all of our bodies are irregular, but some of us are more irregular than others). Clothes sizing are based on a average model of the human body.
2) Fabric interaction with body. Softer fabrics drape in different ways from more rigid fabrics. Also how loose or muscular your flesh is can affect fit if you're in the market for really slim fits (European style).
3) Non-representative snapshots: your measurements in the morning will differ from other times of day, and it will differ over the course of weeks and months, even if your diet is stable.
4) Shrinkage/expansion: depending on material, there is some shrinkage or expansion after the first wash/wear. Although this is mostly a known quantity and good clothiers account for this.
Really good bespoke tailors understand these principles, and make allowances for them as they build your clothing. Also they know to put elastic fabrics in the right places so the suit will still fit even after a large meal.
I think the one way we can get closer to a good fit while being remote is to have pop-up stations/kiosks where we can get multiple 3D scans of our body on separate occasions (kinda like how most bespoke tailors require at least 3 fittings). That still doesn't account for fabric-body interactions, but it gets us a lot closer than recommendation systems.
p.s. the other problem is cultural. Most Americans don't know or care quite as much about fit (because it's not as prized in the culture) as their European counterparts, so their data is going to be skewed slightly towards the left end of the competence curve.