An accurate and consistent scroll bar is a valuable navigation tool, please don't break it.
Opening modals changes the scrollbar length. Opening accordions do too. The scrollbar doesn't "break" when content is added to a page, it just adjusts accordingly. It seems like this is all good, expected behavior; not an a11y concern.
Particularly for the elderly and disabled, but honestly pretty much anyone not familiar with using computers, elements of the user interface suddenly changing seemingly at random is extremely disruptive and disorienting. In the spirit of accesibility, as an engineer you want to eliminate such behaviour because being disrupted and disoriented like that is really unpleasant.
This appears to be postulation on your part rather than actual knowledge of accessibility needs.
The specter of "visually changing things are bad" may hold some water (WCAG requirements do include things along those lines), but in order to discourage folks from using common web components I'd definitely need to see some actual evidence of this phenomenon w.r.t scrolling.
I mean, this whole thread is rife with people telling stories of being terribly annoyed by the shenanigans that infinite scrolling causes. Including issues both involving the scroll bar as well as other forms of navigation that break down with infinite scrolling.
I'm sure you have also experienced moments of disorientation when something you were using (not necessarily infinite scrolling) suddenly decided to do something all on its own and left you to figure out what just happened.
Dragging the scroll bar has its benefits over touch/wheel too: It allows for quicker, longer distance travel with fewer (and in the sense that you just move your arm, also simpler) movements.
Seems fine to me? New content is appended, so the length and your position changes. Scroll bar reflects what’s going on. What’s the problem?