Determinism in a system has been a cornerstone of validation. Even outside of software circles (the term 'test fixture' originally referred to a contraption you fitted hardware into to serve as a mock 'environment' for testing, which could fake inputs and/or register outputs).
> I lament the day that unpredictable icons line a toolbar changing as often as I change a tool.
If you visit booking.com it's unlikely you'll ever see the same website twice. That's because their product team is continuously A/B testing, trying to come up with the best one-size-fits-all solution.
Although their designers' decisions are based on usage data, the data will only tell that button A outperforms button B, not why. And that's because our devices are unable to accurately measure what we're doing and what state we're in.
With the right data algorithms could make layout and even component level design decisions, but tailor them to individual users instead of user personas.
An AI will not predict when I want to draw a construction line in Fusion360 and automatically select or display a clickable widget in a suitable and predictable manner. The correct choice is to give lots of options, visible at a glance, with hotkeys they are reconfigurable.
I lament the day that unpredictable icons line a toolbar changing as often as I change a tool. And this whole discussion happens independent of the styling desires a company wishes to exert, with smooth transitions, animated modals, etc.
Maybe I’ll eat my hat within 5 years, but it’ll be because it has been forced upon us instead of any actual user experience feedback.