I would still argue you did the right thing. Sure it would be nice to always have enough domain expertise to pick the right solution in the first place, but you didn't have that knowledge at the time. This experience cost you a little pain but your system is working and you are now smarter.
A developer will be presented with thousands of situations like this across her career. If she takes the YAGNI/under-engineering approach, it almost always ends up a similar story - some growing pains, some re-engineering, a lesson, and a working system.
On the other hand, if she errors on the side of over-engineering, there's no bound on how far off the rails she can go. At best she ends up with some extra useless code that someone will delete later. At worst it becomes a time sink that consumes the entire budget and schedule.
YAGNI forces you to always hew to the side of under-engineering. And if there really are design considerations that you know are easier to bake in early than retrofit, it's almost always possible to arrange dev stories in such a way to force those design decisions out early.
A developer will be presented with thousands of situations like this across her career. If she takes the YAGNI/under-engineering approach, it almost always ends up a similar story - some growing pains, some re-engineering, a lesson, and a working system.
On the other hand, if she errors on the side of over-engineering, there's no bound on how far off the rails she can go. At best she ends up with some extra useless code that someone will delete later. At worst it becomes a time sink that consumes the entire budget and schedule.
YAGNI forces you to always hew to the side of under-engineering. And if there really are design considerations that you know are easier to bake in early than retrofit, it's almost always possible to arrange dev stories in such a way to force those design decisions out early.