This is very much a to each their own thing. It worked well for you but it also sounds like you had a uniquely active family and community to support this. The primary problem with homeschooling and unschooling is the lack of family support. Either by apathy or by lack of economic opportunity. Or simply their own lack of education preventing them from even being aware of these options (such as finding a community to coordinate with to make up for their on educational deficiencies).
I can't speak for all public schools, my parents deliberately chose better than average schools for my sister and I to attend. But good public schools exist. Good teachers exist. But just like good home schooling and unschooling experiences, this requires either luck of birth (well-to-do parents) or deliberate effort by parents to improve their children's position.
By learning independently of school, I learned way later in life, and learned in a way that was right for my stage of development.
My anecdote about it not being productive is purely subjective, but the science on development theory is clear. If left to their own devices, almost all people learn native language, writing and math on their own timeline in about 100 hours. School makes this process far less effcient, and teaches things like grammer that program the brain to read slowly and without depth.
Obviously my anecdote is hyperbolic, I speak primarily as someone who had a great unschooling experience and then went to school and saw the difference, and a little bit as someone who has read extensive child development theory.