There are charter schools, magnet schools, and academically focused private schools. Ultimately though if your child is too gifted you will need private tutors. You dont need the tutors all day and you can supplement with other more cost effective methods as above.
My city even has a school for girls that are preparing to compete in non academic subjects and that want to practice for half the day. Academics are only half day and then they do whatever their passion is for the rest of the day.
I had the great pleasure of visiting a school based on that model and it allowed a lot of flexibility for students. In my instance there were 4+ hours of "Free-work" blocks everyday, where we could work on whatever we wanted, we just had to make sure we finished certain (personalised) workloads by the end of the week. Pacing and what we wanted to do with our time if we finished early was up to us as a student and we could work on more advanced topics. Students were also encouraged to ask older members of the mixed age class if we got stuck, which made most students quite good at explaining topics to each other while giving older or more advanced students something to do if they didn't want to advance further in their own studies.
I think while schooling is far from ideal, most kids aren't reading physics books in elementary school. Many kids are struggling to cope on several different fronts.
I understand that many kids are not this way, but I was the type that read physics textbooks while in elementary school. We had a mentorship program that paired kids with someone from two grades above (e.g. 1st graders with 3rd graders). My reading level was always higher than my mentors' and it made me furious (and probably insufferable).
I feel like my entire time in school was spent getting it out of the way so I could study things that genuinely interested me. I would love for my children to not feel that same constraint, but instead feel free to go deeper at their own pace.
I'd recommend this talk by Sir Ken Robinson on this topic generally: https://www.youtube.com/watch?v=zDZFcDGpL4U