That conclusion isn’t quite logically rigorous. You need to have a prior on the odds of being a Boltzmann brain in the first place. If that prior is extremely high, your conclusion won’t hold.