It's a potential quasar. It's just that there's not much accretion happening right "now". And it does occasionally flare.[0] Also, I don't believe that we have the data to exclude the possibility that something big is heading inward. We only see stuff that's close enough to get hot enough.

But of course, given that we're near the galactic plane, neither jet points at us. And although, as I recall, the jets are somewhat off-axis, they're not off in our direction.

0) https://arxiv.org/abs/1709.09896

It might become a full-blown quasar when the Andromeda Galaxy collides with the Milky Way ~4.5 billion years from now. We'd better take that into consideration when we start planning for the death of our Sun ;)

Based on what we've observed, one could likely estimate the amount of accretion needed to make Sag A* visible from Earth. Or bright enough to fry us. It'd be a huge stretch for me to do that. But I bet that it's a lot less than Andromeda. I mean, there could be a substantial black hole, maybe 50-100 years out, and I doubt that we could detect it now :)