We don't know for sure, and we probably never will. It's impossible to prove that something is actually a constant over large distances and timescales. You can always go a bit further or wait a bit longer, where it might not be as constant as we expected.

However, most experimental data so far is in agreement on it actually being a constant.

Wikipedia has a section on it: https://en.wikipedia.org/wiki/Fine-structure_constant#Is_the...?