This is typical behavior in Japan when you question someones authority. It doesn't go well with them, it's a direct attack on their honor. You really need to go deeper with him (over dinner or so), but he refuses, so most likely he is wrong snd he knows it.