I think there's a view that his content has changed recently. I used to watch him a lot in previous years as it really was the best place to just hear conversations with interesting people. Now there's a view that he's taken on a particular political lean and has surrounded himself with conspiracy theorists. I doubt he's actually promoting them but his show does put them in front of a large and impressionable audience (in my personal opinion, he is extremely impressionable himself and parrots the views of his guest for the sake of the conversation regardless of his actual belief).

There are also some theories that his sudden giant increase in wealth may have contributed to his shift from the mainstream to the fringe