You're both right.
It's a common story -- people use browser A, start noticing a variety of problems, switch to browser B and everything is so much better. After a while, they start noticing a variety of problems, switch to browser A, and once again everything is so much better.
Those people are not being stupid, that is their real experience.
Sometimes the problem is in a web site (an update starts leaking memory, for example) and whatever browser is running gets blamed.
Sometimes it's just natural human bias—we want to see patterns, and we want there to be a solution ("just switch browsers!"), so we get selective in what we notice and don't notice.
Sometimes it's because profile cruft piles up, and resetting would fix it.
Sometimes the problems really are in one browser and not the other. There are plenty of legitimate problems to be found, and things change pretty rapidly.
But the comparison point is switching browsers, which also loses lots of state.
(In both cases, you can import a subset of your state into your new profile, and you'd probably get most of the performance advantages.)
I believe all browsers have been improving in their resistance to the accumulated state problem, but I also believe that all browsers are still susceptible to it.
Any time I've tried switching away it's more to see if I'm missing out on something... and it turns out I'm not.
In my experience, Chrome has problems with memory, performance, rendering and battery drainage. Every now and then I try to use Chrome (or a derivative) as secondary browser and I give up because it has noticeably higher latency for simple web page loads, and more likely to suffer irritating repaint flashes.