I haven't had any downsides at all, for my use cases. I've heard from a few folks that lack of "labels" is a bummer (if you rely on them in Gmail), and their mobile clients aren't quite as good (though I've never used them).