The bad news is as you say: having mapping data of your private residence siphoned off by third parties, be it government or private industry.
I would say I'm as excited as I am worried about the potential for this technology. That said, I think the privacy implications are very similar to Kinect, which is already here. The only difference perhaps being mobility.
However, my recollection is that there was a reasonably vocal reaction to the privacy implications of Xbox One / Kinect 2 in gaming and tech media, although they mostly related to the timing of the first Snowden leaks. This article is a pretty good example of the media hype around it: http://www.theverge.com/2013/7/16/4526770/will-the-nsa-use-t... .
I think the "always-on" nature was more criticized than the point-cloud/time-of-flight abilities of the camera, but pushback against more and more ways to collect data attached to equipment made by big tech corporations isn't exclusive to Google.
Having point cloud data of your home in general is a complete non-issue, I'm sure most of the time the government already has floor plan data for your home.
The possible benefits of this tech are tremendous, and far outweigh any crazy conspiracy theory downsides.
What you label as "paranoia" and "baseless conspiratorial fear" is better described as "examining the potential consequences of the things we build". You are advocating that we examine only the positive potential uses of the technology we develop and that we ignore the potential consequences, because "we'll work it out and everything will be awesome!", or something.
Perhaps the potential consequences will never come to fruition--but it's still very important that we examine them. There's a real shortage of that going on, which is why some people post negative reactions to these developments rather than something more balanced. Stop trying to dismiss those concerns and that conversation--if you're not interested in hearing it or taking part in it, this is likely the wrong discussion forum for you to be participating in.
The problem, is no longer that someone is "watching you". Its worse. The problem is that someone, one day, could watch your entire life. Then proceed to pull apart events and use them, mostly out of context, against you.
I wouldn't call myself paranoid. I'm not doing anything about it. Honestly, if the data was only retained a month, unless needed in a current investigation, I would agree with you, this wouldn't be worth mentioning. However, retention of data is cheap and the data of your life will probably outlast you. As such, the parent posters in this convo have a point, it is something worth giving attention to.
PS. And to be quite honest, I don't even care that they can look back at my life and scrutinize me for an email or two. The real fear is that they can pull specific quotes or life choices out of context. You as an individual can't even discredit them because you don't have access to the original files (unless the entity does you a favor and shares them with you). So you can show how innocent the same email quote or life choice is while in context. You are instead left with your memory, and hopefully powerful rhetoric to convince a jury you are innocent.