But how do we know that? There is no accountability. They do everything under secrecy. It might be different if we had a third-party, non-governmental agency verifying that they are not violating the 4th amendment.

Why have no accountability and no transparency if they were not going to violate our privacy? I'm flipping the "If you have nothing to hide" bit on them.

But even if there were oversight, the main issue is the potential for abuse. Having all that data in the hands of an entity that can imprison you is a pretty frightening idea. Sure, maybe they're following the rules now, but in desperate times, people can behave immorally and radically. Do not trust other human beings, even if they are under the umbrella of "government". Look at what evil humans have done in the last century alone and you'll agree that we should limit the potential for abuse as much as possible.

The government's position is that both Congress and the courts provide accountability here, and the reason for not being transparent is that they do not want the people who are being monitored to be aware of exactly what capabilities the USA does and does not have.

The rest of your argument is about reality as civil libertarians see it, and not how the government sees it. (Because, after all, they think that the briefing that Congress gets and the oversight of the FISA court is sufficient protection.)