I wouldn't build anything beyond what the server logs, which I do by default in the server config.

Having an access log and error log is sufficient, until you know what you want to query. Writing a script to do some custom query is a few minutes of work and can run over all the files created...ever. Then the person who wanted the query will just as likely realize the question they had is impossible to answer right now or doesn't want to expose the answer, or any other of a myriad of results that aren't "this adds business value".

Stop building more than you need. Get back to the work.

Yes - but people dont. You often get pushback for implementing more the than basic request, you have to defend your decision from someone who didnt think of it, so now they think it cant be good.

Logging, auditing, anything that isnt immediately available to solve the next problem is often disregarded until "wow it would have been nice to know..."

I'm in a camp of "do the minimum". The cost of adding some call to a logger to dump data into file/syslog/etc is microscopic. The cost of adding that data into BigTable and building tooling and web interface and reporting around it is probably higher than the cost of the entire project.

In my experience companies are experiencing data overload and data paralysis because they have tons of data that is absolutely useless now and will be even more useless tomorrow. And they keep adding data about the data because that's the current koolaid everyone is drinking.

I think based on your experience that's a fair line to take; I've spent a lot of time on the ops side, and seen much of "well we did nothing and we're all out of ideas AND budget"

The best answer is somewhere between these two extremes, and I like your approach of starting simple and clear - a file based approach also easily lends itself to centralizing or bootstrapping to another medium - the bigtable approach calls for a migration and potentially even a change in the guarantees made by the underlying system because someone started to repurpose it as not just an audit trail because "of course we can just query it." (ugghgghghgh)

Spreadsheets work with BigTable out of the box in the G-Suite. I love dumping stuff in BigTable for analytics so other non engineering people can easily interact with that data and run all sorts of complex and non complex queries against it with tooling which they are already know, Out-Of-The-Box! This is as cheap as it gets when it comes to building a "web interface and reporting" and the cost is zero.

Does your app handle all the errors that insert into a bigtable could trigger? Does it do retries? Does it queue messages for those retries via a separate queue manager? Does it synchronize the state of the transaction and the state of the transaction inside of the big table? If one succeeded and one failed, what do you do?

That's the extra complexity. If dumps into data lake are that important, then it becomes its own special beast that needs to be fed. If the dumps into the data lake aren't important, then the question to ask "Do I need the data lake at all or am I doing it because everyone else is doing it?" If the answer is "yes" then absolutely build that functionality. If the answer is "Eh" the question becomes much murkier.