News for hackers. Stuff that matters.
Re:Show HN: Query Google Sheet data using PostgreSQL clients by mritchie712 on Tuesday May 31, 2022 @06:36AM (#31567950)
You can query a Sheet with Clickhouse out of the box. Obviously doesn't

    with sheet as (
    select \* 
    from 
    url('https://docs.google.com/spreadsheets/d/1XGCy0tYU5YcEouO09_ErZIyqjA-VJ4pidLZmMmJkEdk/gviz/tq?tqx=out:csv&sheet=Sheet1&range=A:C', CSVWithNames)
    )

    select \* from sheet


https://luabase.notion.site/Query-a-Google-Sheet-905da4e981b...
Re:Show HN: Query Google Sheet data using PostgreSQL clients by whatrocks on Tuesday May 31, 2022 @08:55AM (#31568940)
I use roapi to power my “book list” on my static site. I have it running on an always-on Replit repl. Try it here:

https://charlieharrington.com/library/

Click the “I’m feeling lazy” button if you just want to see some example queries.

I also wrote a tutorial, as well, if you’d like to try out roapi yourself:

https://charlieharrington.com/sql-powered-reading-list/

Re:Show HN: Query Google Sheet data using PostgreSQL clients by hbarka on Tuesday May 31, 2022 @02:05AM (#31566558)
I spent years using Excel but I mostly use Google Sheets now. The cloud integration just feels much more effortless compared to Microsoft’s Office products ported to cloud. I’ve seen departments using GS as a system of record for quickly setting up master data like budgets and forecasts. I wish it had database connectivity the way Excel has ODBC.
Re:Show HN: Query Google Sheet data using PostgreSQL clients by mritchie712 on Tuesday May 31, 2022 @06:42AM (#31567991)
I founded https://seekwell.io/ because I had that same wish a few years backs.

Also worth checking out https://equals.app/

Re:Show HN: Query Google Sheet data using PostgreSQL clients by silvershell on Tuesday May 31, 2022 @11:20AM (#31570557)
Seekwell.io is giving me a 502 Error as of now.
Re:Show HN: Query Google Sheet data using PostgreSQL clients by mritchie712 on Tuesday May 31, 2022 @12:16PM (#31571365)
weird, GCP might have been down a moment? Is it working now?
Re:Show HN: Query Google Sheet data using PostgreSQL clients by cocothem on Tuesday May 31, 2022 @07:04AM (#31568160)
Have you looked at connected sheets? https://cloud.google.com/bigquery/docs/connected-sheets
Re:Show HN: Query Google Sheet data using PostgreSQL clients by hbarka on Wednesday June 01, 2022 @11:03AM (#31582848)
That article describes how to use GS as a front end for BigQuery data. I’m describing the opposite, which is ingesting spreadsheet data into databases.
Re:Show HN: Query Google Sheet data using PostgreSQL clients by nathanwallace on Tuesday May 31, 2022 @06:45AM (#31568006)
Steampipe [1] is an open source Postgres Foreign Data Wrapper with 73 plugins [2] including Google Sheets support [3]. A lot of our users use it to connect custom metadata with other systems - e.g. join ownership info in Google Sheets with tags on AWS resources. Notes: I'm a lead on the project, it's focused on read only.

1 - https://steampipe.io 2 - https://hub.steampipe.io/plugins 3 - https://hub.steampipe.io/plugins/turbot/googlesheets

Re:Show HN: Query Google Sheet data using PostgreSQL clients by Aaronstotle on Tuesday May 31, 2022 @12:35PM (#31571596)
unrelated: I used steampipe to easily automate some compliance audits on AWS, was great to do be able to look at all s3 buckets and make sure they're encrypted.
Re:Show HN: Query Google Sheet data using PostgreSQL clients by hoistbypetard on Tuesday May 31, 2022 @02:16AM (#31566601)
Last time I tried to do the moral equivalent of this (not using SQL, just some custom python), I ran afoul of the rate limits really quickly.

Does this client do something smart to mitigate that, have the limits been relaxed, or is it still a problem?

Re:Show HN: Query Google Sheet data using PostgreSQL clients by houqp on Tuesday May 31, 2022 @02:25AM (#31566639)
We cache all the data in memory in Arrow format so queries don't need to go through google api, it will only hit google api when a data refresh is needed.
Re:Show HN: Query Google Sheet data using PostgreSQL clients by rockwotj on Tuesday May 31, 2022 @02:58AM (#31566798)
How does cache invalidation work?
Re:Show HN: Query Google Sheet data using PostgreSQL clients by houqp on Tuesday May 31, 2022 @03:00AM (#31566812)
It's manual for now, ROAPI is designed for slowly moving datasets. You just hit the data update API to force a refresh.

I have plan to add automated streaming data update in the background, starting with delta lake tables. It should all be very straight forward to implement.

Re:Show HN: Query Google Sheet data using PostgreSQL clients by kristiandupont on Tuesday May 31, 2022 @03:22AM (#31566933)
I have thought about using google sheets as the database for projects many times.

The great thing about it is that you get a very versatile admin UI for free. The downsides however are rate limits and the fact that I can't think of any way to build in some sort of safety that would prevent people from accidentally deleting a bunch of data with a mouse click. I would love to hear if anyone has real life experience with the approach.

Re:Show HN: Query Google Sheet data using PostgreSQL clients by remus on Tuesday May 31, 2022 @06:35AM (#31567936)
> The great thing about it is that you get a very versatile admin UI for free. The downsides however are rate limits and the fact that I can't think of any way to build in some sort of safety that would prevent people from accidentally deleting a bunch of data with a mouse click.

My experience is that this is always the issue with spreadsheets when you try and grow them out to try and behave like more powerful tools. The same functionality (flexibility etc.) that makes them great for 1-5 people to work on means that it's hard to constrain the system as much as you'd like to when you grow it.

Having said that there are things you can do, like using protected ranges in gsheets that only certain users can edit, or input validation that makes it harder to add bad data. The built in version history is also a handy fall back in case someone does bork the sheet and you catch it in time.

Re:Show HN: Query Google Sheet data using PostgreSQL clients by hardwaresofton on Tuesday May 31, 2022 @03:29AM (#31566958)
Check out the following -- I've found them to be very useful:

- Baserow[0]

- NocoDB[1]

Also, modern CMSes that are DB-first (or at the very least try their best not to mangle the DB and provide programmatic access) are basically completely overlapped with this use case and can be good as well.

[0]: https://baserow.io

[1]: https://nocodb.com

Re:Show HN: Query Google Sheet data using PostgreSQL clients by kristiandupont on Tuesday May 31, 2022 @03:39AM (#31567006)
Awesome, thank you!
Re:Show HN: Query Google Sheet data using PostgreSQL clients by snowstormsun on Tuesday May 31, 2022 @04:03AM (#31567117)
I use it in one project to store public metadata of podcasts. The CI/CD pipeline downloads the google sheet via the public url, does some custom parsing and uses the data to build a react app. Worked like a charm for this use case, some fiddling required to get the parsing done right, though. And there is no api limit if your data is public. For using it as a live read-only database I am not sure if it's a good idea. Generally google sheets has a decent history of versions which makes recovery to an older version possible.
Re:Show HN: Query Google Sheet data using PostgreSQL clients by Normal_gaussian on Tuesday May 31, 2022 @03:18AM (#31566906)
This is cool; from reading the descriptions I presume this doesn't support writing - is there any intention to do so?
Re:Show HN: Query Google Sheet data using PostgreSQL clients by Normal_gaussian on Tuesday May 31, 2022 @03:20AM (#31566922)
Nevermind, I just worked out how to get to the overview. ROAPI seems to stand for read-only API :)
Re:Show HN: Query Google Sheet data using PostgreSQL clients by jdonaldson on Tuesday May 31, 2022 @12:21PM (#31571412)
Using this with materialized views could be very useful!