News for hackers. Stuff that matters.
Re:Query Google Sheets as virtual tables with SQLite extensi... by satoshiiii on Saturday March 18, 2023 @08:05AM (#35208379)
Last project I was involved with deals with Google Sheets. They were using it as their multiuser read/write database. I offered to migrate it all to a proper database for FREE during my own personal time outside of work. Blank stares were all I got. I really hope AGI will soon be able to replace those humans that cannot be reasoned with logic. Millions were involved, hundreds of employees, thousands of sales, all being tracked in a couple of sheets. The faster data retrieval alone was worth it if we migrated it to a proper database.
Re:Query Google Sheets as virtual tables with SQLite extensi... by AlecSchueler on Saturday March 18, 2023 @08:26AM (#35208519)
But could the hundreds of employees continue to make changes to the DB if it was moved out of sheets?
Re:Query Google Sheets as virtual tables with SQLite extensi... by satoshiiii on Saturday March 18, 2023 @08:34AM (#35208570)
Actually there were only a handful of people updating it daily and it was my job to make all of those somehow give accurate results. The hundred of employees I was talking about were the ones making the sales, no direct access to those sheets, and their performance is going to be evaluated with the data that was recorded within those sheets. Data retrieval is so slow, a month of sales by 300 salesperson is around 30k rows, we have to connect that to their attendance, the day whether it is a weekday, weekend, holiday, rest day. The store they were assigned. The monthly quota they have, etc.

I lasted 3 months only, got tired of the sudden meetings discussing why can't we give fast and accurate results.

Re:Query Google Sheets as virtual tables with SQLite extensi... by cswhnjidd on Saturday March 18, 2023 @08:47AM (#35208651)
The point the person above was trying to make is that the reason you got "blank stares" was that you were proposing replacing a system where the users could read and write the data, to a system where they can't access it. Everyone saw this but you. Im sure AGI would've seen it too.

Also, the fact that you offered to do it for free makes me question your motivation. I think you were trying to make your own job easier at the cost of the users no longer having the thing that worked for them.

Re:Query Google Sheets as virtual tables with SQLite extensi... by satoshiiii on Saturday March 18, 2023 @09:28AM (#35208973)
They were using the wrong tool for the job. The R in RDBMS for the existing sheets, they were using R as in not machines but humans doing the RELATIONAL work. Yes I was trying to make my job easier, that is why mankind invented RDBMS right? To make our work easier and accurate? No?
Re:Query Google Sheets as virtual tables with SQLite extensi... by zarzavat on Sunday March 19, 2023 @08:23AM (#35218253)
The user interface is usually more important than the backend. If the user interface for a spreadsheet is more powerful, then that can make it a better solution than a database that has a better backend but terrible UI.

It can be very difficult to replace spreadsheets because the UI has had 30+ years of iteration.

Re:Query Google Sheets as virtual tables with SQLite extensi... by nonethewiser on Saturday March 18, 2023 @09:49AM (#35209109)
Can’t they though? Presumably there would be a form that validates inputs that they can use. Part of the problem that OP described is that they were writing directly to the “database” (and OP had to validate input afterwards).
Re:Query Google Sheets as virtual tables with SQLite extensi... by msravi on Saturday March 18, 2023 @08:57AM (#35208717)
So it probably would have been better to give them an excel-like frontend connected to an sql server?
Re:Query Google Sheets as virtual tables with SQLite extensi... by password4321 on Saturday March 18, 2023 @11:00AM (#35209623)
If you could match the same price, security, availability, mobile app/responsive UI support, audit history, comments, read-only sharing, export features, 3rd party add-ons, etc.

The "we will destroy all your accounts across our entire service catalog including any potential of future earnings if you cross our AI abuse detection thresholds" sword of Damocles is obviously unavailable and thus out of scope.

Re:Query Google Sheets as virtual tables with SQLite extensi... by O__________O on Saturday March 18, 2023 @09:20AM (#35208898)
Nothing you’ve described sounds like a valid reason to move “a couple of spreadsheets” to a database system which might require: budget planning, database administrator, system design, server, colocation, query optimization, report system, monitoring, user interface, backup/recovery plan, authentication system, change management logs, user training, business process analysis, etc.
Re:Query Google Sheets as virtual tables with SQLite extensi... by RadiozRadioz on Saturday March 18, 2023 @10:13AM (#35209285)
The two planks of wood across the canyon work fine, there's no valid reason to build a bridge! That would require planning, architects, engineers, safety inspections, construction, etc. Let's just keep ferrying cars across the wood planks, we all know how wood planks work, and we've got plenty of duct tape for when they break!
Re:Query Google Sheets as virtual tables with SQLite extensi... by jeffbee on Saturday March 18, 2023 @10:32AM (#35209429)
This is a deeply flawed analogy. Hosted cloud spreadsheet has effectively perfect availability and durability compared to most database solutions, and never requires maintenance, and costs nothing.
Re:Query Google Sheets as virtual tables with SQLite extensi... by RadiozRadioz on Saturday March 18, 2023 @02:07PM (#35211556)
Google Sheets, the product, may have excellent reliability, but business processes that rely on spreadsheets certainly do not. The point in my analogy was not that the planks of wood were bad products, it's that they were being misused by ignorant people.

Use wood planks as a temporary crossing, don't ferry cars over them. Run your bake sale with spreadsheets, don't run a multimillion business with them.

I don't need to link you to the resources on the astronomical number of problems that are caused by spreadsheet-oriented business processes and shadow IT. Using spreadsheets instead of proper systems shifts the operational burden onto humans, who certainly do not have perfect availability and durability.

Re:Query Google Sheets as virtual tables with SQLite extensi... by jbandela1 on Saturday March 18, 2023 @02:16PM (#35211650)
> but business processes that rely on spreadsheets certainly do not

I wonder if this migration from spreadsheets to databases is what helps ossify business culture in older companies. With spreadsheets, change and iteration are easy with fast cycles. With a database, change is a pain and iteration cycles are very long since everything has to though multiple stages of review, approval, and implementation.

Re:Query Google Sheets as virtual tables with SQLite extensi... by RadiozRadioz on Saturday March 18, 2023 @02:33PM (#35211800)
Interesting idea. I'd say probably yes, to some extent. Bringing back the human element, I'd say the businesses still stuck on spreadsheets are much less open to change than those who are willing to adopt database technology.

While long review cycles for schema changes are annoying, I find it more worrying that most businesses have zero review process at all for spreadsheets.

Re:Query Google Sheets as virtual tables with SQLite extensi... by jeffbee on Saturday March 18, 2023 @02:20PM (#35211693)
That's why your standard IT guy loves databases and hates democratized software like Sheets. You can run a 80% of business in a spreadsheet but IT Guy doesn't want to be disintermediated.
Re:Query Google Sheets as virtual tables with SQLite extensi... by RadiozRadioz on Saturday March 18, 2023 @03:04PM (#35212083)
No, as an actual IT guy myself, our job is to support and improve business technical operations. Most of us want _less_ work. If we wanted to be intermediaries, we'd be middle managers.

I don't know how much time you've spent supporting spreadsheet users, but I work with them daily and I see first-hand the amount of energy being wasted. Stuff that takes them all day, they could do in a single SQL query. Probably wouldn't take them more than a day to learn how.

It's not about democracy, it's about doing things in a sensible way. I'd be happy if they used another "democratic" tool like Microsoft PowerBI and a fully-managed database.

Re:Query Google Sheets as virtual tables with SQLite extensi... by jeffbee on Saturday March 18, 2023 @03:07PM (#35212103)
You're just gatekeeping. You have tautologically defined "sensible" to be the same as your personal preferences.
Re:Query Google Sheets as virtual tables with SQLite extensi... by RadiozRadioz on Saturday March 18, 2023 @03:31PM (#35212306)
Eh, maybe. I'll hazard that my preferences for business technical operations are more informed than those of my users. I seek only to help them.

Side note, I'm quite surprised that somebody in this form is so staunchly in favor of spreadsheets.

Re:Query Google Sheets as virtual tables with SQLite extensi... by gruturo on Saturday March 18, 2023 @10:07AM (#35209246)
I hate to break it to you, but an equally likely outcome of the introduction of AGI would be to replace you instead, and give those users exactly what they asked for, even if it's not optimal (or improving it, but preserving the interface they are used to).
Re:Query Google Sheets as virtual tables with SQLite extensi... by pphysch on Saturday March 18, 2023 @12:19PM (#35210329)
Spreadsheets have their place, not every sheet needs to be migrated to a database application. But some certainly should.
Re:Query Google Sheets as virtual tables with SQLite extensi... by aarondia on Saturday March 18, 2023 @12:26PM (#35210410)
Somewhere before "I had to procure a new VM just to open this Excel file", you should probably migrate to a database + Python
Re:Query Google Sheets as virtual tables with SQLite extensi... by cloudking on Saturday March 18, 2023 @12:22PM (#35210367)
> I really hope AGI will soon be able to replace those humans that cannot be reasoned with logic.

Great use case

Re:Query Google Sheets as virtual tables with SQLite extensi... by NoZebra120vClip on Sunday March 19, 2023 @03:54PM (#35222878)
I am not a lawyer, and I do not know your jurisdiction, but where I come from, it would be illegal for me to work off-the-clock for my employer. It opens up many cans of worms regarding liability, accounting, etc. I have to wonder whether those sorts of things were going through the heads which gave you "blank stares". Did you attempt to compromise by offering to do it on the clock instead?