News for hackers. Stuff that matters.
Re:Google Workspace for Education storage policy changes by jerrac on Friday February 19, 2021 @09:41AM (#26193618)
If any Google employee reads this, how do you check how much data is in a specific drive folder? I haven't found an official method.

I need to free up space, thanks to the new storage policy. It's kinda hard to track down which directories are using the most data when there is no way to check...

Re:Google Workspace for Education storage policy changes by kyrra on Friday February 19, 2021 @11:01AM (#26194482)
(Googler, opinions are my own). I also don't work on drive, these are my rough understanding of things.

There may not be a way to do this yet, but hopefully should be coming someday. One key thing to understand is that Google Drive seems to be implemented as a graph, not a tree. This becomes apparent when you realized that any item (folder or file) could be placed into multiple directories on your drive. This allowed for cycles and lots of other crazy things within the filesystem (as you aren't guaranteed to have a single parent folder).

Now, this behavior finally changed in Sept 2020[1], and they are backfilling all mutli-links to be shortcut (symlink) based. This should finally allow for proper directory navigation and determining size. It also fixes all kinds of other problems that a graph file system causes.

[1] https://cloud.google.com/blog/products/g-suite/simplifying-g...

Re:Google Workspace for Education storage policy changes by CarelessExpert on Friday February 19, 2021 @11:49AM (#26194997)
> This allowed for cycles and lots of other crazy things within the filesystem (as you aren't guaranteed to have a single parent folder).

That doesn't prevent a folder size calculation. It does mean you need to do a graph traversal, and you might need to supplement the feature with a visualization akin to what you get from windirstat/baobab/etc, so users know what's factoring into the calculation.

It also likely means this isn't an instantaneous operation, but again, I look at windirstat/baobab as examples for how to present a UX that's workable.

But that doesn't prevent the feature from being built, and the fact that rclone already has it says to me the gdrive team has simply chosen not to make this functionality available.

Re:Google Workspace for Education storage policy changes by kyrra on Friday February 19, 2021 @11:58AM (#26195100)
I agree that it doesn't prevent it. Traversing a graph is a simple compsci problem, but optimizing it is more complicated. My guess is that it prevent caching of folder sizes, so any size checks require full graph walks, which are not cheap.
Re:Google Workspace for Education storage policy changes by kyrra on Friday February 19, 2021 @02:07PM (#26196703)
Another complication I thought of: permissions. While this may not matter as much for consumer drive product, for Google Drive within an organization, permissions are per object. So 2 people looking at the same folder may see different content. This impacts folder-size, as any given person may only see a subset of the content within a folder.
Re:Google Workspace for Education storage policy changes by deepl_derber on Friday February 19, 2021 @03:05PM (#26197436)
I think it stands to reason that if computing folder sizes was a trivial thing to implement the Drive folks would've done it by now. It's obviously useful.
Re:Google Workspace for Education storage policy changes by CarelessExpert on Friday February 19, 2021 @05:19PM (#26198763)
It equally stands to reason that by not offering this feature, they make it more difficult for users to manage their quota, which means they can drive folks to higher-priced plans. This would be the "malice" take.

Given Google's history with UX and support of existing products, the other possibility is the product is simply understaffed/undermaintained and poorly managed. This is the "incompetence" take.

And again, to emphasize: rclone can do this today. Is it a bit slow (and possibly limited)? Absolutely. But it works, and is how I've turned to managing my own gdrive quota since Google seems to be either unwilling or incapable of offering the functionality themselves.

Re:Google Workspace for Education storage policy changes by deepl_derber on Friday February 19, 2021 @08:45PM (#26200622)
I disagree - rclone can solve a substantially simpler and different problem (i.e. computing the size of a specified hierarchy on-demand). Being able to store and produce sizes of arbitrary folders in Drive for however many millions of users in a reasonable time frame is a substantially more challenging problem, and what we're talking about here. Google probably hasn't provided the on-demand version of this because it's trivial to script with their API.

Given that Google does offer you tools for managing your quota, and knowing it's a very difficult problem to solve, I'm disinclined to believe this is malicious or incompetent.

Re:Google Workspace for Education storage policy changes by CarelessExpert on Friday February 19, 2021 @10:46AM (#26194339)
I'm not a Google employee, but as far as I'm aware, there isn't one, and frankly, at this point I have to wonder if the lack of such a feature is a malicious way of preventing users from managing their storage utilization.

I've also had to resort to using rclone for this, which is absurd.

P.S. And don't forget: the data in the Trash folder counts against your quota. So if you use rclone to get folder sizes and start deleting things, you better be sure to empty the trash.

And that gets very difficult if the Trash has a lot of items in it, as the web interface will only delete a limited number of items from the Trash at a time.

As a result, you might also want to use rclone to delete permanently and bypass the Trash entirely.

Once again, while I tend to assume incompetence over malice for most things, in this case, you really gotta wonder...

Re:Google Workspace for Education storage policy changes by deepl_derber on Friday February 19, 2021 @03:06PM (#26197445)
Trash is now auto-emptied.

You can also view objects by the quota they consume at https://drive.google.com/drive/u/0/quota

Re:Google Workspace for Education storage policy changes by CarelessExpert on Friday February 19, 2021 @05:29PM (#26198874)
> Trash is now auto-emptied.

After 30 days, IIRC. That's not helpful if you're trying to manage quota now, and certainly isn't an excuse for the truly astonishingly bad UX around the trash bin.

> You can also view objects by the quota they consume at https://drive.google.com/drive/u/0/quota

Objects as in individual files. It does not show you which folders are, in aggregate, consuming large amounts of storage. As a result this is utterly useless unless your quota is taken up by a few large objects. I legitimately have no idea what use case they thought they were addressing when they built that view.

Re:Google Workspace for Education storage policy changes by deepl_derber on Friday February 19, 2021 @08:50PM (#26200657)
I think the UX around the trash bin is pretty good and I've always found the quota pane useful.
Re:Google Workspace for Education storage policy changes by jerrac on Saturday February 20, 2021 @12:19PM (#26205886)
Here are my thoughts on the issue. (To be clear, my thoughts have no input from my workplace.)

A) Why wasn't proper usage reporting built in from the start? If you are limiting a user by a usage metric, then you should be giving them a way to manage that usage intelligently. It does sound like Drive is a complicated structure, but, well, that's no excuse for not supporting your users properly.

B) I'm not surprised by Google ending the unlimited plan. But I still consider it bait and switch. Promising "unlimited" in the first place is impossible. It's false advertising. All the companies that do, or did, that, really should be fined. At very least we should hit them with a class action.

C) Google's method of ending the unlimited plan is not very good. They really should have done something like:

"We are ending the unlimited plan. New users will receive 100TB. Current users with more than 100TB of data will be granted a perpetual 120% of their current data usage as of last night at midnight UTC. Current users with less than 100TB, will have a 120TB limit."

That would have been far more fair, and much less of a bait and switch.

Anyway, thanks to everyone who replied. The tips about rclone really helped out.

Re:Google Workspace for Education storage policy changes by IceWreck on Friday February 19, 2021 @10:39AM (#26194263)
Rclone's size command does this.

rclone size gdrive_remote:/path/to/folder

Re:Google Workspace for Education storage policy changes by rmorey on Friday February 19, 2021 @12:12PM (#26195266)
rclone also has the really nifty ncdu command: https://rclone.org/commands/rclone_ncdu/
Re:Google Workspace for Education storage policy changes by jerrac on Saturday February 20, 2021 @12:00PM (#26205669)
Just tried that. It works. Thanks!
Re:Google Workspace for Education storage policy changes by jerrac on Saturday February 20, 2021 @12:00PM (#26205673)
Nice. That worked well. Thanks!