News for hackers. Stuff that matters.
Re:Sensenmann: Code Deletion at Scale by UncleMeat on Saturday April 29, 2023 @04:49PM (#35757030)
"This code has been dead for six months" is a very good heuristic that the code is not relevant. I do occasionally reject the sensenmann CLs, but only very very rarely. This isn't weird code that nobody knows why it exists but it is currently doing something. This is code that cannot execute.
Re:Sensenmann: Code Deletion at Scale by ninjanomnom on Saturday April 29, 2023 @05:37PM (#35757462)
Code that only triggers from a yearly holiday, disaster alerts, leap years, or the like, would have longer periods of going unused and likely be very problematic if removed. Unless by dead code you mean unreachable code in which case it shouldn't exist in the first place and I agree should be removed.
Re:Sensenmann: Code Deletion at Scale by joshuamorton on Saturday April 29, 2023 @05:46PM (#35757557)
Yes, the nice thing about blaze/bazel + sensenmann is that you can very accurately say "this code was not built into a binary that has run in the past 6 months".

Sometimes you still want it (e.g. python scripts that are used every once in a while for ad-hoc things and might go months between uses), but usually the right thing to do is productionize stuff like that slightly more (and also test it semi-regularly to make sure it hasn't broken).

Re:Sensenmann: Code Deletion at Scale by CyberDildonics on Saturday April 29, 2023 @06:50PM (#35758046)
You can probably get most of that by just looking at the atime attribute on the file system.
Re:Sensenmann: Code Deletion at Scale by joshuamorton on Saturday April 29, 2023 @06:57PM (#35758103)
Nah, there's stuff that scans the entire repo regularly for all kinds of interesting purposes, and of that's ignoring the fact that `atime` isn't available or a source of truth in piper.

Like conceptually I believe this could be wrong in both directions, since there's heavy caching of build artifacts, you can totally build a transitive dependency of some file without actually reading the file (and potentially do this for a relatively long period of time, though I don't think that will happen in practice), and stuff will regularly look through large swaths of files that aren't necessarily run.

Re:Sensenmann: Code Deletion at Scale by CyberDildonics on Saturday April 29, 2023 @11:07PM (#35759376)
Like conceptually why wouldn't you turn on atime and like why would you cache an entire file instead of just reading it?
Re:Sensenmann: Code Deletion at Scale by joshuamorton on Sunday April 30, 2023 @04:42AM (#35760737)
> conceptually why wouldn't you turn on atime

Because piper[1] isn't a normal filesystem. It's often accessible through a FUSE-based api, so it appears like a filesystem to some users sometimes, but it can also be accessed over an RPC api. So the concept of "atime" isn't really a fit, because, well, you access the filesystem from a view that is based on your workspace, (think, akin to the git commit hash being in the filepath), and the "real" underlying file isn't necessarily on your machine.

So under a reasonable definition of atime, the atime of most files is never, because on any given arbitrary commit, you don't access/build everything.

> you cache an entire file instead of just reading it

You don't cache the file, you cache the file's outputs, keyed by a hash of the file (or all the files which are deps of a particular output). With bazel, you have a shared build artifact cache that can securely and reliably be shared across every user at a company with tens of thousands of engineers. If I build some target `:foo`, which corresponds to `foo.o`, generated from `foo.c`, but someone built `:foo` five minutes ago, as long as `:foo` hasn't changed (and you can check that the file hasn't changed without reading it because the fancy filesystem stores a hash of the file alongside the actual file), I won't actually read `foo.c` or go though the motions of building `:foo`, I can just pull `foo.o` directly from the object cache and use that in any dependencies, which means that I can build my output without ever invoking a compiler, which is really cheap and fast.

You could argue that upon reading the hash you should update the atime, but that's extremely expensive since now you have to do writes instead of idempotent reads a bunch of times a second.

[1]: https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...

Re:Sensenmann: Code Deletion at Scale by CyberDildonics on Sunday April 30, 2023 @09:15AM (#35762248)
Nah, like conceptually you could check the atime of the cached outputs too
Re:Sensenmann: Code Deletion at Scale by joshuamorton on Sunday April 30, 2023 @02:22PM (#35765202)
I mean that's functionally what they do, just more accurately since it isn't read time, but time it was built into a binary since that information is accessible, and because the object cache isn't filesystem like and doesn't have an "atime".

And also you still need to reverse from a cached output foo.o, to every transitive dependency of that, of which there could be thousands. Which can be done but requires invoking the build system to do. So it's not anything like just checking a timestamp on the file.

Re:Sensenmann: Code Deletion at Scale by CyberDildonics on Sunday April 30, 2023 @04:28PM (#35766314)
Nah, conceptually a file system is already an object cache so you can use the filesystem then like use its atime
Re:Sensenmann: Code Deletion at Scale by joshuamorton on Monday May 01, 2023 @01:07AM (#35769287)
I realize you're trolling at this point, but no a filesystem and an object cache aren't conceptually similar, in case anyone else might be confused.
Re:Sensenmann: Code Deletion at Scale by CyberDildonics on Monday May 01, 2023 @08:37AM (#35771960)
Nah, like conceptually they are both writing serialized bytes and like indexing them so they can conceptually be found easily and like read back. This is probably why you didn't like explain why they are different.
Re:Sensenmann: Code Deletion at Scale by UncleMeat on Saturday April 29, 2023 @08:16PM (#35758578)
That isn't dead code.

Dead code is shit that has a build rule and no other build rules in the entire repo depend on it. Or private functions that have no callers in an application that isn't built with any kind of reflection capabilities.

Re:Sensenmann: Code Deletion at Scale by jaggederest on Saturday April 29, 2023 @08:34PM (#35758664)
Or code that is behind if(false), System.exit or null pointer dereference. Finding those kinds of things is one of the best part of modern compilers, it's nice that they've extended that to the whole ecosystem.
Re:Sensenmann: Code Deletion at Scale by ethanwillis on Sunday April 30, 2023 @06:01AM (#35761136)
Dead code or abandoned project with goals that got forgotten but might still be important?
Re:Sensenmann: Code Deletion at Scale by taspeotis on Saturday April 29, 2023 @06:18PM (#35757814)
Different industries I guess. The new financial year comes around every 12mo. Good luck explaining to the accountants that you deleted their end-of-year reconciliation reports because they didn’t run them every 6mo.
Re:Sensenmann: Code Deletion at Scale by jeffbee on Saturday April 29, 2023 @06:44PM (#35757995)
Wouldn't it be mostly their fault for approving the removal?
Re:Sensenmann: Code Deletion at Scale by jaggederest on Saturday April 29, 2023 @08:37PM (#35758674)
I would expect that those binaries would be built and deployed constantly, even if they just woke up once a day to check whether it was mid-april or whenever the financial year ended. I would also expect that they would be integrated with other reporting that happened on a daily, weekly, monthly, and quarterly basis.

Do you really have an entire application binary that is only deployed and run once a year?

Re:Sensenmann: Code Deletion at Scale by userbinator on Sunday April 30, 2023 @05:09AM (#35760887)
would be built and deployed constantly

That's an enormously stupid waste of resources.

Do you really have an entire application binary that is only deployed and run once a year?

I have ones which are run when the need arises. It could be a year, it could be ten. But I know they work and I don't need to touch them, nor would I want to delete them.

Re:Sensenmann: Code Deletion at Scale by ElliotH on Sunday April 30, 2023 @05:58AM (#35761119)
At Google there’s not really such a thing as code that never changes. Your dependencies will shift beneath you, even if it’s just the compiler itself. More likely though the libraries you use have been updated, a robot or two have come through and applied automated fixes etc etc.

So if we’re using it once a year, (we do have tools like that, maybe not strictly annually but certainly rarely) then you do indeed want it to be automatically built on some kind of recurring basis so it’s ready to go when you need it. And the tooling makes it (relatively) easy to have that happen.

Re:Sensenmann: Code Deletion at Scale by GeneralMayhem on Sunday April 30, 2023 @06:01AM (#35761134)
If your once-a-year binary has any dependencies in common with others, then you need to at the very least be rebuilding and re-running the tests when those shared dependencies change, or else you'll find at the end of the year that it doesn't build, let alone work. And the compiler and stdlib count as shared dependencies.
Re:Sensenmann: Code Deletion at Scale by userbinator on Saturday April 29, 2023 @09:41PM (#35758974)
The fact that you can so very straightforwardly declare that code which hasn't been used in 6 months is irrelevant is extremely revealing about what leads to the deplorable state of "modern" software development.