News for hackers. Stuff that matters.
Re:Glean – System for collecting, deriving and querying fact... by lazamar on Tuesday August 31, 2021 @07:22AM (#28366448)
Glean is focused on storing and querying data about the code. The idea is that you have your own program to collect that data, then you use Glean to store that compactly and to have snappy queries.

You would create entries like "this is a declaration of X", "this is a use of X". Then you can query things like "give me all uses of X" in sub-millisecond time. You hook that up to an LSP server then you get almost zero-cost find-references, jump-to-definition, etc. The snappy queries also mean it becomes possible to perform whole codebase (and cross-language) analysis. That is, answering questions like "what code is not referenced from this root?", "does this Haskell function use anything that calls malloc?" (analysis through the ffi barrier).

One can also attach all kinds of information from different sources to code entities, not only things derived from the source itself. You add things like run-time costs, frequency of use, common errors, etc, and an LSP server could make all of it available right in your editor.

For very large or complex codebases, where it is just too expensive or too complicated to calculate this information locally a system like this becomes very useful.

Re:Glean – System for collecting, deriving and querying fact... by soonnow on Tuesday August 31, 2021 @07:35AM (#28366565)
> For very large or complex codebases, where it is just too expensive or too complicated to calculate this information locally a system like this becomes very useful.

Thanks I guess I get it now. But to enable this functionality you'd need to have some form of frontend or integration into the existing build lifecycle?

Or IDE integration I guess.

Re:Glean – System for collecting, deriving and querying fact... by scns on Tuesday August 31, 2021 @08:39AM (#28367230)
Oh wow, mindblowing stuff. Glad to see tech like this being open sourced, fuels the imagination about possible future scenarios. Do you use it on the Linux Kernel?
Re:Glean – System for collecting, deriving and querying fact... by minxomat on Tuesday August 31, 2021 @08:50AM (#28367374)
A comparable, powerful system (CodeQL) was used recently on the kernel[1] and Chrome. You can learn more about it here: https://codeql.github.com/docs/codeql-overview/about-codeql/

(disclosure: I work at GH on CQL)

[1] https://pwning.systems/posts/sequoia-variant-analysis/

Re:Glean – System for collecting, deriving and querying fact... by X6S1x6Okd1st on Tuesday August 31, 2021 @01:34PM (#28371012)
Re:Glean – System for collecting, deriving and querying fact... by gigatexal on Tuesday August 31, 2021 @09:18AM (#28367693)
Thank you for this summary I was unsure of how this is really useful. That before step is missing I think.