News for hackers. Stuff that matters.
Re:Rust Analyzer in 2018 and 2019 by sqs on Monday December 17, 2018 @11:39AM (#18700006)
We (Sourcegraph) have also found the “dump all the data from the compiler” approach limiting because it doesn’t work for the live editing use case (as the article says), and then that means you have 2 analyzers: one used with editors and one used for offline tools like Sourcegraph, DXR, etc. That bifurcation means each one gets less community involvement. The ideal IMO is to always target the live editing use case and add caching and optimizations that use language-specific knowledge, so that in both cases you’re querying a live server but it can compute the answers quickly. I haven’t worked with Rust specifically, but it’s interesting to see that they came to a similar conclusion.

BTW Sourcegraph is interested in funding this effort and building better Rust support for code browsing and search into our product using https://docs.sourcegraph.com/extensions. I just reached out to the author.

Re:Rust Analyzer in 2018 and 2019 by matklad on Monday December 17, 2018 @11:55AM (#18700119)
The thing is, if you target only live editing case, you can become slower for "batch compilation" case. For source browsers this probably isn't too important, b/c one can add caching using something like LSIF. However for command line compilers a similar caching won't work: in the majority of cases, you compile new code. But adding fast "batch mode" on top of "live" editing seems easier than going in the opposite direction.
Re:Rust Analyzer in 2018 and 2019 by nicoburns on Monday December 17, 2018 @12:30PM (#18700431)
How often does one do non-incremental compiles though?
Re:Rust Analyzer in 2018 and 2019 by losvedir on Monday December 17, 2018 @01:06PM (#18700707)
I think it's pretty common for CI, but I guess that use case can handle a little lower performance.
Re:Rust Analyzer in 2018 and 2019 by smaddox on Monday December 17, 2018 @01:46PM (#18701025)
And for each CI compile, how many times have devs done an incremental compile? Probably a lot.
Re:Rust Analyzer in 2018 and 2019 by woolvalley on Tuesday December 18, 2018 @02:09PM (#18709543)
Well with a good build cache, CI can do things like incremental compiles too. You can even put the cache on the network and have most of your CI use incremental compiles.
Re:Rust Analyzer in 2018 and 2019 by adrianN on Tuesday December 18, 2018 @12:57AM (#18704805)
I've worked on projects that took over an hour to compile. I would not like lower performance for CI. Many workflows require successful CI runs before one can merge for example.
Re:Rust Analyzer in 2018 and 2019 by nicoburns on Tuesday December 18, 2018 @11:40AM (#18707930)
It's really only Rust and C++ that ever take that long, right? IMO, that's simply the compiler being unacceptably slow. I also can't see a slightly less efficient parse tree being a big factor in compile times this kind of length...
Re:Rust Analyzer in 2018 and 2019 by the8472 on Monday December 17, 2018 @01:24PM (#18700837)
Eclipse does make this work for java with ECJ. It gains total and correct knowledge of the code base by instrumenting a compiler designed for incremental compilation. But it's also usable for batch compilation.
Re:Rust Analyzer in 2018 and 2019 by hinkley on Monday December 17, 2018 @07:00PM (#18703425)
There's a whole class of problems around this that I'd like to see get some more attention. It shows up for all kinds of static analysis in code editors, including things as simple as autocomplete.

For instance, if I have a file that compiles and I start typing in the space between two function bodies, I'm probably implementing a new function. In fact that probability is so high that you should just assume that to be the case, even if the code I've written so far doesn't have balanced nesting yet. All decisions and advice should be based on the speculation that the code will eventually represent a new set of blocks, not a reorganization of the remaining ones.

You can't make the same assumption about if statements inside of a function, or about incomplete multiline comments, but you can assume that anywhere from zero to all of the sibling blocks are about to have their scope changed (but not ancestors).

In short, the gap between how I'd describe a commit and how the diff tool represents it also prevents live code analysis from being able to work as well as it might.

Re:Rust Analyzer in 2018 and 2019 by matklad on Tuesday December 18, 2018 @03:00AM (#18705154)
That's not a particularly hard problem, all tools which try to solve it (for example, IntelliJ), solve it easily. For example, here's the syntax tree that rust-analyzer builds for this example:

    fn foo() { }

    fn

    fn bar() { }
Syntax tree (whitespace nodes elided for clarity):

    SOURCE_FILE@[0; 31)
      FN_DEF@[0; 12)
        FN_KW@[0; 2)
        NAME@[3; 6)
          IDENT@[3; 6) "foo"
        ...
      FN_DEF@[14; 16)
        FN_KW@[14; 16)
        err: `expected a name`
      FN_DEF@[18; 30)
        FN_KW@[18; 20)
        NAME@[21; 24)
          IDENT@[21; 24) "bar"
        ...
The problem is, a lot of tools are build with the assumption that the code is correct&complete, and don't try to tackle this problem at all.
Re:Rust Analyzer in 2018 and 2019 by hinkley on Tuesday December 18, 2018 @11:35AM (#18707900)
When llvm was young, one of its design goals was to be useful for live code analysis, to prevent that problem of having two parsers that might not agree.

Sounds like we lost some of that along the way.

Re:Rust Analyzer in 2018 and 2019 by skybrian on Tuesday December 18, 2018 @12:39AM (#18704751)
It seems like parsing both forwards in the normal way and backwards from the end of the file would be helpful? That is, take the longest suffix that parses correctly and assume any syntax errors are before that for error-reporting purposes.

Most programming languages aren't designed for parsing backwards, though!

Re:Rust Analyzer in 2018 and 2019 by hinkley on Tuesday December 18, 2018 @01:14AM (#18704857)
Backward in time, not a linear file scan.
Re:Rust Analyzer in 2018 and 2019 by n-gatedotcom on Monday December 17, 2018 @11:44AM (#18700046)
You guys created language server protocol, right? I think langserver.org is owned by sg?
Re:Rust Analyzer in 2018 and 2019 by sqs on Monday December 17, 2018 @11:47AM (#18700063)
No, Microsoft created LSP, but we’ve built a lot of language servers and added LSP for code intelligence in Sourcegraph (and on code hosts like GitHub and GitLab with https://docs.sourcegraph.com/integration/browser_extension).

(Aside: are you from n-gate.com, as your username suggests?)

Re:Rust Analyzer in 2018 and 2019 by n-gatedotcom on Monday December 17, 2018 @11:49AM (#18700082)
Oh right. I just googled. Why add two canonical sources when the original github does a fine job and is also publicly editable (by pull requests)?
Re:Rust Analyzer in 2018 and 2019 by sqs on Monday December 17, 2018 @11:59AM (#18700158)
Microsoft told us they definitely did not plan to make any kind of language server registry, so we saw a community need for an informational site to exist with more details than the wiki page they had. We’ve heard it’s useful, but we don’t want to create duplication. Want to file an issue on the langserver.org repo for discussion?
Re:Rust Analyzer in 2018 and 2019 by nerpderp82 on Monday December 17, 2018 @12:13PM (#18700281)
Love your about page!
Re:Rust Analyzer in 2018 and 2019 by AlexCoventry on Monday December 17, 2018 @07:36PM (#18703608)
You are violating The Prime Directive.
Re:Rust Analyzer in 2018 and 2019 by n-gatedotcom on Monday December 17, 2018 @09:29PM (#18704098)
What would that be?
Re:Rust Analyzer in 2018 and 2019 by AlexCoventry on Tuesday December 18, 2018 @12:19AM (#18704693)
If you can't figure it out, perhaps you are truly a Hackernews. :-)
Re:Rust Analyzer in 2018 and 2019 by n-gatedotcom on Tuesday December 18, 2018 @01:05AM (#18704830)
I don't see why one should be well versed in stupid hollywood-isms.:)And hackernews is a derogatory term that I will have you know I am very offended by. Better only than web-boy