News for hackers. Stuff that matters.
Re:Welcoming Semmle to GitHub by sneak on Wednesday September 18, 2019 @02:37PM (#21008525)
Nothing is a panacea. Things that help move the needle without requiring tons of time or effort are useful and valuable. I'm really glad to see more efforts in this area.
Re:Welcoming Semmle to GitHub by DannyBee on Wednesday September 18, 2019 @03:52PM (#21009329)
While it's true that there is no pancaea, Semmle will not move the needle on vulnerability finding. This I have extensive data on.

(I mean this in terms of capability, not sudden popularity)

It would move the needle on a bunch else. It is a good tool for sure (and im very happy for them), i just think they will disappoint people by pressing this particular narrative, and wouldn't do so with a different narrative

Re:Welcoming Semmle to GitHub by intern4tional on Wednesday September 18, 2019 @04:58PM (#21010124)
I'd be curious as to why you think that. Are you able to provide more detail on that claim? I have extensive experience with Semmle and my experience drastically differs from you.

With certain languages and a strong and diverse ruleset Semmle has it's strengths. In particular with native code (C, C++) and decent rules I have seen Semmle be very successful at finding certain classes of bugs.

Re:Welcoming Semmle to GitHub by lvh on Wednesday September 18, 2019 @06:30PM (#21011083)
The Datalog part is interesting! Do they have a bunch of rules to make graph queries work nicely, like Datomic pull syntax or maybe some pattern matching syntactic sugar? Is the underlying thing still an EAVT store? Is any of that information publicly available?
Re:Welcoming Semmle to GitHub by dantiberian on Thursday September 19, 2019 @12:15AM (#21013237)
It looks like they have reasonable docs on their query language, in particular https://help.semmle.com/QL/learn-ql/about-ql.html#properties... has some info on the QL language.

https://help.semmle.com/lgtm-enterprise/user/help/generate-d... says "LGTM generates a database for each commit stored in a repository. Each database is a relational database that represents the structure of the codebase for a specific revision, or snapshot, of the code.", though a triple store could qualify as relational here. I couldn't find much more than that about the implementation details though.

Re:Welcoming Semmle to GitHub by lvh on Thursday September 19, 2019 @09:23AM (#21015890)
Right. I found those docs but they didn’t look like datalog queries at all. Of course that doesn’t mean they don’t compile down to datalog :)
Re:Welcoming Semmle to GitHub by mphi on Wednesday September 18, 2019 @11:33PM (#21013038)
Could you share the details of your experience? My experience has been quite the opposite.

I have been using Semmle daily to automate much of the vulnerability discovery process and I am extremely satisfied.

We run it over millions of lines of Java code and have not yet run into scale or perf problems.

Developing custom queries and defining security invariants in a logic language is, quite honestly, a joy.

Re:Welcoming Semmle to GitHub by lawnchair_larry on Wednesday September 18, 2019 @06:56PM (#21011331)
I would love to hear more about this data, as everyone I know who has used it for vulnerability finding has very good things to say. Semmle have also demonstrated its capabilities with some high profile examples.
Re:Welcoming Semmle to GitHub by tptacek on Wednesday September 18, 2019 @07:19PM (#21011535)
Who have you talked to about it? Outside Mozilla and Microsoft?
Re:Welcoming Semmle to GitHub by lawnchair_larry on Wednesday September 18, 2019 @10:13PM (#21012580)
I’d rather not name drop individuals or companies, partly because I don’t know that these entities want their business relationships public, but neither of the companies you named. I’ve also used it myself (you can too, at lgtm.com). I’m aware that MS is a customer, but I don’t think I’ve talked to anyone there about their experiences with it. As far as static analysis goes, which is inherently limited, it’s far better than anything else I’ve tried (which is most of them).
Re:Welcoming Semmle to GitHub by tptacek on Thursday September 19, 2019 @11:51AM (#21017351)
Fair! Thanks!
Re:Welcoming Semmle to GitHub by UncleMeat on Wednesday September 18, 2019 @04:28PM (#21009763)
Semmle does not scale, both in terms of their index design and their overall system design. This makes it poorly suited for truly global program properties and much better suited for things like refactoring.
Re:Welcoming Semmle to GitHub by muricula on Wednesday September 18, 2019 @05:06PM (#21010202)
It's being run frequently across the entire Windows OS repo. I have heard there is more work to be done to make it scale better, but it can scale.
Re:Welcoming Semmle to GitHub by intern4tional on Wednesday September 18, 2019 @05:39PM (#21010525)
Am Microsoft.

Mountains were moved to make it scale, but that has been achieved. Semmle can scale with work - it just takes a lot of effort and code.

Re:Welcoming Semmle to GitHub by DannyBee on Wednesday September 18, 2019 @09:38PM (#21012389)
Yeah, I think that is at least one of the issues. For us, for what it provides, it is not worth the time/effort vs just building our own tools or other options.
Re:Welcoming Semmle to GitHub by lawnchair_larry on Wednesday September 18, 2019 @06:50PM (#21011287)
What had to be done to make it scale?
Re:Welcoming Semmle to GitHub by mynegation on Wednesday September 18, 2019 @09:34PM (#21012372)
I was not part of this effort but I did scale another static analysis tool for industrial size codebases and have a patent on it.

To simplify a bit you can think of most static analysis algorithms in terms of graph problems where nodes are statements and functions and edges are flow of control and calls. On large codebases the amount of edges, nodes, and calculated data is just too big to keep in memory. The trick is to break the graph intelligently into parts, calculate some sort of summary information for each of them, distributing between cpus or computers, move up to the supegraph of graphs and perform higher level calculations on it.