News for hackers. Stuff that matters.
Re:PullRequest pulls in $8M Series A just months after scori... by antirez on Friday April 13, 2018 @05:58PM (#16834241)
I'm not sure what the intended audience for this is exactly, but I know for sure that in certain software that have an high degree of complexity and moving parts, to see if the PR is sane you need to be an expert of that code base. No way to make "local" decisions. (cit. Butterfly Effect)

Maybe for things like mobile applications or React or stuff like that, where there are many patterns to follow, it simpler to see if something may be wrong, but I still expect an high degree of false negatives.

Re:PullRequest pulls in $8M Series A just months after scori... by carterehsmith on Friday April 13, 2018 @07:22PM (#16834778)
Hmmm so how do you scale this business of reviewing code?

At the beginning, maybe you sign up some super high-end 10X developers to do reviews. Also run some tools like SpotBugs or Coverity or something.

Then when the ammount of reviews goes up by 10X, what are you going to do? You can't possibly find many 10X developers that are willing to review random crap code.

So you end up reselling cloud-based Coverity licenses (SpotBugs is free).

Also, this only works if the company is willing to share its code with a random startup. Which is kind of a no-go for large shops. So who do you sell to? Large shops won't allow access to their code. Startups would, but they either won't pay in principle (hey we can use SpotBugs or Coverity) or won't pay much.

Can you make a business of it? I don't know, but there are some investments being made, so I guess there is a better story than the one I see.

Re:PullRequest pulls in $8M Series A just months after scori... by lyal on Friday April 13, 2018 @08:09PM (#16834996)
Disclosure: I'm Lyal Avery, founder of PullRequest.

On functional terms, we have a couple of thousand reviewers that have signed up to review. Our tooling helps them work faster; long term, we'll also provide this tooling to internal terms.

So far, we have about a dozen of the Fortune 500 signed up; larger teams, for a variety of reasons, are more open in some cases (one of the many learnings of this startup).

Edit: signed not signing.

Re:PullRequest pulls in $8M Series A just months after scori... by carterehsmith on Friday April 13, 2018 @09:08PM (#16835234)
First off, I wish you all the best.

"we have about a dozen of the Fortune 500 signing up"

Let's be honest. What this means is that someone in the Fortune 500 clicked on your email advertisement. Maybe installed your software.

Do you have anyone in Fortune 500 running critical business on your platform?

> we have a couple of thousand reviewers that have signed up to review.

Anyone can sign up a couple thousand "Mechanical Turk" reviewers in no time. Or "RentACoder" or whatever. What is the value provided? Where I work, the value of "review" provided by someone not deeply involved in the code is "0". Which is why you ask some relevant people to review the code.

Re:PullRequest pulls in $8M Series A just months after scori... by lyal on Friday April 13, 2018 @09:57PM (#16835389)
Yes; we most definitely have teams from Fortune 500s running mission critical code through out platform. We will be releasing a case study from one in the next quarter.

I would say the difference on the reviewer side of our platform is in quality. We are working to surface this more.

Re:PullRequest pulls in $8M Series A just months after scori... by austinjp on Friday April 13, 2018 @06:58PM (#16834645)
Indeed. I'm really curious about the example I saw on their front page, apparently showing a bot commenting that "accessing this ImageView outside the main thread may cause crashes in this app". Is this genuinely possible to achieve?
Re:PullRequest pulls in $8M Series A just months after scori... by lyal on Friday April 13, 2018 @08:09PM (#16834998)
Ah! Bot in this case is the label applied to any application within Github.
Re:PullRequest pulls in $8M Series A just months after scori... by austinjp on Saturday April 14, 2018 @04:53AM (#16836550)
Okay didn't realise that, thanks. So it's humans. Well, yes, in that case I fully agree with others' comments about sustainability and scalability.
Re:PullRequest pulls in $8M Series A just months after scori... by lyal on Friday April 13, 2018 @08:25PM (#16835069)
Definitely a concern within the business -- we're trying to solve this through a combination of tooling and reassignment of reviewers to the same projects. Within tooling, we use analysis to pair developers with expertise against a project (if you've built a middleware stack in python ten times, it becomes easier to gain context and identify issues).

Totally true in places like mobile/react, or projects where teams are unlikely to have significant internal expertise despite technology in production (see a midsized game studio running an erlang server for chat) providing value is easy. We believe our approach is working, and will continue to get better.

(On a personal level -- huge fan of your work!)

Re:PullRequest pulls in $8M Series A just months after scori... by antirez on Monday April 16, 2018 @05:13AM (#16847671)
Thanks for replying directly! Btw what I said is definitely not a limit for your business IMHO, because even if you continue to have an high rate of false negatives, many businesses may find valuable to find some bugs using your service: to lower the amount of imperfections can be already good enough. After all no code review is able to spot any bug. It's just that in complex software with many moving parts, one should understand that probably two kinds of reviews are needed, more "local" reviews that can still spot certain bugs, and other conceptual reviews made by people which are very expert in the code base. Moreover, I think that for large customers, you may even have a product that involves developers becoming experts of a single code base of a large customer to provide in-depth reviews. Good luck!
Re:PullRequest pulls in $8M Series A just months after scori... by kornish on Friday April 13, 2018 @07:40PM (#16834886)
Totally. I'm also curious about consistency: if you get reviewers who prefer different idioms and stylistic points between review, what does that do for codebase style over time? I wonder how their quality control operates on the reviewer side.
Re:PullRequest pulls in $8M Series A just months after scori... by lyal on Friday April 13, 2018 @08:12PM (#16835014)
We both internally and have customer rate reviews -- we also establish corpus details with customers during the onboarding process. On the whole, we believe it's valuable to stay within the structure established within a team and nudge things forward rather than making wholesale changes to disrupt.
Re:PullRequest pulls in $8M Series A just months after scori... by encoderer on Friday April 13, 2018 @07:54PM (#16834931)
If that’s what you’re covering or expecting from code reviews you’re doing it wrong.

The only right answer here is automated code formatting and linting.

Re:PullRequest pulls in $8M Series A just months after scori... by aassddffasdf on Friday April 13, 2018 @08:07PM (#16834987)
That is not what is meant by "idioms and stylistic aspects". Take, e.g. `o.map(v => ...).getOrElse(v2)` vs `o match case Some(v) => ... case None => v2`. The former is idiomatic and preferred over the latter (but of course some jackass somewhere will disagree and argue incessantly about it).
Re:PullRequest pulls in $8M Series A just months after scori... by kornish on Friday April 13, 2018 @08:11PM (#16835011)
Exactly right. I'm not referring to which line the braces go on, but rather how domains get modeled, how errors are handled, how functional or imperative the code is, etc.
Re:PullRequest pulls in $8M Series A just months after scori... by paulddraper on Saturday April 14, 2018 @09:50PM (#16840644)
Side point, but AFIAK `fold()()` is most idiomatic in Scala.
Re:PullRequest pulls in $8M Series A just months after scori... by Nullabillity on Sunday April 15, 2018 @04:47PM (#16844863)
And yet, `match` is the only one that gets correct type inference.

`.map(ifSome).getOrElse(ifNone)` infers the type from `ifSome`, and forces `ifNone` to conform.

`.fold(ifNone)(ifSome)` infers the type from `ifNone`, and forces `ifSome` to conform.

Meanwhile, `match { case Some(x) => ifSome(x); case None(x) => ifNone }` will correctly infer from both cases at once, and give you the greatest common ancestor.

Re:PullRequest pulls in $8M Series A just months after scori... by paulddraper on Tuesday April 17, 2018 @06:19PM (#16862806)
Yeah, I agree. ¯\_(ツ)_/¯
Re:PullRequest pulls in $8M Series A just months after scori... by philip1209 on Friday April 13, 2018 @08:03PM (#16834966)
I think a big use case could be letting non-technical project owners get insight into whether their project is on-track and good. Really important for people who outsource projects on Upwork or something.

However, it does seem like the bigger need is the right code the first time. Of course, that's impossible, but this seems like a secondary need rather than the primary issue.

Re:PullRequest pulls in $8M Series A just months after scori... by siruncledrew on Sunday April 15, 2018 @12:03AM (#16841135)
Exactly. It may take a ton of time to properly review code in certain complex cases, which would be super costly. That kind of specialization seems too much for an average "Code Review as a Service" engagement.

Aside from that, I'm not sure if the employee developers will get annoyed with having a third-party come in and "tell them how to do things" or potentially mistakenly point out non-issues. I see why management may want to do this, but I wonder whether the developers will be pleased having a third-party put on their turf.

Re:PullRequest pulls in $8M Series A just months after scori... by agitator on Friday April 13, 2018 @06:03PM (#16834271)
hmm... I don't know if this would ever be worth the money. Any real engineering outfit knows that the value in code reviews isn't just in uncovering syntactic or logical flaws, but actual engineering decisions. And part of the evolution and learning process of a team is in code reviewing together. I think if you really don't give a shit about code maintainability, but want a working product, then yeah this is for you.

Companies like Atlassian should allow discounts on their bitbucket product if you allow an AI to learn from it, and then they should sell an advanced Lint or auto review feature based on training data from real code reviews. I think that's where the money is at. I wouldn't replace code reviews, but I'd use AI to sell a product that handles all the low hanging fruit for engineers so when it comes to code reviews, they only need to worry about the higher level stuff.

Re:PullRequest pulls in $8M Series A just months after scori... by jeremyjh on Friday April 13, 2018 @06:33PM (#16834463)
I agree. This sounds to like a way to get almost none of the benefits of code review at a marginally lower cost.
Re:PullRequest pulls in $8M Series A just months after scori... by bboreham on Friday April 13, 2018 @06:39PM (#16834517)
You’re basically describing Source{d} https://www.sourced.tech
Re:PullRequest pulls in $8M Series A just months after scori... by whoisjuan on Friday April 13, 2018 @06:45PM (#16834562)
They must be having some sort of decent traction to close these rounds, so that means that they have a strong use case. But honestly, I fail to understand what's that use case.

What kind of value do you get from an isolated code review from someone who is totally not familiarized with your business logic or your code base? Can someone here explain what would be a potential use case for this service?

Re:PullRequest pulls in $8M Series A just months after scori... by koolba on Friday April 13, 2018 @06:48PM (#16834582)
I have no firsthand knowledge on them in particular, but I know that large financial institutions are sorely lacking in this department. If they can pay someone and get something in return then it’s a net win. It doesn’t have to be great. Just enough that an exec can justify a line on a PowerPoint for “N bugs eliminated”.

Combine that with nose bleed pricing for said big banks and this would be a cash cow.

Re:PullRequest pulls in $8M Series A just months after scori... by poulsbohemian on Friday April 13, 2018 @06:51PM (#16834601)
But see that's the crazy irony... I guarantee you those large financial institutions used to have teams for this kind of thing, then decided they were a bottleneck and too costly, so they offshored the work or got rid of the function altogether. So they reap what they sow.
Re:PullRequest pulls in $8M Series A just months after scori... by hyperpallium on Friday April 13, 2018 @08:07PM (#16834989)
Annual license would be far less than the total cost of employing that team (roughly 2x salary).

Welcome to AI redundancy.

Re:PullRequest pulls in $8M Series A just months after scori... by sheeshkebab on Friday April 13, 2018 @06:52PM (#16834607)
Big financial institutions would have the least of interest of giving this kind of access to source code for reviews by who knows who from some cloud service.
Re:PullRequest pulls in $8M Series A just months after scori... by carterehsmith on Friday April 13, 2018 @07:32PM (#16834832)
> They must be having some sort of decent traction to close these rounds, so that means that they have a strong use case.

Hard to tell. Closing rounds is not a guarantee of anything. VC model is to throw shit against the wall and see what sticks. The model is sound. So, say, Juicero took $150M and did not stick. But Dropbox did stick. It will hopefully pay for all Juiceros and other failed investments, and then some.

Re:PullRequest pulls in $8M Series A just months after scori... by poulsbohemian on Friday April 13, 2018 @06:50PM (#16834590)
Much like the slow food movement, it feels like we need a slow software movement. The article talks about the compressed cycles leaving no room for quality assurance. Sigh. After the dotCom crash, QA / test teams went away in a lot of places because it was deemed too expensive. Time marches on and now we have a generation brainwashed into thinking unit testing and "code reviews" constitute QA, and apparently want us to buy into the idea that it "takes too long" so we'd better just outsource it. Jeez I feel old.
Re:PullRequest pulls in $8M Series A just months after scori... by lyal on Friday April 13, 2018 @08:32PM (#16835115)
Yes; unfortunately QA was used here, when QA doesn't capture what we do. Probably my fault in the interview.
Re:PullRequest pulls in $8M Series A just months after scori... by htd23 on Friday April 13, 2018 @06:29PM (#16834445)
Speaking of automating the process, "Learning From Large Codebases" https://www.srl.inf.ethz.ch/raychev_thesis.pdf won the Honorable Mention for the 2016 ACM Doctoral Dissertation Award; www.deepcode.ai is building on the technology.

From their webpage: "We automatically learn from millions of available software programs and use this knowledge to make key suggestions on how to improve your code"

Re:PullRequest pulls in $8M Series A just months after scori... by ckdarby on Saturday April 14, 2018 @07:10PM (#16840104)
Has anyone applied this to a working prototype? Unless I'm mistaken I just see ~120 pages of theory?
Re:PullRequest pulls in $8M Series A just months after scori... by 383888 on Monday April 16, 2018 @04:15PM (#16852303)
Re:PullRequest pulls in $8M Series A just months after scori... by chiefalchemist on Friday April 13, 2018 @06:23PM (#16834405)
On one hand, having the fresh eyes of an objective third party review code feels like a great idea.

What I'm stuck on is mostly the legal implications.

1) Do companies want a relatively unknown poking around in their secret sauce? Ages ago, my father worked on a number of projects at Bell Labs where parts for custom prototypes where sourced to different vendors so none knew exactly what was being built. Yes, now is not then. None the less how will this risk be mitigated (beyond contracts and background checks)?

2) And what if something does go wrong? The reviewer(s) miss something and eventually the shit hits the fan. Now what?

No show stoppers. But friction enough to be curious how they fare.

Re:PullRequest pulls in $8M Series A just months after scori... by paulie_a on Friday April 13, 2018 @06:31PM (#16834456)
Your secret sauce is your sales team and ability to get customers to use the features. Outside of dropping secret keys etc, your actual implementation in source code is nearly irrelevant. If you depend on keeping your code a total secret you are in for a world of hurt. Ex-employees certainly have copies laying around that could be leaked intentionally or just nabbed by some malware
Re:PullRequest pulls in $8M Series A just months after scori... by agitator on Friday April 13, 2018 @08:05PM (#16834980)
That's an odd position to take.... If there was nothing special about the code you worked on, then the product is probably not all that significant.

I mean, why work on something if there is no technology differentiating you?

Re:PullRequest pulls in $8M Series A just months after scori... by paulie_a on Saturday April 14, 2018 @04:19AM (#16836460)
> i mean, why work on something if there is no technology differentiating you?

Technology doesn't differentiate, value propositions do.

Re:PullRequest pulls in $8M Series A just months after scori... by fizwhiz on Friday April 13, 2018 @07:24PM (#16834800)
you do realize that trade secrets are a thing, right?
Re:PullRequest pulls in $8M Series A just months after scori... by hyperpallium on Friday April 13, 2018 @08:10PM (#16835005)
just not in enterprise code
Re:PullRequest pulls in $8M Series A just months after scori... by dchichkov on Friday April 13, 2018 @09:19PM (#16835268)
Congrats to PullRequest! They are solving a really good problem!

A shameless plug, if you are interested investing into an intellectual property of an 'underdog' competitor, contact me - [email protected] - I must have been doing something right, as I've had the exact same idea at the time and applied/interviewed with YC in the same batch as PullRequest. You can see the timeline for yourself at the internet archive of articoder.com.

Re:PullRequest pulls in $8M Series A just months after scori... by nnd on Friday April 13, 2018 @06:31PM (#16834457)
I wonder if there is a market for indie developers looking for a code review for their side projects?
Re:PullRequest pulls in $8M Series A just months after scori... by scottharveyco on Friday April 13, 2018 @08:27PM (#16835089)
This was my first thought, having a second person look at a particularly tricky PR could be quite useful sometimes.
Re:PullRequest pulls in $8M Series A just months after scori... by megadeth on Friday April 13, 2018 @05:50PM (#16834184)
How’re they achieving this ? Static analysis, linting ?
Re:PullRequest pulls in $8M Series A just months after scori... by drchiu on Friday April 13, 2018 @05:51PM (#16834193)
It looks like they use real live human reviewers. Seems like a great remote job position.
Re:PullRequest pulls in $8M Series A just months after scori... by lyal on Friday April 13, 2018 @08:16PM (#16835025)
Yes; one of the reasons I created the company was to give flexibility to people I've seen drop out of the industry for one reason or another (be it having kids, illness, etc). It's a real shame when life condition means that years of incredible experience go away.
Re:PullRequest pulls in $8M Series A just months after scori... by 505aaron on Friday April 13, 2018 @06:23PM (#16834406)
I am guessing the pitch went "It is like Uber for code review" /s

Seriously though, I am curious if the quality will remain high as they scale. I wish them luck. I would bet one of the big players acquires them within 1-2 years.

Re:PullRequest pulls in $8M Series A just months after scori... by lyal on Friday April 13, 2018 @08:29PM (#16835096)
Thanks much -- this is a company, product, and project where we have the liberty and passion to say that quality is what we care the most about.

Also: I'm lame, I went with "We're the Hotmail of Food Delivery." Still not sure how that deck worked out ;-) /s

Re:PullRequest pulls in $8M Series A just months after scori... by UK-Al05 on Friday April 13, 2018 @07:40PM (#16834889)
Yeah, it's hard enough to find quality Devs as it is. When your entire business is about high quality Dev review your going to need a lot of them.
Re:PullRequest pulls in $8M Series A just months after scori... by austinjp on Friday April 13, 2018 @07:05PM (#16834683)
Their front page shows "pullrequest [bot]" commenting on a commit. Very curious as to exactly how they're making that work.
Re:PullRequest pulls in $8M Series A just months after scori... by UK-Al05 on Friday April 13, 2018 @07:39PM (#16834878)
Defiantly humans. They have a link to sign up as a code reviewer.
Re:PullRequest pulls in $8M Series A just months after scori... by balls187 on Friday April 13, 2018 @07:58PM (#16834946)
I'd love to know how many code reviewers are developers who are performing code-reviews during their 9-5 job.
Re:PullRequest pulls in $8M Series A just months after scori... by zapita on Friday April 13, 2018 @06:07PM (#16834301)
I like the idea and think this could grow into a successful business... But my god what a terrible name.

I can't stand it when businesses appropriate terms with a well-established definition... From Salesforce calling their Ai product Einstein (complete with disgraceful cartoon mascot), to the recently launched "Stdlib"... Is it so hard to create your own unique brand and invest in that?

Re:PullRequest pulls in $8M Series A just months after scori... by kraftman on Friday April 13, 2018 @06:08PM (#16834308)
Naming things is hard!
Re:PullRequest pulls in $8M Series A just months after scori... by stephengillie on Friday April 13, 2018 @07:12PM (#16834724)
You could call it Pandora Nexus.

This reminds me of Zuckerberg naming his bot "Jarvis" - which I learned about a week after learning the dev lead on my team made a bot named "Jarvis". Unoriginal names are a pet peeve of mine. We have numerous pantheons and fandoms, yet everyone uses the same handful of names.

Re:PullRequest pulls in $8M Series A just months after scori... by dchichkov on Friday April 13, 2018 @09:24PM (#16835289)
I would guess it was YC that had suggested the name. Am I right?
Re:PullRequest pulls in $8M Series A just months after scori... by lyal on Thursday April 19, 2018 @06:00PM (#16880553)
No.