Re:PullRequest pulls in $8M Series A just months after scori...
by antirez on Friday April 13, 2018 @05:58PM
(#16834241)
I'm not sure what the intended audience for this is exactly, but I know for sure that in certain software that have an high degree of complexity and moving parts, to see if the PR is sane you need to be an expert of that code base. No way to make "local" decisions. (cit. Butterfly Effect)
Maybe for things like mobile applications or React or stuff like that, where there are many patterns to follow, it simpler to see if something may be wrong, but I still expect an high degree of false negatives.
Re:PullRequest pulls in $8M Series A just months after scori...
by carterehsmith on Friday April 13, 2018 @07:22PM
(#16834778)
Hmmm so how do you scale this business of reviewing code?
At the beginning, maybe you sign up some super high-end 10X developers to do reviews. Also run some tools like SpotBugs or Coverity or something.
Then when the ammount of reviews goes up by 10X, what are you going to do? You can't possibly find many 10X developers that are willing to review random crap code.
So you end up reselling cloud-based Coverity licenses (SpotBugs is free).
Also, this only works if the company is willing to share its code with a random startup. Which is kind of a no-go for large shops. So who do you sell to? Large shops won't allow access to their code. Startups would, but they either won't pay in principle (hey we can use SpotBugs or Coverity) or won't pay much.
Can you make a business of it? I don't know, but there are some investments being made, so I guess there is a better story than the one I see.
Re:PullRequest pulls in $8M Series A just months after scori...
by lyal on Friday April 13, 2018 @08:09PM
(#16834996)
Disclosure: I'm Lyal Avery, founder of PullRequest.
On functional terms, we have a couple of thousand reviewers that have signed up to review. Our tooling helps them work faster; long term, we'll also provide this tooling to internal terms.
So far, we have about a dozen of the Fortune 500 signed up; larger teams, for a variety of reasons, are more open in some cases (one of the many learnings of this startup).
Re:PullRequest pulls in $8M Series A just months after scori...
by carterehsmith on Friday April 13, 2018 @09:08PM
(#16835234)
First off, I wish you all the best.
"we have about a dozen of the Fortune 500 signing up"
Let's be honest. What this means is that someone in the Fortune 500 clicked on your email advertisement. Maybe installed your software.
Do you have anyone in Fortune 500 running critical business on your platform?
> we have a couple of thousand reviewers that have signed up to review.
Anyone can sign up a couple thousand "Mechanical Turk" reviewers in no time. Or "RentACoder" or whatever. What is the value provided? Where I work, the value of "review" provided by someone not deeply involved in the code is "0". Which is why you ask some relevant people to review the code.
Re:PullRequest pulls in $8M Series A just months after scori...
by lyal on Friday April 13, 2018 @09:57PM
(#16835389)
Yes; we most definitely have teams from Fortune 500s running mission critical code through out platform. We will be releasing a case study from one in the next quarter.
I would say the difference on the reviewer side of our platform is in quality. We are working to surface this more.
Re:PullRequest pulls in $8M Series A just months after scori...
by austinjp on Friday April 13, 2018 @06:58PM
(#16834645)
Indeed. I'm really curious about the example I saw on their front page, apparently showing a bot commenting that "accessing this ImageView outside the main thread may cause crashes in this app". Is this genuinely possible to achieve?
Re:PullRequest pulls in $8M Series A just months after scori...
by lyal on Friday April 13, 2018 @08:25PM
(#16835069)
Definitely a concern within the business -- we're trying to solve this through a combination of tooling and reassignment of reviewers to the same projects. Within tooling, we use analysis to pair developers with expertise against a project (if you've built a middleware stack in python ten times, it becomes easier to gain context and identify issues).
Totally true in places like mobile/react, or projects where teams are unlikely to have significant internal expertise despite technology in production (see a midsized game studio running an erlang server for chat) providing value is easy. We believe our approach is working, and will continue to get better.
Re:PullRequest pulls in $8M Series A just months after scori...
by antirez on Monday April 16, 2018 @05:13AM
(#16847671)
Thanks for replying directly! Btw what I said is definitely not a limit for your business IMHO, because even if you continue to have an high rate of false negatives, many businesses may find valuable to find some bugs using your service: to lower the amount of imperfections can be already good enough. After all no code review is able to spot any bug. It's just that in complex software with many moving parts, one should understand that probably two kinds of reviews are needed, more "local" reviews that can still spot certain bugs, and other conceptual reviews made by people which are very expert in the code base. Moreover, I think that for large customers, you may even have a product that involves developers becoming experts of a single code base of a large customer to provide in-depth reviews. Good luck!
Re:PullRequest pulls in $8M Series A just months after scori...
by kornish on Friday April 13, 2018 @07:40PM
(#16834886)
Totally. I'm also curious about consistency: if you get reviewers who prefer different idioms and stylistic points between review, what does that do for codebase style over time? I wonder how their quality control operates on the reviewer side.
Re:PullRequest pulls in $8M Series A just months after scori...
by lyal on Friday April 13, 2018 @08:12PM
(#16835014)
We both internally and have customer rate reviews -- we also establish corpus details with customers during the onboarding process. On the whole, we believe it's valuable to stay within the structure established within a team and nudge things forward rather than making wholesale changes to disrupt.
Re:PullRequest pulls in $8M Series A just months after scori...
by aassddffasdf on Friday April 13, 2018 @08:07PM
(#16834987)
That is not what is meant by "idioms and stylistic aspects". Take, e.g. `o.map(v => ...).getOrElse(v2)` vs `o match case Some(v) => ... case None => v2`. The former is idiomatic and preferred over the latter (but of course some jackass somewhere will disagree and argue incessantly about it).
Re:PullRequest pulls in $8M Series A just months after scori...
by kornish on Friday April 13, 2018 @08:11PM
(#16835011)
Exactly right. I'm not referring to which line the braces go on, but rather how domains get modeled, how errors are handled, how functional or imperative the code is, etc.
Re:PullRequest pulls in $8M Series A just months after scori...
by Nullabillity on Sunday April 15, 2018 @04:47PM
(#16844863)
And yet, `match` is the only one that gets correct type inference.
`.map(ifSome).getOrElse(ifNone)` infers the type from `ifSome`, and forces `ifNone` to conform.
`.fold(ifNone)(ifSome)` infers the type from `ifNone`, and forces `ifSome` to conform.
Meanwhile, `match { case Some(x) => ifSome(x); case None(x) => ifNone }` will correctly infer from both cases at once, and give you the greatest common ancestor.
Re:PullRequest pulls in $8M Series A just months after scori...
by philip1209 on Friday April 13, 2018 @08:03PM
(#16834966)
I think a big use case could be letting non-technical project owners get insight into whether their project is on-track and good. Really important for people who outsource projects on Upwork or something.
However, it does seem like the bigger need is the right code the first time. Of course, that's impossible, but this seems like a secondary need rather than the primary issue.
Re:PullRequest pulls in $8M Series A just months after scori...
by siruncledrew on Sunday April 15, 2018 @12:03AM
(#16841135)
Exactly. It may take a ton of time to properly review code in certain complex cases, which would be super costly. That kind of specialization seems too much for an average "Code Review as a Service" engagement.
Aside from that, I'm not sure if the employee developers will get annoyed with having a third-party come in and "tell them how to do things" or potentially mistakenly point out non-issues. I see why management may want to do this, but I wonder whether the developers will be pleased having a third-party put on their turf.
Re:PullRequest pulls in $8M Series A just months after scori...
by agitator on Friday April 13, 2018 @06:03PM
(#16834271)
hmm... I don't know if this would ever be worth the money. Any real engineering outfit knows that the value in code reviews isn't just in uncovering syntactic or logical flaws, but actual engineering decisions. And part of the evolution and learning process of a team is in code reviewing together. I think if you really don't give a shit about code maintainability, but want a working product, then yeah this is for you.
Companies like Atlassian should allow discounts on their bitbucket product if you allow an AI to learn from it, and then they should sell an advanced Lint or auto review feature based on training data from real code reviews. I think that's where the money is at. I wouldn't replace code reviews, but I'd use AI to sell a product that handles all the low hanging fruit for engineers so when it comes to code reviews, they only need to worry about the higher level stuff.
Re:PullRequest pulls in $8M Series A just months after scori...
by whoisjuan on Friday April 13, 2018 @06:45PM
(#16834562)
They must be having some sort of decent traction to close these rounds, so that means that they have a strong use case. But honestly, I fail to understand what's that use case.
What kind of value do you get from an isolated code review from someone who is totally not familiarized with your business logic or your code base? Can someone here explain what would be a potential use case for this service?
Re:PullRequest pulls in $8M Series A just months after scori...
by koolba on Friday April 13, 2018 @06:48PM
(#16834582)
I have no firsthand knowledge on them in particular, but I know that large financial institutions are sorely lacking in this department. If they can pay someone and get something in return then it’s a net win. It doesn’t have to be great. Just enough that an exec can justify a line on a PowerPoint for “N bugs eliminated”.
Combine that with nose bleed pricing for said big banks and this would be a cash cow.
Re:PullRequest pulls in $8M Series A just months after scori...
by poulsbohemian on Friday April 13, 2018 @06:51PM
(#16834601)
But see that's the crazy irony... I guarantee you those large financial institutions used to have teams for this kind of thing, then decided they were a bottleneck and too costly, so they offshored the work or got rid of the function altogether. So they reap what they sow.
Re:PullRequest pulls in $8M Series A just months after scori...
by sheeshkebab on Friday April 13, 2018 @06:52PM
(#16834607)
Big financial institutions would have the least of interest of giving this kind of access to source code for reviews by who knows who from some cloud service.
Re:PullRequest pulls in $8M Series A just months after scori...
by carterehsmith on Friday April 13, 2018 @07:32PM
(#16834832)
> They must be having some sort of decent traction to close these rounds, so that means that they have a strong use case.
Hard to tell. Closing rounds is not a guarantee of anything. VC model is to throw shit against the wall and see what sticks. The model is sound. So, say, Juicero took $150M and did not stick. But Dropbox did stick. It will hopefully pay for all Juiceros and other failed investments, and then some.
Re:PullRequest pulls in $8M Series A just months after scori...
by poulsbohemian on Friday April 13, 2018 @06:50PM
(#16834590)
Much like the slow food movement, it feels like we need a slow software movement. The article talks about the compressed cycles leaving no room for quality assurance. Sigh. After the dotCom crash, QA / test teams went away in a lot of places because it was deemed too expensive. Time marches on and now we have a generation brainwashed into thinking unit testing and "code reviews" constitute QA, and apparently want us to buy into the idea that it "takes too long" so we'd better just outsource it. Jeez I feel old.
Re:PullRequest pulls in $8M Series A just months after scori...
by htd23 on Friday April 13, 2018 @06:29PM
(#16834445)
Speaking of automating the process, "Learning From Large Codebases" https://www.srl.inf.ethz.ch/raychev_thesis.pdf won the Honorable Mention for the 2016 ACM Doctoral Dissertation Award; www.deepcode.ai is building on the technology.
From their webpage: "We automatically learn from millions of available software programs and use this knowledge to make key suggestions on how to improve your code"
Re:PullRequest pulls in $8M Series A just months after scori...
by chiefalchemist on Friday April 13, 2018 @06:23PM
(#16834405)
On one hand, having the fresh eyes of an objective third party review code feels like a great idea.
What I'm stuck on is mostly the legal implications.
1) Do companies want a relatively unknown poking around in their secret sauce? Ages ago, my father worked on a number of projects at Bell Labs where parts for custom prototypes where sourced to different vendors so none knew exactly what was being built. Yes, now is not then. None the less how will this risk be mitigated (beyond contracts and background checks)?
2) And what if something does go wrong? The reviewer(s) miss something and eventually the shit hits the fan. Now what?
No show stoppers. But friction enough to be curious how they fare.
Re:PullRequest pulls in $8M Series A just months after scori...
by paulie_a on Friday April 13, 2018 @06:31PM
(#16834456)
Your secret sauce is your sales team and ability to get customers to use the features. Outside of dropping secret keys etc, your actual implementation in source code is nearly irrelevant. If you depend on keeping your code a total secret you are in for a world of hurt. Ex-employees certainly have copies laying around that could be leaked intentionally or just nabbed by some malware
Re:PullRequest pulls in $8M Series A just months after scori...
by dchichkov on Friday April 13, 2018 @09:19PM
(#16835268)
Congrats to PullRequest! They are solving a really good problem!
A shameless plug, if you are interested investing into an intellectual property of an 'underdog' competitor, contact me - [email protected] - I must have been doing something right, as I've had the exact same idea at the time and applied/interviewed with YC in the same batch as PullRequest. You can see the timeline for yourself at the internet archive of articoder.com.
Re:PullRequest pulls in $8M Series A just months after scori...
by lyal on Friday April 13, 2018 @08:16PM
(#16835025)
Yes; one of the reasons I created the company was to give flexibility to people I've seen drop out of the industry for one reason or another (be it having kids, illness, etc). It's a real shame when life condition means that years of incredible experience go away.
Re:PullRequest pulls in $8M Series A just months after scori...
by 505aaron on Friday April 13, 2018 @06:23PM
(#16834406)
I am guessing the pitch went "It is like Uber for code review" /s
Seriously though, I am curious if the quality will remain high as they scale. I wish them luck. I would bet one of the big players acquires them within 1-2 years.
Re:PullRequest pulls in $8M Series A just months after scori...
by zapita on Friday April 13, 2018 @06:07PM
(#16834301)
I like the idea and think this could grow into a successful business... But my god what a terrible name.
I can't stand it when businesses appropriate terms with a well-established definition... From Salesforce calling their Ai product Einstein (complete with disgraceful cartoon mascot), to the recently launched "Stdlib"... Is it so hard to create your own unique brand and invest in that?
Re:PullRequest pulls in $8M Series A just months after scori...
by stephengillie on Friday April 13, 2018 @07:12PM
(#16834724)
You could call it Pandora Nexus.
This reminds me of Zuckerberg naming his bot "Jarvis" - which I learned about a week after learning the dev lead on my team made a bot named "Jarvis". Unoriginal names are a pet peeve of mine. We have numerous pantheons and fandoms, yet everyone uses the same handful of names.
Maybe for things like mobile applications or React or stuff like that, where there are many patterns to follow, it simpler to see if something may be wrong, but I still expect an high degree of false negatives.
At the beginning, maybe you sign up some super high-end 10X developers to do reviews. Also run some tools like SpotBugs or Coverity or something.
Then when the ammount of reviews goes up by 10X, what are you going to do? You can't possibly find many 10X developers that are willing to review random crap code.
So you end up reselling cloud-based Coverity licenses (SpotBugs is free).
Also, this only works if the company is willing to share its code with a random startup. Which is kind of a no-go for large shops. So who do you sell to? Large shops won't allow access to their code. Startups would, but they either won't pay in principle (hey we can use SpotBugs or Coverity) or won't pay much.
Can you make a business of it? I don't know, but there are some investments being made, so I guess there is a better story than the one I see.
On functional terms, we have a couple of thousand reviewers that have signed up to review. Our tooling helps them work faster; long term, we'll also provide this tooling to internal terms.
So far, we have about a dozen of the Fortune 500 signed up; larger teams, for a variety of reasons, are more open in some cases (one of the many learnings of this startup).
Edit: signed not signing.
"we have about a dozen of the Fortune 500 signing up"
Let's be honest. What this means is that someone in the Fortune 500 clicked on your email advertisement. Maybe installed your software.
Do you have anyone in Fortune 500 running critical business on your platform?
> we have a couple of thousand reviewers that have signed up to review.
Anyone can sign up a couple thousand "Mechanical Turk" reviewers in no time. Or "RentACoder" or whatever. What is the value provided? Where I work, the value of "review" provided by someone not deeply involved in the code is "0". Which is why you ask some relevant people to review the code.
I would say the difference on the reviewer side of our platform is in quality. We are working to surface this more.
Totally true in places like mobile/react, or projects where teams are unlikely to have significant internal expertise despite technology in production (see a midsized game studio running an erlang server for chat) providing value is easy. We believe our approach is working, and will continue to get better.
(On a personal level -- huge fan of your work!)
The only right answer here is automated code formatting and linting.
`.map(ifSome).getOrElse(ifNone)` infers the type from `ifSome`, and forces `ifNone` to conform.
`.fold(ifNone)(ifSome)` infers the type from `ifNone`, and forces `ifSome` to conform.
Meanwhile, `match { case Some(x) => ifSome(x); case None(x) => ifNone }` will correctly infer from both cases at once, and give you the greatest common ancestor.
However, it does seem like the bigger need is the right code the first time. Of course, that's impossible, but this seems like a secondary need rather than the primary issue.
Aside from that, I'm not sure if the employee developers will get annoyed with having a third-party come in and "tell them how to do things" or potentially mistakenly point out non-issues. I see why management may want to do this, but I wonder whether the developers will be pleased having a third-party put on their turf.
Companies like Atlassian should allow discounts on their bitbucket product if you allow an AI to learn from it, and then they should sell an advanced Lint or auto review feature based on training data from real code reviews. I think that's where the money is at. I wouldn't replace code reviews, but I'd use AI to sell a product that handles all the low hanging fruit for engineers so when it comes to code reviews, they only need to worry about the higher level stuff.
What kind of value do you get from an isolated code review from someone who is totally not familiarized with your business logic or your code base? Can someone here explain what would be a potential use case for this service?
Combine that with nose bleed pricing for said big banks and this would be a cash cow.
Welcome to AI redundancy.
Hard to tell. Closing rounds is not a guarantee of anything. VC model is to throw shit against the wall and see what sticks. The model is sound. So, say, Juicero took $150M and did not stick. But Dropbox did stick. It will hopefully pay for all Juiceros and other failed investments, and then some.
From their webpage: "We automatically learn from millions of available software programs and use this knowledge to make key suggestions on how to improve your code"
What I'm stuck on is mostly the legal implications.
1) Do companies want a relatively unknown poking around in their secret sauce? Ages ago, my father worked on a number of projects at Bell Labs where parts for custom prototypes where sourced to different vendors so none knew exactly what was being built. Yes, now is not then. None the less how will this risk be mitigated (beyond contracts and background checks)?
2) And what if something does go wrong? The reviewer(s) miss something and eventually the shit hits the fan. Now what?
No show stoppers. But friction enough to be curious how they fare.
I mean, why work on something if there is no technology differentiating you?
Technology doesn't differentiate, value propositions do.
A shameless plug, if you are interested investing into an intellectual property of an 'underdog' competitor, contact me - [email protected] - I must have been doing something right, as I've had the exact same idea at the time and applied/interviewed with YC in the same batch as PullRequest. You can see the timeline for yourself at the internet archive of articoder.com.
Seriously though, I am curious if the quality will remain high as they scale. I wish them luck. I would bet one of the big players acquires them within 1-2 years.
Also: I'm lame, I went with "We're the Hotmail of Food Delivery." Still not sure how that deck worked out ;-) /s
I can't stand it when businesses appropriate terms with a well-established definition... From Salesforce calling their Ai product Einstein (complete with disgraceful cartoon mascot), to the recently launched "Stdlib"... Is it so hard to create your own unique brand and invest in that?
This reminds me of Zuckerberg naming his bot "Jarvis" - which I learned about a week after learning the dev lead on my team made a bot named "Jarvis". Unoriginal names are a pet peeve of mine. We have numerous pantheons and fandoms, yet everyone uses the same handful of names.