Just out of curiosity, what's the biggest job of this type you've personally handled?

We're talking thousands of restaurants that we wanted pictures of food from, each of the restaurants had dozens of images we could pull. So tens of thousands of images needed to be sifted through, I figured with the right tooling, myself and my cofounder could put together something really nice that would only need an hour or so of maintenance a day to keep up.

So I built a pipeline that used very basic and easy to build and maintain 'dumb' Rails asset pipeline pages to present data for sifting. Go to the endpoint, it shows you the name of the restaurant and a bunch of images, you select one, type in a name for the dish, and it saves it to the database and puts up another page of images.

It took me bitching up a storm to get him to even look at it. He complained about how long he thought it would take, while I just got to work. Took maybe three weeks to prototype our app. One thing I learned in the process is that if you're looking at a bunch of Southern food, for some reason the picture of shrimp and grits always looks the most appetizing.

I was well on my way to classifying and figuring out novel ways to present the data when I had to make the determination that there wasn't good cofounder fit. So now I work with CNN.

But now all my side projects revolve around ways to get human attention to improve automated tasks. I suppose one of these days I'll get the right idea and/or the right cofounder and I'll give it another go.

There's a wealth of usable information out there on the web that one can build businesses on top of if one only wants to apply a little elbow grease to clean it and turn it into data. It's far easier to scrape data with a regular web browser with a custom browser extension than to try to build out headless infrastructure. But no one wants to do it.

Our main initiative was creating a heuristic based classifier (think lots of regex). At my own initiative, I trained ML classifiers while we worked on it. As development went on, the ML classifiers were rapidly catching up with the heuristic based one. Unfortunately it was kind of a one off data processing task, and when time ran out the regex machine was still in the lead.

I was modestly proud of the legalese DSL generator I wrote up. The lawyers didn't even know they were writing coffeescript as they typed out what documents were, what key dates were, etc. :D

That coffeescript formed the basis of our accuracy testing suite. It was as fundamental as it was huge. That team ended up creating a couple thousand tests in less than a month.

The task of verifying correctness of a pull was much more time consuming than this one. Even if it only took 30 seconds to verify (optimistic), that's 290 hours. Which isn't necessarily all that much for an organization of verifiers (or Amazon Turk), but it is a lot for an individual.

Maybe that should be the cost. But perhaps some things you might be fine with letting a bot do (after manual verification of a statistical sampling, and thorough testing).

The project is currently on hold.

The entire purpose of automation (meaning, automation in general, not just business automation) is to take humans out of the equation for mundane and repetitive tasks[1], and have them deal only with exception scenarios and edge cases cannot be properly handled by the machine (either by design or due to system limitation).

Guess what happens when you have humans approve each and every automated action like you suggest? You defeat the purpose of automation, and users end up hating the system because mundaneness and repetition are reintroduced, except in a different context.

[1]The reason you want to do this is because the more mundane and repetitive a task is, the more likely people are to make mistakes, and mistakes can be costly. In fact they are often more costly than the labor itself.

But - picking the right photo for some restaurant, as GP stated in another comment? Make good UI and perform manually. Alternatively, train NN in background - but it might not make sense in a startup world where the effort for this would be prohibitively high. In the end it's the photos that matter to your business, not that fancy photo picker algorithm that took ages to develop and that you can't sell to anyone else.

That’s not a good example though because it is unlikely to be repetitive and mundane. It’s a decision most restaurenteurs make at most a few times for each restaurant.

So, PRs then?