News for hackers. Stuff that matters.
Re:Automation Should Be Like Iron Man, Not Ultron by erikb on Saturday May 21, 2016 @08:03AM (#11744393)
I remember around 2010 people adviced the same principle but with different words. At this time Go was still a game that couldn't be beaten by AI alone, so it was taken as example. And people found that if you let a person play, he can only achieve that much, mostly disappointing compared to perfection (with some exceptions), if you write an AI it won't be able to solve all the problems themselves. But if you combine them, using a human player that gets calculation backup by AI, that draws diagrams ( heat maps in case of Go), that gathers data, this is what enables the human to perform on professional level (about 3dan pro if I remember correctly) despite being a total beginner (15 kyu on Kiseido Go Server).

I completely agree with that and believe that the profesional and scientific world is actually also developing in that direction.

Re:Automation Should Be Like Iron Man, Not Ultron by jfoutz on Saturday May 21, 2016 @11:23AM (#11744950)
In 1968 Douglas Engelbart used the wonderfully space aged term "brain amplification" in his mother of all demos.

In practice, i'm not sure how it actually works. Systems work can be automated away. And it becomes brittle. Something like emacs or vim absolutely feel like iron man armor. I can't really see what vim for systems work would look like.

My vague thoughts are, maybe build systems that give lots of information about what's happening right now, and how it's different than the past. Lots of commands for introspection. some commands for intervention. At scale, picking a specific machine to remove from load balancing pool or restart or destroy and reallocate at a keystroke might be handy.

It's an interesting perspective, but the difference seems subtle in most practical contexts.

Re:Automation Should Be Like Iron Man, Not Ultron by falcolas on Saturday May 21, 2016 @11:39AM (#11745025)
Here's one example from my experience:

There are many MySQL tools which are able to automate the process of failing over between a master and a slave. Github initially chose a solution which would trigger this failover automatically, when a master was unresponsive for a particular amount of time. One bad query came in, and the tool did it's job by failing over from the master to the slave. Here's the problem - the query was automatically retried against the slave, causing it to fail over back to the master. Repeatedly.

The solution? Page when MySQL became unresponsive, let a human initiate the failover, and let the tools handle the failover itself (which is a very complex dance to perform manually). I think they eventually moved to a model where it would fail over once per few hours, and alert if the condition occurred again.

Re:Automation Should Be Like Iron Man, Not Ultron by MikeTLive on Saturday May 21, 2016 @03:30PM (#11745841)
The same questions a human will ask can be encoded into your admin automation. It is only as powerful or accurate as the process allows.

If you don't already have a procedural flowchart build one. Have the human document the questions asked and answers followed.

Re:Automation Should Be Like Iron Man, Not Ultron by falcolas on Saturday May 21, 2016 @06:46PM (#11746537)
Only if you can foresee the problems and solutions to encode into the system. If you miss one, the system will fall down or make the wrong decision.
Re:Automation Should Be Like Iron Man, Not Ultron by XorNot on Saturday May 21, 2016 @10:49PM (#11747185)
That's a problem of historical data. What are you really doing when something "seems unusual"?
Re:Automation Should Be Like Iron Man, Not Ultron by virgilp on Monday May 23, 2016 @06:58AM (#11753027)
You're using your best judgment?
Re:Automation Should Be Like Iron Man, Not Ultron by sampo on Saturday May 21, 2016 @03:32PM (#11745849)
> In 1968 Douglas Engelbart used the wonderfully space aged term "brain amplification"

And Steve Jobs in 1990, "And that’s what a computer is to me ... it’s the equivalent of a bicycle for our minds."

https://www.youtube.com/watch?v=ob_GX50Za6c#t=25s

Re:Automation Should Be Like Iron Man, Not Ultron by danudey on Sunday May 22, 2016 @12:41AM (#11747394)
Here's how I built our deployment system at work for our various interdependent packages and services, each of which is a git repository (and where libraries are symlinked into those git repositories as needed).

First, I built a tool called checkout_package which clones/updates a package and then recursively installs and symlinks all of its dependencies. Devs used this for local installations as well as on production servers.

After that had its problems ironed out and everything was working, I built a system called command_runner, which was a simple JSON-based RPC daemon. This had the ability, among other things, to call out to command_runner and return the result back to the callee. Now our developers could run command_runner, which we knew worked (and knew how it worked), on remote machines without having to log in. A simple shell script was all that was required to loop over every machine to update them all.

After that, we built a web interface that could let you choose which packages you needed to update, and it would run command_runner in X number of threads to do the updates concurrently. It also returns the output from command_runner (which is extremely terse if everything went fine) back to the user of the web interface so they could tell when something went wrong, and retry if necessary.

We also use monit. I added functionality to command_runner to run and parse 'monit summary' to see what was installed on and running/not running on a given server, and to run monit commands (start, stop, or 'restart', i.e. stop and then start) on a given server. From there, we followed the same pattern: shell script to iterate, integrate into web interface.

Eventually, we tied the two together. After you deploy, it takes you to a version of the services management page that only shows (and already has selected) the services which are, or depend on, the packages you deployed.

None of our system abstracts away any of the details of what's happening; it merely provides an easier way to do what you would be doing anyway, and provides more information than you would normally have (for example, the package deployment page can show you the git diff between what's live and what's HEAD, so that you know whose changes you're about to deploy). Because the devs still use checkout_package at the command line in their local environments, it requires that tool to remain both usable and sufficiently user-friendly, and for nothing to be wrapped up into any other layers of automation that is necessary for doing the fundamentals.

This way, we have managed to prevent our deployment process from becoming impossible to reproduce by hand if necessary (e.g. when deploying/rolling back changes to our deployment system).

The system provides extra information that you wouldn't have had, does some tasks for you that you don't have to do manually anymore, and does stuff for you all at once instead of having to do the same task over and over again per server, but fundamentally it's more like cruise control than a self-driving car.

Re:Automation Should Be Like Iron Man, Not Ultron by Houshalter on Saturday May 21, 2016 @08:13PM (#11746834)
That works because computer game playing was totally different than how humans play. Humans can recognize common patterns, and computers can brute force many possible positions and evaluate them. The two complement each other.

But now AlphaGo beat humans using neural networks that can learn to recognize common patterns similar to humans. And it was combined with tree search to evaluate many more possible positions than a human could.

So I doubt human and AlphaGo would have any advantage over just AlphaGo. In the future the same will become true in many other domains.

Re:Automation Should Be Like Iron Man, Not Ultron by donlzx on Saturday May 21, 2016 @08:48PM (#11746936)
I actually think AlphaGo is a perfect example of Human-machine cooperation. Deep neural network is the easier part, the intuition and architecture design by Humans are the most critical part.
Re:Automation Should Be Like Iron Man, Not Ultron by barkingdog on Saturday May 21, 2016 @09:06PM (#11746990)
Here's where your analogy breaks down: what if the state of AlphaGo enhancers was as good as AlphaGo AIs? Could the current state of the art AI beat a human working with a state of the art augmented AI based assistant?

The lack of an answer is an interesting question, and I wonder how much it has to do with cultural concerns. Obviously, "machine beats man at intelligent task" is a lot sexier in some ways.

Re:Automation Should Be Like Iron Man, Not Ultron by rqebmm on Saturday May 21, 2016 @10:33PM (#11747165)
> I doubt human and AlphaGo would have any advantage over just AlphaGo

I know what you're saying, but I think it's important to remember that AlphaGo was created by humans. The nature of Go pattern recognition was almost assuredly factored into the creation of the algorithm.

Re:Automation Should Be Like Iron Man, Not Ultron by srtjstjsj on Sunday May 22, 2016 @12:32AM (#11747374)
No. AlphaGo used a general-purpose algorithm with basic Go rules and novice-strategy optimizations.