Re:The Art of Automation
by tuatoru on Saturday September 26, 2020 @06:35PM
(#24602712)
Randall Munroe has done at least two cartoons about this.
My own experience has led me to the point where if I'm thinking about automating something, I stop and ask "is there a way that this can be done away with completely?"
The most reliable, easiest to maintain scripts are those with zero lines of code. They also perform best and consume least resources.
With a bit of thought and/or flexibility in how things are achieved, it's surprising how often this question produces a good result.
Re:The Art of Automation
by tyingq on Sunday September 27, 2020 @09:14AM
(#24606383)
This certainly holds true for a lot of approval workflows. I've been in big companies where my ~$300 Visio purchase flowed to a VP for approval. They could have easily done an auto-approve based on job title. It's not like the VP, who had hundreds of people on his team, would know if I needed it or not.
Re:The Art of Automation
by jrott on Sunday September 27, 2020 @12:37PM
(#24607899)
Approval workflows have so many problems. I've seen people that have to make tons of approvals but it's not there real job just approve everything that comes through since they don't want to break things. Almost completely destroyed the security point of making people review stuff for access control. It does fix things in a pure compliance sense so it'll continue.
Re:The Art of Automation
by jka on Sunday September 27, 2020 @08:06AM
(#24606010)
Your comment predicts the next few waves of no/low-code automation, I think.
Early phases will involve the introduction of no/low-code automations that integrate existing APIs, services, and systems within businesses for the purposes of automating existing processes.
That's good for the APIs, systems and automation platforms because they become essential and encumbent components of operating those businesses.
The later, and I suspect much longer, phases will involve identification of business processes that aren't really needed and that can be removed by thinking about the ways that organizations function in entirely new ways, or that can be streamlined.
By that point some of the integrations may have become so baked-in that they'll require significant maintenance to update. And as we see with some cloud-based IoT devices, it's possible for service providers to encounter outages or cease operating, in which case internal businesses processes may halt.
All this is similar to the gradual automation and improvement of physical assembly lines; it's a case of applying the same principles to organizations. And yep, the cheapest, most efficient and reliable component is one that doesn't have to exist.
Re:The Art of Automation
by ethbr0 on Monday September 28, 2020 @03:28AM
(#24613936)
So, there's something of a story behind this.
To simplify things, the early phase you mentioned is simply computing, and has been proceeding since the 1960s. The latter phase is called business process management (aka BPM) [1] and has been a thing since ~1980(?).
In promise, BPM says "Encode all your business processes and their relationships into our tool, and we'll optimize and automate them." In practice, BPM is doomed to failure because (1) it's invariably implemented in an organization by management consultant types who care more about billable hours than actual solutions & (2) it's fundamentally an imperfect abstraction over your actual business processes, leading to leaks / hacks in implementation, leading to yet another system that further slows down the business.
SAP is illustrative here, with the adage that smart businesses change their business to fit SAP, rather than try to change SAP to fit their business.
Re:The Art of Automation
by rubin55 on Sunday September 27, 2020 @04:08AM
(#24605054)
A thing I usually find lacking in others’ “automate-all-the-things” scripts/apps/pipelines/etc is documentation. Without good documentation you only temporarily kept time-consuming chaos/manual-labor at bay at the cost of more complexity/entropy in the future. In my opinion, without docs, it eventually becomes a burden on (future) colleagues instead of a nicely consistent automated-away process. Don’t automate (just) for yourself, automate for others, where docs empower others to be autonomous and able to manage said automation without your head around.
Re:The Art of Automation
by DavidPeiffer on Sunday September 27, 2020 @09:34AM
(#24606489)
I agree documentation is super important! It's I recently started at a new company and have again been implementing the framework described in Manual Work is a Bug[1], summarized as:
1) Document the Steps
2) Create Automation Equivalents
3) Create Automation
4) Self-Service and Autonomous Systems
When I started going through stuff with my coworker, I made a Word document with bullets and sub-bullets for all the steps in a process. When they unexpectedly needed to take a week leave of absence a month into the job, I was able to complete almost every routine part of the job. It took a ton of stress off of them!
Across all my professional jobs, I find #1 to be the hardest. My process is to write down the high-level steps, then strictly follow the documentation next time I do the process, adding more and more detail every time.
Once it's decently documented, I record how long each step takes, and add comments like "This needs to be better documented" or "This is the most tedious step, and would be pretty simple to automate". Timing each step lets me start and complete something between meetings, filling in gaps.
Overall I've found the system to be really helpful. It makes transitioning job duties to new employees trivial and has saved me literally hundreds of hours over just a few years in the professional workforce.
Re:The Art of Automation
by dgudkov on Sunday September 27, 2020 @05:48AM
(#24605472)
Good point. EasyMorph (I'm the founder) auto-generates human-readable HTML documentation on workflows in plain English. It's not perfect, but not bad either.
Re:The Art of Automation
by nimrody on Sunday September 27, 2020 @12:52PM
(#24608021)
There was a post here a few weeks ago reminding us of the dangers of this approach. I don't have the link but the essence is here:
After a process gets automated, people forget the details of how to do the work manually. Now, when the automation breaks (and it will break at some point. It takes a huge amount of work to make these things failsafe and scripting doesn't really encourage robustness) -- no one remembers the details and fixing the issue is more difficult and requires specialized knowledge that is no longer common in the organization.
Re:The Art of Automation
by gglitch on Sunday September 27, 2020 @01:30PM
(#24608374)
This sounds to me like not a failure of automation as such, but of process/knowledge management by the team that builds the automation. No process is permanently exempt from entropy.
Re:The Art of Automation
by mxuribe on Sunday September 27, 2020 @01:21PM
(#24608272)
Totally agree, and i've seen this many times (and have had my teams "clean up" after others). however, what can help mitigate - maybe not fully prevent - in some cases are the following 2 approaches:
1. Build automation so that it is NOT such a blackbox. I even go so far as to build automation with scripting languages...that allows myself/my team to at least peek at the source code to see what is happening.
2. And, of course documentation. i know this is simplistic of me to state...but it is invaluable nonethelss. I should also clarify that i mean there should exist documentation both for users, or by extension those who benefit from the automation, as well as, for developers/sys admins/sys maintainers (say, within the code itself).
Re:The Art of Automation
by iamwil on Sunday September 27, 2020 @04:52AM
(#24605209)
I started curating a bunch of automation stories, use-cases, and ideas at https://automationcookbook.io. It is usually more business process automation, rather than personal automation, though.
Re:The Art of Automation
by asicsp on Sunday September 27, 2020 @04:35AM
(#24605145)
ATBS is also free to read online [0] and the author has _automated_ to provide free coupons [1] (applicable during first week of a month) for the online course on udemy
I'd also suggest realpython [2] for resources on web scraping
Re:The Art of Automation
by wombatmobile on Sunday September 27, 2020 @08:00AM
(#24605978)
What you are describing is experiential change. Instead of feeling beholden to a task, which in your equation is negative time, automation frees you to have opportunity for other activities that you might experience as positive time.
Another way to view these experiences is that they all take place in the finite amount of time that you have here on Earth. That time doesn't change. Only your experience of it changes.
You can control your experiences by cultivating your thoughts.
For example, you might enjoy the time it takes you to design and build a system for automating a task. Contrast that experience with that of somebody who hates the idea of doing work to automate a task, and would rather just get the task over and done with, and never consider automation.
Who is right, and who is wrong?
You could calculate an answer by applying your maths, and that would be one answer, which might come out in your favour, or might not, depending on whether you could automate a task in less time than it would take to perform the task.
But you already know that you're coming out ahead, even if it takes you 10 times longer to design and implement the automation than it does to perform the task manually, because you enjoy the process of automating tasks. That's because you've cultivated your thoughts.
Re:The Art of Automation
by samus on Sunday September 27, 2020 @08:51AM
(#24606258)
> But you already know that you're coming out ahead, even if it takes you 10 times longer to design and implement the automation than it does to perform the task manually, because you enjoy the process of automating tasks. That's because you've cultivated your thoughts.
This is only true under the assumption that people automate stuff for fun. Businesses have a different incentive. They want automation to enable their pricey employees to take care of more tricky tasks, or to simply require fewer of them in the first place. A business won't give leave to their employees to automate processes just because they enjoy automating stuff.
Re:The Art of Automation
by wombatmobile on Sunday September 27, 2020 @04:42PM
(#24609990)
You are writing about commercial profit and loss, which is potentially infinite in duration, and reducible down to one integer.
I am writing about a human being experiencing a finite life, expansively.
Must these two domains be mutually exclusive, or conflicting? If business only existed for profit, they might be, but is that the only reason businesses exist?
Re:The Art of Automation
by walterbell on Sunday September 27, 2020 @01:44PM
(#24608500)
Low-cost NFC tag stickers can be used to trigger iPhone shortcuts, with an inch of proximity. iOS shortcuts can make SSH connections to issue commands via shell scripts on local Linux servers.
Re:The Art of Automation
by contingencies on Sunday September 27, 2020 @03:40PM
(#24609537)
That would be a closed source stack and device dependent solution. You could also use QR codes to trigger URI access which in turn ran arbitrary code on arbitrary devices. That would be an open source stack and device agnostic solution.
Re:The Art of Automation
by walterbell on Sunday September 27, 2020 @04:00PM
(#24609694)
NFC can trigger an interrupt to launch code/app, so proximity is sufficient. NFC exists on multiple platforms. QR codes would require launching an app to read the code, it would be faster for the user to launch the automation shortcut directly.
There is also the issue of security. Which platform hardware+software would you entrust to pull the trigger on an automation sequence with physical side-effects?
My own experience has led me to the point where if I'm thinking about automating something, I stop and ask "is there a way that this can be done away with completely?"
The most reliable, easiest to maintain scripts are those with zero lines of code. They also perform best and consume least resources.
With a bit of thought and/or flexibility in how things are achieved, it's surprising how often this question produces a good result.
Early phases will involve the introduction of no/low-code automations that integrate existing APIs, services, and systems within businesses for the purposes of automating existing processes.
That's good for the APIs, systems and automation platforms because they become essential and encumbent components of operating those businesses.
The later, and I suspect much longer, phases will involve identification of business processes that aren't really needed and that can be removed by thinking about the ways that organizations function in entirely new ways, or that can be streamlined.
By that point some of the integrations may have become so baked-in that they'll require significant maintenance to update. And as we see with some cloud-based IoT devices, it's possible for service providers to encounter outages or cease operating, in which case internal businesses processes may halt.
All this is similar to the gradual automation and improvement of physical assembly lines; it's a case of applying the same principles to organizations. And yep, the cheapest, most efficient and reliable component is one that doesn't have to exist.
To simplify things, the early phase you mentioned is simply computing, and has been proceeding since the 1960s. The latter phase is called business process management (aka BPM) [1] and has been a thing since ~1980(?).
In promise, BPM says "Encode all your business processes and their relationships into our tool, and we'll optimize and automate them." In practice, BPM is doomed to failure because (1) it's invariably implemented in an organization by management consultant types who care more about billable hours than actual solutions & (2) it's fundamentally an imperfect abstraction over your actual business processes, leading to leaks / hacks in implementation, leading to yet another system that further slows down the business.
SAP is illustrative here, with the adage that smart businesses change their business to fit SAP, rather than try to change SAP to fit their business.
[1] https://en.m.wikipedia.org/wiki/Business_process_management
https://xkcd.com/1205/
It questions whether one should automate something that will only need to be done a few times.
1) Document the Steps 2) Create Automation Equivalents 3) Create Automation 4) Self-Service and Autonomous Systems
When I started going through stuff with my coworker, I made a Word document with bullets and sub-bullets for all the steps in a process. When they unexpectedly needed to take a week leave of absence a month into the job, I was able to complete almost every routine part of the job. It took a ton of stress off of them!
Across all my professional jobs, I find #1 to be the hardest. My process is to write down the high-level steps, then strictly follow the documentation next time I do the process, adding more and more detail every time.
Once it's decently documented, I record how long each step takes, and add comments like "This needs to be better documented" or "This is the most tedious step, and would be pretty simple to automate". Timing each step lets me start and complete something between meetings, filling in gaps.
Overall I've found the system to be really helpful. It makes transitioning job duties to new employees trivial and has saved me literally hundreds of hours over just a few years in the professional workforce.
[1] https://queue.acm.org/detail.cfm?id=3197520
https://easymorph.com
After a process gets automated, people forget the details of how to do the work manually. Now, when the automation breaks (and it will break at some point. It takes a huge amount of work to make these things failsafe and scripting doesn't really encourage robustness) -- no one remembers the details and fixing the issue is more difficult and requires specialized knowledge that is no longer common in the organization.
1. Build automation so that it is NOT such a blackbox. I even go so far as to build automation with scripting languages...that allows myself/my team to at least peek at the source code to see what is happening.
2. And, of course documentation. i know this is simplistic of me to state...but it is invaluable nonethelss. I should also clarify that i mean there should exist documentation both for users, or by extension those who benefit from the automation, as well as, for developers/sys admins/sys maintainers (say, within the code itself).
Highly recommend Automate the Boring Stuff with Python (https://amzn.to/3mVuKSx) if you are interested in automation + coding (from beginner level).
Have automated banking tasks and programmatically getting information from the internet (aka scrapping).
Amazing what you can do with python libraries - beautiful soup, selenium...
I'd also suggest realpython [2] for resources on web scraping
[0] https://automatetheboringstuff.com/2e/
[1] https://inventwithpython.com/automateudemy
[2] https://realpython.com/tutorials/web-scraping/
You can automate a task and thereby cause the task to happen without your involvement, but that doesn't give you more time. It just automates a task.
The amount of time you have is still that which happens between being born and dying.
Another way to view these experiences is that they all take place in the finite amount of time that you have here on Earth. That time doesn't change. Only your experience of it changes.
You can control your experiences by cultivating your thoughts.
For example, you might enjoy the time it takes you to design and build a system for automating a task. Contrast that experience with that of somebody who hates the idea of doing work to automate a task, and would rather just get the task over and done with, and never consider automation.
Who is right, and who is wrong?
You could calculate an answer by applying your maths, and that would be one answer, which might come out in your favour, or might not, depending on whether you could automate a task in less time than it would take to perform the task.
But you already know that you're coming out ahead, even if it takes you 10 times longer to design and implement the automation than it does to perform the task manually, because you enjoy the process of automating tasks. That's because you've cultivated your thoughts.
This is only true under the assumption that people automate stuff for fun. Businesses have a different incentive. They want automation to enable their pricey employees to take care of more tricky tasks, or to simply require fewer of them in the first place. A business won't give leave to their employees to automate processes just because they enjoy automating stuff.
I am writing about a human being experiencing a finite life, expansively.
Must these two domains be mutually exclusive, or conflicting? If business only existed for profit, they might be, but is that the only reason businesses exist?
timeGained = (manualTimeSpent * n) - (timeToAutomate)
- Repetitive manual labor is prone to failure. If failure is expansive and automated task could be better just for the additional safety.
- Once it's automatized the frequency at which the task is done can be increased.
- Once something is automatized, other things become easier to automatized.
There is also the issue of security. Which platform hardware+software would you entrust to pull the trigger on an automation sequence with physical side-effects?
Trust? NIH. Something I wrote myself, only available on the local wireless network segment, probably.