News for hackers. Stuff that matters.
Re:I Accidentally Deleted 7TB of Videos Before Going to Prod... by bambax on Thursday May 05, 2022 @07:53AM (#31272157)
Yes. Also, maybe not have a delete action in the middle of a script. It's usually better to build a list of items to be deleted. In that case, two lists: items to be deleted, items to be kept. Then compare the lists:

- make sure the sum of their lengths == number of total current items

- make sure items_to_be_kept.length != 0

- make sure no two items appear in both lists

- check some items chosen at random to see if they were sorted in the correct list

At this point the only possible mistake left is to confuse the lists and send the "to_be_kept" one to the delete script; a dry run of the delete list can be in order.

Re:I Accidentally Deleted 7TB of Videos Before Going to Prod... by ectopod on Thursday May 05, 2022 @11:48AM (#31274875)
This. The original approach can fail horribly if there's a problem on the server when you run the script for real. Your code can be perfect but that's no guarantee the server will always return what it ought to.
Re:I Accidentally Deleted 7TB of Videos Before Going to Prod... by pc86 on Thursday May 05, 2022 @08:05AM (#31272267)
I've had good success with this approach, have two distinct scripts generate the two lists, then in addition to your items here also checking that every item appears in one of the lists.
Re:I Accidentally Deleted 7TB of Videos Before Going to Prod... by ufo on Thursday May 05, 2022 @09:38AM (#31273157)
What do you recommend, to not get intro trouble if there are spaces or newlines in the file names?
Re:I Accidentally Deleted 7TB of Videos Before Going to Prod... by marcosdumay on Thursday May 05, 2022 @11:43AM (#31274821)
Try not to delete stuff with Bash.

This is the most reliable way. Bash has a few niceties for error handling, but if you are using them, you would probably fare better in another language.

If you do insist on Bash, quote everything, and use the "${var}" syntax instead of "$var". Also, make sure you handle every single possible error.

Re:I Accidentally Deleted 7TB of Videos Before Going to Prod... by ricardobeat on Thursday May 05, 2022 @05:01PM (#31278368)
`set -e` will abort on any error, anywhere in the pipeline. It’s a must for any critical script.
Re:I Accidentally Deleted 7TB of Videos Before Going to Prod... by kevinmgranger on Thursday May 05, 2022 @11:37AM (#31274719)
Don't use a shell script.
Re:I Accidentally Deleted 7TB of Videos Before Going to Prod... by ufo on Thursday May 05, 2022 @11:50AM (#31274902)
Do you mean, always pass the list directly to the next script via function calls, without writing it to an intermediate file / pipeline?
Re:I Accidentally Deleted 7TB of Videos Before Going to Prod... by kevinmgranger on Friday May 06, 2022 @12:02PM (#31286752)
I'm being flippant, because shell scripts are so inherently error prone they're to be avoided for critical stuff like this.

If you _absolutely_ must use a shell script:

0. Use shellcheck, which will warn you about many of the below issues: https://www.shellcheck.net/

1. understand how quoting and word splitting work: https://mywiki.wooledge.org/Quotes

2. if piping files to other programs, using `-print0` or equivalent (or even better, if using something like find, its built in execution options): https://mywiki.wooledge.org/UsingFind

3. Beware the pitfalls (especially something like parsing `ls`): https://mywiki.wooledge.org/BashPitfalls

(warning: the community around that wiki can be pretty toxic, just keep that in mind when foraying into it.)

Re:I Accidentally Deleted 7TB of Videos Before Going to Prod... by plonk on Thursday May 05, 2022 @02:32PM (#31276924)
Yes, use the list argument to Python’s subprocess.run for example. It’s much easier to not mess up if your arguments don’t get parsed by a shell before getting passed.