News for hackers. Stuff that matters.
Re:Show HN: fcp – A significantly faster alternative to cp(1... by formerly_proven on Wednesday June 16, 2021 @07:30AM (#27527284)
For SSDs this is probably the right thing (maybe use uring tho?)

For hard drives this is the opposite of what you want. For hard drives, especially when copying large trees, you only want one thread to access the disk, and preferably sort its stat()s and open/read/close() by inode.

Also this tool seems to lack most options that cp(1) has.

Re:Show HN: fcp – A significantly faster alternative to cp(1... by CWuestefeld on Wednesday June 16, 2021 @09:56AM (#27528531)
options that cp(1) has

OK, this is something that's bugged me for a long time. I've been in the field forever, but working on Windows. So I've seen this kind of thing many times before, particularly when referencing man, and it seems completely cryptic.

The way linux people occasionally put a parenthetic number after a generic command seems to me one of the ways that normal civilians, and even pros like me, are intimidated into staying away from serious linux usage.

So what gives? What the heck is the parenthetic number for?

Re:Show HN: fcp – A significantly faster alternative to cp(1... by lvkv on Wednesday June 16, 2021 @10:06AM (#27528614)
The number denotes what section of the manual the command (or function, etc.) comes from. Aside from that, it also helps clear up potential confusion about what you’re talking about—a good example is stat(1) (the command) and stat(2) (the system call). Without the number, you’d have to rely on context clues to infer which stat was the relevant one. https://unix.stackexchange.com/questions/3586/what-do-the-nu...
Re:Show HN: fcp – A significantly faster alternative to cp(1... by duped on Wednesday June 16, 2021 @10:00AM (#27528567)
It refers to the manpage section. Good SO post: https://unix.stackexchange.com/questions/3586/what-do-the-nu...
Re:Show HN: fcp – A significantly faster alternative to cp(1... by andrewmcwatters on Wednesday June 16, 2021 @11:20AM (#27529517)
I can’t lie, if you spend most of your time in Windows, there’s probably diminished reasons to understand manpages, but for those that do any degree of meaningful work on a Linux or Unix (e.g. macOS) system, not knowing how to navigate man pages separates entire classes of developer professionalism to me.

It’s night and day working with people who know how to be approachable in development, but also realize you need to understand some basic flags for everyday tools and not turn to Google or Stack Overflow for what is directly in your manpages.

Admittedly those who cannot stay approachable concoct all sorts of shell hell for everyone, even those who are experienced.

Excellent question, by the way. I wish we’d get some more candid questions like these on HN when relevant to the thread, like this one. It’s a learning opportunity for all readers.

Re:Show HN: fcp – A significantly faster alternative to cp(1... by unknown_error on Wednesday June 16, 2021 @01:44PM (#27531328)
What's wrong with Stack over man pages? Most man pages don't you give advice or context, just the raw reference, but sometimes the best way to do something is not even the tool you originally looked up (say, scp vs rsync).

Stack combines the reference (which flags) with the expertise (of educated guesses by sometimes wiser users).

I would never trust a man page over a good Stack answer. (edit: as a web dev who uses *nix all day, but has no desire to become a full fledged sysadmin... Just give me the tool I need to get the job done without sacrificing performance or security. I don't need to know every possible gnu tool and flag and distro dependent options...)

Re:Show HN: fcp – A significantly faster alternative to cp(1... by andrewmcwatters on Wednesday June 16, 2021 @04:59PM (#27533398)
You’re certainly better off reading both.

But I have a bigger issue with the fact that developers who use Stack Overflow are more often than not just going to copy and paste and not understand what they’re doing.

And that’s predominantly how the site is used.

Yeah, there are plenty of manpages that are verbose, but frankly a lot of them are easier to parse than reading hacked together Stack Overflow answers that don’t explain how the poster arrived at their answer.

They literally don’t show their work.

Re:Show HN: fcp – A significantly faster alternative to cp(1... by amluto on Wednesday June 16, 2021 @10:00AM (#27528568)
It’s the man page section. cp(1) is documented in …/man1/…. You can type man 1 cp.

Roughly, 1 is unprivileged shell commands, 2 is syscalls, 3 is library functions, and 8 is administrative commands.

Re:Show HN: fcp – A significantly faster alternative to cp(1... by vthriller on Wednesday June 16, 2021 @08:24AM (#27527682)
> preferably sort its stat()s and open/read/close() by inode

A better approach would be to use information about location of data extents. Luckily there's already a crate that can help with that: https://github.com/the8472/platter-walk

Re:Show HN: fcp – A significantly faster alternative to cp(1... by formerly_proven on Wednesday June 16, 2021 @09:27AM (#27528279)
That's better for reading the data, but doesn't work for ordering stat()s because looking up extent information requires reading the inode, but that's what we're trying to make linear-ish at that point.
Re:Show HN: fcp – A significantly faster alternative to cp(1... by the8472 on Wednesday June 16, 2021 @02:06PM (#27531590)
you can optimize for both. obtain directory entries, sort by inode, do a batch stat (ideally via io_uring) and fiemap and then sort again by extent order.

And if you only need the `d_type` instead of the full stat struct, then you get that for free from getdents(2) on some filesystems, this is what platter_walk does.

Re:Show HN: fcp – A significantly faster alternative to cp(1... by dekhn on Wednesday June 16, 2021 @11:09AM (#27529357)
one of the things that surprised me about large cluster filesystems (although it's obvious in retrospect) is that the solution to scaling performance is many threads, not one, because the backend can handle many concurrent requests. In fact at one place I worked, the cluster filesystem wouldn't even take you seriously until you hit it with GB/s of requests.
Re:Show HN: fcp – A significantly faster alternative to cp(1... by Svetlitski on Wednesday June 16, 2021 @11:17AM (#27529469)
Definitely valid points, as I mentioned in a comment below I'll be updating the README to mention this is geared towards systems with SSDs.

As for lacking most of cp(1)'s options, that's a deliberate choice. My goal with fcp is to make the most common use case of cp(1) (i.e. copy some files/directories with no options) fast, not to replace cp(1) completely.