News for hackers. Stuff that matters.
Re:Transfer.sh – File sharing from the command line by ktm5j on Wednesday June 20, 2018 @10:08AM (#17355474)
I like it! But for me ssh already provides a simple and secure way to move files from one place or another..

  tar -cf - ./files.txt ./orDir/ | ssh host "(cd /dest/dir; tar -xf -)"
Given that ssh it so ubiquitous I think this will always be my go to.
Re:Transfer.sh – File sharing from the command line by Sir_Cmpwn on Wednesday June 20, 2018 @12:28PM (#17356861)
I use netcat a lot with my friends, we don't even have to do the SSH dance.

On the receiving end:

    nc -vll 0.0.0.0 12345 | pv | tar xv
On the sending end:

    tar cv files... | pv | nc -N "destination IP" 12345
pv is a nice tool that just reports on the rate and number of bytes moving through the pipe, optional if you don't have it. Throw in gpg --symmetric + gpg --decrypt for good measure if you want to encrypt it on the wire with a password.
Re:Transfer.sh – File sharing from the command line by zymhan on Wednesday June 20, 2018 @12:52PM (#17357069)
That works assuming the client has a public IP address or port forwarding set up, or you're on a local network together.
Re:Transfer.sh – File sharing from the command line by yjftsjthsd-h on Wednesday June 20, 2018 @03:40PM (#17358977)
That throws your files over the wire with no encryption, signing, or integrity verification. It might well be fine for your use, but I don't think I'd ever be comfortable with it.
Re:Transfer.sh – File sharing from the command line by Sir_Cmpwn on Wednesday June 20, 2018 @03:46PM (#17359041)
Did you miss the part of my comment where I explained how to add encryption?
Re:Transfer.sh – File sharing from the command line by yjftsjthsd-h on Wednesday June 20, 2018 @05:29PM (#17359955)
...yep, I only read the command itself and skimmed right over that. My bad.
Re:Transfer.sh – File sharing from the command line by Giroflex on Wednesday June 20, 2018 @10:31AM (#17355735)
What about good old scp?

    scp file.txt user@host:/dest/dir/
Re:Transfer.sh – File sharing from the command line by alex_young on Wednesday June 20, 2018 @10:40AM (#17355809)
Or rsync?

rsync -zvh file.txt user@host:/dest/dir/

So many ways to do this :)

Re:Transfer.sh – File sharing from the command line by coatmatter on Wednesday June 20, 2018 @11:25AM (#17356262)
rsync has always been my favourite because it makes the most sense to me (and the --help/man page is easy to read).

rsync -n -avh --progress source destination:~/asdf/ for a dry run followed by ctrl-p, ctrl-a, alt-f, alt-d, alt-d to remove the -n flag and then execute that for the real thing.

Occasionally though, I'll also use sftp if I'm just pulling one thing - perhaps even after sshing to the remote machine.

For all of these, SSH keys should be set up (and desktop logins secured) to make life easier.

As for Android, adb push and adb pull -a seems to work better than mtp:// or AirDroid in my experience.

Re:Transfer.sh – File sharing from the command line by LukeShu on Wednesday June 20, 2018 @12:41PM (#17356981)
After all these years, I still can't keep straight when I need a trailing slash in rsync, and when I need to not have it.
Re:Transfer.sh – File sharing from the command line by coatmatter on Wednesday June 20, 2018 @07:54PM (#17360965)
If you think of it in terms of archives and whether you want to "extract" into the current directory, or a new directory within the current one; that might help.

rsync source destination will plonk the entire source directory and put it inside destination as a neat bundle.

rsync source/ destination will take the contents of source (but not the directory source itself) and plonk it in destination

I found the info page a little dry but it does describe it succintly:

    rsync -av /src/foo /dest
    rsync -av /src/foo/ /dest/foo
For some reason though, my head freaks out when it sees "foo" and "bar", but all they're saying is that it does the same thing.

If in doubt though, just chuck everything into the destination ~/temp/ or ~/asdf/ and sort it out later.

To be honest though, most of the time I just use fish shell's autosuggestions to guide me along.

Re:Transfer.sh – File sharing from the command line by zymhan on Wednesday June 20, 2018 @12:52PM (#17357074)
What if you don't have network and login access to the other person's computer?
Re:Transfer.sh – File sharing from the command line by ktm5j on Wednesday June 20, 2018 @11:55AM (#17356530)
I usually go straight to tar & ssh over scp since you can do directories where you can't with scp. But yeah, if you just have some files scp works.
Re:Transfer.sh – File sharing from the command line by mijamo on Wednesday June 20, 2018 @12:01PM (#17356596)
You can obviously send directories with scp, just use -r. Or am I missing something?
Re:Transfer.sh – File sharing from the command line by charlchi on Wednesday June 20, 2018 @02:54PM (#17358516)
Recently used it to copy half a terabyte of stuff on my home network. Unsure about the exact specification but it supported the same flags as cp as far as I could tell.
Re:Transfer.sh – File sharing from the command line by ktm5j on Thursday June 21, 2018 @04:12PM (#17368857)
Well I'll be damned.. not sure what lead me to think this.
Re:Transfer.sh – File sharing from the command line by codegeek on Wednesday June 20, 2018 @03:09PM (#17358658)
Yes -r works for copying directories. I do it all the time
Re:Transfer.sh – File sharing from the command line by LukeShu on Wednesday June 20, 2018 @12:42PM (#17356994)
scp performs very slowly with directories containing many small files; "tar|ssh tar" does fine with that.
Re:Transfer.sh – File sharing from the command line by palebluedot on Wednesday June 20, 2018 @11:35AM (#17356354)
> tar -cf - ./files.txt ./orDir/ | ssh host "(cd /dest/dir; tar -xf -)"

You should use '&&' instead of ';' on the host side. That way you don't accidentally dump your transfer contents into a wrong directory if the existing host dir doesn't exist.

e.g.: tar cf - stuff | ssh host "(cd /dest/dir && tar xf -)"

Re:Transfer.sh – File sharing from the command line by mnarayan01 on Wednesday June 20, 2018 @11:38AM (#17356383)
Or just skip the shell entirely:

  tar -C /dest/dir -xf -
Re:Transfer.sh – File sharing from the command line by ktm5j on Wednesday June 20, 2018 @11:58AM (#17356555)
Yeah that's the right way to do it, but I always forget which flag it is for tar and this is faster than pulling up the manpages ;)
Re:Transfer.sh – File sharing from the command line by reacharavindh on Wednesday June 20, 2018 @01:22PM (#17357422)
Could you give a full example of how this would work? Receiver and sender..
Re:Transfer.sh – File sharing from the command line by int0x80 on Wednesday June 20, 2018 @01:57PM (#17357859)

    tar cf - src/ | ssh $host "tar -C /dest/dir -xf -"
Re:Transfer.sh – File sharing from the command line by reacharavindh on Wednesday June 20, 2018 @04:08PM (#17359246)
Do you have any idea about a crazy fast way to transfer files between two machines on the internal network?

No encryption necessary No integrity checks No resume support Just plain and as fast it could get..

I need to try the netcat version but I'm hoping someone could show me a concurrent version of it that is mad fast.

Re:Transfer.sh – File sharing from the command line by int0x80 on Wednesday June 20, 2018 @04:57PM (#17359685)
You can use nc (netcat). If you have a 10G ethernet you will not saturate it because you will be limited by disk IO (I get 1Gb/s disk reads for a M.2 SSD). I you can read from disk faster, netcat alone will not saturate a 10G link, probably will hit a 3Gb/s limit (depending on your hw). You will need parallel transfers (xargs, a bit of scripting etc). You can also try rsync, maybe it is good enough for you.
Re:Transfer.sh – File sharing from the command line by mnarayan01 on Thursday June 21, 2018 @10:50AM (#17365660)
From your source machine:

  < /path/to/source ncat remote-host 8001
On your destination machine:

  ncat -l 8001 > /path/to/dest
Though in most situations with files of reasonable size, you're probably going to be better off running through `gzip -c`.
Re:Transfer.sh – File sharing from the command line by ktm5j on Wednesday June 20, 2018 @11:58AM (#17356564)
Good point!
Re:Transfer.sh – File sharing from the command line by clord on Wednesday June 20, 2018 @11:11AM (#17356130)
This particular tool seems useful for sharing files with people who might not be comfortable with the command-line, or where you don't have an account on the receiver's computer, since it produces a regular http link to download the data.
Re:Transfer.sh – File sharing from the command line by ktm5j on Wednesday June 20, 2018 @11:56AM (#17356544)
That makes sense, I hadn't thought about that.
Re:Transfer.sh – File sharing from the command line by welly on Wednesday June 20, 2018 @11:36AM (#17356357)
> ssh already provides a simple way

:-D

> tar -cf - ./files.txt ./orDir/ | ssh host "(cd /dest/dir; tar -xf -)"

:-O

Re:Transfer.sh – File sharing from the command line by bencoder on Wednesday June 20, 2018 @11:53AM (#17356511)
Re:Transfer.sh – File sharing from the command line by djsumdog on Wednesday June 20, 2018 @12:12PM (#17356720)
2007. Wow. That's a lot in Internet time.
Re:Transfer.sh – File sharing from the command line by lou1306 on Wednesday June 20, 2018 @12:36PM (#17356939)
It is a lot in computer time as well. The other gem in that thread IMHO is how they marketed Dropbox as a replacement for USB thumb drives (which it also is, but it just goes to show how niche the concept of cloud was 11 years ago).
Re:Transfer.sh – File sharing from the command line by smellf on Wednesday June 20, 2018 @12:41PM (#17356985)
Yeah, and that guy is actually still active on HN and is always decent whenever someone calls him out about his ancient Dropbox comment.
Re:Transfer.sh – File sharing from the command line by reacweb on Wednesday June 20, 2018 @10:42AM (#17355826)
If you can trust to have gnu tar on both side, you can reduce typing. I usually do: tar c files.txt orDir | ssh -C host tar x -C /dest/dir
Re:Transfer.sh – File sharing from the command line by coatmatter on Wednesday June 20, 2018 @11:38AM (#17356382)
What about just typing rsync -havz --progress source destination?

-h for human-readable numbers -a for archive mode -v and --progress for verbose info -z for compression during transfer

Add a -n for a dry run if required.

https://linux.die.net/man/1/rsync

One of the people who came up with it - Andrew Tridgell - was more or less "responsible" for Linux and BitKeeper parting ways, which in turn ultimately lead to the creation of Git. I think it's a fascinating story. Excellent tools.

The progress / speed display is the main thing that keeps me coming back to rsync even though other tools might manage the same job - not seeing the progress of a copy is what had me searching for that solution in the very first place.

Re:Transfer.sh – File sharing from the command line by phre4k on Wednesday June 20, 2018 @02:10PM (#17358006)
Yo dawg, I heard you like progress, so check out rsync's

--info=progress2

You can use it since rsync version 3.1.0.

Re:Transfer.sh – File sharing from the command line by coatmatter on Wednesday June 20, 2018 @07:44PM (#17360921)
Hah I've seen that in the man a few times but always feel too lazy to call it up. Next time I'll give it another go!
Re:Transfer.sh – File sharing from the command line by vram22 on Wednesday June 20, 2018 @06:25PM (#17360397)
>Ha ha, I remember writing a command like that (made it into a script for repeated use, with args) when one of our HP-UX servers in the company where I worked then, had a DAT drive failure. The script allowed us to take backups of the source code and data on one such box from another box (connected on the same network), until the bad drive was replaced a few days later.

Didn't know about nc or rysnc at the time. Good to see those other solutions, and good that there are many ways to do it, with different pros and cons.

Re:Transfer.sh – File sharing from the command line by apocalypstyx on Wednesday June 20, 2018 @10:59AM (#17356022)
Reminds me of how they left recursions out of the 'cp' command on plan9, so the recommended way to copy directories is:

@{cd fromdir && tar cp .} | @{cd todir && tar xT}

Re:Transfer.sh – File sharing from the command line by djsumdog on Wednesday June 20, 2018 @12:11PM (#17356702)
I have a lot of friends who don't know what the fuck SSH is. For them, these services are great at sending them files.