News for hackers. Stuff that matters.
Re:A Unix Utility You Should Know About: Pipe Viewer by aston on Monday February 02, 2009 @02:34PM (#462739)
I just learned about the --forest switch for ps. If you use it, you'll get a tree representing all of the parent and child processes running rather than just a straight up list.
Re:A Unix Utility You Should Know About: Pipe Viewer by raamdev on Monday February 02, 2009 @03:13PM (#462804)
Thanks! I've been using a tool called pstree for the same thing (I didn't even realize ps had such a feature). It looks like the ps utility on OS X doesn't support --forest though, so pstree is still useful there.

http://freshmeat.net/projects/pstree/

Re:A Unix Utility You Should Know About: Pipe Viewer by Dobbs on Tuesday February 03, 2009 @03:42AM (#463720)
The --forest or as later mentioned the -f does not exist in the freeBSD version of ps (I'm assuming all bsd's).

If my understanding is correct OSX uses the freebsd utilities so that would be your reason.

Re:A Unix Utility You Should Know About: Pipe Viewer by palish on Monday February 02, 2009 @04:29PM (#462913)
--forest? Seriously?

Metaphors are sometimes confusing. Call it what it is. "Reflection" should have been called "type information", for example, and "--forest" should have been called "--nested".

Re:A Unix Utility You Should Know About: Pipe Viewer by yan on Monday February 02, 2009 @05:09PM (#462983)
A process tree is known to almost everyone who uses Unix. A forest is, in my opinion, a good name for a collection of trees.

Reflection is the ability of something to introspect. The term "self-reflection" dates back to 1652[1], so why change it? "Type information" is inaccurate because reflection can mean more than just getting information about types, like interfacing with the runtime or getting information about the language implementation.

I don't think concepts should be misnamed at the expense of accuracy.

[1] http://www.merriam-webster.com/dictionary/self-reflection

Re:A Unix Utility You Should Know About: Pipe Viewer by palish on Monday February 02, 2009 @08:07PM (#463274)
The term "self-reflection" dates back to 1652, so why change it?

I'm saying it shouldn't have been used in the first place. It's an awkward, confusing metaphor. Programs aren't "reflecting", because "self-reflection" is something that only a sentient creature can perform. Programs currently have no concept of "self". I'd say "reflection" is less accurate than "type information", which covers at least 95% of use cases, if not more.

I don't think concepts should be misnamed at the expense of accuracy.

Neither do I. Also, all else being equal, less metaphors are better.

Re:A Unix Utility You Should Know About: Pipe Viewer by rkowalick on Tuesday February 03, 2009 @02:47PM (#464517)
In graph theory, a collection of disjoint trees is called a forest.
Re:A Unix Utility You Should Know About: Pipe Viewer by lacker on Tuesday February 03, 2009 @12:21AM (#463557)
You're talking about a tool whose name is "ps". Unusual names for commands are par for the Unix course.
Re:A Unix Utility You Should Know About: Pipe Viewer by nailer on Tuesday February 03, 2009 @03:01AM (#463685)
Process.

List.

Move.

Copy.

Tape Archiver.

They're not unusual, they're short.

Re:A Unix Utility You Should Know About: Pipe Viewer by FlorinAndrei on Monday February 02, 2009 @03:02PM (#462784)
Or -f

As in:

ps axf

Re:A Unix Utility You Should Know About: Pipe Viewer by ivank on Monday February 02, 2009 @04:09PM (#462881)
htop can also do this.
Re:A Unix Utility You Should Know About: Pipe Viewer by pkrumins on Monday February 02, 2009 @09:25AM (#462283)
ask hacker news: what other utilities do you know that others don't? :)
Re:A Unix Utility You Should Know About: Pipe Viewer by yan on Monday February 02, 2009 @10:36AM (#462380)
A short list of what I like and use:

the command 'tee': let's you see what is happening at an intermediate stage of a pipeline (I originally thought your blog post would be about tee)

the -S flag to objdump: If you're trying to learn how compilers work or assembly, the '-S' flag to objdumb is absolutely beautiful. If you compile a binary with debugging symbols (-g in gcc), 'objdump -S binary' will intersperse the assembly with the original code, letting you see what each line compiled into.

I use 'pushd' and 'popd' bash builtins. It lets you remember current directories and have a stack of locations that you've been. Very valuable when you're jumping around a large source tree.

Even though not a command, I usually follow ls by "-ltr". That arranges it such that the most recently modified file is on the bottom. Very easy to see what files were modified.

"du -sk * | sort -rn" for a nice listing of directory sizes.

Pressing "esc, ." (That's the Esc key, followed by a period. Not together) in bash would insert the previous command's argument. i.e.:

  bash# ls /some/really/long/path/to/a/huge/file.c
  bash# vi [esc, .]    # <-- that would insert the path
  bash# vi $_          # <-- same effect
'tree' is nice to picture a directory hierarchy.
Re:A Unix Utility You Should Know About: Pipe Viewer by gcv on Monday February 02, 2009 @10:52AM (#462411)
Looks like the Esc-. trick works in zsh as well. I didn't know about it. Cool. Thank you.

With regard to pushd and popd: these tricks work in zsh also. In addition, (again, zsh-specific) you can set these options:

  setopt auto_pushd         # automatic directory stack
  setopt pushd_minus        # swap +/- for pushd and popd
  setopt pushd_silent       # no output after pushd and popd
  setopt pushd_to_home      # pushd with no arguments goes to ~
  setopt pushd_ignore_dups  # no duplicates in directory stack
With these options set, every time you change directories using cd, the directory goes into your directory stack, basically executing an implicit pushd. Then, move around the directory structure a little, and try typing 'cd -<Tab>' (meaning type 'cd -' and press Tab for autocompletion). It'll list entries in your directory stack by number (so -1 is your previous directory, -2 is the one before, and so on). It's a handy wrapper for bare pushd/popd functionality.

Bash might offer similar functionality, but I don't know much about bash.

Re:A Unix Utility You Should Know About: Pipe Viewer by nuclear_eclipse on Monday February 02, 2009 @12:24PM (#462547)

    setopt pushd_to_home      # pushd with no arguments goes to ~
I would personally dislike that, because without that option set, calling pushd (at least in Zsh) with no argument will actually rotate the top two elements on the directory stack, which can be extremely handy for switching between two places, rather than needing to constantly call `pushd +1` or `pushd -1`.

e.g.:

    $ pu ~
    /home/user
    $ pu workspace/project
    /home/user/workspace/project /home/user
    $ pu /tmp/data
    /tmp/data /home/user/workspace/project /home/user
    $ pu
    /home/user/workspace/project /tmp/data /home/user
    $ po
    /tmp/data /home/user
    $ po
    /home/user
note that I alias pu="pushd" and po="popd" for easier shell usage...
Re:A Unix Utility You Should Know About: Pipe Viewer by likpok on Monday February 02, 2009 @11:15AM (#462452)
I doubt it. One of the big features of zsh over bash is the tab completion is fantastic (arguments, URLS, etc). With a good .zsh and .zshrc, it is an amazing experience.
Re:A Unix Utility You Should Know About: Pipe Viewer by nuclear_eclipse on Monday February 02, 2009 @12:36PM (#462564)
My only complaint with Zsh's tab completion, which is usually far superb compared to Bash, is that package-name completion is much slower in Zsh as compared to Bash, at least on the Debian and Ubuntu systems that I use. Typing `aptitude install build-<Tab>` in Bash almost immediately completes "build-essential", but Zsh can take anywhere from 3-15 seconds to search through the package cache before completing the same package name.

Other than that, I absolutely love Zsh's completion system, and the awesome capabilities it provides, like context-sensitive Git argument completion, such as `git add <Tab>` listing only the files that are either changed since the last commit, or not yet tracked by Git. Insanely useful!

Re:A Unix Utility You Should Know About: Pipe Viewer by nixme on Monday February 02, 2009 @02:05PM (#462689)
I find M-. (on most keyboards, hold Alt and press .) to be faster than Esc,. although using Esc as a "metafy-next-key" is useful for broken terminals.
Re:A Unix Utility You Should Know About: Pipe Viewer by _phred on Monday February 02, 2009 @04:08PM (#462877)
!$ is the substitution in most shells for the last arg of the last command. I use it all the time as it's really easy to type.
Re:A Unix Utility You Should Know About: Pipe Viewer by jsrn on Monday February 02, 2009 @09:42AM (#462312)
mmv (moves multiple files based on a simple pattern replacement language). Example:

    mmv "*.mp3" "old_#1.mp3"
prefixes all files in the current directory ending with .mp3 with 'old_'. Btw., the quotes around the arguments are necesssary because mmv uses some of the same metacharacters as the shell does.

Multiple substitutions are allowed, e.g.:

    $ mmv "*.*" "#1___#2"
renames blub.xyz to blub___xyz.
Re:A Unix Utility You Should Know About: Pipe Viewer by jarekr on Tuesday February 03, 2009 @03:09AM (#463692)
Zsh has a module called zmv which lets you do things like:

zmv '(??)-(*).mp3' '$1 - ${(C)2:gs/_/ }.mp3'

Which would change "01-foo_bar_baz.mp3" into "01 - Foo Bar Baz.mp3"

Re:A Unix Utility You Should Know About: Pipe Viewer by nuclear_eclipse on Monday February 02, 2009 @12:18PM (#462537)
I've been looking for something like that for years!
Re:A Unix Utility You Should Know About: Pipe Viewer by moe on Monday February 02, 2009 @12:54PM (#462599)
I commonly use qmv (from renameutils):

qmv, qcp - Rename or copy files quickly, editing the file names in a text editor

Found nothing better for bulk renames because you get to use all the pattern/replace power of your favorite editor instead of learning some wierd specialized grammar through trial & error.

This is how to remove all spaces from mp3 filenames using vim:

qmv -f do *.mp3

:%s/ /_/g

:wq

Re:A Unix Utility You Should Know About: Pipe Viewer by jsrn on Monday February 02, 2009 @01:00PM (#462610)
for Emacs users, there is wdired mode, which makes a directory listing editable just like a normal Emacs buffer.
Re:A Unix Utility You Should Know About: Pipe Viewer by yters on Monday February 02, 2009 @05:42PM (#463058)
A lot of what I find lacking in a standard terminal, Emacs makes up for with its shell and terminal emulators. I find that an editable buffer with the contents of my entire session to be eminently powerful when combined with keyboard macros and the search/replace commands.
Re:A Unix Utility You Should Know About: Pipe Viewer by puffythefish on Monday February 02, 2009 @07:13PM (#463200)
Has anyone had any luck getting this to compile for OS X (Leopard)? make seems to fail on an error for me.
Re:A Unix Utility You Should Know About: Pipe Viewer by mark_h on Monday February 02, 2009 @03:37PM (#462833)
Andrew Myers has a shell script called mvsed that does something similar, using sed to specify the patterns.

Found it: http://www.cs.cornell.edu/andru/useful.html

Re:A Unix Utility You Should Know About: Pipe Viewer by Erwin on Monday February 02, 2009 @03:57PM (#462862)
There's a 'rename' script that comes with Perl that also does this, which may be useful if you want some really fancy renames. E.g. rename '$_=lc' *.MP3 to change case to lowercase.
Re:A Unix Utility You Should Know About: Pipe Viewer by nickb on Monday February 02, 2009 @03:40PM (#462839)
Anyone know where to find a source tarball that compiles under Leopard? All I've found is a bunch of precompiled Linux RPMs...
Re:A Unix Utility You Should Know About: Pipe Viewer by ruby_roo on Monday February 02, 2009 @06:57PM (#463175)
Update your MacPorts. It was just added:

http://trac.macports.org/changeset/46335

Re:A Unix Utility You Should Know About: Pipe Viewer by jsrn on Monday February 02, 2009 @05:41PM (#463056)
hm - no source tarball, but perhaps others find this Leopard binary useful:

http://www.joel.lopes-da-silva.com/articles/computer-science...

(including man pages etc.) Caveat: I didn't try it myself (I don't have Leopard).

Re:A Unix Utility You Should Know About: Pipe Viewer by pwk on Monday February 02, 2009 @12:20PM (#462539)
Oh, and of course netcat (nc). Indispensable for debugging anything TCP/IP based.

  echo -n 'GET / HTTP/1.0\r\n\r\n' | nc news.ycombinator.com 80
Of course there are more specific tools for specific protocols, like curl or wget for http, but nc is the swiss army knife.
Re:A Unix Utility You Should Know About: Pipe Viewer by yan on Monday February 02, 2009 @05:12PM (#462992)
nc is also awesome for quick and dirty file/directory transfer over the wire:

  host1 (receiving): nc -l 6123 > tar xfz -
  host2 (sending  ): tar cfz - directory/ | nc host2 6123
Re:A Unix Utility You Should Know About: Pipe Viewer by alnayyir on Wednesday February 04, 2009 @12:54AM (#465309)
you know, it's cool that you do that (quite neat in fact), but why is this better than scp'ing something, pray tell?
Re:A Unix Utility You Should Know About: Pipe Viewer by meredydd on Wednesday February 04, 2009 @05:40AM (#465563)
My favourite two reasons for using it:

1. I often tend to use it when I want to pipe the output into a command. If it's a big transfer, I don't really want to have to stream several gigabytes of data onto disk, then off it, then back on again. Restoring a database dump is one example:

  me@host1$ nc -l -p <port> < db_dump.sql

  postgres@host2$ nc <host> <port> | psql db-name

2. The other big advantage over scp is that sometimes, your target (deliberately) isn't accessible directly from an SSH login.

In the example above, I want to process that data as the 'postgres' user, but there's no way I'm allowing direct logins as that user. 'root' is, of course, the other common example.

Re:A Unix Utility You Should Know About: Pipe Viewer by yan on Wednesday February 04, 2009 @08:49AM (#465711)
Firstly, if the two machines are on the same switch this method is noticeably faster than scp'ing something large. The actual main reason though, is that this isn't limited to tarring over a network. What it comes down to, is you can almost seamlessly have a shell pipeline and one of those stages jumps a machine boundary.
Re:A Unix Utility You Should Know About: Pipe Viewer by alnayyir on Wednesday February 04, 2009 @02:50PM (#466356)
I see. Gotta love netcat.
Re:A Unix Utility You Should Know About: Pipe Viewer by gcv on Monday February 02, 2009 @09:32AM (#462294)
I was happy to discover rlwrap. Not every REPL or utility-with-its-own-prompt out there supports readline by default.
Re:A Unix Utility You Should Know About: Pipe Viewer by pwk on Monday February 02, 2009 @12:10PM (#462527)
I find xargs handy, it lets you use the output of one command as arguments for another command. As an example, I have a little alias that I use to grep through my ruby/rails source trees:

  alias gr 'find . -name \*.rb -o -name \*.rhtml -o -name \*.erb -o -name \*.rjs | xargs egrep \!*'
(backticks can be used similarly, but can run into command line length limits)
Re:A Unix Utility You Should Know About: Pipe Viewer by jacquesm on Monday February 02, 2009 @04:30PM (#462916)
not really a utility but a very handy one liner to check if someone is being annoying on a http server:

tail -1000 logfile | cut -d ' ' -f 1 | sort | uniq -c | sort -n

Bots should stand out in the last couple of lines and have a linecount that is substantially higher than a normal user.

Re:A Unix Utility You Should Know About: Pipe Viewer by migpwr on Monday February 02, 2009 @10:43AM (#462398)
This sounds useful but I don't like the fact that it's named "pv". I've always associated "pv" with physical volume and lvm... might be a little confusing.
Re:A Unix Utility You Should Know About: Pipe Viewer by kaens on Monday February 02, 2009 @11:08AM (#462437)
You could create an alias for it, like

    alias eta='pv'
Re:A Unix Utility You Should Know About: Pipe Viewer by akkartik on Monday February 02, 2009 @01:33PM (#462649)
I made a graphical wrapper a few years ago -- call it from a terminal, but if you have X it won't munge stderr.http://akkartik.name/bash.html#pipeview
Re:A Unix Utility You Should Know About: Pipe Viewer by wmf on Monday February 02, 2009 @02:15PM (#462713)
For more pipe tricks, check out bidilink:

http://0pointer.de/lennart/projects/bidilink/

Re:A Unix Utility You Should Know About: Pipe Viewer by cliffy on Monday February 02, 2009 @10:11AM (#462349)
Is there an equivalent utility for Windows?
Re:A Unix Utility You Should Know About: Pipe Viewer by newt0311 on Monday February 02, 2009 @10:24AM (#462366)
Wouldn't windows have to have a usable shell first?
Re:A Unix Utility You Should Know About: Pipe Viewer by gnaritas on Monday February 02, 2009 @10:43AM (#462399)
It does, it's called cygwin and it's the first thing you install after getting windows to boot.
Re:A Unix Utility You Should Know About: Pipe Viewer by likpok on Monday February 02, 2009 @11:13AM (#462446)
Cygwin is a bloated piece of crap, especially if you run Vista/Server 200[3-8]. See my other comment about SUA.
Re:A Unix Utility You Should Know About: Pipe Viewer by gnaritas on Monday February 02, 2009 @01:03PM (#462612)
And yet, it is vastly better than Windows actual command line and extremely useful for managing and developing on Windows boxes. I find your claim of it being a bloated piece of crap simply untrue.
Re:A Unix Utility You Should Know About: Pipe Viewer by gojomo on Monday February 02, 2009 @01:44PM (#462666)
How is cygwin worse (more bloated or 'crap') on Vista/Server than other Windows versions?
Re:A Unix Utility You Should Know About: Pipe Viewer by nailer on Tuesday February 03, 2009 @03:07AM (#463691)
* It's redundant as there's posh out of the box, which has built in aliases for common Unix commands

* It reimplements common Windows services

* Doesn't separate content from presentation

* Doesn't integrate with other server software

* Lacks the reporting tools posh has - eg, sending a table to Excel.

Re:A Unix Utility You Should Know About: Pipe Viewer by gnaritas on Thursday February 05, 2009 @04:13PM (#468747)
Powershell is not a unix, just because it can do some of the same things and has aliases for some of the commands doesn't make it a drop in replacement. Cygwin is a drop in replacement that lets me run the same scripts I run on my Linux boxes, i.e. it is not redundant.

I'll take cygwin over powershell any and every day of the week.

Re:A Unix Utility You Should Know About: Pipe Viewer by nailer on Friday February 06, 2009 @07:25AM (#469650)
I didn't state that Powershell was a Unix. Instead, I implied it's a superior way to control Windows from a command line, which it is.

Running Linux scripts is a double edged sword: you can use existing scripts, but they can't access the whole Windows API, are too long due to unnecessary regex work, and can't report as easily.

Re:A Unix Utility You Should Know About: Pipe Viewer by gnaritas on Saturday February 07, 2009 @09:20PM (#471709)
I don't want to control Windows from the command line, I want to control my own applications, access the file system, and run and manage services.

Of course powershell integrates with Windows better than cygwin, the power of cygwin is that it makes Windows a usable Unix, which is far more important to me than controlling Windows.

Re:A Unix Utility You Should Know About: Pipe Viewer by gojomo on Monday February 02, 2009 @04:47PM (#462945)
I'm asking a genuine question, so an answer would be more civil than a downvote.
Re:A Unix Utility You Should Know About: Pipe Viewer by nailer on Tuesday February 03, 2009 @03:04AM (#463688)
There's an object shell included by default in both Windows 7 and 2008.

There is none in Linux - you end up having to find regexs to match data rather than simply piping to where and asking for the field you want.

Re:A Unix Utility You Should Know About: Pipe Viewer by likpok on Monday February 02, 2009 @11:12AM (#462444)
SUA seems to be what my windowsy friends talk about. It's in newer versions of windows. It natively supports many UNIX apps, and has source compatibility with UNIX.
Re:A Unix Utility You Should Know About: Pipe Viewer by quoderat on Monday February 02, 2009 @10:39AM (#462390)
Agreed. Even powershell is so slow. And needlessly complex.
Re:A Unix Utility You Should Know About: Pipe Viewer by ralph on Saturday February 07, 2009 @09:40AM (#471143)
Sometimes, pv(1) doesn't help. For example, `tar cf - foo | bzip2 -9v >foo.tar.bz2'. You don't know the size of the data that needs to pass down the pipe. But I sometimes find watching tar(1) open the files to read is handy; `strace -e trace=open $(pidof tar)'.
Re:A Unix Utility You Should Know About: Pipe Viewer by Keyframe on Monday February 02, 2009 @05:56PM (#463083)
anyone have an idea how this might work in tandem with dd, for example dd if=/dev/urandom of=1GB.bin bs=1M count=1024
Re:A Unix Utility You Should Know About: Pipe Viewer by blasdel on Tuesday February 03, 2009 @02:56PM (#464540)
Easy!

  dd if=/dev/urandom bs=1M count=1024 | pv | dd of=1GB.bin
Re:A Unix Utility You Should Know About: Pipe Viewer by pkrumins on Wednesday February 04, 2009 @11:45AM (#465971)
or simpler:

    dd if=/dev/urandom bs=1M count=1024 | pv > 1GB.bin
Re:A Unix Utility You Should Know About: Pipe Viewer by Keyframe on Tuesday February 03, 2009 @06:29PM (#464944)
thanks!