News for hackers. Stuff that matters.
Re:Stop using tail -f (mostly) by pimlottc on Monday March 30, 2015 @12:33PM (#9290645)
I was introduced to less +F a while ago and it is quite nice, but there is one simple "feature" of tail -f that I miss quite a lot: being able 'mark' the log with gaps by hitting enter a few times. This is especially handy when you have to first load a page or warm up the app before performing the operation you're interested in watching, letting you separate the earlier output from the lines generated by what you're testing.

With less, you have to keep track of the current position by timestamp or other unique message, and it's easy to lose your place when the output starts streaming in. With tail, you only need a moment to mark your spot and then it's visually distinct even as more message come in.

Re:Stop using tail -f (mostly) by evincarofautumn on Monday March 30, 2015 @12:46PM (#9290741)
With less, you can use m<letter> to mark the current position and '<letter> to go to a marked position. h will show you lots of useful help.
Re:Stop using tail -f (mostly) by jimktrains2 on Monday March 30, 2015 @12:54PM (#9290799)
GP isn't meaning it like that, they mean being able to add some spaces so that the next log entries stand out for quick visual identification as you're making changes and reloading the application.
Re:Stop using tail -f (mostly) by toddkaufmann on Monday March 30, 2015 @01:55PM (#9291267)
Use '/' (search) to enter a regexp to match a pattern on that line, and it will be highlighted.
Re:Stop using tail -f (mostly) by lgas on Monday March 30, 2015 @02:43PM (#9291625)
This has nothing to do with what is being discussed in this subthread.
Re:Stop using tail -f (mostly) by teraflop on Monday March 30, 2015 @02:50PM (#9291672)
Sure it does. If your log lines are distinct (e.g. they have timestamps or unique IDs) then you can use less's search highlighting to provide a visual marker for a specific line, similar to what you can do by manually inserting a bunch of blank lines on the console.

This trick doesn't work if your log file has a bunch of identical lines and you want to keep an eye on their rate, though.

Re:Stop using tail -f (mostly) by gojomo on Monday March 30, 2015 @03:33PM (#9291988)
Having to remember & type a timestamp has much more mental overhead (planning & memory) than "scan/scroll back to last block of vertical whitespace".

That's why suggestions of either named-marks or back-searches aren't considered equally-attractive alternatives to marking the scrollback with a batch of <return>s.

Re:Stop using tail -f (mostly) by shawabawa3 on Monday March 30, 2015 @03:17PM (#9291872)
imagine the scenario: "I want to see everything that happens in a single request"

Enter method:

  1. Press enter a bunch of times
  2. Reload browser
  3. Press enter a bunch of times and scroll up
Your method:

  1. Search for last line in output to highlight it?
  2. Reload page
  3. Try and figure out where stuff starts and ends with loads of visual noise
Re:Stop using tail -f (mostly) by teraflop on Monday March 30, 2015 @05:53PM (#9292870)
I won't argue that for this specific use case, tail isn't friendlier than less.

But the original poster posted a useful tip, and is now getting aggressive downvotes and comments like "This has nothing to do with what is being discussed in this subthread." I think that's unwarranted.

Re:Stop using tail -f (mostly) by MadcapJake on Monday March 30, 2015 @04:10PM (#9292234)
less method:

  1. ma
  2. Reload browser
  3. 'a
less method with two marks:

  1. ma
  2. Reload browser
  3. mb
  4. 'a
Re:Stop using tail -f (mostly) by ajanuary on Monday March 30, 2015 @04:45PM (#9292470)
Unfortunately mark doesn't seem to work while you're following.
Re:Stop using tail -f (mostly) by shangxiao on Monday March 30, 2015 @04:13PM (#9292254)
A bunch of blank lines is a lot more grokable than timestamps lost in a bunch of text.
Re:Stop using tail -f (mostly) by bch on Monday March 30, 2015 @12:55PM (#9290812)
An example of some of the vi commands that less(1) has adopted. You can also:

  * page forward and backward, including using numeric prefixes to indicate a lineno or indicate "repeat 'n' times".
  * launch vi if less(1) is working on a file (versus (eg) stdout of some process)
  * search fwd/backward
  * start examining a new file w/o leaving less(1)
  * ...
Re:Stop using tail -f (mostly) by pjungwir on Monday March 30, 2015 @02:12PM (#9291405)
Another less-vs-vi question:

As a junior programmer I got chewed out by my IT department because I was examining a (production) log file in vi. The sysadmin told me I should use less (actually more---this was on Solaris in ~2001). To this day I only read log files with less, but I've never figured out his objection. Negatives to using vi I can imagine are:

- I might write to the log file. That seems like a real worry, although I could also say `vi -R` to prevent it.

- Starving the production system of memory. I'm pretty sure he expressed this concern. Any insight into whether it is legit? Is less actually any better?

Obviously you really should have a log shipping & aggregation service so you can read logs offline, etc, but not every project is large enough for that, nor every org organized enough. So for the sake of argument my premise is, "Assuming you want to read a log on production . . ."

Re:Stop using tail -f (mostly) by annnnd on Tuesday March 31, 2015 @02:33AM (#9294685)
From sysop standpoint there is a huge difference:

1) for user: less chance (pun intended) to actually change the file when all you wanted was to read it

2) for sysadmin: if sysadmin sees "less somefile.log" in bash history, he knows the user just read the log. If he sees "vi somefile.log" then he doesn't know if the user has also changed the log file (maybe not even knowing it).

The assumption is that you deal with non-malicious users who just make mistakes (which is often the case).

Re:Stop using tail -f (mostly) by notfoss on Tuesday March 31, 2015 @09:50AM (#9296102)

  for sysadmin: if sysadmin sees "less somefile.log" in bash history, he knows the user just read the log. If he sees "vi somefile.log" then he doesn't know if the user has also changed the log file (maybe not even knowing it).
In case you didn't know, you can invoke an editor from within less by pressing 'v'. And that wouldn't get registered in the shell history ;)
Re:Stop using tail -f (mostly) by pjungwir on Tuesday March 31, 2015 @10:39AM (#9296491)
Ha ha, this reminds me of the days I had sudo access to `vi` but not a lot of other commands, on a box that IT didn't really want to support. . . .
Re:Stop using tail -f (mostly) by annnnd on Tuesday April 07, 2015 @09:30AM (#9333845)
> The assumption is that you deal with non-malicious users...
Re:Stop using tail -f (mostly) by notfoss on Tuesday April 07, 2015 @09:53AM (#9333988)
It was there previously? Sorry I missed it. In any case, I wouldn't call the action I mentioned malicious. There is no hack involved, no improvisation either. Simply using a built-in feature of `less`.
Re:Stop using tail -f (mostly) by pjungwir on Tuesday March 31, 2015 @05:00AM (#9295065)
Thanks for your reply! Since I do a lot of devops I've tried over the years to formulate a mental model of how a sysadmin thinks and what they care about. I'd never have thought about this auditability concern, so it's something to add to my list! :-)
Re:Stop using tail -f (mostly) by vbezhenar on Monday March 30, 2015 @02:20PM (#9291449)
On large files less fires up immediately and vim is quite slow. I think that vim is loading an entire file in the memory while less just loads visible chunk. May be other vi implementations are more efficient with large files.

Besides — vim really shouldn't be used on log files. It's editor, not viewer.

Re:Stop using tail -f (mostly) by pjungwir on Monday March 30, 2015 @03:44PM (#9292060)
> Besides — vim really shouldn't be used on log files. It's editor, not viewer.

I think the two concerns I stated are probably legit, but this one is boring to me. Vim is nicer than less for reading files. I have more navigation commands. I can yank part of the file and save it somewhere else. I can pipe a range of lines through awk/etc. I can switch between files. I can split my screen. Etc. Some of these are probably available in less too (more than more(1) supported in 2001), but I doubt all, and I already know the commands in vim. I'm interested in not clobbering my logs and not crashing the server, but if you tell me vim is an editor not a viewer, I'll ask Why?

Btw re-reading my words I don't mean to sound combative. But the point of my original question was to understand. I've been cargo-culting "use less for logs" for 14 years already.

Re:Stop using tail -f (mostly) by kragen on Monday March 30, 2015 @04:55PM (#9292548)
You can save a part of the file, pipe a range of lines, or switch between files in less, too, and the commands are mostly the same as in vi. It'd sure be nice to be able to split the screen, though!
Re:Stop using tail -f (mostly) by EdiX on Monday March 30, 2015 @03:23PM (#9291920)
> I think that vim is loading an entire file in the memory while less just loads visible chunk

No, vim also only keeps part of the file loaded, however it will try to count how many lines the file has.

Re:Stop using tail -f (mostly) by robmil on Monday March 30, 2015 @04:44PM (#9292462)
Vim ships with a command called `view` that will start it in read-only mode, and makes it very much a viewer — not just an editor.
Re:Stop using tail -f (mostly) by dredmorbius on Monday March 30, 2015 @09:56PM (#9293940)
True, and I like using this for syntax-colored views of files. But it's got an annoying tendency to switch to edit mode with very little fuss.

Sometimes I want a viewer to just be a viewer.

Speaking of which: are there any Linux/Unix viewers that do have generalized syntax highlighting support?

Re:Stop using tail -f (mostly) by notfoss on Tuesday March 31, 2015 @02:07PM (#9298581)
You can give vimpager a try. It even supports vimrc. But I have felt it to be considerably slower than less.
Re:Stop using tail -f (mostly) by sukilot on Tuesday March 31, 2015 @11:16AM (#9296825)
chmod :)
Re:Stop using tail -f (mostly) by dredmorbius on Thursday April 02, 2015 @07:32PM (#9313800)
It took me way too long to grok that.
Re:Stop using tail -f (mostly) by oimaz on Monday March 30, 2015 @04:43PM (#9292453)
With vim, I usually use 'vim -u NONE <file-name>' which skips the startup file and makes loading zippy
Re:Stop using tail -f (mostly) by ryen on Monday March 30, 2015 @04:44PM (#9292463)
It may also try to do syntax highlighting and other formatting stuff, taking up even more resources and cpu time.
Re:Stop using tail -f (mostly) by sliverstorm on Monday March 30, 2015 @04:01PM (#9292179)
And an editor should not be used on log files?

Frequently, frequently I am in the position that I need to manipulate an overly-verbose log file to condense out the information I want. I could spend a half hour concocting wizard-like shell invocations, OR I could do it interactively in five minutes with vi...

Re:Stop using tail -f (mostly) by blueblob on Monday March 30, 2015 @02:51PM (#9291678)
I think your idea of `vi -R` or `vim -R` is a good idea. `less` uses less memory and may load faster but looking at logs for things like valgrind vim will give you syntax highlighting that I am not sure you can get with `less`.
Re:Stop using tail -f (mostly) by dcohenp on Monday March 30, 2015 @03:45PM (#9292069)
BTW, starting vim as "view" (through e.g. a symlink) is equivalent to "vim -R", precisely for this use case [0].

[0] http://vimdoc.sourceforge.net/htmldoc/starting.html#view

Re:Stop using tail -f (mostly) by blueblob on Monday March 30, 2015 @09:14PM (#9293784)
I have used `view` but it didn't give me syntax highlighting. On my system (archlinux) view is actually a symlink to ex (which must change its behavior based on its name?).

    readlink /usr/bin/view
    ex
Re:Stop using tail -f (mostly) by hrktb on Monday March 30, 2015 @04:13PM (#9292256)
I think Edix's answer is the most pragmatic. Also, someone at work actually crashed a service by opening the log files in vim.

I think the problem was that the memory and processor were already getting stomped on (thus the need to look at the logs) and vim tried to do a lot of fancy stuffs to get more info on the file as a whole.

Re:Stop using tail -f (mostly) by dexxtreme on Monday March 30, 2015 @02:20PM (#9291450)
If I'm logging into a server later to resolve an issue, one thing I will probably check is the command history. If you open a file with "vi", how do I know that you only looked at it, and did not make any changes to it? When I see "less" in the command history, I know for certain that it was a read-only operation.
Re:Stop using tail -f (mostly) by sukilot on Tuesday March 31, 2015 @11:17AM (#9296843)
Why do you let revs login to production machines. That's your first problem.
Re:Stop using tail -f (mostly) by cstoner on Monday March 30, 2015 @02:23PM (#9291473)
It's a good practice for sure, but your sysadmin was probably a little over-hash.

I'm generally in the habit of using `view` to do read-only vim. `less` works as well.

Re:Stop using tail -f (mostly) by dTal on Tuesday March 31, 2015 @02:45PM (#9298917)
Funny - I always used to use less, until this popped up on HN:

http://seclists.org/fulldisclosure/2014/Nov/74

Now I mostly use vi.

Re:Stop using tail -f (mostly) by bkeroack on Monday March 30, 2015 @02:37PM (#9291580)
I would never "yell" at anybody for using vi vs less, but I might curiously ask their reasoning behind it. I'm generally for less bikeshedding and cargo cultism when possible. Whatever works to get the job done.
Re:Stop using tail -f (mostly) by nodata on Monday March 30, 2015 @02:24PM (#9291483)
Yes, but cpu is normally the problem. vim is very inefficient with long lines, combine that with all of vim's features (syntax highlighting, etc.) and you have the potential to make a problem worse.
Re:Stop using tail -f (mostly) by freehunter on Monday March 30, 2015 @02:22PM (#9291472)
I don't know his exact reasoning, but the biggest thing I can think of you already touched upon: there's more of a danger that you can write to a file accidentally with vi vs with less.

Beyond that, I would probably correct a junior employee as well. Even if there's nothing wrong with it, it's not the right tool for the job. When I first started I got 'yelled' at for checking to see if a machine was on the network using tracert instead of ping. It works, but it's not the right tool for the job.

Re:Stop using tail -f (mostly) by sukilot on Tuesday March 31, 2015 @11:14AM (#9296798)
The sysadmin was low knowledge.

You should download the log file and view it locally. You should never run ad hoc commands on a live production sytem.

And there is no reason that you should have edit privileges on the log file anyway.

Re:Stop using tail -f (mostly) by dredmorbius on Monday March 30, 2015 @09:28PM (#9293840)
'&pattern' filtering is huge for me.

Display only lines which match the pattern; lines which do not match the pattern are not displayed. If pattern is empty (if you type & immediately followed by ENTER), any filtering is turned off, and all lines are displayed. While filtering is in effect, an ampersand is displayed at the beginning of the prompt, as a reminder that some lines in the file may be hidden.

Re:Stop using tail -f (mostly) by pjungwir on Monday March 30, 2015 @02:03PM (#9291330)
Just the other day I was looking for a way to do this in less:

yank 10 lines, open a new file, paste the 10 lines, save the file.

Is that possible? My Googling last week didn't find a good solution.

Re:Stop using tail -f (mostly) by bch on Monday March 30, 2015 @02:35PM (#9291567)
If you wanted to grab 10 lines from /var/log/messages, starting at the first line that has "pjungwlr" in it, you could:

  $ less /var/log/msgs
  /pjungwlr^M
  ^G (note line informational line numbers)
  v (launch vi)
  [double check your line #s, etc]
  :d1,. (delete from 1st line, to current line)
  10j (go down 10 lines)
  :d,$ (delete to EOF)
  :w my_newfile
  :q
Re:Stop using tail -f (mostly) by technofiend on Monday March 30, 2015 @03:37PM (#9292016)
This is such a common problem there's a stackoverflow article on it.

http://stackoverflow.com/questions/17908555/printing-with-se...

(Actually, there's probably more than one despite their really good duplication grooming, but this was the first hit.)

Re:Stop using tail -f (mostly) by pjungwir on Monday March 30, 2015 @03:49PM (#9292101)
`v` was the missing key for me. Looks like it even preserves my position in the file. Thanks!
Re:Stop using tail -f (mostly) by kragen on Monday March 30, 2015 @04:57PM (#9292560)
Usually I `ma|acat > newfile` in less, which saves by default a whole screenfull. There must be a better way to do this in less.
Re:Stop using tail -f (mostly) by alinspired on Monday March 30, 2015 @03:04PM (#9291791)
If you're using screen/tmux as myself, tail -f remains a good option, as you can enter gaps or search, or do 1000 more things as compared to plain terminal session
Re:Stop using tail -f (mostly) by warrenm on Tuesday March 31, 2015 @11:42AM (#9297093)
That doesn't change the actual log file, however - it's only visual whilst in the screen buffer
Re:Stop using tail -f (mostly) by jmathai on Monday March 30, 2015 @08:28PM (#9293594)
On OSX/iterm you can do command+k to clear the screen and less lets you navigate up if you need.