News for hackers. Stuff that matters.
Re:Useful Unix commands for exploring data by dpeck on Wednesday August 27, 2014 @12:21PM (#8232947)
No the case with ruby at all, if you're reading the whole file into memory theres a good chance you're doing it wrong.

check out yield and blocks

Re:Useful Unix commands for exploring data by barrkel on Wednesday August 27, 2014 @01:03PM (#8233227)
The problem is that the most obvious way of doing it - File.readlines('foo.txt').map { ... }.select { ... } etc. - is not stream-oriented.
Re:Useful Unix commands for exploring data by riffraff on Wednesday August 27, 2014 @04:59PM (#8234684)
arguably, it's trivial to make that stream oriented

    open('tmp.rb').each_line.lazy.map {...}.select {...}
the problem with processing big files with ruby (in my humble experience) is usually that it's still slow enough that "preprocessing with grep&uniq" is worthwhile.
Re:Useful Unix commands for exploring data by barrkel on Thursday August 28, 2014 @05:19AM (#8236493)

    > open('tmp.rb').each_line.lazy
    NoMethodError: undefined method `lazy' for #<Enumerator: #<File:Procfile>:each_line>
Not everybody is using Ruby 2.0.
Re:Useful Unix commands for exploring data by Terretta on Wednesday August 27, 2014 @12:52PM (#8233143)
> if you're reading the whole file into memory theres a good chance you're doing it wrong

GP: "working with file lines _naively_ in Ruby"

Re:Useful Unix commands for exploring data by dpeck on Wednesday August 27, 2014 @03:28PM (#8234224)
ah, my bad, read that as natively and chalked it up to odd wording.