News for hackers. Stuff that matters.
Re:CLI: Improved by orf on Thursday August 30, 2018 @03:41PM (#17880051)
Is anyone else impressed with the quality (and speed!) of some of the tools written in Rust? I'm an avid user of fd and bat, the former being ridiculously fast. Often I find something on github, I'm impressed by the quality of the documentation, features, UI etc, then lo and behold it's written in Rust.

Another one potentially for this list is tokei[1] I was trying to count the code in our repos at work and used the venerable 'cloc' utility. It took over 5 minutes. Looking around I found tokei, written in rust. Same-ish results (more accurate actually) took 10 seconds.

1. https://github.com/Aaronepower/tokei

Re:CLI: Improved by the8472 on Thursday August 30, 2018 @05:31PM (#17880967)
I had a case where tar was too slow for my needs. Rust did let me cobble the right syscalls and threads together to make it more HDD-friendly and faster while forcing me to handle all the gritty filesystem error cases from the start. Lo and behold, about 4 times faster on an idle system and orders of magnitude faster on a busy system. It just would not have been possible to do it by combining shell utilities and too finicky in C.
Re:CLI: Improved by carlmr on Friday August 31, 2018 @02:33AM (#17883400)
That's the thing with Rust, it makes you work hard right from the start to make everything right, but when you have something working it's usually fast, too.
Re:CLI: Improved by thethirdone on Thursday August 30, 2018 @05:56PM (#17881153)
Do you have the code posted online? I would like to take a look through it.
Re:CLI: Improved by the8472 on Thursday August 30, 2018 @05:59PM (#17881177)
Re:CLI: Improved by tretiy3 on Thursday August 30, 2018 @07:03PM (#17881581)
Only 117 lines of code? Impressive.
Re:CLI: Improved by the8472 on Thursday August 30, 2018 @07:08PM (#17881622)
I have moved the meat of the optimizations to separate libraries, they're linked in the readme.
Re:CLI: Improved by xxpor on Thursday August 30, 2018 @05:59PM (#17881182)
I'm surprised they said ack || ag over ripgrep. I'm a total convert. It's AMAZINGLY fast.
Re:CLI: Improved by KingMob on Friday August 31, 2018 @05:27PM (#17888713)
In the comments, the author admitted he hadn't heard of ripgrep, and would give it a try.
Re:CLI: Improved by steveklabnik on Thursday August 30, 2018 @03:57PM (#17880174)
IIRC, loc (in rust) might be even faster than tokei at times? I always forget, and I’m sure I saw an old benchmark...

We have a whole working group this year focused on making the experience of writing CLIs awesome, so hopefully we’ll see even more great tools in the future!

Re:CLI: Improved by cgag on Friday August 31, 2018 @12:23AM (#17882940)
Loc (mine) is usually faster in my tests but doesn't handle comments that start in strings, so if there's something like x="/*" it can be way off, so I usually still point people at tokei.

I need to try implementing the string thing. It'd be a lot more fun to try to compete on speed if we were the same on accuracy.

Re:CLI: Improved by vram22 on Thursday August 30, 2018 @04:56PM (#17880676)
>We have a whole working group this year focused on making the experience of writing CLIs awesome

Can you explain what you mean by this? Something like the people are going to focus on language and library features that help with writing CLIs? Or something else?

Asking because CLIs are one of my interests.

Re:CLI: Improved by epage on Thursday August 30, 2018 @09:05PM (#17882152)
We've tried to dig into various problematic areas of writing CLIs in Rust [0], worked to create or improve libraries [1], and are working on writing a "book" for Rust-based CLIs [2].

You can reach out to us on gitter [3].

[0]: https://github.com/rust-lang-nursery/cli-wg/issues

[1]: https://github.com/rust-clique/ and https://github.com/assert-rs/

[2]: https://rust-lang-nursery.github.io/cli-wg/index.html

[3]: https://gitter.im/rust-lang/WG-CLI

The areas we're trying to finish up for Rust 2018 Edition:

- Get clap (arg parsing) to 3.0 [4]

- Get assert_cmd [5] / assert_fs [6] to 1.0

- Finish work on man-page generation [7]

- Make it easier to package binaries [8] and document the CI for it [9]

[4]: https://github.com/rust-lang-nursery/cli-wg/issues/41

[5]: https://github.com/assert-rs/assert_cmd/

[6]: https://github.com/assert-rs/assert_fs

[7]: https://github.com/rust-lang-nursery/cli-wg/issues/38

[8]: https://github.com/crate-ci/cargo-tarball

[9]: https://github.com/crate-ci/crate-ci.github.io

Re:CLI: Improved by vram22 on Friday August 31, 2018 @02:21PM (#17887756)
Thanks for the detailed answer. Will check out some of those links.
Re:CLI: Improved by isaacaggrey on Thursday August 30, 2018 @04:58PM (#17880700)
Here is the Rust CLI working group announcement: https://internals.rust-lang.org/t/announcing-the-cli-working...

Rust has different community working groups that focus on improving the Rust ecosystem in different ways this year: https://internals.rust-lang.org/t/announcing-the-2018-domain...

Re:CLI: Improved by vram22 on Thursday August 30, 2018 @05:28PM (#17880946)
Thank you, will take a look.
Re:CLI: Improved by Aaronepower on Friday August 31, 2018 @08:15AM (#17884856)
Well luckily I recently published a new comparison benchmark[0]. :)

The TL;DR is that loc is faster by a few hundred milliseconds depending on repository size, but as cgag mentions doesn't have comment in string detection so can be quite off in its metrics, for example on the Rust repo Tokei says it has 643,754 lines of code where as loc says it's 635,849.

[0]: https://github.com/Aaronepower/tokei/blob/master/COMPARISON....

Re:CLI: Improved by steveklabnik on Friday August 31, 2018 @09:50AM (#17885548)
Awesome! :)
Re:CLI: Improved by dijit on Thursday August 30, 2018 @03:59PM (#17880198)
I wonder if there's a name for this phenomenon.

New and not-popularly adopted languages tend to have more senior people learning and developing software using them; this leads to people (read: recruiters) looking for the people with experience.. People equate the quality of software with something innate with the language, or that the developers are exceptional. Then after being popularly adopted it slumps in perceived elegance and goes on a downward spiral until it ends up like PHP, Ruby or JAVA. (not to undersell the folks who predominantly use those languages, I'm just picking languages that I've seen follow this pattern)

Re:CLI: Improved by adwhit on Thursday August 30, 2018 @06:08PM (#17881246)
A more generous interpretation might be that Rust has sparked a CLI renaissance, since it lets developers write CLIs that have that satisfying zip that previously was only possible in C. None of PHP, Ruby or Java is responsive enough for a good CLI tool (also, static compilation is a must for wide deployment).
Re:CLI: Improved by vram22 on Thursday August 30, 2018 @07:58PM (#17881867)
FreePascal has been around for a long time and supports CLI programming. Maybe not very much support in the basic language and stdlib, but the essentials are there (CLI arg handling and file I/O). Third-party libraries may have more and you can always write your own. Both speed and size of binaries are good. I had compiled some simple CLI programs and they were under 100K, maybe under 50K. It's also supposed to be quite cross-platform, though I have not checked that out.

Also, I don't know Rust (but have read that it is somewhat difficult to learn); for the basics, FreePascal (FP) may be easier than Rust to learn and start using, because although it (FP) has advanced language features, for many basic CLI programs, the simpler procedural features should be enough.

https://freepascal.org/

Re:CLI: Improved by jaco8 on Thursday August 30, 2018 @09:36PM (#17882279)
You would be very, very pleased with Nim.

https://nim-lang.org/

Re:CLI: Improved by vram22 on Friday August 31, 2018 @08:44AM (#17885053)
Thanks. I had seen some HN threads on it, but had not really checked it out. Will do so.
Re:CLI: Improved by jetti on Thursday August 30, 2018 @07:09PM (#17881625)
Not to mention you get a language that is enjoyable but also compiles to native code. My first Rust project was a Haml parser and I created a CLI for it too that is similar to the Ruby version. It is nice being able to ship the 8Mb executable and not have to worry about users having the right runtime.
Re:CLI: Improved by carlmr on Friday August 31, 2018 @02:39AM (#17883422)
Also don't forget Cargo. It makes it so much easier to work with than installing C/C++ dependencies.
Re:CLI: Improved by jetti on Friday August 31, 2018 @09:52AM (#17885558)
Cargo is awesome too. Cargo reminds me of Mix (from the Elixir ecosystem). The one thing that Mix has on Cargo is that it is super simple and straight forward to extend Mix. Cargo may be the same way but from what I've seen it seems more complicated.
Re:CLI: Improved by steveklabnik on Friday August 31, 2018 @10:01AM (#17885629)
That’d make sense given the common ancestry in Bundler :)

Do you have any docs on what you mean by extensibility here? I’m curious!

Re:CLI: Improved by jetti on Friday August 31, 2018 @10:17AM (#17885750)
Sure! Take a look at Mix.Task [0]. You basically name your module Mix.Tasks.X where X is whatever you want the command to be and then include "use Mix.Task" and implement the run function and you have now extended Mix. You call the task my running "mix X" and it will run the code in the run function. I created a small task that generates ORM models for Ecto (Elixir's de facto ORM library) using Mix tasks [1]. So the user types mix plsm and they can generate the code.

[0] https://hexdocs.pm/mix/Mix.Task.html [1] https://github.com/jhartwell/Plsm

Re:CLI: Improved by steveklabnik on Friday August 31, 2018 @01:50PM (#17887535)
Ah ha! Thanks. So yeah, if you have an executable in your PATH named cargo-foo, then “cargo foo” will execute it. But we’re planning on having that style of functionality in the future as well, the cod name is “tasks”, but it still doesn’t have an RFC.
Re:CLI: Improved by jetti on Friday August 31, 2018 @01:54PM (#17887564)
I'll definitely want to check out that RFC when/if it opens up. I didn't realize that cargo did that with the executables, that's good to know. Thanks!
Re:CLI: Improved by pjmlp on Friday August 31, 2018 @02:17AM (#17883333)
Just like any other compiled language.

GCC's static linking issues with glibc don't apply to other compilers.

Re:CLI: Improved by jetti on Friday August 31, 2018 @09:50AM (#17885542)
No, not like any other compiled language. Just like any other native compiled language, maybe, but Java and C# are also compiled yet require a large runtime in order to run any application. An 8Mb executable written in Rust is going to be 8Mb total. An 8Mb executable written in C# or Java is going to weigh much more than that when you take into account the framework. Now, one could argue that it is almost a given for the JVM to be installed on a given computer but you can't say the same for .NET when doing cross-platform development.
Re:CLI: Improved by pjmlp on Friday August 31, 2018 @10:34AM (#17885896)
Java and C# are also native compiled languages, with linker support.

You just have to look properly for those options.

The only thing is that not all of them are free beer.

Re:CLI: Improved by shimon on Thursday August 30, 2018 @04:49PM (#17880620)
The Python Paradox (2004): http://www.paulgraham.com/pypar.html

The name may not have aged well...

Re:CLI: Improved by akrasuski1 on Friday August 31, 2018 @01:47AM (#17883237)
This could actually be caused by disk caching effects. Have you tried running cloc second time afterwards?
Re:CLI: Improved by rofrol on Friday August 31, 2018 @05:16AM (#17884091)
I tried cloc multiple times. tokei much faster. Try it yourself.
Re:CLI: Improved by hultner on Friday August 31, 2018 @07:49AM (#17884737)
tokei is fast but isn't the smartest tool (at least for python). I compared it to pygount (and looked on individual files). Total lines is correct but it's grossly miscalculating code vs comments resulting in almost 2x total actual SLOC for a semi large project I manage.

I manually counted a few files for comparison, I believe tokei doesn't understand python-doc-comments and think it's code.

Re:CLI: Improved by prakashdanish on Friday August 31, 2018 @12:11AM (#17882898)
Since we're on the subject, I would ask this question I've been meaning to ask for a while. I love writing command line tools and I've written a few in Python but the performance isn't there.

Would you suggest me to learn Rust in order to write CLI programs? I'm also looking at haskell for the same but after this thread, I'm really thinking of going the rust way. Ideas?

Re:CLI: Improved by cgag on Friday August 31, 2018 @12:31AM (#17882967)
I personally would strongly recommend going the rust route having spent a decent amount of time learning Haskell. I think youll spend more time working on real problems and learn more about low level programming, in addition to the code being much faster.
Re:CLI: Improved by lyqwyd on Friday August 31, 2018 @12:56AM (#17883046)
I would say it depends on your goals. If performance is your primary concern, then go with rust, one of rust’s primary focuses is performance.

I don’t think of Haskell as the best language for command line dev, but I’m far from expert in either language.

Re:CLI: Improved by mixmastamyk on Friday August 31, 2018 @01:59AM (#17883273)
Also try cython.
Re:CLI: Improved by vram22 on Thursday August 30, 2018 @05:51PM (#17881121)
Have you done any comparison (even informal) of similar tools written in Rust and Go?
Re:CLI: Improved by bostonvaulter2 on Thursday August 30, 2018 @10:32PM (#17882453)
A very informal analysis of bat and ccat is that ccat (written in Go) has choked on large files multiple times (which resulted in panics), while bat hasn't crashed on me yet. Of course I've only tried bat on two files so far ;)
Re:CLI: Improved by vram22 on Thursday August 30, 2018 @08:02PM (#17881887)
I'm asking because although I know Rust is likely to be faster than Go, I'm interested in the amount of difference.
Re:CLI: Improved by burntsushi on Thursday August 30, 2018 @08:13PM (#17881936)
You might be interested in Ben Boyter's journey to write a source code line counter:

https://boyter.org/posts/sloc-cloc-code/

https://boyter.org/posts/why-count-lines-of-code/

Re:CLI: Improved by vram22 on Friday August 31, 2018 @02:19PM (#17887737)
Thanks, will take a look at those.
Re:CLI: Improved by topspin on Thursday August 30, 2018 @05:47PM (#17881093)
tokei is actually the very first Rust program that I found 'in the wild' while searching for a tool. I had run many Rust programs prior to then, but they had always been in the context of experimenting with Rust.