lcdoutlet

Re:Why GNU grep is fast (2010) on Thursday July 23, 2020 @11:06AM (#23928099)

Thank you for your reply. Again, I am big fan of your work specfically rg and fst.

I apologize for mentioning this without submitting bug reports. I will do so in the near future.

The dataset can be found here. https://dumps.wikimedia.org/enwiki/20200420/enwiki-20200420-...

In general when I start a search, the patterns are somewhat pathological. For example when learning about a new codebase I might start with 10-100 regex and 100+ keywords. With each iteration the complexity is reduced until I find the most relevant parts of the codebase.

I know rg performs significantly better than grep out of the box. I think grep by default is compiled without optimizations and does not use concurrency. I would be interested in comparing the performance characteristics between the tools in more detail.

Re:Why GNU grep is fast (2010) on Thursday July 23, 2020 @09:05AM (#23926910)

I am a big fan of ripgrep. I think the tool is great for quick simple searches. On larger data sets with sufficiently complicated regex patterns the performance gap is small. I have also encountered situations using rg on med-large datasets where the files may contain Unicode, regular expressions, random garbage... and rg failed. I don't recall a situation like this where the error handling in grep didn't recover and move on to the next file.

Here is an example showing the performance gap to be small using (enwiki-20200401-pages-articles-multistream.xml 72G) split into 24 files on a ramdisk. I ran this 50 times and the results are similar.

du -sch . 72G . 72G total

time ls|xargs -i -P24 rg -c "<.?>" {} real 0m6.841s user 2m8.766s sys 0m5.637s

time ls|xargs -i -P24 grep -Ec "<.?>" {} real 0m6.168s user 1m35.058s sys 0m24.177s

Re:Finding bugs in Haskell code by proving it on Sunday December 10, 2017 @06:32PM (#15894116)

http://www.cl.cam.ac.uk/~jrh13/hol-light/

The author John Harrison who works at Intel has used his theorem proving application hol-light in doing verification on large code bases.

Re:Finding bugs in Haskell code by proving it on Sunday December 10, 2017 @05:28PM (#15893835)

Nice Post!

I think tools that enable formal reasoning about code are incredibly valuable.

Just going through the process the author was able to find and fix three bugs. Could you imagine a pipeline where people submitted code that had already been proven to be correct?

Sounds amazing!

Re:Googler: Google Search, Site Search, News from the terminal on Tuesday March 14, 2017 @04:44PM (#13871308)

Cool project!

I currently use http://surfraw.alioth.debian.org/ along with w3m and it works great but does not appear to be currently maintained. Would you have any interest in adding similar search options to your application? Query Wikipedia, HN or Stack Overflow.... directly?

Re:Ask HN: What do I need to know to get started with TensorFlow? on Tuesday November 10, 2015 @01:03AM (#10537848)

To install http://tensorflow.org/get_started/os_setup.md

They have some general walkthrough tutorials http://tensorflow.org/tutorials/mnist/beginners/index.md

and a nice overview on machine learning. you will need to setup an account. It is free and very informative. https://www.coursera.org/learn/machine-learning/home/week/1

hundreds of resources for machine learning can be found here http://www.machinelearningsalon.org/assets/the-machine-learn...

Re:Ask HN: I have $25,000 to invest on Saturday February 07, 2015 @07:08PM (#9015872)

If you are still looking for a place to invest. I have a good business model and good references from a previous investor.