News for hackers. Stuff that matters.
Re:Show HN: Anycomplete by jxy on Tuesday November 29, 2016 @03:54PM (#13066201)
If you have your shell ready,

    curl -sS 'https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=aurora' \
        | json_pp | sed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p}'
Re:Show HN: Anycomplete by jsrn on Tuesday November 29, 2016 @06:04PM (#13066959)
Nice!

To run this with macOS, I had to use the GNU version of sed. I installed it with

    $ brew install gnu-sed
And it is then called with 'gsed' instead of 'sed'.

As an avid Perl programmer, I had json_pp in my $PATH - for everyone else - it is here: https://metacpan.org/pod/JSON::PP

You can install it with cpanm:

    $ cpanm JSON::PP
If you don't have cpanm, you can install it with

    $ curl -L https://cpanmin.us | perl - App::cpanminus
The modified command line from above then becomes:

    $ curl -sS 'https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=aurora' \
        | json_pp | gsed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p}'
Here is a little Bash function to encapsulate this:

    $ function c() { curl -sS "https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=$1" | json_pp | gsed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p}'; }
Which then allows you to use it like this:

    $ c hacker
    hacker news
    hacker typer
    hackerrank
    hackerman
    hackers movie
    hacker fare
    hackerone
    hackers cast
    hacker pschorr
For spaces in your query, use a '+':

    $ c New+York
    new york times
    new york and company
    new york giants
    new york post
    new york daily news
    new york weather
Re:Show HN: Anycomplete by ekns on Thursday December 01, 2016 @02:23AM (#13077816)
Improved version using jq. This properly URL encodes the query parameter and uses a much simpler sed command. Additionally the API returns UTF-8 encoded data when the user agent is specified. Requires curl >= 7.18.0.

    function c() {
        url='https://www.google.com/complete/search?client=hp&hl=en&xhr=t'
        # NB: user-agent must be specified to get back UTF-8 data!
        curl -H 'user-agent: Mozilla/5.0' -sSG --data-urlencode "q=$*" "$url" |
            jq -r .[1][][0] |
            sed 's,</\?b>,,g'
    }
Example:

    $ c ':)' ':('
    ) ( meaning
    ) ( emoticon
    ) ( ͡° ͜ʖ ͡°)
    ) ( emoticon meaning
Re:Show HN: Anycomplete by eridius on Tuesday November 29, 2016 @07:22PM (#13067413)
You can use BSD sed by just inserting a semicolon into the sed pattern:

  sed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p;}'
(Edited to remove previous advice about inserting a newline)
Re:Show HN: Anycomplete by kevinastone on Tuesday November 29, 2016 @06:28PM (#13067129)
Would think jq[0] is more suitable if you're going to install a cli json parser.

[0]: https://stedolan.github.io/jq/

Re:Show HN: Anycomplete by jsrn on Tuesday November 29, 2016 @06:32PM (#13067160)
Here is the command with jq instead of json_pp

    $ curl -sS 'https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=aurora' | jq . | gsed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p}'
(see my other comment about gsed vs sed)
Re:Show HN: Anycomplete by kevinastone on Wednesday November 30, 2016 @12:08AM (#13068623)
You don't need sed, you can do it all in jq:

    curl -sS 'https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=aurora' | jq -r '.[1] | .[][0] | sub("</?b>"; ""; "gi")'
Re:Show HN: Anycomplete by eridius on Tuesday November 29, 2016 @07:22PM (#13067418)
Here's the above modified to work with BSD sed:

  curl -sS 'https://www.google.com/complete/search?client=hp&hl=en&xhr=t&q=aurora' | jq . | sed -nE '/<\/?b>/{s```g;s`"|,|^ *``g;p;}'
(Edited to remove previous advice about inserting a newline)
Re:Show HN: Anycomplete by devoply on Tuesday November 29, 2016 @03:59PM (#13066237)
I have learned the hard way never to type stuff like this into a shell because I might not like the results. It looks fine, but never again. ;)
Re:Show HN: Anycomplete by curiousgal on Tuesday November 29, 2016 @04:17PM (#13066350)
What happened?
Re:Show HN: Anycomplete by devoply on Tuesday November 29, 2016 @04:22PM (#13066384)
This was like 13 years ago, but it in essence rm -rvf / ... but it was a obfuscated one liner that some dick put out there as a joke. Result was having to restore a site and lose a few weeks of posts.
Re:Show HN: Anycomplete by verandaguy on Tuesday November 29, 2016 @05:10PM (#13066673)
It's generally an easy way to introduce malicious shell scripts into remote machines operated by people who don't know about this.

There's a blog dedicated to shaming sites that do this: https://curlpipesh.tumblr.com/#_=_

Re:Show HN: Anycomplete by niij on Wednesday November 30, 2016 @12:49AM (#13068759)
Have you ever downloaded and installed a program through a web browser? Is there a practical difference between that and piping to sh, besides a GUI?
Re:Show HN: Anycomplete by OskarS on Wednesday November 30, 2016 @08:46AM (#13070570)
If you install through a GUI or a package manager, it will often check a cryptographic signature to validate that you downloaded the genuine thing, and you weren't man-in-the-middle'd with some malware. Also, it will check that nothing was corrupted in transit.

But generally speaking, you're right. You're taking a risk every time you run executable code from the internet, and if you don't want to live in a paranoid state of fear constantly, you have to balance security and convenience. If you check that it says "https" in the beginning and that you trust the source, then you're probably fine.

Re:Show HN: Anycomplete by __s on Tuesday November 29, 2016 @06:07PM (#13066980)
But they aren't piping to sh..
Re:Show HN: Anycomplete by kirushik on Wednesday November 30, 2016 @09:17AM (#13070857)
The infamous Barmin's patch doesn't have any `sh` mentions either:

``` echo "test... test... test..." | perl -e '$??s:;s:s;;$?::s;;=]=>%-{<-|}<&|`{;;y; -/:-@[-`{-};`-{/" -;;s;;$_;see' ```

Re:Show HN: Anycomplete by cyphar on Tuesday November 29, 2016 @06:27PM (#13067124)
Yeah, but if there's a code execution vulnerability in any of the tools in the pipeline it's as good as piping to sh.
Re:Show HN: Anycomplete by duaneb on Tuesday November 29, 2016 @09:03PM (#13067838)
Yup! Using curl at all is downright dangerous.

/s

Re:Show HN: Anycomplete by cyphar on Tuesday November 29, 2016 @09:25PM (#13067952)
Well, it's all about risk assessment. Curl has had RCE security bugs before[1]. That doesn't mean curl is "downright dangerous" it just means that "it's safe because it doesn't pipe to sh" is also not a correct thing to say.

[1]: http://blog.volema.com/curl-rce.html

Re:Show HN: Anycomplete by verandaguy on Wednesday November 30, 2016 @10:29AM (#13071464)
Incidentally, cURL just had a major security audit. The dev is doing due diligence to avoid RCE vulnerabilities.
Re:Show HN: Anycomplete by duaneb on Wednesday November 30, 2016 @10:14AM (#13071352)
Piping to jq and sed is just fine. I'm not sure where sh comes into this...
Re:Show HN: Anycomplete by cyphar on Wednesday November 30, 2016 @11:33PM (#13077194)
Unless there's a bug in curl, jq or sed that leads to an RCE, which is what my point was.
Re:Show HN: Anycomplete by TeMPOraL on Wednesday November 30, 2016 @02:48AM (#13069145)
I suppose 'devoply pattern-matched "curl -sS <sth> | <sth else>" to "don't do that". I think that because that's exactly what I did now; it took me a few seconds to realize that - this time - the invocation is mostly harmless.