News for hackers. Stuff that matters.
Re:The time might come when we add some JSON specific comman... by CobrastanJorji on Thursday January 20, 2022 @01:17PM (#30012518)
If you regularly do command line JSON requests, I'm a big fan of HTTPie. It's so much easier to use correctly. https://httpie.io/docs/cli/examples

For example, here's a JSONy POST request with cURL:

curl -s -H "Content-Type: application/json" -X POST https://api.ctl.io/v2/authentication/login --data '{"username":"YOUR.USERNAME","password":"YOUR.PASSWORD"}'

Here's that same request with HTTPie:

http POST https://api.ctl.io/v2/authentication/login username=YOUR.USERNAME password=YOUR.PASSWORD

Re:The time might come when we add some JSON specific comman... by parhamn on Thursday January 20, 2022 @01:24PM (#30012617)
After this change the equivalent would be:

    curl -XPOST --jp username=YOUR.USERNAME password=YOUR.PASSWORD https://api.ctl.io/v2/authentication/login
Which isn't too far from your desired outcome (notably, without relying on argument position for meaning). Although, I guess the argument that "curl is already installed on almost every server", sorta gets moot because I imagine it will take a while for most distros to move to the latest curl that will support --json/--jp
Re:The time might come when we add some JSON specific comman... by jedberg on Thursday January 20, 2022 @01:36PM (#30012759)
It's a lot easier for a distro to update a default utility than to add a new one though. I suspect having updated curl would happen faster than adding httpie to the list of default utilities.
Re:The time might come when we add some JSON specific comman... by derefr on Thursday January 20, 2022 @02:01PM (#30013086)
Especially singe HTTPie is a python package. curl exists even in plenty of embedded systems, in many Docker base-images, etc.; but scripting runtimes like Python generally don't.
Re:The time might come when we add some JSON specific comman... by sneak on Thursday January 20, 2022 @04:38PM (#30015451)
I wish this were the case. Unfortunately for us grownups that run stable/LTS distros, non-security updates like these don't usually make it to us for like 5 years or something.

There's a happy medium, and we're not in it.

Re:The time might come when we add some JSON specific comman... by fragmede on Friday January 21, 2022 @05:31AM (#30021676)
It depends. How long did to take for bash4-isms to get adopted? Sometimes the update time for more basic utilities, for all systems can take a long time. (Apple switching to zsh doesn't help here.)
Re:The time might come when we add some JSON specific comman... by bonkabonka on Thursday January 20, 2022 @02:23PM (#30013452)
Except if it's an enterprisey distro, it won't get updated. Specifically RHEL 7 ships 7.29.0 and cherry-picks features (TLS handshakes) to backport to their nine-year-old version.
Re:The time might come when we add some JSON specific comman... by thwarted on Thursday January 20, 2022 @01:55PM (#30013011)
The exact opposite is more likely true, depending on the nature of the change. A new utility has no current usage so there are no in-the-wild backwards compatibility concerns. The system python on redhat based distros was stuck at 2 for a long time because some of the sysadmin utilities relied on it and all those deps needs to be vetted or converted to either a newer python version or their own shipped version of the runtime that didn't use the general purpose python install.
Re:The time might come when we add some JSON specific comman... by jedberg on Thursday January 20, 2022 @02:09PM (#30013193)
Python is a huge outlier in this case. Very rarely do you have a default tool that can't be upgraded, and this case was only because Python is an interpreted language.

Curl is a command line tool. As long as they only add new functionality, there is very little that prevents an upgrade.

Re:The time might come when we add some JSON specific comman... by jsight on Thursday January 20, 2022 @02:01PM (#30013082)
There's a big difference between a bump to the latest curl version and a jump between two major versions of a language runtime. The Python 2->3 transition took years and many distros kept both versions.

Most dependencies get updated much more quickly. It wouldn't even shock me if this change got picked up mid-cycle.

Re:The time might come when we add some JSON specific comman... by bonkabonka on Thursday January 20, 2022 @02:19PM (#30013365)
Unhappily RedHat won't. Heck, RedHat has gone well out of their way to backport updated TLS handshakes (1.2 at least) to their 2013 version (7.29.0) of curl.
Re:The time might come when we add some JSON specific comman... by jsight on Monday January 24, 2022 @11:27AM (#30059789)
I'm assuming that you mean for RHEL? That's a little bit of a special case.
Re:The time might come when we add some JSON specific comman... by ericbarrett on Thursday January 20, 2022 @02:09PM (#30013194)
This is only a valid comparison if curl deprecates or moves options, as Python 3 did with language features and standard library functions.
Re:The time might come when we add some JSON specific comman... by jlundberg on Thursday January 20, 2022 @03:10PM (#30014043)
And that -X POST is actually not needed. Adding any payload will make curl do a POST request.
Re:The time might come when we add some JSON specific comman... by mback2k on Thursday January 20, 2022 @03:53PM (#30014797)
Re:The time might come when we add some JSON specific comman... by floatingatoll on Thursday January 20, 2022 @02:47PM (#30013773)
nit for understanding's sake: is the second --jp missing?
Re:The time might come when we add some JSON specific comman... by parhamn on Thursday January 20, 2022 @03:51PM (#30014747)
Not per the linked wiki [1]. But it is still being designed it seems.

[1] https://github.com/curl/curl/wiki/JSON#--jp-part

Re:The time might come when we add some JSON specific comman... by jicea on Thursday January 20, 2022 @01:51PM (#30012962)
Shameless plug, we've built hurl [1] to add some syntactic sugar over curl, with plain text. For example, you can write a text file post.hurl:

    POST https://api.ctl.io/v2/authentication/login
    {
      "username": "YOUR.USERNAME",
      "password": "YOUR.PASSWORD"
    }
and it will send the same POST request with a json body:

   hurl post.hurl
You can add asserts on the response too:

    POST https://api.ctl.io/v2/authentication/login
    {
      "username": "YOUR.USERNAME",
      "password": "YOUR.PASSWORD"
    }
    HTTP/1.0 200
    [Asserts]
    jsonpath "$.status" == "LOGGED"
Under the hood, we use libcurl and a Rust binding. The HTTP engine is curl because curl is awesome!

[1] https://github.com/Orange-OpenSource/hurl

Re:The time might come when we add some JSON specific comman... by e12e on Thursday January 20, 2022 @07:37PM (#30017606)
Hurl probably has many fine uses - but note that curl reads json from file just fine:

  curl -d @post.json http://example.com
Re:The time might come when we add some JSON specific comman... by na85 on Thursday January 20, 2022 @02:22PM (#30013420)
>hurl

Awesome name.

Re:The time might come when we add some JSON specific comman... by tinselcity on Thursday January 20, 2022 @05:51PM (#30016471)
Re:The time might come when we add some JSON specific comman... by db65edfc7996 on Thursday January 20, 2022 @01:21PM (#30012581)
I have recently been trying a port of HTTPie in Rust, xh [0] (which is needlessly hard to find in a websearch). I am a Python guy, but love having single executable tools.

[0] https://github.com/ducaale/xh

Re:The time might come when we add some JSON specific comman... by kbd on Thursday January 20, 2022 @02:12PM (#30013258)
Happy to hear about xh, thanks for mentioning it. I recently ran into a problem where HTTPie didn't support HTTP/2, so I had to fall back to curl. The ticket for HTTP/2 in HTTPie is still open https://github.com/httpie/httpie/issues/692

Happy to see that xh supports HTTP/2 out of the box.

Re:The time might come when we add some JSON specific comman... by petepete on Thursday January 20, 2022 @01:32PM (#30012705)
xh is great, just like HTTPie but incredibly fast.

I alias it to http so it's memorable and the commands make more sense though.

Re:The time might come when we add some JSON specific comman... by lambdaba on Thursday January 20, 2022 @01:33PM (#30012714)
Since htttpie can't do nested JSON assignments I like to use jo, like this:

  $ jo foo=$(jo bar=qux) | http example.com
Piped JSON automatically uses POST
Re:The time might come when we add some JSON specific comman... by treesciencebot on Thursday January 20, 2022 @03:12PM (#30014087)
(HTTPie maintainer here)

You might be in luck since Nested JSON support is going to be star feature of our upcoming release. Here is a sneak peek:

$ http --offline --print=B pie.dev/post \ search[type]=client \ search[stars]:=50000 \ search[platforms][]=Web \ search[platforms][]=Desktop \ search[platforms][]=Mobile \ search[platforms][]=CLI

{ "search": { "platforms": [ "Web", "Desktop", "Mobile", "CLI" ], "stars": 50000, "type": "client" } }

We are rolling a brand-new mini-language that integrates really well with the existing request building syntax, but also features stuff like JSON type-safety and amazing error messages for basic syntax errors.

Re:The time might come when we add some JSON specific comman... by CobrastanJorji on Thursday January 20, 2022 @03:38PM (#30014551)
That sounds great! Any chance of HTTP/2 or HTTP/3 support?
Re:The time might come when we add some JSON specific comman... by treesciencebot on Thursday January 20, 2022 @04:28PM (#30015309)
It will probably be post 3.0, since the underlying HTTP interface we use (https://pypi.org/project/requests) does not support HTTP/2. We are currently discussing how to migrate from that to something like httpx, without causing any change of user-visible behavior.
Re:The time might come when we add some JSON specific comman... by CGamesPlay on Thursday January 20, 2022 @01:51PM (#30012950)
This is amazing! Definitely adding to my toolbox.
Re:The time might come when we add some JSON specific comman... by darrenf on Thursday January 20, 2022 @02:12PM (#30013260)
For longer than I can recall I've had a `post_json` alias like this:

    curl -H'Content-Type: application/json' -d @- << 'JSON'
Invoke with

    post_json <url>
Then just paste or write JSON without caring about quotes etc, finish the heredoc, done.
Re:The time might come when we add some JSON specific comman... by paledot on Thursday January 20, 2022 @10:27PM (#30018775)
TIL you could open (and not close) a heredoc string in an alias.
Re:The time might come when we add some JSON specific comman... by tomsmeding on Thursday January 20, 2022 @02:24PM (#30013457)
Don't you need `<<'JSON'` instead of `<<JSON`? Otherwise any $ in the json body would be interpreted as a shell variable reference.
Re:The time might come when we add some JSON specific comman... by darrenf on Thursday January 20, 2022 @03:04PM (#30013969)
Yes, I mistyped on my phone. Edited, thanks!
Re:The time might come when we add some JSON specific comman... by Jaepa on Thursday January 20, 2022 @02:08PM (#30013188)
curl's primary advantage is that its pretty much universal.

I can hand another dev a curl statement without having to worry if they have the requisite software to reproduce the call.

Also Postman, Insomnia, Sentry, & Swagger all support ability export to curl.

Re:The time might come when we add some JSON specific comman... by adolph on Thursday January 20, 2022 @03:41PM (#30014594)
This example isn't fair/steelmanning cURL.

  -s: if HTTPie disables progress bar by default, that's just a different design choice, the advantage is in the eye of the beholder
  --data '{"user...: likewise, HTTPie's default to JSON is a design choice. I wouldn't say the default is superior
  -H "Content-Type...: likewise, if HTTPie adds this header by default, thats just getting in my way when I don't want that header
  -X POST: you don't need to specify that in cURL if using --data
Re:The time might come when we add some JSON specific comman... by aembleton on Friday January 21, 2022 @06:36PM (#30031386)
HTTPie can be even more compact by using the command https to indicate to use that protocol. This makes the command look like:

https POST api.ctl.io/v2/authentication/login username=YOUR.USERNAME password=YOUR.PASSWORD

Re:The time might come when we add some JSON specific comman... by mimimi31 on Thursday January 20, 2022 @01:23PM (#30012608)
I think --data implies -X POST, so that part at least would be unnecessary.
Re:The time might come when we add some JSON specific comman... by CobrastanJorji on Thursday January 20, 2022 @01:31PM (#30012694)
Ahh, I believe you're right. "PUT" would have been a better example, in that case. But I suppose getting the cURL syntax wrong helps my point that I find the cURL syntax confusing.
Re:The time might come when we add some JSON specific comman... by Diti on Thursday January 20, 2022 @05:03PM (#30015769)
HTTPie does not support multi-valued keys in query strings (sending several field[]= for example). It is very annoying and, for consistency, I would rather use cURL (instead of mixing syntax with HTTPie).
Re:The time might come when we add some JSON specific comman... by jkbr on Friday January 21, 2022 @01:15PM (#30027585)
If I’m understanding your point correctly, HTTPie has support for multi-value keys in query strings.

You can use the `name==value` query parameter syntax [0]:

  $ http --offline pie.dev/get AAA==AAA AAA==BBB
  GET /get?AAA=AAA&AAA=BBB HTTP/1.1
Or simply:

  $ http --offline 'pie.dev/get?AAA=AAA&AAA=BBB'
  GET /get?AAA=AAA&AAA=BBB HTTP/1.1
[0]https://httpie.io/docs/cli/querystring-parameters
Re:The time might come when we add some JSON specific comman... by mmcnl on Thursday January 20, 2022 @06:04PM (#30016628)
I appreciate HTTPie but it can never beat the fact that you curl is almost always installed by default.
Re:The time might come when we add some JSON specific comman... by 1vuio0pswjnm7 on Thursday January 20, 2022 @09:28PM (#30018457)
There is a different way, without curl or other curl-like programs, without Python, Go, or the like. It is smaller and faster, IME as a non-developer.

   post api.ctl.io/v2/authentication/login < 1.txt|openssl s_client -connect api.ctl.io:443
post is a 702-character shell script. printf is a built-in.

      #!/bin/sh
      (  
      y=Connection;n=0;while read x;do
      x1=${1#*//};x2=${x1%%/*};x3=${x1#*/};
      x=$(printf "%s" "${x%%:*}:";echo "${x#*:}");
      if test x"${x3}" = x"${x2}";then x3="";fi;
      printf "%s\r\n%s\r\n%s\r\n%s\r\n" \
      "POST /${x3} HTTP/1.1" \
      "Host: ${x2}" \
      "Content-Type: application/json" \
      "Content-Length: ${#x}";
      if [ $n -gt 1 ];then
      printf "%s\r\n\r\n%s\r\n" "$y: keep-alive" "$x";else
      printf "%s\r\n\r\n%s\r\n" "$y: close" "$x";fi;
      export n=$((n+1));
      done;
      if [ $n -gt 1 ];then
      printf "%s\r\n%s\r\n%s\r\n" \
      "GET /robots.txt HTTP/1.0" \
      "Host: ${x2}" \
      "$y: close";fi;
      )
Data to be posted may be stored in a one-line file named "1.txt"

    cat > 1.txt
    { "username": "YOUR.USERNAME", "password": "YOUR.PASSWORD" }
    ^D
Examine the request

    post api.ctl.io/v2/authentication/login < 1.txt
POST the request

    post api.ctl.io/v2/authentication/login < 1.txt|nc -vvn 127.1 80
Alternatively, data to be posted can be read from stdin

    echo '{ "username": "YOUR.USERNAME", "password": "YOUR.PASSWORD" }' \
    |post api.ctl.io/v2/authentication/login \
    |nc -vvn 127.1 80
If the data to be posed is JSON formatted as multiple lines such as

    { 
      "username": "YOUR.USERNAME", 
      "password": "YOUR.PASSWORD" 
    }
    
then

    (tr -d '\12' < 1.txt;echo) \
    |post api.ctl.io/v2/authentication/login \
    |nc -vvn 127.1 80
For TLS we use proxy listening on 127.0.0.1, e.g., stunnel, sslsplit, haproxy, etc.

    cat > 1.cfg
    pid=/tmp/1.pid
    [ x ]
    accept=127.0.0.1:80
    client=yes
    connect=64.15.182.200:443
    options=NO_TICKET
    options=NO_RENEGOTIATION
    renegotiation=no
    sni=
    sslVersion=TLSv1.3
    ^D

    stunnel 1.cfg
Re:The time might come when we add some JSON specific comman... by 1vuio0pswjnm7 on Thursday January 20, 2022 @09:11PM (#30018346)
There is a different way, without curl or other curl-like programs, without Python, Go, etc. It is smaller and faster, IME as a non-developer.

   post api.ctl.io/v2/authentication/login < 1.txt|openssl s_client -connect api.ctl.io:443 -ign_eof
post is a 702-character shell script. printf is a built-in.

      #!/bin/sh
      (  
      y=Connection;n=0;while read x;do
      x1=${1#*//};x2=${x1%%/*};x3=${x1#*/};
      x=$(printf "%s" "${x%%:*}:";echo "${x#*:}");
      if test x"${x3}" = x"${x2}";then x3="";fi;
      printf "%s\r\n%s\r\n%s\r\n%s\r\n" \
      "POST /${x3} HTTP/1.1" \
      "Host: ${x2}" \
      "Content-Type: application/json" \
      "Content-Length: ${#x}";
      if [ $n -gt 1 ];then
      printf "%s\r\n\r\n%s\r\n" "$y: keep-alive" "$x";else
      printf "%s\r\n\r\n%s\r\n" "$y: close" "$x";fi;
      export n=$((n+1));
      done;
      if [ $n -gt 1 ];then
      printf "%s\r\n%s\r\n%s\r\n" \
      "GET /robots.txt HTTP/1.0" \
      "Host: ${x2}" \
      "$y: close";fi;
      )
Data to be posted may be stored in a one-line file named "1.txt"

    cat > 1.txt
    { "username": "YOUR.USERNAME", "password": "YOUR.PASSWORD" }
    ^D
Examine the request

    post api.ctl.io/v2/authentication/login < 1.txt
POST the request

    post api.ctl.io/v2/authentication/login < 1.txt|nc -vvn 127.1 80
Alternatively, data to be posted can be read from stdin

    echo '{ "username": "YOUR.USERNAME", "password": "YOUR.PASSWORD" }' \
    |post api.ctl.io/v2/authentication/login \
    |nc -vvn 127.1 80
If the data to be posted is JSON formatted as multiple lines such as

    { 
      "username": "YOUR.USERNAME", 
      "password": "YOUR.PASSWORD" 
    }
    
then something like

    (tr -d '\12' < 1.txt;echo) \
    |post api.ctl.io/v2/authentication/login \
    |nc -vvn 127.1 80
For TLS we use proxy listening on 127.0.0.1, e.g., stunnel, sslsplit, haproxy, etc. This way we only ever have to type a single address and port, i.e., 127.1 80, or short alias for the hostname, e.g., echo 127.0.0.1 p >> /etc/hosts.

    cat > 1.cfg
    pid=/tmp/1.pid
    [ x ]
    accept=127.0.0.1:80
    client=yes
    connect=64.15.182.200:443
    options=NO_TICKET
    options=NO_RENEGOTIATION
    renegotiation=no
    sni=
    sslVersion=TLSv1.3
    ^D

    stunnel 1.cfg
Re:The time might come when we add some JSON specific comman... by 1vuio0pswjnm7 on Friday January 21, 2022 @10:35PM (#30033159)
Apologies for unintentional duplicate. Happens sometimes when I forget to change default cache settings in text-only browser.