News for hackers. Stuff that matters.
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @05:39AM (#23688333)
From time to time, I want a quick tool to help me compare EC2 instances price. https://ec2instances.info/ works great but it's somewhat slow and didn't have a way to just `curl` from terminal.

So I develop this small tool which you can do thing like

    curl https://ec2.shop
    curl https://ec2.shop?region=us-west-2&filter=t2,m4
Re:Show HN: Curl https://ec2.shop by JimDabell on Tuesday June 30, 2020 @07:57AM (#23689159)
Are you manually detecting Curl specifically? If I use anything other than Curl, e.g. HTTPie or wget, I get the HTML page, even if I provide an `Accept: text/plain` request header. If I use one of those tools and spoof the Curl user agent string, I get the text/plain response. You should probably just look at the Accept request header, it’s there specifically so clients can tell you what kind of response they want.
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @11:50AM (#23691273)
Great idea.

I just implement it: https://github.com/yeo/ec2.shop/commit/ab258aef3ac2ad3d45b26...

Now by default it's text mode, so request by http client library or tool like curl, wget, httppie all get text bersion.

Only when detecting `Accept` contains `html` and user agent is safari/chrome/firefox etc it send html version.

Re:Show HN: Curl https://ec2.shop by karmakaze on Tuesday June 30, 2020 @02:12PM (#23692977)
There should be no reason to check user agent if already checking `Accept`. That's usually more for browser quirks or allow/blocking.
Re:Show HN: Curl https://ec2.shop by jasonjayr on Tuesday June 30, 2020 @08:22AM (#23689320)
There should be a new mime type for this: text/x-ansi
Re:Show HN: Curl https://ec2.shop by gruez on Tuesday June 30, 2020 @11:16AM (#23690910)
but why "ansi"? ANSI is only an encoding. You can have html that's in ansi, and you can have something like OP's that's not in ansi (eg. if you add emojis).
Re:Show HN: Curl https://ec2.shop by geofft on Tuesday June 30, 2020 @01:20PM (#23692414)
This is "ANSI" as in ANSI X3.64 escape sequences, not "ANSI" as in a character encoding. You can output color to curl on an ANSI-compatible terminal but not to something expecting actual plain text like a browser.
Re:Show HN: Curl https://ec2.shop by skissane on Wednesday July 01, 2020 @04:48AM (#23699321)
I think text/ecma48 would be a better name. There are lots of ANSI standards. (I guess text/ansi-x3.64 might also work, but text/ecma48 seems simpler.)
Re:Show HN: Curl https://ec2.shop by jasonjayr on Wednesday July 01, 2020 @08:37AM (#23700663)
Yes, I should have been more specific. text/x-term-${TERM} probably would be better, since it could be any number of terminal types...
Re:Show HN: Curl https://ec2.shop by nitrogen on Tuesday June 30, 2020 @02:13PM (#23692988)
You might be thinking of ASCII?
Re:Show HN: Curl https://ec2.shop by gruez on Tuesday June 30, 2020 @02:35PM (#23693228)
Re:Show HN: Curl https://ec2.shop by nitrogen on Tuesday June 30, 2020 @06:26PM (#23695487)
From the wiki: "The phrase ANSI character set has no well-defined meaning and has been used to refer to the following, among other things"

So if you are thinking of CP-437 or Windows-1252 I suppose that's fair, but generally in the context of terminal apps "ANSI" without qualification refers to the escape sequences.

Now I am curious though, did ANSI the standards body have any official character encoding standards?

Re:Show HN: Curl https://ec2.shop by simedw on Tuesday June 30, 2020 @07:38AM (#23689052)
Useful tool!

Do you not need to escape the ampersand in the second example? otherwise bash will run it in the background and skip the `filters=t2,m4` part.

Re:Show HN: Curl https://ec2.shop by bdcravens on Tuesday June 30, 2020 @11:32AM (#23691062)
Yes, but many terminals will escape it for you if you paste in.
Re:Show HN: Curl https://ec2.shop by brodouevencode on Tuesday June 30, 2020 @09:24AM (#23689715)
And it has to be infinitely better than using the pricing API.
Re:Show HN: Curl https://ec2.shop by tinhspace on Tuesday June 30, 2020 @05:57AM (#23688428)
Thank you so much. Going to try this one, very useful tool.
Re:Show HN: Curl https://ec2.shop by kissgyorgy on Tuesday June 30, 2020 @10:59AM (#23690711)
You can do cool things with this very quickly. For example, I always wondered the price differences between different regions, you can compare two instance prices quickly:

    diff -u <(curl -sS "https://ec2.shop?region=eu-central-1" | grep "t3.large" ) <(curl -sS "https://ec2.shop?region=us-east-1" | grep "t3.large")
Re:Show HN: Curl https://ec2.shop by echeese on Tuesday June 30, 2020 @07:05AM (#23688848)
The prices column is sorted lexicographically, instead of numeric, i.e. 1, 10, 2
Re:Show HN: Curl https://ec2.shop by GlitchMr on Tuesday June 30, 2020 @08:53AM (#23689496)
This can be dealt with by using a different comparison function, say `.sort((a, b) => a.localeCompare(b, 'en', {numeric: true}))`.
Re:Show HN: Curl https://ec2.shop by ohashi on Tuesday June 30, 2020 @07:35AM (#23689024)
They all are
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @09:13PM (#23696693)
I fixed this sorting issue :-). Give it another try.
Re:Show HN: Curl https://ec2.shop by solatic on Tuesday June 30, 2020 @07:37AM (#23689032)
Color output seems to be tuned for dark terminal backgrounds. Memory, vCPUs, and Storage are nearly unreadable on a white background.
Re:Show HN: Curl https://ec2.shop by hnlmorg on Tuesday June 30, 2020 @07:50AM (#23689124)
I've moaned about this with other CLI tools as well. I wish trend of using RGB values in ANSI escape codes would die because as ugly as the 16*3 colour fields¹ are, they are at least customisable so people can use palettes that suite their terminal and eyesight.

You don't get any such opportunity if the developers hard code the colour values in (and it's worse with a tool like this because you can't even set an environmental variable to change the tools behaviour)

¹ sixteen colours plus bright and dark variants

Re:Show HN: Curl https://ec2.shop by ASVVVAD on Tuesday June 30, 2020 @08:10AM (#23689244)
Can you link me to a way to use colours that change with the terminal colour preset? I haven't come across anything like that or maybe didn't pay attention. It would be really helpful

You said you can't even set an env variable to change them so I assume that there is an even better way?

Re:Show HN: Curl https://ec2.shop by hnlmorg on Tuesday June 30, 2020 @08:29AM (#23689360)
Wikipedia has a good section on the different methods of describing colour: https://en.wikipedia.org/wiki/ANSI_escape_code#3/4_bit

I've got some Go code that you can reuse if you want to the escape codes in Go: https://github.com/lmorg/murex/blob/master/utils/ansi/codes....

Just bare in mind that any numbers you see in documentation are sent as ASCII values rather than integers. eg `ESC[31m` (red text) should be sent to the terminal as

    []byte{  // http://www.asciitable.com/
        27,  // ESC character code
        91,  // '['
        51,  // '3'
        49,  // '1'
        109, // 'm'
    }

Though, as you know, most languages will have some syntactic sugar to translate characters to their ASCII values, eg

    'm' == 109
    Asc("m") == 109
    ...etc...
so at least you don't have to write all those values by hand.
Re:Show HN: Curl https://ec2.shop by ASVVVAD on Tuesday June 30, 2020 @03:14PM (#23693695)
Thanks for the explanation and the snippets ^^
Re:Show HN: Curl https://ec2.shop by kristopolous on Tuesday June 30, 2020 @11:55AM (#23691356)
On Linux, in X, you can use xresources. Otherwise there's termcap
Re:Show HN: Curl https://ec2.shop by ASVVVAD on Tuesday June 30, 2020 @03:13PM (#23693686)
totally forgot about xresources! thanks for the reminder
Re:Show HN: Curl https://ec2.shop by numlock86 on Tuesday June 30, 2020 @08:37AM (#23689392)
Thanks for the heads up. Colored output in terminal programs is a no go.
Re:Show HN: Curl https://ec2.shop by kolinkorr839 on Tuesday June 30, 2020 @09:56AM (#23690021)
I have a white background too. Is there a workaround for this?
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @11:55AM (#23691351)
I disable the colorize so it should be your default terminal now.
Re:Show HN: Curl https://ec2.shop by allis10 on Tuesday June 30, 2020 @07:46AM (#23689095)
Sounds like you played a stupid game and won a stupid prize.
Re:Show HN: Curl https://ec2.shop by dang on Tuesday June 30, 2020 @03:02PM (#23693548)
Could you please read and follow the site guidelines? https://news.ycombinator.com/newsguidelines.html

We're hoping to have a different sort of discussion than internet default here.

Re:Show HN: Curl https://ec2.shop by shitloadofbooks on Tuesday June 30, 2020 @08:38AM (#23689402)
I really like this, but the horizontal lines are not helpful at all and just double the number of lines for no benefit.

You can obviously

    curl ec2.shop | grep -v ─
but I'm not sure I want to do that every time and I'm not exactly sure why, but when I do that, I can't select the entire row, only a column at a time (zsh on iTerm 2).
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @11:54AM (#23691341)
I turn off the color :-). And I must say it much nicer without color. Thank you.
Re:Show HN: Curl https://ec2.shop by anamexis on Tuesday June 30, 2020 @09:03AM (#23689566)
I also really like this, and have some feedback:

I don't find the colors useful. They aren't contextual at all, it's just certain columns have certain colors, for no apparent reason.

Re:Show HN: Curl https://ec2.shop by cheeaun on Tuesday June 30, 2020 @08:09AM (#23689232)
Is this open sourced?

I personally try looking into implementing a curl "interface" to one of my projects, realised that it actually need to check the user-agent (of curl and a plethora of curl alternatives) which I find kind of weird...

Re:Show HN: Curl https://ec2.shop by codetrotter on Tuesday June 30, 2020 @08:27AM (#23689347)
> realised that it actually need to check the user-agent

Like someone suggested for the featured page as well, I recommend you make use of the accept header.

For example, assuming your interface is outputting freeform text (since the goal was to have something readable in terminal), you could use something like this:

    Accept: text/vnd.myproject.tui+plain; version=1.0
(Substitute “myproject” with the actual name of your project.)
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @10:54AM (#23690658)
Hey, yes it's 100% open source.

https://github.com/yeo/ec2.shop

The reason it check `curl` agent is because it share the same endpoint (root url) for both of `curl` page and browser version(the one with dropdown, grid etc)

Re:Show HN: Curl https://ec2.shop by cheeaun on Tuesday June 30, 2020 @11:10AM (#23690840)
I see, thanks!
Re:Show HN: Curl https://ec2.shop by soerface on Tuesday June 30, 2020 @12:34PM (#23691846)
Neat! The prices are without currency symbols - I guess it's always USD? Or maybe dependent on the region…? Would be nice if the $ symbol is printed, and even better if I could see the prices of my european servers in european currency - maybe add a param to select the currency?

It would be very helpful if you would support the `Accept: application/json` header. This way, we could use it in combination with jq to do arbitrary filtering:

    curl -L -H "Accept: application/json" ec2.shop | jq .
Re:Show HN: Curl https://ec2.shop by YoavR7 on Tuesday June 30, 2020 @07:22AM (#23688949)
Cool! You should probably add http support and not only https. Writing `curl ec2.shop` is easier than `curl https://ec2.shop`
Re:Show HN: Curl https://ec2.shop by saurik on Tuesday June 30, 2020 @07:24AM (#23688957)
For HTTPS to truly be meaningful we need to stop supporting HTTP as an on-ramp, to prevent people from just hijacking that initial unencrypted connection and sending anything they want.
Re:Show HN: Curl https://ec2.shop by Znafon on Tuesday June 30, 2020 @07:30AM (#23688995)
If you MITM and the user agent send an HTTP request for ec2.shop it does not matter whether the webserver supports HTTP or not, you can send a fake HTTP response either way.
Re:Show HN: Curl https://ec2.shop by hnlmorg on Tuesday June 30, 2020 @07:46AM (#23689093)
That was the GPs point.
Re:Show HN: Curl https://ec2.shop by mschuster91 on Tuesday June 30, 2020 @07:35AM (#23689019)
Anyone who likes to prevent that can submit their site to the HSTS preload list. Chrome, Firefox and Edge use a shared one, the only two relevant other agents (Safari and curl) unfortunately don't though.
Re:Show HN: Curl https://ec2.shop by flatiron on Tuesday June 30, 2020 @07:47AM (#23689104)
This is about curl though.
Re:Show HN: Curl https://ec2.shop by xxpor on Tuesday June 30, 2020 @11:40AM (#23691160)
(speaking only for myself, not my employer)

I truly do not care if someone goes through the effort to MITM my curl of ec2.shop to inject fake prices or something like that.

There's nothing here that's going to be executed, it'll just be printed or grepped.

In theory you could exploit a 0 day in curl or my terminal or something like that, but I think if you truly think about the risks and tradeoffs here it's really not worth worrying about.

If curl had an hsts list to make this irrelevant that'd also be cool.

Re:Show HN: Curl https://ec2.shop by toyg on Tuesday June 30, 2020 @02:17PM (#23693048)
> There's nothing here that's going to be executed

... yet.

Imagine someone using it to find "the biggest size available under $1" and then taking that value to execute some other script.

Re:Show HN: Curl https://ec2.shop by hnlmorg on Tuesday June 30, 2020 @08:04AM (#23689203)
These days curl should probably default to https. Or at least give you an environmental variable where you can define the default protocol (libcurl does offer something similar: https://curl.haxx.se/libcurl/c/CURLOPT_DEFAULT_PROTOCOL.html)
Re:Show HN: Curl https://ec2.shop by throwaway8941 on Tuesday June 30, 2020 @08:09AM (#23689237)

    alias curl='curl --proto-default https'
Re:Show HN: Curl https://ec2.shop by shawabawa3 on Tuesday June 30, 2020 @09:20AM (#23689673)
curl -L ec2.shop works for just 2 extra chars
Re:Show HN: Curl https://ec2.shop by baliex on Tuesday June 30, 2020 @07:37AM (#23689035)
Could you update the `...&filter...` example to wrap the URL in quotes so that the `&` isn't interpretted by the shell as sending the process to the background?
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @10:55AM (#23690662)
Ah crap, yes, going to update it.
Re:Show HN: Curl https://ec2.shop by elemenophy on Tuesday June 30, 2020 @08:15AM (#23689281)
this is an issue of ZSH.
Re:Show HN: Curl https://ec2.shop by hnlmorg on Wednesday July 01, 2020 @06:28AM (#23699936)
That is a POSIX standard and not something specific to zsh
Re:Show HN: Curl https://ec2.shop by wolfgang42 on Tuesday June 30, 2020 @11:07AM (#23690811)
No, plain sh also uses the ampersand as a job control operator.
Re:Show HN: Curl https://ec2.shop by nnx on Tuesday June 30, 2020 @12:05PM (#23691475)
Interesting tool. Would be really useful if it supported Spot prices, which are even more of a PITA to find using Amazon’s UI.
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @12:41PM (#23691924)
Yes, it isn't that hard to support Spot prices but the price change too often so I have to scrape it for every 5 minutes. Right now, I haven't had the automation to run this yet but it's doable and my pain point too =)
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @10:56AM (#23690681)
Wow, thank you for the feedback. I'm going to implement lot of features suggest in this page.

Since I cannot edit original comment, yes, the site is open source: https://github.com/yeo/ec2.shop

Re:Show HN: Curl https://ec2.shop by dxxvi on Tuesday June 30, 2020 @10:19AM (#23690254)
Can I be sure that the prices from this endpoint are accurate (i.e. if Amzn changes the prices, this endpoint will reflect that change)? Off-topic: if I request a spot instance, is there any way (API/cli) to know how much Amzn charges me every hour for that instance?
Re:Show HN: Curl https://ec2.shop by gen220 on Tuesday June 30, 2020 @10:48AM (#23690567)
I've built something like this in the past (I was running a beefy server for my friends, and wanted them to know how much it was costing us at an hourly rate, before they ran it for 3 weeks straight).

It's definitely possible and not very challenging, but the API Documentation for prices was pretty weak when I built it (a year or two ago, written in Go if it makes a difference). The google cloud prices API was similarly gross. It was kinda fun, but I wouldn't enjoy maintaining that code. In reality, it turns out that $/hour isn't a super straightforward metric (there are many dimensions that go into it), so it's not super straightforward to query.

Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @10:59AM (#23690709)
I think the price is accurate. It's fetched directly from this Amazon endpoint. https://github.com/yeo/ec2.shop/blob/master/download.sh#L9

I'm going to add spot instance. Will just need to figure out what URL AWS is fetching

Re:Show HN: Curl https://ec2.shop by zkirill on Tuesday June 30, 2020 @07:25AM (#23688958)
Super! It would be great to see price per month as well.
Re:Show HN: Curl https://ec2.shop by css on Tuesday June 30, 2020 @10:53AM (#23690639)
Very neat. Anyone know why

    curl -s https://ec2.shop | grep 't2'
would work, but

    curl -s https://ec2.shop | grep 'm4'
does not? Both commands work if I write the curl result to a file, but when I pipe from curl, the pattern `m4` matches all instances of the number 4.
Re:Show HN: Curl https://ec2.shop by wolfgang42 on Tuesday June 30, 2020 @11:01AM (#23690739)
It’s not matching every number 4, just the ones at the start of a column. The m is coming from “ESC[31m”, the escape code used to set the color of the text in the column.
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @11:10AM (#23690846)
Thank you, this is a bug.

The color code for blue is `"\033[34m` so it always match m4 for anything :(.

Just push a fix by disabling color as suggestion in here as well

Re:Show HN: Curl https://ec2.shop by css on Tuesday June 30, 2020 @11:12AM (#23690864)
Ah, got it, thanks for the clarity.
Re:Show HN: Curl https://ec2.shop by xhrpost on Tuesday June 30, 2020 @02:27PM (#23693150)
Neat, didn't realize how much the options for ec2 have grown over time. Can those instances with 100Gbps networking actually push that to upstream transit (ie. the Internet)? Or is that mostly for internal network communication between instances and other AWS services?
Re:Show HN: Curl https://ec2.shop by nine_k on Tuesday June 30, 2020 @07:41PM (#23696120)
The examples miss the opening quote before the URLs.

Illustration: http://dmitry.cheryasov.info/random/missing-quote-mark.jpg

Re:Show HN: Curl https://ec2.shop by ryanmccullagh on Tuesday June 30, 2020 @10:01PM (#23696917)
Nice list. Recently, I loaded all the prices into an Excel document to make things easier for projecting costs and determining prices. I always use the following formula for estimating monthly costs on Amezmo.

$Hourly * 750

750 being the number of hours in 1 month.

Re:Show HN: Curl https://ec2.shop by afshinmeh on Tuesday June 30, 2020 @08:08AM (#23689226)
Nice! thanks for using https://gridjs.io :)
Re:Show HN: Curl https://ec2.shop by m00dy on Tuesday June 30, 2020 @07:24AM (#23688956)
now curl is the new black ? :) This is not my first time that I see curl based services.
Re:Show HN: Curl https://ec2.shop by alexellisuk on Tuesday June 30, 2020 @08:38AM (#23689403)
Could you have it retain the sort order when changing region?
Re:Show HN: Curl https://ec2.shop by kolinkorr839 on Tuesday June 30, 2020 @10:00AM (#23690053)
Is there a way to show the output in json format?
Re:Show HN: Curl https://ec2.shop by mike-cardwell on Tuesday June 30, 2020 @12:25PM (#23691731)
Would be nice if both of these work (they don't):

curl https://ec2.shop -H 'Accept: application/json'

curl https://ec2.shop?json

Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @01:17PM (#23692371)
It should work that way now :-). Thanks for suggestion.
Re:Show HN: Curl https://ec2.shop by kureikain on Tuesday June 30, 2020 @01:16PM (#23692367)
Per suggest by mike-cardwell, I implemented this:

  curl https://ec2.shop -H 'Accept: application/json'
  curl 'https://ec2.shop?json'
  curl 'https://ec2.shop?txt'
Re:Show HN: Curl https://ec2.shop by JosephRedfern on Tuesday June 30, 2020 @07:48AM (#23689110)
Very hand. Can you add sorting?
Re:Show HN: Curl https://ec2.shop by hnlmorg on Tuesday June 30, 2020 @08:10AM (#23689246)
...or remove the pretty line glyphs so this tool can be used in a normal CLI pipeline (eg `sort`)
Re:Show HN: Curl https://ec2.shop by certifiedloud on Tuesday June 30, 2020 @08:11AM (#23689251)
You can always pipe to sort
Re:Show HN: Curl https://ec2.shop by kstrauser on Tuesday June 30, 2020 @11:51AM (#23691290)
Bravo. This is really lovely!
Re:Show HN: Curl https://ec2.shop by vmception on Tuesday June 30, 2020 @10:46AM (#23690551)
I like this trend