News for hackers. Stuff that matters.
Re:How I Took an API Side Project to 250M Daily Requests by babuskov on Saturday July 01, 2017 @04:23PM (#14678801)
I'm baffled why anyone would use this, when you can import data in a database and run it on your own server?

I mean, you might spend 20 minutes more to set it up, but you are safe from having to rely on 3rd party service.

Anyway, kudos to coderholic for creating this and sharing the story.

Re:How I Took an API Side Project to 250M Daily Requests by gfodor on Saturday July 01, 2017 @04:46PM (#14678928)
- Database needs to be purchased

- Database needs to be distributed to your servers

- Database can become out of date easily

- Database lookup requires going to local disk and having a relatively fast access path/cache for lookups

- In general, a local database requires a large amount of effort compared to just running a curl in your PHP code.

If you are actually going to use a database, the proper solution does not look like "put it on your webservers" anyway, it looks like "put it on a separate service with a fast caching layer" etc etc. So in other words, the proper solution to decouple yourself from a 3rd party API is to... build a 1st party API.

In other words, not a 20 minute job. For small shops, a quick curl during the page load is a 20 minute job.

Re:How I Took an API Side Project to 250M Daily Requests by heipei on Saturday July 01, 2017 @05:39PM (#14679190)
There are many ways to quickly get and use a GeoIP DB like MaxMind, most requiring way less than 20 minutes. This is just one example of something I started using recently:

docker run -p 8080:8080 -d fiorix/freegeoip curl localhost:8080/json/1.2.3.4

(Repo: https://github.com/fiorix/freegeoip)

Re:How I Took an API Side Project to 250M Daily Requests by callumjones on Saturday July 01, 2017 @07:10PM (#14679618)
Where are you going to host it, how will you cycle the container to keep it up to date, and will you need to load balance it?

Running a Docker container in production is not a 20 minute job.

Re:How I Took an API Side Project to 250M Daily Requests by deno on Sunday July 02, 2017 @04:41AM (#14681541)
Just put HAProxy in front of it. It’s not rocket science.

https://www.haproxy.com/solutions/microservices.html

Re:How I Took an API Side Project to 250M Daily Requests by callumjones on Monday July 03, 2017 @01:57PM (#14690162)
Setting up HAProxy in a production environment with discovery for the docker container may not be rocket science but it’s certainly not 20 minutes.
Re:How I Took an API Side Project to 250M Daily Requests by motet_a on Saturday July 01, 2017 @06:43PM (#14679481)
Yes, Docker is great but learning it can definitely require more than 20 minutes. A `curl ipinfo.io` is still simpler.
Re:How I Took an API Side Project to 250M Daily Requests by deno on Sunday July 02, 2017 @04:45AM (#14681557)
This is the tax you’re paying for that every time:

  Connected to 35.165.108.15:443
  
  HTTP/1.1 200 OK
  Server: nginx/1.8.1
  Access-Control-Allow-Origin: *
  Content-Type: application/json; charset=utf-8
  Date: Sun, 02 Jul 2017 08:54:40 GMT
  X-Content-Type-Options: nosniff
  Connection: keep-alive
  
  Body discarded
  
    DNS Lookup   TCP Connection   TLS Handshake   Server Processing   Content Transfer
  [      4ms  |         244ms  |        518ms  |            525ms  |             0ms  ]
              |                |               |                   |                  |
     namelookup:4ms            |               |                   |                  |
                         connect:248ms         |                   |                  |
                                     pretransfer:767ms             |                  |
                                                       starttransfer:1292ms           |
                                                                                  total:1292ms
Re:How I Took an API Side Project to 250M Daily Requests by gfodor on Sunday July 02, 2017 @03:16PM (#14683896)
The OP was "baffled" why anyone would ever use this API. Clearly, the API's success shows that sometimes this tax you highlight is well worth it. (That was my point as well.) Nobody is claiming the tax doesn't exist, just that it shouldn't be baffling that a rational actor would choose to pay the tax in exchange for the corresponding benefits.

(Not to mention with very minimal effort you can usually avoid the majority of the specific tax of latency you mention, by doing things like parallelizing the request with other work or doing it asynchronously to the user's interface.)

Re:How I Took an API Side Project to 250M Daily Requests by wittedhaddock on Sunday July 02, 2017 @08:40AM (#14682231)
What tool (or arg passed to curl maybe) allowed you to generate that response? The graph is great
Re:How I Took an API Side Project to 250M Daily Requests by deno on Sunday July 02, 2017 @04:18PM (#14684145)
https://github.com/reorx/httpstat

It’s great for checking something quickly, otherwise you have the same thing in browser’s dev tools.

Re:How I Took an API Side Project to 250M Daily Requests by jhgg on Saturday July 01, 2017 @09:57PM (#14680374)
It is kinda a 20 minute job. On production, where I work, we just bake the maxmind DB into our app server images (it updates when we redeploy - which is a couple times a week) and use the maxmind client to read from that DB file.

Was just an extra step in our build pipeline.

Re:How I Took an API Side Project to 250M Daily Requests by user5994461 on Tuesday July 04, 2017 @12:48PM (#14696899)
You're misinterpreting "database".

In this case the geoIP database is like a 1MB CSV file, provided for free to the entire world by maxmind (a major network provider).

You can put give that file to many servers like nginx/apache out of the box, they will start adding a header with the country and the city of the client.

What this service is doing is effectively looking up the ip block in that csv file and returning the result as JSON.

Re:How I Took an API Side Project to 250M Daily Requests by timdorr on Saturday July 01, 2017 @04:46PM (#14678927)
In environments where you don't control the whole stack, such as a plugin or library code.
Re:How I Took an API Side Project to 250M Daily Requests by matt2000 on Saturday July 01, 2017 @04:36PM (#14678870)
Where is this data available for download?
Re:How I Took an API Side Project to 250M Daily Requests by guessmyname on Saturday July 01, 2017 @04:43PM (#14678908)
Re:How I Took an API Side Project to 250M Daily Requests by alam2000 on Sunday July 02, 2017 @04:03AM (#14681440)
You can also try the IP2Location LITE

http://lite.ip2location.com

Re:How I Took an API Side Project to 250M Daily Requests by matt2000 on Sunday July 02, 2017 @09:04AM (#14682307)
Thanks!
Re:How I Took an API Side Project to 250M Daily Requests by theoneone on Saturday July 01, 2017 @04:42PM (#14678901)
Good question!