News for hackers. Stuff that matters.
Re:How I Took an API Side Project to 250M Daily Requests by gfodor on Saturday July 01, 2017 @04:46PM (#14678928)
- Database needs to be purchased

- Database needs to be distributed to your servers

- Database can become out of date easily

- Database lookup requires going to local disk and having a relatively fast access path/cache for lookups

- In general, a local database requires a large amount of effort compared to just running a curl in your PHP code.

If you are actually going to use a database, the proper solution does not look like "put it on your webservers" anyway, it looks like "put it on a separate service with a fast caching layer" etc etc. So in other words, the proper solution to decouple yourself from a 3rd party API is to... build a 1st party API.

In other words, not a 20 minute job. For small shops, a quick curl during the page load is a 20 minute job.

Re:How I Took an API Side Project to 250M Daily Requests by heipei on Saturday July 01, 2017 @05:39PM (#14679190)
There are many ways to quickly get and use a GeoIP DB like MaxMind, most requiring way less than 20 minutes. This is just one example of something I started using recently:

docker run -p 8080:8080 -d fiorix/freegeoip curl localhost:8080/json/1.2.3.4

(Repo: https://github.com/fiorix/freegeoip)

Re:How I Took an API Side Project to 250M Daily Requests by callumjones on Saturday July 01, 2017 @07:10PM (#14679618)
Where are you going to host it, how will you cycle the container to keep it up to date, and will you need to load balance it?

Running a Docker container in production is not a 20 minute job.

Re:How I Took an API Side Project to 250M Daily Requests by deno on Sunday July 02, 2017 @04:41AM (#14681541)
Just put HAProxy in front of it. It’s not rocket science.

https://www.haproxy.com/solutions/microservices.html

Re:How I Took an API Side Project to 250M Daily Requests by callumjones on Monday July 03, 2017 @01:57PM (#14690162)
Setting up HAProxy in a production environment with discovery for the docker container may not be rocket science but it’s certainly not 20 minutes.
Re:How I Took an API Side Project to 250M Daily Requests by motet_a on Saturday July 01, 2017 @06:43PM (#14679481)
Yes, Docker is great but learning it can definitely require more than 20 minutes. A `curl ipinfo.io` is still simpler.
Re:How I Took an API Side Project to 250M Daily Requests by deno on Sunday July 02, 2017 @04:45AM (#14681557)
This is the tax you’re paying for that every time:

  Connected to 35.165.108.15:443
  
  HTTP/1.1 200 OK
  Server: nginx/1.8.1
  Access-Control-Allow-Origin: *
  Content-Type: application/json; charset=utf-8
  Date: Sun, 02 Jul 2017 08:54:40 GMT
  X-Content-Type-Options: nosniff
  Connection: keep-alive
  
  Body discarded
  
    DNS Lookup   TCP Connection   TLS Handshake   Server Processing   Content Transfer
  [      4ms  |         244ms  |        518ms  |            525ms  |             0ms  ]
              |                |               |                   |                  |
     namelookup:4ms            |               |                   |                  |
                         connect:248ms         |                   |                  |
                                     pretransfer:767ms             |                  |
                                                       starttransfer:1292ms           |
                                                                                  total:1292ms
Re:How I Took an API Side Project to 250M Daily Requests by gfodor on Sunday July 02, 2017 @03:16PM (#14683896)
The OP was "baffled" why anyone would ever use this API. Clearly, the API's success shows that sometimes this tax you highlight is well worth it. (That was my point as well.) Nobody is claiming the tax doesn't exist, just that it shouldn't be baffling that a rational actor would choose to pay the tax in exchange for the corresponding benefits.

(Not to mention with very minimal effort you can usually avoid the majority of the specific tax of latency you mention, by doing things like parallelizing the request with other work or doing it asynchronously to the user's interface.)

Re:How I Took an API Side Project to 250M Daily Requests by wittedhaddock on Sunday July 02, 2017 @08:40AM (#14682231)
What tool (or arg passed to curl maybe) allowed you to generate that response? The graph is great
Re:How I Took an API Side Project to 250M Daily Requests by deno on Sunday July 02, 2017 @04:18PM (#14684145)
https://github.com/reorx/httpstat

It’s great for checking something quickly, otherwise you have the same thing in browser’s dev tools.

Re:How I Took an API Side Project to 250M Daily Requests by jhgg on Saturday July 01, 2017 @09:57PM (#14680374)
It is kinda a 20 minute job. On production, where I work, we just bake the maxmind DB into our app server images (it updates when we redeploy - which is a couple times a week) and use the maxmind client to read from that DB file.

Was just an extra step in our build pipeline.

Re:How I Took an API Side Project to 250M Daily Requests by user5994461 on Tuesday July 04, 2017 @12:48PM (#14696899)
You're misinterpreting "database".

In this case the geoIP database is like a 1MB CSV file, provided for free to the entire world by maxmind (a major network provider).

You can put give that file to many servers like nginx/apache out of the box, they will start adding a header with the country and the city of the client.

What this service is doing is effectively looking up the ip block in that csv file and returning the result as JSON.