News for hackers. Stuff that matters.
Re:How I Took an API Side Project to 250M Daily Requests by rickduggan on Sunday July 02, 2017 @02:58AM (#14681261)
This is super cool. I use a similar API to provide a client-side service called IP Request Mapper (https://chrome.google.com/webstore/detail/ip-request-mapper/...). Coming soon to a Show HN near you.

What it does is show where every asset on a web page is loaded from. It allows you to visualize how many different requests go into building just one web page. While it's gotten much better, the Houston Chronicle (https://chron.com) used to make about 500 individual requests to build its home page. It's down to about 125.

It's best to run it across two different monitors, with IP Request Mapper on one monitor and your "normal" browser window on another. Then enter any URL and watch the map start populating based on the geolocating every request made by the page.

But it's projects like ipinfo.io that make these other things possible. Standing on the shoulders of giants and all that...kudos to you, coderholic.

Re:How I Took an API Side Project to 250M Daily Requests by heipei on Sunday July 02, 2017 @03:07AM (#14681291)
That's the same motivation that started me to build https://urlscan.io. I wanted an easy way for everyone to visualise the amount, size and destinations of the various HTTP requests that a single page-load triggers. Incidentally I also created a tool that is being used by a lot of Security / Phishing researchers. If you ever want some inspiration for additional IP / domain annotation sources, check it out. I should really do a "Show HN" soon ;)

Meanwhile, this is a scan for a particularly noisy German newspaper website (faz.net): https://urlscan.io/result/f23e2e7e-e1eb-4591-9794-92f97957dd...

This website contacted 35 IPs in 7 countries across 24 domains to perform 302 HTTP transactions. Of those, 51 were HTTPS (17 %) and 35% were IPv6. The main IP is 92.123.94.227, located in European Union and belongs to AKAMAI-ASN1. In total, 4 MB of data was transfered, which is 9 MB uncompressed. It took 2.51 seconds to load this page. 16 cookies were set, and 42 messages to the console were logged.

Re:How I Took an API Side Project to 250M Daily Requests by sumedh on Monday July 03, 2017 @03:50AM (#14686601)
Any domain I enter, i just get this

invalid csrf token Error code 403

Re:How I Took an API Side Project to 250M Daily Requests by random_rr on Sunday July 02, 2017 @09:43AM (#14682457)
Very cool tool. Thanks for posting this!
Re:How I Took an API Side Project to 250M Daily Requests by rickduggan on Sunday July 02, 2017 @07:39PM (#14684887)
Agreed. heipei, love what you did here.
Re:How I Took an API Side Project to 250M Daily Requests by sAbakumoff on Sunday July 02, 2017 @06:39AM (#14681889)
How is it different from the dev tools network feature that is available in every browser?
Re:How I Took an API Side Project to 250M Daily Requests by rickduggan on Sunday July 02, 2017 @07:36PM (#14684872)
Not sure if you are asking me or heipei. For my project, it shows the requests on a map, where the requests in dev tools aren't geolocated and mapped. Same basic principle, just a different view.
Re:How I Took an API Side Project to 250M Daily Requests by sAbakumoff on Monday July 03, 2017 @01:36AM (#14686066)
Okay, why is it helpful? Or it is just for entertaining?
Re:How I Took an API Side Project to 250M Daily Requests by rickduggan on Tuesday July 04, 2017 @01:26AM (#14693716)
I'd call it more interesting than helpful (not that it's not helpful). It gives insight into how a web page is actually built -- while I'm sure most readers here understand it, that's not true for the general population.

It might make developers think twice about how they build sites if they could see how overly complex they get. As I mentioned, the Houston Chronicle site used to require about 4x the number of requests as it does now so someone did some optimization.

Re:How I Took an API Side Project to 250M Daily Requests by sAbakumoff on Tuesday July 04, 2017 @03:21PM (#14697743)
Reducing number of resources loaded by a web page is the basics of the performance optimization.