News for hackers. Stuff that matters.
Re:What technologies are under your site by tectonic on Saturday July 09, 2011 @11:07PM (#2746918)
Hey guys, I made this site and just gave a talk about it at SHDH. Someone must have submitted it. Thanks for all your feedback, I really appreciate it!
Re:What technologies are under your site by mmaunder on Sunday July 10, 2011 @12:23AM (#2747028)
Awesome! Suggestions:

Do a reverse lookup on the site's IP and add matchers for the hostnames that show up.

Mention if they're round robining their DNS.

Add nmap OS fingerprinting.

Do a traceroute and log the IP of the closest router (final hop) to the site and add matching for that.

Add a wiki interface and build a crunchbase like app.

Add archiving of data and monitoring over time (as netcraft did in their original app).

Re:What technologies are under your site by JoshTriplett on Sunday July 10, 2011 @12:29AM (#2747034)
nmap OS fingerprinting seems a bit aggressive, but reverse-DNS checking seems like a great way to check which hosting platform a site uses.
Re:What technologies are under your site by 16s on Sunday July 10, 2011 @09:38AM (#2747740)
nmap is too aggressive. It's a prelude to actual hacking attempts and labeled by IDS systems as such. Don't use it for this or you may end up in legal trouble.
Re:What technologies are under your site by tectonic on Sunday July 10, 2011 @04:21PM (#2748535)
Agreed. UnderTheSite makes a specific point of only considering information that would be returned when a user's browser hits a website. It doesn't scan or probe ports / urls.
Re:What technologies are under your site by tectonic on Sunday July 10, 2011 @12:56AM (#2747079)
Thanks for the great feedback!
Re:What technologies are under your site by known on Sunday July 10, 2011 @03:55AM (#2747300)
Re:What technologies are under your site by jf on Sunday July 10, 2011 @01:26AM (#2747119)
Awww, I wish I didn't have to leave SHDH before the lightning talks started!

I wrote a Ruby program to do something similar to what you're doing: https://github.com/jpf/domain-profiler - If you ever start profiling sites using information from places other than what the server returns, perhaps what I've done can help inspire you?

Re:What technologies are under your site by adrianwaj on Sunday July 10, 2011 @02:04AM (#2747166)
Is there a working example of domain profiler please?
Re:What technologies are under your site by tectonic on Sunday July 10, 2011 @01:26AM (#2747120)
Thanks, I'll check it out!
Re:What technologies are under your site by dorkitude on Sunday July 10, 2011 @04:49AM (#2747363)
Cool site! I'm having a lot of fun. However, I was confused by this flow:

First: http://drktd.com/8Jgn

After clicking the 'Add It!' link: http://drktd.com/8Iaf

The wording on the previous page had me expecting a page for manually adding technologies to my site's stack a la Bagcheck.com.

Re:What technologies are under your site by tectonic on Sunday July 10, 2011 @04:50AM (#2747367)
Ah! This is a feature that I plan to add. Do you want to select technologies from a list, or write free-form text about your technology stack?
Re:What technologies are under your site by JoshTriplett on Sunday July 10, 2011 @03:49PM (#2748447)
I'd like to suggest avoiding manual additions of technology for as long as possible. Focus on adding more ways to match specific technologies. After all, a site could always advertise more technologies in server headers or meta generator tags.
Re:What technologies are under your site by tectonic on Sunday July 10, 2011 @04:22PM (#2748537)
What if we propose an extension to humans.txt called technologies.txt where a site can self-describe their stack?
Re:What technologies are under your site by JoshTriplett on Thursday July 14, 2011 @07:07PM (#2765571)
A mechanism for sites themselves to advertise their stack seems like a great idea (though I'd prefer it not occur via fixed URLs like humans.txt or robots.txt, but via headers or meta tags). I'd just suggest not allowing arbitrary additions to a site's stack without any way to verify them.
Re:What technologies are under your site by tectonic on Sunday July 10, 2011 @02:02AM (#2747162)
Follow @underthesite for updates as I implement all of the great suggestions that you guys have made here.
Re:What technologies are under your site by thomasfl on Sunday July 10, 2011 @04:48PM (#2748592)
Would love to see if it could detect more scripting languages and frameworks than PHP.
Re:What technologies are under your site by jbhelms on Sunday July 10, 2011 @12:14AM (#2747017)
It incorrectly stated that my site uses cold fusion.
Re:What technologies are under your site by MJR on Sunday July 10, 2011 @01:04AM (#2747089)
What's the site?
Re:What technologies are under your site by jbhelms on Sunday July 10, 2011 @02:05AM (#2747167)
Re:What technologies are under your site by suyash on Sunday July 10, 2011 @12:40AM (#2747060)
I posted it buddy, I liked your demo a lot!
Re:What technologies are under your site by tectonic on Sunday July 10, 2011 @12:57AM (#2747080)
Thanks :)