News for hackers. Stuff that matters.
Re:Facebook was used as a proxy by web scraping bots by coderholic on Thursday November 05, 2020 @11:31PM (#25004918)
At https://host.io we scrape every registered domain once a month, and make the meta data available freely over an API. You could use that to get a title for a domain (although not for a URL that's not the main domain), eg:

    $ curl https://host.io/api/web/facebook.com?token=$TOKEN
    {
      "domain": "facebook.com",
      "rank": 2,
      "url": "https://www.facebook.com/",
      "ip": "157.240.11.35",
      "date": "2020-08-26T17:39:17.981Z",
      "length": 160817,
      "encoding": "utf8",
      "copyright": "Facebook © 2020",
      "title": "Facebook - Log In or Sign Up",
      "description": "Create an account or log into Facebook. Connect with friends, family and other people you know. Share photos and videos, send messages and get updates.",
      "links": [
        "messenger.com",
        "oculus.com"
      ]
   }
See https://host.io/docs for more details about the API and what else you can do with it (eg. find backlinks to domains, domains with the same adsense ID etc)

There are no comments on this comment yet.