News for hackers. Stuff that matters.
Re:Does the Google Bot index css hidden divs? yes. by kolbusa on Wednesday September 28, 2011 @11:28AM (#3048331)
This seems rather intuitive. Does the google bot even parse css? Is there any evidence that it ignores non-displayed text present in HTML?
Re:Does the Google Bot index css hidden divs? yes. by infinity on Wednesday September 28, 2011 @12:18PM (#3048612)
Googlebot and some other search engine crawlers also crawl css stylesheet files, I have seen evidence in my server log files.

There are several patents describing how a search engine might analyse the visual appearance and structure of a website, with a special view to give some parts of the page content (the main content) a higher weight or priority then the rest (like footer text and navigation). I would expect that the algorithms used by Google will give a good impression about the page rendering. What conclusions are drawn from this and how it affects the ranking of individual pages for certain search queries, who knows, ... pretty much about this is kept secret and there is a lot of speculation.

Many search engine patents are explored and explained by Bill Slawski on his blog "SEO by the sea":

http://www.seobythesea.com/

That there are these patents does of course not tell us, that search engines are working exactly in this or another way, but I think it often gives a good impression about what might be going on behind the scenes.

Re:Does the Google Bot index css hidden divs? yes. by blauwbilgorgel on Wednesday September 28, 2011 @02:54PM (#3049345)
Further reading:

  Techniques for approximating the visual layout of a web 
  page and determining the portion of the page containing 
  the significant content 
http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Se...

  Automatic Visual Segmentation Of Webpages
http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sec...

  Techniques for approximating the visual layout of a web 
  page and determining the portion of the page containing 
  the significant content 
http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Se....

  VIPS: a Vision-based Page Segmentation Algorithm
http://research.microsoft.com/apps/pubs/default.aspx?id=7002...

  Classifying functions of web blocks based on linguistic features
http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sec...

  How a Search Engine Might Identify the Functions of 
  Blocks in Web Pages to Improve Search Results
http://www.seobythesea.com/2011/02/how-a-search-engine-might...
Re:Does the Google Bot index css hidden divs? yes. by cooperadymas on Wednesday September 28, 2011 @11:57AM (#3048516)
Yes, it does take into account text that is hidden from view. This used to be (still is in some places) a deceptive tactic to rank a page higher in the search results. You could hide a massive block of keywords out of site of your visitor, or hide links to sites that you want ranking higher. Alternatively, more malicious people would (and still do) hack into a website in order to hide links in this same manner. The website owner isn't likely to notice them if they aren't visible on a page.

Google is pretty good at figuring out what is there for legitimate reasons versus spam.

Re:Does the Google Bot index css hidden divs? yes. by jonknee on Wednesday September 28, 2011 @12:57PM (#3048814)
Yes, Googlebot also parses Javascript. All part of the SEO arms race.