News for hackers. Stuff that matters.
Re:How Googlebot crawls JavaScript by erikb on Tuesday May 12, 2015 @05:24AM (#9530420)
But how! I don't know about other people here, but in our company we haven't figured out how to parse (for testing of course) dynamic websites. All tools, including free tools like Selenium and paid tools like QF-Test, seem to not be able to understand how it works, or our web developers are not able to code dynamic web like it should be coded.
Re:How Googlebot crawls JavaScript by hpaavola on Tuesday May 12, 2015 @06:07AM (#9530586)

    pip install robotframework-selenium2library
test.txt:

    *** Settings ***
    Library           Selenium2Library

    *** Test Cases ***
    My First Test
        [Setup]    Open Browser    http://google.com    firefox
        Input Text    name=q    Robot Framework
        Click Button    name=btnG
        Wait Until Page Contains Element    ires
        Click Link    Robot Framework
        [Teardown]    Close All Browsers
and then execute

    pybot test.txt
Selenium2 and Robot Framework are pretty neat when it comes to web testing.
Re:How Googlebot crawls JavaScript by maerten on Tuesday May 12, 2015 @06:28AM (#9530654)
We're using CasperJS (basically a simple interface to PhantomJS), works great for us. Have a look here: http://casperjs.org/
Re:How Googlebot crawls JavaScript by onion2k on Tuesday May 12, 2015 @05:32AM (#9530454)
I use nightwatch.js ( http://nightwatchjs.org/ ). It's a layer on top of Selenium that makes browser testing a lot more straightforward. If you start with small, straightforward tests and build testable things from there you code will improve.
Re:How Googlebot crawls JavaScript by erikb on Tuesday May 12, 2015 @06:34AM (#9530680)
The problem is that tester and developer aren't the same person. This way the tests are much better at finding expectations the developer didn't have, but it's harder to convince the developers to code more testable, because they don't know the pains of testing.
Re:How Googlebot crawls JavaScript by onion2k on Tuesday May 12, 2015 @07:43AM (#9530961)
The point of testing is not to prove the code doesn't work. It's to prove the code does work. That subtle but important difference is the key to good testing.

Finding a problem with code is useful, but it's extremely limited. You might find 100 bugs, but if there's 101 bugs your product has the potential to fail completely. It's so much more useful to define a framework of things that the code has to do properly and make sure it does do them all properly. To that end, testing should come first - define what the code needs to do, write tests to make sure it does those things (automated unit tests where possible, but at the very least well defined processes for how you make sure it works), and then write the code to actually do it. Any developer who isn't interested in proving their code works, and will continue to work as it becomes more complex, is a terrible developer.

tl;dr If you want to fix testing don't write any code until you know how you're going to test it works.

Re:How Googlebot crawls JavaScript by danielsamuels on Tuesday May 12, 2015 @05:44AM (#9530506)
WebDriverWait and expected_conditions.presence_of_element_located are your friends. Here's some sample code from our automation project: https://gist.github.com/danielsamuels/a39e0fef4e15d2ab04b5
Re:How Googlebot crawls JavaScript by LunaSea on Tuesday May 12, 2015 @06:19AM (#9530628)
You should look into Phantom.js or Zombie.js.
Re:How Googlebot crawls JavaScript by amelius on Tuesday May 12, 2015 @09:02AM (#9531372)
Google made chrome, so I'm sure they know how to render webpages, and execute JavaScript. It's all mostly open source so you could use it too.
Re:How Googlebot crawls JavaScript by bhartzer on Tuesday May 12, 2015 @09:55AM (#9531747)
Yes, Google made Chrome. Keep in mind that they created Googlebot, and Chrome is really only a "slimmed down" version of Googlebot. Chrome came about because of Googlebot.
Re:How Googlebot crawls JavaScript by erikb on Wednesday May 13, 2015 @06:08AM (#9537863)
Or maybe they did it to get full access to the users (having access to your customers through other providers sks), get a more stable web experience (which increases the numbers of all users and thereby the number of google ad viewers), and have more power in the definition of web standards?
Re:How Googlebot crawls JavaScript by motti on Tuesday May 12, 2015 @12:17PM (#9532795)
I don't think this is true at all.