News for hackers. Stuff that matters.
Re:What Test Engineers Do at Google: Building Test Infrastru... by sly010 on Friday November 18, 2016 @03:50PM (#12989969)
In a large graph you would re-test the same part of the code over and over, because many top level inputs is going to do trigger the same code-path deep down in the graph, which is wasteful. In a sense they memoize service call answers, so any tested code-path only needs to run once.

If they actually do this across their code they will never have to run "real service calls", because those services are also tested the same way. It's mocking all the way down.

Re:What Test Engineers Do at Google: Building Test Infrastru... by bit_logic on Friday November 18, 2016 @04:15PM (#12990137)
Paraphrasing the famous quote: There are only two hard problems in CS: naming things and cache expiration.

An issue I see with this is that there's a potential window when tests pass with bad data. For example, tests using a mock and the mock is periodically verified with a service call. The mock could become bad data, but won't be marked as bad until the next service call verification. Until that happens, the tests using the mock will all pass. It's not clear from the article how they address this.

Re:What Test Engineers Do at Google: Building Test Infrastru... by dmoy on Friday November 18, 2016 @05:02PM (#12990462)
... and off-by-one errors.
Re:What Test Engineers Do at Google: Building Test Infrastru... by paulddraper on Friday November 18, 2016 @07:18PM (#12991124)
Perhaps Google tracks the changes to the dependencies, and reruns tests against a a real service when it changes.

That's how I would do it, at least.

If you have task-oriented build system with dependencies (like Google does with Blaze), it'd fit right in.

Though depending on how you deploy, you may have old and new services at different versions. Assuming a level backwards/forwards compatibility may be reasonable.

Re:What Test Engineers Do at Google: Building Test Infrastru... by jaawn on Monday November 21, 2016 @11:50AM (#13006793)
I imagine that if the service call to verify mock data fails, it should retroactively invalidate all tests based on that mock data, or at least this should be understood to be the case by the people reviewing test reports.