pwojnaro

This becomes even harder if your execution environment is non-deterministic by definition.

Imagine test automation on physical devices (ex-founder of a mobile app testing SaaS company here) where system0level events that can't be programmatically detected or dismissed can delay, interrupt or alter the outcome of your tests.

Dealing with exceptions to make execution outcomes more stable seem like an obvious choice, but what if it's not feasible due to the complexity of the environment? We found that accepting variance and extrapolating test results from multiple executions in the same (or identical) environment was the only way forward.

Thanks, I missed that. Looks like Apple added the section on leasing (3. Leasing for Permitted Developer Services) in the SLA for Big Sur in November 2020.

It's entirely possible that the 24hr minimum is driven by the operational overhead, not just the licensing terms. Since this is bare metal, re-provisioning between customer sessions might require some level of human interaction, and this needs to be baked into the cost model of such a service.

Very cool. With Frank, Calabash and Zucchini already out there (to name just a few), the iOS UI automation space is getting pretty crowded. Having said that, it is definitely nice to see something that is not based on Cucumber. Any ETA on the Android?