Re:GTA III and Vice City fully reverse engineered: re3
by mschuster91 on Sunday February 14, 2021 @11:06AM
(#26133458)
Lots of things are at play here (excuse the bad pun):
* SaaS stuff can be built very quickly (if you use cached node_modules or PHP composer's vendor directory) and deployed to parallel CI test runners that don't have much requirements. In contrast, your typical game takes much longer to (re)compile and as soon as you touch anything that requires graphics, you need some sort of (beefy) GPU in the server. Anyone who has ever tried virtualization with PCIe passthrough knows what a pain that is to set up, not to mention NVIDIA explicitly disallowing to virtualize their GPUs.
* It's easy to do unit tests for simple entities and operations (e.g. collision detection, physics)... but games are full of global state mutated from across the place.
* For everything that tests visual (or, god forbid, audio) output, computers are bad at testing. Change a texture? Great, now you have to re-check all your tests if they're still accurate and working. Humans are, for now, still better.
* Hardware-specific bugs: these are one of a goddamn nightmare. For most software, it behaves identically across computers - let's take most Docker containerized software... it will essentially run on anything that's amd64. But games integrate with OSes, especially the OS 3D stack (DX, OpenGL, Vulkan, Metal and on top of that the GPU drivers and GPU firmware) so incredibly deep that there are lots and lots of potential issues and workarounds. It's simply impossible to test for these.
* Closed source components in the loop: Games ship with all sorts of blobs, from video codecs to sound effect libraries. These are in your address space, but you can't really test them.
* Human tests are cheaper than writing test suites and setting up CI test farms (see the first point in my list).
* SaaS stuff can be built very quickly (if you use cached node_modules or PHP composer's vendor directory) and deployed to parallel CI test runners that don't have much requirements. In contrast, your typical game takes much longer to (re)compile and as soon as you touch anything that requires graphics, you need some sort of (beefy) GPU in the server. Anyone who has ever tried virtualization with PCIe passthrough knows what a pain that is to set up, not to mention NVIDIA explicitly disallowing to virtualize their GPUs.
* It's easy to do unit tests for simple entities and operations (e.g. collision detection, physics)... but games are full of global state mutated from across the place.
* For everything that tests visual (or, god forbid, audio) output, computers are bad at testing. Change a texture? Great, now you have to re-check all your tests if they're still accurate and working. Humans are, for now, still better.
* Hardware-specific bugs: these are one of a goddamn nightmare. For most software, it behaves identically across computers - let's take most Docker containerized software... it will essentially run on anything that's amd64. But games integrate with OSes, especially the OS 3D stack (DX, OpenGL, Vulkan, Metal and on top of that the GPU drivers and GPU firmware) so incredibly deep that there are lots and lots of potential issues and workarounds. It's simply impossible to test for these.
* Closed source components in the loop: Games ship with all sorts of blobs, from video codecs to sound effect libraries. These are in your address space, but you can't really test them.
* Human tests are cheaper than writing test suites and setting up CI test farms (see the first point in my list).