News for hackers. Stuff that matters.

A comment on CD Projekt Red is under investigation. Parent comment

Re:CD Projekt Red is under investigation by lumost on Monday January 11, 2021 @03:54PM (#25736698)
I wonder how much of this is broken QA throughout the industry. I burned ~60 hours in cyberpunk on a ps4 pro, the content of the game was fun - but the bugs were pretty dumb. Many of the worst bugs originated in story pathways that would only be triggered if various conditions had occurred (which undoubtedly changed during development).

From a testing perspective It seems like it would require an impossible amount of QA time to vet all of the quest paths as a player, and it would be easy to miss game breaking bugs if QA testers were using manipulated save files. Issues like the bad police AI only crop up once in the main game, but are pretty noticeable throughout free roam.

If players want games to get bigger, will we need smarter and more automated QA tools? what would these look like?

[ Parent | on HN ]
Re:CD Projekt Red is under investigation by pythonaut_16 on Monday January 11, 2021 @04:01PM (#25736867)
> Q: Open-world games are often really buggy, because there’s just so much going on. But I experienced very little of that in my time with Breath of the Wild. How did you pull that off? Was it just a really extensive QA process?

> Dohta: There was another point that we developed during our QA process. We came up with a number of scripts that would basically allow the game to be played automatically, and allow Link to run through various parts of the game automatically. And as that was happening, on the QA side of things, if a bug did appear I’d suddenly get a flood of emails about it. That was one tool that we found to be really handy.

https://www.theverge.com/2017/3/11/14881076/the-legend-of-ze...

Breath of the Wild used a tool to do automated run throughs as part of their bug testing suite. This is just a quote from one interview, but if you do a bit of Googling you can find some good information about their development and planning process.

[ Parent | on HN ]
Re:CD Projekt Red is under investigation by duhi88 on Monday January 11, 2021 @06:19PM (#25739465)
From what I understand, they also promised a bunch of features that were never implemented. Maybe they're still to come, but it sounds like they are months away from fixing all the bugs, let alone implementing better AI and other things players complained about.

I bet it would have been met with less backlash if they delayed only the PS4/Xbox S versions. Players must expect some Day 0 bugs from games this large.

[ Parent | on HN ]
Re:CD Projekt Red is under investigation by fpgaminer on Monday January 11, 2021 @06:01PM (#25739110)
> Many of the worst bugs originated in story pathways that would only be triggered if various conditions had occurred

I think the absolute worst bug I encountered was because of this, but I don't believe the story pathway that triggers it was uncommon. In fact, that condition apparently has a major effect on the story later.

Basically it was part of a main story quest where you have to wait a day for a character to call you before the quest can continue. But I never got a call. I saw a lot of threads on the issue, and it seems the bug happens if you did some optional dialogue right beforehand. But unlike other optional dialogue in the game, this optional dialogue was really hard to miss. I had to actively run past an NPC to avoid it.

I ended up losing about an hour of gameplay from that bug :/

I agree with you that it was likely because they changed something last minute and never checked it again; a QA failure for sure. And my theory is the COVID situation made QA testing a nightmare. Still ... that bug was brutal. Every "wait for X to call you" quest after made me anxious.

[ Parent | on HN ]
Re:CD Projekt Red is under investigation by ethbr0 on Monday January 11, 2021 @04:02PM (#25736886)
It's not rocket science. It's devs treating QA like second-class citizens and substituting (cheap) person-hours for proper technical tools.

Any sequence of events can be represented as a directed graph.

Any event check can be validated against that directed graph as feasible.

Instead, Bethesda (equally guilty) and CDPR seem to let their devs add whatever checks, and then trust QA to untangle and validate the infinite number of combinations.

tl;dr - open world games are incompatible with traditional QA methods and tools

[ Parent | on HN ]