Programmer: Loading times are really slow, I want to look into it next sprint.
Management: Feature X is higher priority, put it in the backlog and we'll get to it.
Aside from being generally shameful, the real kicker was that this was a "website performance & reliability" company x__x
Many of us, after having run out of other crap to do, would sit around and wonder if the "B" grade network engineers were assigned to run the company LAN, or the ones we sent onsite were as incompetent.
Internal IT is almost invariably a cost center, the technicians providing the service you are selling to customers are working in a profit center. So, yeah, probably that plus be managed in a way which focussed on minimizing cost not maximizing internal customer satisfaction or other quality metrics.
In this case it was a JSON, but using .NET so Newtonsoft is at lease efficient. The issues were many cases of: * Converting string to lowercase to compare them as the keys were case insensitive. (I replaced with a StringComparison.OrderinalCaseInsensitve) * Reduntant Dictionary's * Using Dictionary<Key, Key> as a Set. replaced with HashSet.
The data wasn't meant to be that big, but when a big client was migrated it ended up with 200MB of JSON. (If there data was organised differently it would've been split accross many json blobs instead)
It would also be nice to handle it alls as UTF8 like System.Text.Json does. That would half all the strings saving a fair bit. (I mean the JSON blob it starts with is converted to UTF-16 because .NET)
But I could be wrong. I've only worked with sites and apps not gaming.
I don't think you choose to become a manager at one of the world's top video game companies if you don't love video games.
Hell, I don't think one of the world's top video game companies would hire you as a manager if you didn't convince them, in interviews, that you understand and love their products by using them yourself.
Using your own company's products whenever possible is generally a pretty important part of being a manager.
Same thing happened with memory consumption. The problem was ignored until we regularly had background jobs using > 4 GB of memory, causing cascading failures of our EC2 instances and a whole bunch of strange behaviour as the OOM killer sniped random processes.
Programmer(s): Can we set aside some time to fix the long loading times?
Management: No, that won't earn us any money, focus on adding features
I work LiveOps and usually long loading times are something that we would take seriously, as it negatively impacts the reputation of the game.
Management: So, what can we do about the loading times?
Programmer(s): That's just how long it takes to load JSON. After all, the algorithm/function couldn't be more straightforward. Most of the complaints are probably coming from older hardware. And with new PC's and next-gen consoles it probably won't be noticeable at all.
Management: OK, guess that's that then. Sucks but nothing we can do.
Management had no idea of knowing whether this is true or not -- they have to trust what their devs tell them. And every time over the years someone asked "hey why is loading so slow?" they get told "yeah they looked into it when it was built, turns out there was no way to speed it up, so not worth looking into again."
And I'm guessing that while Rockstar's best devs are put on the really complex in-game performance stuff... their least experienced ones are put on stuff like... loading a game's JSON config from servers.
I've seen it personally in the past where the supposedly "easy" dev tasks are given to a separate team entirely, accountable to management directly, instead of accountable to the highly capable tech lead in charge of all the rest. I've got to assume that was basically the root cause here.
But I agree, this is incredibly embarrassing and unforgiveable. Whatever chain of accountability allowed this to happen... goddamn there's got to be one hell of an internal postmortem on this one.