Ask HN: How do game companies keep their files organized for a large-scale game?

Re:Ask HN: How do game companies keep their files organized ... by jtolmar on Thursday February 04, 2016 @04:59PM (#11037630)

The best way to do it that I've found is to arrange a folder hierarchy of "where" rather than "what." So there's no one place with all the game's rock models, but rather firelands/towerofeternalflame/courtyard/rock01.3ds and so forth. "Where" doesn't have to mean a literal location; it could be playerbases/horde/rock.3ds or something. Though for most games, everything not tied to the player really can just be its literal location. It's crucial that these directories have a mix of models, textures, and whatever else you need, instead of a mass of parallel structures.

If there are automated tools, they should run on the entire asset directory and dump to a new directory in whatever hierarchy makes sense to the tool. When that turns out to be slow, check "file modified" timestamps in the tool instead of dumping required knowledge on the artist.

Getting artists to actually use source code control instead of having personal folders full of things like rock01_retouched_new_final_final.3ds is a bit of a pipedream, but you can at least put the asset directory under source control and make that the only way to see things in the engine.

[ Parent | on HN ]

Re:Ask HN: How do game companies keep their files organized ... by solipsism on Thursday February 04, 2016 @05:11PM (#11037715)

Getting artists to actually use source code control instead of having personal folders full of things like rock01_retouched_new_final_final.3ds is a bit of a pipedream

BS. Just hire better artists. It sounds like hand-waving, but I've worked with both kinds of artists. You're building games, so if you focus on only hiring artists based on their portfolio you might end up with a bunch of stubborn artists who need to be hand-held and worked around constantly. If instead you focus on hiring artists who are good at art AND who can demonstrate a propensity or desire to learn new workflows and and to be flexible, then you'll end up with a bunch of team players.

This is a common complaint about artists, but it has more to do with culture and expectations than anything else.

[ Parent | on HN ]

Re:Ask HN: How do game companies keep their files organized ... by blablablame on Thursday February 04, 2016 @05:40PM (#11037867)

Agreed. While a few artist keep a few _01 _02 files around, mostly as backup/undo files, my last few years in the industry, they all used some kind of version control (maybe not git, but perforce and svn with their visual tools)

[ Parent | on HN ]

Re:Ask HN: How do game companies keep their files organized ... by dev360 on Thursday February 04, 2016 @06:24PM (#11038087)

Sounds like perforce is common when there are lots of binary files - is this mostly due to storage concerns?

[ Parent | on HN ]

Re:Ask HN: How do game companies keep their files organized ... by vvanders on Thursday February 04, 2016 @06:30PM (#11038120)

Bunch of factors:

  - Really good binary performance(light years ahead of git).
  - Proper locking for files that can't be merged(basically everything in the art dir).
  - Proxy support for co-located teams.

Game repos can easily approach multiple TB just for the art assets. In that area Perforce is once of the few VCS that scales appropriately.

[ Parent | on HN ]

Re:Ask HN: How do game companies keep their files organized ... by malkia on Thursday February 04, 2016 @06:43PM (#11038188)

+1. Also easy graphical interface - P4Win/P4V, and easy to understand the system of changelists. It also helps that older changelists mean things submitted in the past, newer in the future - e.g. in contrast with digest/fingerprint schemes like other systems use.

[ Parent | on HN ]

Re:Ask HN: How do game companies keep their files organized ... by vvanders on Thursday February 04, 2016 @08:14PM (#11038685)

Yup forgot about that but a good point as well.

[ Parent | on HN ]

Re:Ask HN: How do game companies keep their files organized ... by sytse on Thursday February 04, 2016 @07:19PM (#11038385)

Yes, perforce is really good with large binary files. Only recently Git LFS allowed for git repositories to store large binaries as well.

[ Parent | on HN ]

Re:Ask HN: How do game companies keep their files organized ... by ido on Friday February 05, 2016 @02:29AM (#11040103)

We use git with lfs for our game. It has on occasion been a bit of a pain, however perforce is too...unfortunately there isn't a really great solution atm, just a bunch of more or less flawed ones.

[ Parent | on HN ]

Re:Ask HN: How do game companies keep their files organized ... by baszalmstra on Friday February 05, 2016 @03:41AM (#11040308)

Can you comment on the problems you ran into with LFS?

[ Parent | on HN ]

Re:Ask HN: How do game companies keep their files organized ... by ido on Friday February 05, 2016 @08:19AM (#11041164)

sometimes it/git would mess up for unknown reasons and people will get the text file with a pointer to the lfs-hosted asset instead of the asset itself.

Also it's very slow for small files, and the filtering is according to file extension.

So e.g. if I have some massive mp3s for (sometimes many minutes of) music but also many (100s-1000s) small ones for sound effects downloading 1000 small mp3s take a couple seconds per file even if each is only ~40kb big (normally download rate is a couple MB/sec).

[ Parent | on HN ]