I have a couple small projects on the backburner at work that require me to reverse engineer file formats. I've been able to decode a few basic details out of the files by looking for patterns in a hex editor but the finer details are still escaping me.
It gets tougher when there are anti-debugging protections in place but that generally happens only in the video games industry.
If the format is simple enough though, you will be fine with a good hex editor and some pattern recognition.
Your other option is something like QuickBMS - on mobile, can't link.
As a counterpoint, the community I founded around Hearthstone (http://hearthsim.info/) is extremely open source friendly and on excellent terms with the developers. It's really about how you decide to manage it.
To echo this with code, I've open sourced my collection of game archive unpackers. Given how similar they all are, this is a typical example of how to add a new decoder:
https://github.com/shish/pyge/blob/master/archive/PackDat3.p...
(Note that two of those 6 short strings are user-friendliness metadata and can be omitted, and one of the others is redundant -- "Magic Bytes", "Header Format" and "Directory Entry Format" are all you really need in most cases)
The work consisted mainly in reverse engineering WoW's file formats, protocols etc in order to understand, index and organize data, and to be able to produce "datamined" patch notes as soon as new game files were available.
I got to work with and learn from people a thousand times smarter than me, it was by far my favourite job and it's why I still do various Blizzard-related work today.
It also creates an interesting relationship with the game's developers. I think on that front, many studios could learn a lot about working with the people interested in the technical aspects of their game, rather than working against them. Most get it horribly wrong, few of them get it right at all - Riot is one of the few that do, I wish I enjoyed LoL. :)
http://www.eurogamer.net/articles/2013-12-06-avalanche-gives...
vs
http://www.eurogamer.net/articles/2015-08-10-rockstar-bans-g...
Studios that get too large for their own sake and get completely disconnected from their userbase. They no longer see the players, the fun or even the game - they just see the numbers. Business guys thinking a video game company can be run like a bank.
The devs get blamed for it, too; as if they had anything to do with it. These decisions are made high above them.
> At HandmadeCon @mike_acton brought up the game data format problem. I've reverse engineered hundreds of proprietary formats from games, and he's right that all I would need to make things simple is a basic description of the file data. Just struct declarations are enough. No encryption has stopped me, I always find the data I need, and your data all has the same general architecture, so there's no reason a game developer should be afraid of the public knowing how their game's data is laid out. This zeitgeist of closely guarding that information is doing you more harm than good — moddability is a key feature of many successful games, and all modders want is docs.
--
There's a lot of other industries that could stand to learn from this. Reinventing the wheel sucks. Doing your own, proprietary thing in your corner, taking on extra maintenance work, extra work to modify existing tools to make them compatible with your own image/archive/whatever format... it all sucks and nobody wins.