Some examples:
https://github.com/wader/jqjq https://github.com/wader/catgolf
This seems like a pretty niche need with only some hobbyists motivated enough to work on it. Is there a broader application than your use case? Otherwise I think thats why the existing software for this isnt great.
On the other hand, it’s surprising to me that “grab sounds from a wii game” is so niche! My gut felt like it would be slightly more complicated than unzipping a tarball, but my gut didn’t expect it to be a programming challenge worthy of a small competition.
POS deserved all of it if not more.
Still, garnishing someone’s wages the rest of his life seems out of proportion. But I admit it’s harder to defend someone that made a livelihood out of holding peoples’ data hostage.
But it being a paid tool means there are fewer template contributions from 13 year olds, which if we are all honest make up the majority of unpaid open source contributions - they simply have more spare time.
1: https://www.sweetscape.com/010editor/
2: https://www.sweetscape.com/010editor/repository/templates/
3: https://github.com/tge-was-taken/010-Editor-Templates/tree/m...
4: https://wiki.redmodding.org/cyberpunk-2077-modding/modding-k...
I have a hard time imagining a 13 year old using that particular tool buying it. In fact there is a good chance it is used by the crackers themselves.
Still waiting to this day for FuSoYa to release the source code of Lunar Magic.
About a central database of binary parsers, I've been wanting this for ages too. The closest I ever found was augeas, but that's for configuration files.
I'm working on something, that is a open template format for binary file formats. It is usable today as a universal file extractor, with some bugs and limitations.
Check it out at https://github.com/martinlindhe/feng
I'm sure other communities have popped up that I haven't heard of, too. There's lots of interest in unifying forensic parsing under open work.
Check it out at https://github.com/martinlindhe/feng
Fully agree on the need for such a database. The problem is that it's been tried, but lacked traction. The latest one I've seen is Kaitai format files[1], that can be used in visualizers or to auto-generate parsers.
Anyway, I then wrote my own tools and put them on GitHub. And documented the reverse engineered file formats. And then got pull requests from people that want to help!
I was just voicing my bewilderment at this other culture. Don't understand why one would want to live like that.
This entire genre of tools has been a long time point of interest for me. In addition to making a couple OSDs and contributing some tiny improvements to Kaitai, I also have my own binary schema library for Go, restruct, which, biased or not, remains my favorite way to poke at arbitrary formats, since it's really easy to sketch stuff out and read and write to files quickly. It's basically Go's encoding/binary but with struct tags for more advanced things.
Same for me! It turned into a long journey and I am working on a solution that I am very happy about. I am yet to announce it publicly, but i been meaning to for so long already.
If you would like to check it out I would be happy!
You can use it to map / view content from a format there is a template for. Alot of common formats is already included and you can extend it using your own templates.
That said, it does exist in some form.
https://doc.kaitai.io/serialization.html
Restruct supports serialization. I use it all the time to hack on proprietary formats.
I do want to add some kind of runtime format support and i'm working adding kaitai support but it's not ready yet, it's not an easy thing to do :) but i've made very good progress. ideally it will be something like: fq -d format.ksy <query> file
Subscribe or keep an eye on this issue for updates https://github.com/wader/fq/issues/627
And feel free to ask any questions!
I was trying to rip some sounds from a wii game called Rhythm Heaven, and it’s ridiculous how primitive the tech is. By that I mean the programming community’s tech. If you want to extract some assets, you’d better be running Windows, and you’ll need to download some random exe from mediafire made by Jared, a 13yo that coded the extractor in C in his spare time. This is only a very slight exaggeration; Windows being a requirement isn’t.
Hopefully projects like this will standardize all binary formats once and for all.
Actually, this is a good opportunity to ask: how would one contribute a binary parser to fq? If I wrote one for wii sound files, can I just submit a PR or is there some other process?
EDIT: https://github.com/wader/fq/blob/master/doc/dev.md documents the development side of things. I’m more interested in the project itself — if someone puts in the work to make a decoder for an obscure binary format, will it get merged (assuming it’s high quality) or is this only for popular formats?