Thanks.
Did you just crawl random sites and search for .mp3 content on them? Or did you have a set of pre-defined search sites to craw?
Thanks for clearing that up, good luck with your site!
Do you plan to document/opensource you work ?
Also we pack each individual hash before storing in Elasticsearch and gained at least 50% storage space this way.
Our Fingerprint data is quite different from theirs(unreliable ID3 tags, N versions of same track) which is why we needed some tweaks. So far the matching is still far from perfect...
Whether we will open source the whole thing at some point we don't know yet.
I know at some point we did adapt the Solr end (for example, we removed the N most occurring codes) for speed optimizations.
Many users of Echoprint in the wild have adapted the python matching logic for their use case as well as changed the hash update rate on the codegen. A great modification to watch was Sunlight Labs' "Ad Hawk", which ID'd commercials: https://github.com/sunlightlabs/adhawk
Our matching algorithm is based on the open source echoprint-codegen fingerprinting method, which we have built our own stack around:
- Replaced Solr/Tokyo Tyrant with Elasticsearch
- Reimplemented matching-logic
- Crawlers search multiple sources for audio files to be indexed (mp3s arent stored long term, only fingerprinted then deleted)
- Indexing about 1 new track per second
- Found method to verify unrealiable ID3 tags (in progress, current database also includes unferified)
- mogilefs as primary data store for fingerprints
- perl everything
We also provide a free music identification API.
Any feedback would be much appreciated!