Re:Show HN: Zingg – open-source entity resolution for single...
by rishsriv on Wednesday February 09, 2022 @12:13PM
(#30275137)
Thanks for pointing me to the performance numbers!
No open datasets that I'm aware of for fuzzy geocoordinate matching, unfortunately
Re:Show HN: Zingg – open-source entity resolution for single...
by sonalgoyal on Wednesday February 09, 2022 @12:42PM
(#30275539)
Hmm..guess we will wait along and keep an eye on such datasets.
Thanks for liking zingg, super excited to hear this :-) Here are some performance numbers. https://docs.zingg.ai/docs/setup/hardwareSizing.html
We see performance varies by a) Number of attributes to match b) Size of data c) Type of matching and the features we compute for each d) Hardware and cluster size
Although we do not do matching across languages like English with Chinese, we have tested Zingg quite rigorously with Chinese, Japanese, Hindi, German and other languages and it seems to work out of the box. Likely due to the inbuilt Java unicode support and the ML based learning.
You make a great point about continuous variables like lat/long, age etc. Age seems to work, again due to integer differences and the learning. Have not tried lat/long yet. Would you have any dataset you could recommend for testing?