yodaarjun

This month I've been working on automating the data collection and processing for my property research website [0]. I'm testing out DuckDB to process geospatial data, and so far the process has been quite pleasant.

As long as you don't need many advanced spatial functions (where PostGIS shines), and you make sure to minimize the use of GDAL to perform operations, it parallelizes processing quite nicely (most queries on large datasets saturate as many cores as you throw at it).

[0] https://wynds.com.au

Tantivy is great! I was using Postgres FTS with trigrams to index a few hundred thousand address strings for a project of mine [0], but this didn’t scale as well as I’d hoped with a couple million addresses. Replaced it with the tantivy cli [1] and it works a charm (ms searches on a single core vm).

[0]: https://wynds.com.au [1]: https://github.com/quickwit-oss/tantivy-cli

Uh oh. HN crashed the website again

Just tried it out. It's really good Would love to see how you build this out!

hackernews traffic crashed the host :( Cant wait to try tho!

For Deep Learning, deeplearning.ai has launched a free course on Coursera, which you may want to check out.

Hi Bilters. Sure, I'll update the website with more details.