News for hackers. Stuff that matters.
Re:Using less memory to look up IP addresses in Mess With DNS by maxmouchet on Sunday October 27, 2024 @11:54AM (#41963473)
Another solution is to use an MMDB (“MaxMind DB”) file [1] which is essentially a binary tree + deduplicated values (same as idea 3.1).

There are several free ASN MMDBs [2,3] but you can also build your own MMDB files from any Prefix->Value mapping with the mmdbwriter library [4] or a CLI tool built on top of it like mmdbctl [5].

Assuming the ASN MMDB is fully loaded in memory, it would use around 60MB.

[1] https://maxmind.github.io/MaxMind-DB/

[2] https://dev.maxmind.com/geoip/docs/databases/asn/

[3] https://ipinfo.io/products/free-ip-data-downloads

[4] https://github.com/maxmind/mmdbwriter

[5] https://github.com/ipinfo/mmdbctl

(I work for IPinfo, but there are lots of other companies offering MMDB files).

Re:Using less memory to look up IP addresses in Mess With DNS by seligman99 on Sunday October 27, 2024 @01:10PM (#41964013)
This is the same idea I used for my Cloud IP lookup tool [1], lets it all work in browser with a small file to search against

[1] https://cloud-ips.s3-us-west-2.amazonaws.com/index.html

Re:Using less memory to look up IP addresses in Mess With DNS by maxmouchet on Sunday October 27, 2024 @02:22PM (#41964492)
Oh this is nice, and a cool use of HTTP range requests!
Re:Using less memory to look up IP addresses in Mess With DNS by ignoramous on Sunday October 27, 2024 @05:10PM (#41965716)
Oh, then you'll love: Hosting SQLite databases on GitHub Pages or any static file hoster (2021), https://news.ycombinator.com/item?id=27016630
Re:Using less memory to look up IP addresses in Mess With DNS by zdyxry on Monday October 28, 2024 @06:38AM (#41969677)
cool
Re:Using less memory to look up IP addresses in Mess With DNS by mattpallissard on Sunday October 27, 2024 @03:39PM (#41965035)
"MaxMouchet DB"
Re:Using less memory to look up IP addresses in Mess With DNS by gallypette on Monday October 28, 2024 @03:51AM (#41968640)