News for hackers. Stuff that matters.

A comment on Researcher uncovers one of the biggest password dumps in recent history.

Re:Researcher uncovers one of the biggest password dumps in ... by TacticalCoder on Thursday January 18, 2024 @10:23AM (#39042815)
Anybody runs the HIBP password DB locally? (ideally with this latest treasure trove) I saw some converted it to a Bloom filter (which makes lots of sense: for all the passwords on which it answers 'definitely not in set', you know there's no false positive and in case it'd answer 'potentially in set' you could still query manually against the online DB).

I'll search online but if a fellow HNer runs it offline, I'm all ears...

P.S: I've got Gbit/s FTTH as well as servers in datacenters so downloading tens of gigabytes ain't an issue

[ Parent | on HN ]
Re:Researcher uncovers one of the biggest password dumps in ... by andrewla on Thursday January 18, 2024 @12:59PM (#39045070)
Curious about your use case; using their online service [1] is up-to-date and reveals almost no information.

[1] https://news.ycombinator.com/item?id=39044339

[ Parent | on HN ]
Re:Researcher uncovers one of the biggest password dumps in ... by tylerchr on Thursday January 18, 2024 @11:53AM (#39044125)
I wrote a thing for this back when you could download the whole hash database as a single torrent, but I haven’t checked it since they moved over to the PwnedPasswordsDownloader system. This doesn’t use any probabilistic data structures though, it just packs the database into the smallest binary file I could come up with.

https://github.com/tylerchr/pwnedpass

[ Parent | on HN ]
Re:Researcher uncovers one of the biggest password dumps in ... by aaronax on Thursday January 18, 2024 @11:20AM (#39043645)
It is only tens of gigabytes, no need for fiber to handle this. The 37GB of files can be downloaded in 1 hour on an 83mbps link.

(1gbps is 450GB/hour, useful for estimating things)

[ Parent | on HN ]
Re:Researcher uncovers one of the biggest password dumps in ... by myself248 on Thursday January 18, 2024 @12:28PM (#39044625)
My favorite rule of thumb is that 100Mbps is about 1TB/day.
[ Parent | on HN ]
Re:Researcher uncovers one of the biggest password dumps in ... by lorenzbrun on Friday January 19, 2024 @04:26AM (#39053419)
I have a project which acts as a local cache for the HIBP database.

https://github.com/lorenz/hibp-cached

It downloads and continually updates from the upstream database while serving the identical API. On a fast link it can download the entire thing in a few hours.

It just uses a giant BoltDB file to store compressed chunks.

[ Parent | on HN ]