Re:533M Facebook users' phone numbers and personal data have...
by b212 on Saturday April 03, 2021 @06:25PM
(#26685294)
Could you please tell me how did you convert it to sqlite? I've got a huge 1 GB txt file that crashes my comp every time I try to search for myself there :( Thank you!
[ Parent
| on HN ]
Re:533M Facebook users' phone numbers and personal data have...
by banana_giraffe on Saturday April 03, 2021 @06:37PM
(#26685375)
Super hacky python script I used to turn the text files into a sqlite database:
Re:533M Facebook users' phone numbers and personal data have...
by kuyan on Sunday April 04, 2021 @12:47AM
(#26686960)
Thanks, this worked for me! Here's my contribution; I tweaked your script a little for memory usage. https://pastebin.com/4hA8VACe
Re:533M Facebook users' phone numbers and personal data have...
by shazzdeeds on Sunday April 04, 2021 @04:04PM
(#26692412)
Thanks. I tweaked the script to include an incremental printout across files and also creating an index on phone num to easily search.
Re:533M Facebook users' phone numbers and personal data have...
by bobbylarrybobby on Saturday April 03, 2021 @07:02PM
(#26685516)
I suggest using a shell program like grep (or even the search feature of less), as shell programs are notoriously good at lazily seeking through files to keep memory use low.
Re:533M Facebook users' phone numbers and personal data have...
by knolan on Saturday April 03, 2021 @06:48PM
(#26685427)
Firstly don’t do something like open it in notepad. 1GB text files are not exactly difficult to work with once you use a proper text editor or parsing tools.
Re:533M Facebook users' phone numbers and personal data have...
by throwmusic2366 on Saturday April 03, 2021 @07:19PM
(#26685607)
Does it open with Geany?
It's FLOSS. Available on Windows, Mac and Linux.
Re:533M Facebook users' phone numbers and personal data have...
by cable2600 on Sunday April 04, 2021 @01:56AM
(#26687229)
I used to use Borland's Brief to edit large files. But it is commercial. There is a free version in beta here https://sourceforge.net/projects/grief/ Grief. It pages the file in and out of memory to save space.
Re:533M Facebook users' phone numbers and personal data have...
by datavirtue on Saturday April 03, 2021 @06:45PM
(#26685411)
Try Ultra Edit, free trial. It can read and search massive text files without crashing. Quite responsive.on 10GB files.
Re:533M Facebook users' phone numbers and personal data have...
by max_hammer on Sunday April 04, 2021 @02:23AM
(#26687336)
You can use `awk` for any sql like operations on file.