News for hackers. Stuff that matters.
Re:Show HN: Private, in-browser, CSV-SQL query runner using ... by pdyc on Sunday November 03, 2024 @01:04AM (#42031267)
crashed with big file https://www.papaparse.com/resources/files/big.csv i evaluated sqlite for my tool and wrote about it here https://newbeelearn.com/blog/million-rows-csv-debug-story/

BTW i checked your sqlite.wasm to be quite compact, just 340kb are you doing custom compilation.

Re:Show HN: Private, in-browser, CSV-SQL query runner using ... by freakynit on Sunday November 03, 2024 @01:25AM (#42031316)
Hey, thanks for trying out.

I did check out with the csv you linked. It worked for me. https://ibb.co/cb3SFbR. Before making it live, I did already test it with 100MB+ CSV's having like 10+ columns.

I also checked my sentry integration, but couldn't find any errors to your issue. Seems to me like OS/browser level ram restriction. In case you have anything to help me with debug this, do share..

Im using regular sqlite-wasm. No custom stuff.

Thanks..

Re:Show HN: Private, in-browser, CSV-SQL query runner using ... by itsbrex on Sunday November 03, 2024 @05:18AM (#42032223)
Works on my iPhone with the ‘big.csv’ file as well
Re:Show HN: Private, in-browser, CSV-SQL query runner using ... by freakynit on Sunday November 03, 2024 @07:44AM (#42032739)
Thanks a lot for trying it out and verifying :)
Re:Show HN: Private, in-browser, CSV-SQL query runner using ... by pdyc on Sunday November 03, 2024 @12:26PM (#42034457)
i tested on machine with 4gb ram, also i modified the query to remove the limit i.e. "select * from data"

can you let me know from where you are picking sqlite wasm? i checked here https://sqlite.org/download.html and sqlite wasm shared here is approx 830kb

Edit: never mind checked here https://github.com/sql-js/sql.js/releases/tag/v1.12.0 and its indeed around 330kb.

Re:Show HN: Private, in-browser, CSV-SQL query runner using ... by kfajdsl on Sunday November 03, 2024 @08:00PM (#42037469)
Loading that CSV and running `select * from data` left me with 879MB of memory usage on the tab.