News for hackers. Stuff that matters.
Re:Show HN: Dropbase 2.0 – Turn offline files into live data... by mikorym on Tuesday August 18, 2020 @02:23PM (#24202245)
I've used Pandas recently, not sure if this will ever help you, but dictionaries are much faster if you continually add rows.

pandas.concat and similar functions for appending to a Pandas dataframe can be quite slow. Just mentioning this in case you ever encounter this issue; maybe you don't need to. In my case I changed to dictionaries for important parts and 2 mins changed to 5 seconds execution time.

However, in my case, I have to change logical row structure and not just read in rows as is.

Re:Show HN: Dropbase 2.0 – Turn offline files into live data... by ayazhan on Tuesday August 18, 2020 @03:09PM (#24202932)
thank you for advice! we'll consider it