News for hackers. Stuff that matters.
Re:Csvlens: Command line CSV file viewer. Like less but made... by nickjj on Saturday January 06, 2024 @12:30PM (#38893305)
One thing I often do with CSVs is sum up all or specific rows in a column.

For example maybe you're doing end of year taxes and now you have this large CSV export from your bank or payment provider with multiple categories and you want to get the totals for certain things.

In a GUI tool it's really easy to sort by a column and drag your mouse to select what you want and see it summed in real time.

Oftentimes things aren't clean enough to have 100% confidence that you can solve this with an automated script because maybe something is spelled slightly different but it's really the same thing. This feels like one of those things where spending a legit 10-15 minutes once a year to do it manually is better than trying to account for every known and unknown edge case you could think of. The stakes are too high if you get it wrong since it's related to taxes.

Has anyone found a really good standalone basic spreadsheet app that "just works" which isn't Microsoft Excel that works on Windows or Linux? I don't know why but Libre and Open Office both struggle to parse columns out in certain types of CSVs and the sorting behavior is typically a lot worse than Google's spreadsheet app but I'd like to remove some dependence on using Google.

Re:Csvlens: Command line CSV file viewer. Like less but made... by hectormalot on Saturday January 06, 2024 @01:30PM (#38893850)
If you want to use the CLI, Visidata might be what you want. It does have a bit of a learning curve. Beyond that, I've found it quite handy to do quick data explorations. e.g. there are shortcuts for histograms, filtering, x-y plots, etc.

[1]: https://www.visidata.org

Re:Csvlens: Command line CSV file viewer. Like less but made... by AB1908 on Saturday January 06, 2024 @12:50PM (#38893462)
I've never tried this myself but you could try importing the CSV in sqlite and then run aggregate queries over it. Does that sound useful?
Re:Csvlens: Command line CSV file viewer. Like less but made... by nickjj on Saturday January 06, 2024 @01:05PM (#38893604)
You can but it's back to depending on code to get the totals. This is one spot where IMO being able to visualize the data by seeing the rows and have immediate feedback on the sum is useful. You can continue dragging or use CTRL + SHIFT clicking to select more stuff as needed while letting your brain decide what should be grouped together.

With a SQL / code approach you have to account for these things without being able to see them and then adjust the code afterwards to include your custom groupings. It ends up taking more time. If the categories didn't change every year it would for sure be worth it to code up a solution since you'll know the edge cases by looking at the existing CSV but it's a moving target because it could change next year.

Re:Csvlens: Command line CSV file viewer. Like less but made... by mongol on Saturday January 06, 2024 @12:57PM (#38893520)
Was about to suggest the same
Re:Csvlens: Command line CSV file viewer. Like less but made... by snidane on Saturday January 06, 2024 @01:49PM (#38894048)
There is sc-im, which is the closest you'll get to a full spreadsheet app in terminal with vi controls.

https://github.com/andmarti1424/sc-im

Re:Csvlens: Command line CSV file viewer. Like less but made... by wobblykiwi on Saturday January 06, 2024 @01:00PM (#38893552)
I haven't dug too much into the tool, but could you use something like Datasette for that?
Re:Csvlens: Command line CSV file viewer. Like less but made... by e12e on Saturday January 06, 2024 @03:00PM (#38894827)
Have you tried gnumeric?
Re:Csvlens: Command line CSV file viewer. Like less but made... by nickjj on Saturday January 06, 2024 @03:24PM (#38895086)
Nope, but I just tried. I went to their site and noticed there's no binaries for Windows. The site is also served over HTTP (not HTTPS) and the default experience for apt installing it on Ubuntu 22.04 didn't work due to a bunch of packages no longer existing:

    E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/e/evince/evince-common_42.3-0ubuntu3_all.deb  404  Not Found [IP: 91.189.91.83 80]
    E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/poppler/libpoppler118_22.02.0-2ubuntu0.2_amd64.deb  404  Not Found [IP: 91.189.91.83 80]
    E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/poppler/libpoppler-glib8_22.02.0-2ubuntu0.2_amd64.deb  404  Not Found [IP: 91.189.91.83 80]
    E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/g/ghostscript/libgs9-common_9.55.0%7edfsg1-0ubuntu5.5_all.deb  404  Not Found [IP: 91.189.91.83 80]
    E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/g/ghostscript/libgs9_9.55.0%7edfsg1-0ubuntu5.5_amd64.deb  404  Not Found [IP: 91.189.91.83 80]
    E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/e/evince/libevdocument3-4_42.3-0ubuntu3_amd64.deb  404  Not Found [IP: 91.189.91.83 80]
    E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/e/evince/libevview3-3_42.3-0ubuntu3_amd64.deb  404  Not Found [IP: 91.189.91.83 80]
    E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/e/evince/evince_42.3-0ubuntu3_amd64.deb  404  Not Found [IP: 91.189.91.83 80]
    E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/w/webkit2gtk/libjavascriptcoregtk-4.0-18_2.42.1-0ubuntu0.22.04.1_amd64.deb  404  Not Found [IP: 91.189.91.83 80]
    E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/w/webkit2gtk/libwebkit2gtk-4.0-37_2.42.1-0ubuntu0.22.04.1_amd64.deb  404  Not Found [IP: 91.189.91.83 80]
Going to go with nope on this one.
Re:Csvlens: Command line CSV file viewer. Like less but made... by e12e on Saturday January 06, 2024 @03:57PM (#38895375)
Gnumeric isn't in main on Debian/Ubuntu anymore?

Ed: looks like it is in universe on Ubuntu - you have universe enabled?

https://packages.ubuntu.com/search?keywords=gnumeric

Ed: Did you apt update? Core pieces of evince missing sounds very strange?

Re:Csvlens: Command line CSV file viewer. Like less but made... by lowbloodsugar on Saturday January 06, 2024 @01:45PM (#38894004)
Jupyter?
Re:Csvlens: Command line CSV file viewer. Like less but made... by hermitcrab on Sunday January 07, 2024 @08:48AM (#38901149)
Have you looked at data wrangling software like Easy Data Transform, Knime or Alteryx? It is specifically written for cleaning, merging, summing and reshaping tabular data. They can all handle CSV and Excel files (plus a lot more).