If you use hledger, you can actually write a csv importer directly in the tool: https://hledger.org/import-csv.html

I wrote my own idempotent parser before this existed but would give this a try first

I've kind of been thinking that there needs to be an open source version of Yodlee/Plaid. I keep picking up some project like Paisa and then realise that I have to figure out how to import things automatically. I don't really have the patience to do it manually.

But for a few months now I've been thinking an OSS service that does this might be great for such OSS projects. Maybe I'll eventually start it.

I think the problem is that you need to establish a business relationship with the giant banking systems to get them to give you access and unless you have contacts or a way into that world this is very hard to do. I don't know how you'd pull it off in an OSS project.

Not to be that guy that tries to force LLMs into everything, but after automatically importing bank transactions, LLMs like GPT are very powerful in extracting information from the ill-formed, non-standard, transaction description, and subsequently classifying it. You can help train it by manually identifying and classifying the first few, and let it do the rest of the job. I've tried it out myself and works well for me!

What software have you used that does this?

So far every time I've relied on automatic categorization for this sort of thing it fails horribly. I don't think I've used anything that's GPT based though.

> What software have you used that does this?

I wrote my own script that uses the GPT API. For automating bank transcation downloads, it's just a cronjob that runs ever X hours and scrapes the information from the banking website.

Would you be willing to share the prompts you use?

How would LLMs really help though if all you get is usually an order number? I mean, it can probably figure out that an order from McDonalds is food related (though does it get categorized as "Everyday Lunch" or "Going out with Friends" or "Work Events"?) and Geico will probably be some sort of Insurance - but that would just be a large database of common payees, no real need for any AI. That said, if you're importing years worth of data, it would be a great starting point to get ballpark estimates of where your money goes, so that has value.

And then there's e.g., "Amazon Transaction AB2314ACWERF" which could be a new Fridge ("Household Appliances"), a 3D Printer ("Hobby Expenses"), a video game ("Entertainment Expenses"), or a giant double-headed adult massager ("Fax Machine Maintenance") - but the bank statement wouldn't have enough information.

Yep, that's one of its deficiencies! For me, something like GPT helps in triaging and "cleaning up" bulk of the transactions, with little micro-categorization left to be manually done.

I played around with this quite a bit with chat gpt 4 with confirmation instructions, and after a bit of time it starts going haywire. I played around with just creating in a simple csv format, transactions date, name, category (asking it to categorize it), and amount. After a couple runs it started going haywire and hallucinating with transactions I never created.

I created an internal rails clone of financier.io and just created a spreadsheet web area i could copy/paste mass transactions in so i could add a batch at a time ( I suppose I could upload a csv also, but the problem is every bank has different formats)

I have tried this and it works very well. I just copy paste my investments data from website and put it as a comment and usually copilot will figure out how to format it as a ledger transaction based on the previous transactions.

I am not sure if it will work with bulk import though. It's easy to spot mistakes with single entry, hard to do when you have lot of them

> Do people who use this kind of software manually enter every transaction they do every day, or something?

Yes, exactly that. You can do automatic imports of your bank statements but that's arguably more hassle.

It takes 10 minutes at the end of my day to record all my transactions and it gives me a complete understanding of where my money is going.

> It takes 10 minutes at the end of my day to record all my transactions

I’d love to be able to do this, but realistically (for me), I’d need something that works reliably on my worst days. If I’m out late, on vacation for a few days, dealing with some on-call thing all night, etc., I don’t want to be beholden to a 10-minute per day commitment that accumulates linearly. Discipline can only get me so far in the face of chaos—automation can take me the rest of the way there.

This would be a lot harder if you have a spouse and want to track your family finances. It would require both people working together to keep it up to date, and finding that time is tough with kids.

For all the concerns and problems over tools like Mint, having all 10-20 accounts we have automatically sync their data into the system makes it so much easier to manage.

We don't find it hard. We sit down one or two times during the week in the evening and I enter all our transactions into KMyMoney. It takes 5-10 minutes to do once you get the hang of it. This is across 5-10 accounts. You might have a little more work to do with 10-20 accounts.

On Paisa (ledger). I like it! But...KMyMoney has such a nice interface for recurring/future payments and reports (that I can edit to suit our needs) that I'm struggling to see the point of ledger behind gui for our use case. It does look nice though.

I only do it every few days but pretty much the same. I find it's too easy to lose track of how much I've been spending, so by doing it manually I always notice before it becomes a problem. It also helps me save because I notice right away if I get charged for a subscription I'm not using and I can cancel it.

I do the bank import once a week. I set aside an hour on my calendar to update all of my spending accounts. It usually takes more like 15-20 minutes unless I find something unexpected.

For longer term accounts like my 401k, IRAs, and brokerage accounts, I track money going in our out but only update the gains/losses every 6 months or annually on Dec 31st.

10 minutes a day is a lot! That's 300 minutes a month. I do it about once a month and I probably do 2-3 hours.

At least with ledger cli, the format for a transaction is simple enough that a just make a script that reads the CSV that I can export from my bank’s website.

But to your point, it’s still a lot of customization and manual work, I never keep up with it for more than a month or two

I used to host an instance of Firefly III, which has a perfectly usable mobile web interface.

This way, I would log every transaction immediately rather than wait until I get home.

Yes, I do this for over 15 years every day. You can easily train yourself, it never "breaks" and takes only a few seconds.

I download my bank/cc statements once every few months and run them through some homemade cleanup scripts, then I could technically plug it into a tool like this.

I also have some clean up scripts, but the worst part is I have to manually select the tables to extract in the PDF using Tabula. There's still some serious manual reconciliation between credit card and bank statements. I feel good doing it, but it's a slog.

Technically, I might be able to get something similar by using some website to download everything, but honestly I don't trust them. I don't want all of my banking, investment, and spending information consolidated in one place for some tech bro douche bag to sell off to some richer fucks and then lose it to some hackers.

https://tabula.technology

Paisa supports PDF import (very rough implementation), but the problem is actually simpler compared to what tabula is trying to solve. I just need a 2 dimensional array and then the template can easily filter out non financial data by checking if specific column is Date. I have tried it personally on a few PDFs, it worked well except one which had a two column layout.

The paid accounting tools usually will have integration with your banks via Plaid or similar

I do this since ~20 years with a simple Calc-Table. Easy overview and simple to search.

I've been attempting to build a little hobby budgetting app the past couple weeks that could automatically sync from my bank since none of the OSS options seem to be able to this and require manual logging.

I thought this would be fairly straightforward but it's been anything but that - what few open formats there were for this thing are all being sun-setted and most US based banks seem to only allow API access of any kind to established companies. The only real option is to go through Plaid which still seems to require initiating a business relationship with them to get through all the red tape.

For whatever reason, if your in the US, real time syncing of this sort of thing just isn't an option at the personal level.

Only other solution I can think of at this point is to manually scrape with a puppeteer script.

There is a middle ground I think - all my banks/credit cards (7 or 8 accounts spread across 4-5 providers) offer csv downloads of transactions. I haven't tried to automate downloading the csv, but once the csv exists locally automation becomes straightforward. But yes, for real-time automated gathering of the data, this wouldn't be appropriate I imagine.

Splitwise user here

Most banks (at least in the US) allow a data export, usually in Quicken/OFX or CSV format. I download the transactions from my accounts once a week or so and import it into the tool. (PayPal gets charged to my credit card, so it shows up on the credit card import, no need to track PayPal separately)

I currently use YNAB4, and I've assigned categories to Payees, so when it sees a transaction to e.g., my local Pub, it knows to categorize it as Food. For stuff like Amazon or eBay, I need to manually categorize each transaction, but that takes a minute or two.