I think to some extent it's possible to reduce cognitive load even on these simpler actions just because you have to see less unrelated data on what you asked about.

And these type of requests consume very little tokens in comparison to almost any programming work.

[delayed]

Yes, I definitely agree. Though there's a lot more stuff that I can trust them with nowadays, that I couldn't before.

Not just not easier, but also a lot slower.

The UX experience for basically all LLMs that I've interacted with is having the AI sneak in where I don't want it and end up taking 10 extra seconds to do what used to be a 0.1ms operation.

The worst place for AI is in search bars, yet that seems to be the first place every product manager wants to put it. I hate waiting forever to get results because AI is poorly searching in the foreground.

In contrast, Confluence's AI bot has been a huge help to me.

... One wonders if that is due to the great implementation of the bot, or due to the less than stellar results of confluence search.

In Bitbucket, Rovo tells me it can’t write a PR description because it can’t see the code.

It does automatically write the PR in the first place but if you delete that description then try to have Rovo make a new one it fails hard.

Gemini in Google Docs/Workspace is so frustrating like this. I can get Claude to manipulate a Google Doc using the API, no problem, but talk to Gemini, the AI agent in the same browser window and it says it can't do anything [0]

[0] as of a couple of months ago when I last tried reformatting a slide deck using Gemini (failed) and then Claude (succeeded)

For your protection, citizen.

In contrast, Confluence's AI bot is a living nightmare to me.

Rovo makes changes I don't want that I have to undo and its "search" is bafflingly low quality compared to the old search experience, but most of all it just makes every page take longer to load and puts annoying UI elements in the way of what I actually want to interact with.

It mirrors every single other in-app AI bot experience I've had to date - less useful than the functionality they replaced with it, slow and annoying.

I’m a heavy AI user at work and I agree, Atlassian’s bot is painful. It’s also “gullible.” It will fail to do something and then just assume it cannot, that it cannot create pages or make issues or whatever.

The only one I’ve enjoyed is Grafana’s. It’s been a joy to use tbh.

The fail to refusal pipeline is frustratingly common with these in-app agents.

The “copilot” instances built into Microsoft’s programs are almost entirely useless due to this.

That’s because a full turn with the AI takes about as long as Confluence does to respond to a click.

We got something called "glean". It is better than Confluence search, because confluence search is utter crap. It's just a keyword scan with no concept of relevance.

I feel like I have to keep reminding people at work that LLMs are more expensive and have higher latency and less predictable behavior than plain old deterministic software.

Those tradeoffs must be considered when deciding whether a feature requires regular software or AI.

[delayed]

Or they just flat out don't work. I was on an insurance website recently, and getting big annoying popups to use their AI agent for help with the site or my plan, even encouraging me to ask questions. I did have one, very simple, "what's my plan's copay?"

"Sorry, I do not have that information."

So then, "Ok, where do I find it?"

"Sorry, I do not have that information."

What the hell? What is this for? Of course, someone will point out "it's because of regulations" or some technical limitation, sure, I can understand that, but then I must ask the question then why did you think it was worth building?

[delayed]

I've had the bot of a "modern" holiday home tell me some rubbish about luggage storage. Fortunately, I knew that there would be a human fallback during working hours so it worked out OK. But who thought that it was a good idea to have a shitty bot to ask about important things?

It's being sold everywhere to people fed up with dealing with other people who waste their time. Because that really IS a problem.

Most (Note: Not all!) questions you get when you work in a "customer service" role come from lazy, unobservant, or dumb people. So pre-AI, a large business needs an army of people to answer the phone and do things like walking confused people through, step-by-step, how to reset their password. "Did you check your email? Did you check the Spam folder? Did you give it 5-10 minutes?"

So AI is sold with the promise that you can just put it in front of your human agents, and then you'll only need 1/5 the staff (let's set aside the unfortunate economic implications for the other 80%) because most people's questions are stupid and trivial to solve.

Problem is, the dummies can't identify and divert themselves to the bot up front, so the person who loses most (other than the CS agents who lost their jobs) is people like you or me, who only contact customer service when we are reasonably sure there is no way to solve our own problem on the website/the app/by ourselves.

[delayed]

Once is a fairly long while, I do have a duh moment. But usually I hesitate and hesitate because i'm sure I'll get some low-level CS person working from a script. The upside such as it is is that I'm (rarely) pleasantly surprised.

Somehow it feels like it's real (for some people) in experience. I've dealt with both Spectrum/TWC and Verizon FiOS support within the last couple of years and been pleasantly surprised that I could talk 'tech' with the first person I encountered and we cut right through all the troubleshooting bullshit to get things fixed.

Spectrum - we had a voice and a data cable modem (due to location and cabling requirements). Lost the phone service. Called in and the first things I offered them were the MAC addresses off both modems and which one was supposed to be which. That set off the "shit, he knows what he's doing" moment and we got both modems running correctly in less than ten minutes, including IVR and hold time.

Verizon - had a weird ticket that ended up being a NOC issue. Explained the odd backstory of events, a ticket that happened a couple days prior that I was part of a regional outage, and the Tier 1 guy had me hop on a video call to point my camera at my computer at the ONT where I basically did everything without him asking me to (cabling direct, pulling up IP status info, etc, live reboots) and he ran it by a NOC guy remotely and it ended up being a bigger ticket above me. Took an extra day of a regional outage, but that data helped them get rolling.

It's nice to read the happy path cases like yours. The bad experiences tend to stick in our memories much more, but if I'm being honest, I have also sometimes had calls with bright, helpful people who weren't limited to script-reading and reboot prompting. Shoutout to the Tier 1 guys and girls out there who have taken it upon themselves to be truly helpful.

FWIW, my question would have been in any decent FAQ but there wasn't one. It was simply: I have to return my keycard when leaving the room in the morning, but I need to store some luggage until the evening. How do I get into the building and into the storage room? IIRC, the main door was just open at daytime, but the luggage room needed a keycard (really smart system, you clowns). Fortunately, there was a guard / service person when I came to pick up my stuff and I more or less knew that in advance based on previous days.

I had a bot tell me to click something in the chat when it had sent me an email... The whole experience was so bad that the company behind it lost my business.

I called my dentist because I was sick and wanted to move my appointment. You can't talk to a human anymore, it's all ai, after a bit of back and forth it assured me it had canceled my appointment and booked a new one 2 days later. I was actually surprised it went so well

The next day I get an email at 7pm telling my appointment was successfully rescheduled for 1pm (the same day), none of these match either my og appointment nor the rescheduled one

When I went to the dentist for the appointment the ai assistant supposedly rebooked, they told me there was a mistake and I was assigned a kids dentist.

In the end I had to rebook a third appointment with the receptionist

Dentists are plentiful and none of them really know what they are doing (see recent experiment posted here on HN that has The data to support this asserrion).

I switch at the slightest sign of friction.

I had this a while ago; had to actually physically go to the dentist to move my appointment (with the receptionist, who used to answer the phone). Absolutely ridiculous. Particularly as, if they actually want to automate this, well, _calendar software has existed for decades_. No need to use AI bullshit.

seems to me like having a chatbot is the 2020s version of the 2000s "having a website" or 2010s "having a facebook page"

except it's worse, if you manage to get some info out of it, it could all be just hallucinated, you were told a lie and no one is responsible.

it's like the only purpose is to erect a barrier between the company and its clients.

They’re building very little. They’ve been sold/told AI is good at customer service. They usually only bother pointing it to their knowledge base and FAQs and can only really answer questions if the answer is in that

Okay, now do "all my transactions this year for retail stores"? Or "All luxury / deductible purchases"?

Of course when you arbitrarily limit the problem domain to some specific example, you can make any tool seem pointless and stupid. It's when you expand your goals to things that aren't trivial that you get benefits.

Do we need not Mathematica or NumPy because calculators can add just fine?

Then you need to double check all the results because LLMs can't be trusted to accurately select retail stores or deductible purchases.

Well that's a different argument, now isn't it?

What time does it save you if you still have to read through just as many entries to validate as you do to sort?

If you're the type who double-checks your work and this is important: 40% of the time.

If you're the type who doesn't: probably 90% of the time.

This feels like goal-post moving now, with questions less genuinely curious about an answer and more trying to "get" me. How am I supposed to answer that, it depends on the task and the model and the harness and specific human being using it.

Use it or don't, I don't care, but at least be genuine in the discussion.

Do I need to? If I moderately underestimate my entertainment spending because my Apple+ subscription got categorized as a grocery purchase, that doesn't seem like a big problem.

If you don't care to that level then why not put your finger in the air and just come up with a made up estimate yourself?

Like most people, my finger in the air estimates are hugely biased towards underestimating how much I spend. An AI budget app that has like 50% accuracy would probably still not be useful, but 90% accuracy is fine, and while I have not gone around trying to audit AI budget apps I suspect the accuracy for good products is even better than that.

If you're gonna go the extra mile and use an AI-powered system to be able to cross-check data more in depth than a run of the mill report from your bank, it means you want 100% accuracy, not 90%.

The funny thing is that these expanded examples were already features in the budget app I use before AI came along.

What’s the budget app that lets you ask for transactions at luxury stores? Or retail establishments vs online?

Pretty much any banking app will classify the transactions nowadays?

Literally zero banking apps will classify them as "luxury" or "retail establishment" unless you set that up in advance.

What frustrates me is that there's the potential to bake much more information into transaction history.

There's a huge spec (Level III data) that lets the merchant provide full line item detail with every credit card purchase.

Corporate cards get it, but consumers don't. The payment ecosystem often doesn't even submit it if the card isn't identified as a corporate card.

If you had that, you'd probably need way less "smarts" to build budgeting tools-- no need to try to reconcile bank records with order reciepts.

> a lot of AI solutions aren’t easier to do than just doing the thing

Eh, I could see a case where an AI makes ordering for a group easier.

its a classical effect of having a hammer and recognising everything as a nail to whack. its pretty tricky to find a problem well suited to a tool like LLMs.

i programming stuff using LLM of which the resultant programs are deterministic and dont use LLMs or tokens is a better or easier approach to having gains. because the resulting program can be easier to use and doesnt cost a lot to run. Automations are a good target, or integratig libraries into other programs. tedious things u know well how to do but take time to do manually. then once done fix a problem or workflow that u need fixed by the solution yielded.

ive not seen 1 app that does calls to an LLM that i find useful, apart from harnesses that produce code (which is only recently becoming more appealing and might depend on how good and fast u can write code, at which im not the best nor the fastest.)

Recently laid off and not a big AI guy… but to illustrate your points:

Tailoring + proofing resumes to _manually_ apply to jobs is a great application of LLMs in my experience. But I wouldn’t want an AI to apply for jobs for me.

Funny thing about that. This time last year I was applying for jobs and kept my same old latex generated flawed human resume that I’ve been using 20 years or so.

Recruiters said it was standing out in the tech sector. Apparently AI likes to embellish way too much so you don’t trust it.

> But I wouldn’t want an AI to apply for jobs for me.

why? they’re using ai for most of the hiring process. it’s only fair for us to do it too.

Not that you're wrong, but it creates a race to the bottom. More and more people with use AI to apply because more and more people used AI to apply.

My wife has to submit a bunch of translated documents for procurement processes. Noone really reads those but they are a hard requirement. These are usually in DOCX or sometimes PDF with tables, product images etc. So I built her a little applet which (after clicking two buttons) sends the XML or PDF to an LLM in chunks and it spits back the same structure but with translated text. Of course there are edge cases where it doesn’t work perfectly but I’d say it works 95% of the time in a good enough quality to submit them. Saves her roughly 0.5-1 hour per job and much cheaper than other alternatives.

I’m doing something similar at work. Thousands of contracts, need to extract billing terms/logic. It’s scattered throughout each contract but has no summary, each contract can be hundreds of pages of non applicable legalese. The problem is, once I run it, can I trust the output without manual verification? How will I know if it skipped/missed something from a contract?

I would build a review dashboard that shows you the extracted data on one side and the visual snippet of the contract on the other. Quick to double check, and you still save time over opening the contract and searching for the right section and extracting it by hand or via OCR.

We have done something similar where it highlights the terms, basically each extracted term is linked to a specific highlight in the contract, and has them side by side like you said and you can jump around the contract by clicking on the extracted term. This helps when the agent needs HITL clarification on a term, as it gets you there quick and you can tell it more information to aid interpretation. But, if it missing something completely, I’d never know without reading the whole contract and validating every term I found the agent noticed. We’ve done some training sets and it does help and get better but the large variation of contracts means there’s little standardization.

I think we’re on path to just do random full review audits of maybe 10% to get a baseline confidence level and then just note it as a risk. Still a large manual task, we’ll probably outsource it.

I sat through an egregious demo of an AI add on for documentation. The flow was load doc (e.g. pdf) -> data filled into a specialized form view. To really wow me, they showed how you could type in the AI chat interface "change the title to 'abc'" and then it would make the change.

This is a major player in the industry, not some random kid.

This is where I see a lot of AI stuff right now honestly.

It's just "I want the computer to figure this out for me even if it takes longer than the old tools, because I cannot be bothered to learn how to use the old tools"

I've encountered this a lot

I think though if you re-phrase this to “I am lazy…” (the CORE common trait of every great SWE I have priviledge of working with in the last 30 years) “… and I do not want to manually do any work I can automate” things look a lot different through that lens

And then there’s the mindset of “the existing toolsets don’t fit my mindset very well, so I’ll have AI recreate them in a way that only makes sense to me and then harass everyone to adopt the new tool because I understand it.”

Someone didn’t like how our Jira tickets and Confluence docs were structured so they vibe-coded a new ticketing system that has a custom web UI with a Microsoft Access database on the back-end. Management and Customers don’t like it for a good range of reasons but he requires everyone who works on his projects to use it.

Another person didn’t like our build script for generating a templated Ansible role so they made some kind of MCP adapter or harness to ask you canned questions on what your role is named and what features to include, then scrapes our docs to get the required directory structure and generates it. It does literally the same thing but sometimes does it wrong and consumes tokens regardless.

It’s like the XKCD comic about competing standards, but accelerated for any person who is minorly inconvenienced and has API tokens to an AI service.

> click on the search bar/payee filter and type “McDonald’s.” Sure but you still have to open a dedicated app and that the search is just a string search.

I recently setup Muse (just seems to have the easiest out of the box connectors) to connect to all my accounts over Plaid. I can ask questions like: * Review all recurring transactions. * Show overview of last 12 months of spending by category. * Review last months CC transactions. * Estimate next months expenses.

And then in the same app, I automatically get personal financial and budget information that's actual useful. I can adjust the frequency and type of content just by updating the agent's goal.

I tried using YNAB. It felt so cumbersome compared to just asking AI for the info. I had to make a $300 fake expense to zero out an account minimium balance, I can't treat an investment account as a spending account (without some weird hacks).

Meanwhile, the same AI can cleanup my email inbox, watch my calendar, etc.

And I spent like ~2 days having Claude (over remote-control) create a Monte-Carlo simulation for retirement (including Tax calculations). Works great, I can read the code and make whatever changes I need.

The most friction I still encounter is the connector itself being too opinionated. Plaid's API tries to predict future expenses based on what's detected as a recurring charge; this of course was incorrect if similar Amazon.com charge amounts occur on similar days.

IMO purpose built apps are dead. A sandbox of tools that AI can use is far more flexible. Spending $20/month only needs to replace 3-4 other paid single purpose apps (or identify a single $20 recurring service to cancel) to be net-positive. Plus you don't have to configure it, just ask questions.

But that's you, a developer, who not only understands what they actually _need_ in order to have a good budgeting app and retirement planning app, for lack of a better word, but also are willing to spend the time setting it up, understand how it functions, and get it functional.

> And I spent like ~2 days having Claude (over remote-control) create a Monte-Carlo simulation for retirement (including Tax calculations). Works great, I can read the code and make whatever changes I need.

This is the biggest indicator of the divide between the AI-enthusiastic developer and the average person.

The average person isn’t going to want to sit down and spend two days to fine-tune code or a spreadsheet to accomplish this. They want an easy-to-navigate interface that is dead-simple and ready to go with the ability to get an answer done in five minutes or less. If they hit any snags in the first 30 seconds they’ll ditch it and move to the next solution. Why should they have to spend their time creating a duplicate solution when there are so many ready-to-go alternatives already out there? They want the path of least resistance.

True, but by "two days" I mean I asked Claude to do it from my phone while watching TV in the evening after work.

I think for people who can quickly understand and describe systems/code, AI is very magical. So many personal projects would be otherwise too tiring except for the 0.1% OSS developer personality that can spend 4+ hours a day working on stuff. Now AI means many projects can be implemented at the speed of thought.

Even still, the average person is going to have no idea how to prompt for such a thing. Do you think normal people know what monte-carlo simulations are?

I still use Monarch Money (similar to YNAB) but any time I have a real question I just download my data and ask Claude Code questions about my transactions. It works very well and it's not much effort.

There's a symmetry to things that happened in the mid-aughts as at that time there were some shifts in computing habits as well. For instance I knew a bunch of folks at the time who, instead of typing "www.website.com" into the URL bar, would open up google and type "www.website.com" and then click the top link.

I'd ask why and they'd just shrug, like *obviously* google is the front page for everything on the web.

Your anecdote is spot on and it feels like the people who are doing things that way are channeling similar energy.

That's the whole point. Convert the most basic and mundane thing into a revenue-generating subscription.

> Something that is perhaps “underreported” is that a lot of AI solutions aren’t easier to do than just doing the thing.

Even apps can be strictly worse than lower tech solutions.

A local car service I can text with “I need a car to airport XYZ tomorrow at 7am” beats every app flow.

Until they are a no-call no-show at 5am to go to the airport. Then you’re a Rideshare Co customer for life because there’s tracking and they will keep you in the loop.

Look at this person who has never been screwed by Rideshare Co before.

I have never had my car service miss a pickup in 20 years. (Though I've slept through an alarm clock a couple of times.) But, then I don't really have reliable Uber or taxi service where I live.

That's how I feel about LLMs for programming, too. I've tried using them to write programs for me, but every time, when I finish up it winds up being a wash in terms of time+effort, or it took more time+effort than just writing it myself.

Yahoo Mail tried to add an "AI summary" to the top of all emails. The two line summary was worse than merely reading the two lines in the original email.

I'm not sure about what you use but my primary credit card (a major American bank) has an incredibly annoying interface for searching previous transactions.

Going to the grocery store is a similar example. For a while I was big on getting my groceries delivered. But then I started to realize that the amount of time it took to fill my cart was basically the same amount of time it took in the actual store, so I wasn’t really saving much time or convenience at all… plus I was missing out on some intrinsic joy of discovering and picking things myself.

Agreed, I also enjoy going through the line and making small talk with the cashier. Mostly because of remote work post-COVID. We have delivery this year with a newborn in the house, but now that they are almost 1 I don't think I am going to renew.

Up thread I just made a comment about how for some things I see talking verbally to an AI agent as the lowest friction solution, but I think I'll always be a physical grocery store browsing guy. I got groceries delivered a few times, years ago when my wife and I had covid at the same time. The experience was worse in every way for me than going down the aisles and putting things in my cart.

yeah, unless you’re doing with food what steve jobs did with sweaters i don’t see how someone can say they save time with groceries.

i can understand it with the “i’m gonna optimize everything to death crowd” probably eat the same thing every week. i couldn’t do it though, i get bored with the same food. its also why i can’t wear the same sweater every day lol.

An AI bot can jump straight to the thing you want, like "How much did I spend at McDonalds over the last year?"

It's worth considering whether you really want the full list of MCD transactions or is that just a step you're used to as a means to getting the answer you want.

So can grep

I had a conversation the other day with some who told me that one very useful thing with AI now is that you can use it to find hotels in a city. My obvious reaction was that this has been a one or two button press thing on google maps for a very long time. They replied well yeah but now with AI you can tell it to only find rooms that don't go over a certain price.

I think a good chunk of people just do not ever try to understand what buttons do, ever, and it's maddening.

[delayed]

Cognitive load. To get something out of a computer pre-ai you needed to enter precise command. Sometimes too precise - looking at you tar.

Here you can phrase it anyway you like and even typo at writing mcdonalds won't stop the machine from giving you the right answer.

i was thinking about this the other day. so many things are just easier analog style.

i think we fell into a bit of mass cult.

obviously some are easier digital, but it absolutely is not always the case.

concert tickets was the one that hit me the other day. someone had given me physical tickets to a show. i just pulled them out of my pocket, they scanned them and … that was it. i hadn’t done an analog concert ticket since pre-covid times so i kinda forgot how easy it can be. no trying to remember which app i used or which password or searching through email… just pulled them out and that was it. and bonus points i have an actual ticket stub.