- building outputs for price fairness (based on publicly available labor data)
- scope match (is vendor over/under scoping user's intent)
- risk (vendor risk, timeline, price variability, etc.)
- value (some combination of price, service, longevity, etc.)
I don't get much hallucinations in my testing, but overall it's pretty complex pipeline since it is broken down into so many steps.
Take price fairness, for example. i feel like the human part is core to this work. it ultimately all comes down to a test of reasonableness. A wide brush for costs is sometimes used because it keeps things trackable by the humans involved. An AI is able to generate an amount of work thats unreasonable to verify. At the end of the day, pricing is a negotiation not a logic puzzle.
If it does work though, i think it could open a huge door for Cost Plus Fixed Fee contracts which seem like the fairest contract type but often come with too much burden of paper work compared to the more popular firm fixed price option
I had a really complex negotiation for car repairs (goodwill warranty, balancing a long list of repairs/recalls etc) which was pretty time sensitive. If I had already had my service record in a structured format along with the manufacturer's policies I feel like I could have responded with better preparation. Same for any other big maintenance items on the house, mortgage, insurance, etc.
And then there's the flip side--what do my policies and healthcare/loyalty plans cover that I'm not taking advantage of? What can be combined towards my goals etc.
I do have the concept of an "asset" which could be a car, house, etc. and with enough basic info it's pretty easy for the LLM to cross reference common problems, or at least suggest questions that you should follow up on.
I'm leaving intent pretty free-form for now, the most friction I'm willing to add is 2 things:
- Basic enum preferences around budget and flexibility to help with prompting
- A claude code style "a few questions" follow up
Any additional form friction I think gets too complex.
It's funny, a lot of my research has been from subreddits for auto, homeownership, questions for people who work in trades, etc. Every time someone asks "is this quote fair", the response from the experts is almost always "But what do you want"
So in a time-sensitive repairs scenario, intent could "What get's my car safe to drive again for daily commute.... or for a long roadtrip". The output analysis could recommend which fixes are highest priority, where work could be split up, delayed etc.
On top of it all, the most important thing to consider is intent -> An emergency plumbing visit is often very different than a proactive upgrade.
edit: spelling