I think this is a real problem for these AI tools. If you can’t separate the signal from the noise, it doesn’t provide any real value, like an out of range FM radio station.
Things aren't quite so black and white in reality.
Catching errors/validating work is obviously a different process when they're coming from an AI vs a human, but I don't see how it's fundamentally that different here. If the outputs are heavily cited then that might go someway into being able to more easily catch and correct slip-ups
Previously there was alot of stress/pressure which might or might not have led to sloppy work (some consultants are of a high quality). With this, there will be no stress which will (always?) lead to sloppy work. Perhaps there's an argument for the high quality consultants using the tools to produce accurate and high quality work. There will obviously be a sliding scale here. Time will tell.
I'd wager the end result will be sloppy work, at scale :-)
Same problem I have with code models, honestly. We already have way too much boilerplate and bad code; machines to generate more boilerplate and bad code aren't going to help.
Consultants aren't the ones doing the fact-checking, that falls to the client, who ironically tend to assume the consultants did it.
Lots of kids don’t want to do homework. That said, previously many would because there wasn’t another choice. But now they can just ask ChatGPT for the answers they’ll write that down verbatim with zero learning taking place.
Caring isn’t a binary thing or works in isolation.
Sure, but if you're a professional you have to care about your reputation. Presenting hallucinated cases from ChatGPT didn't go very well for that lawyer: https://www.nytimes.com/2023/05/27/nyregion/avianca-airline-...
Doing a proper fact check is as much work as doing the entire research by hand, and therefore, this system is useless to anyone who cares about the result being correct.
> I don't see why this changes.
And because of the above this system should not exist.
Same thing as misinformation. A sufficient quantitative difference becomes a qualitative difference at some point.
The majority of consultant reports are complete rubbish. Low signal-to-noise, quantity-over-quality, low accuracy slop.
LLMs are innoculating people to this kind of low information value content.
People who produce LLM quality output, are now being accused of using LLMs, and can no longer pretend to be adding value.
The result of this is going to be higher quality expectations from consultants and a shaking out of people who produce word vommit rather than accurate, insightful, contextually relevent information.
I suspect this can be done simply by poisoning a query with supplemental suggestions of sources to use in a RAG, many of which don't even have to be publicly available but are made accessible to the LLM (perhaps by submitting hidden URLs that mislead the summary along with the query).
But even after such a practice is uncovered and roundly maligned, that won't stop the infinite supply of net con men from continuing to inject their poisons into the background that drives deep research, so long as the LLM maker doesn't actively oppose this practice actively and publicly -- which none of them have been willing to do with any other LLM operational details so far.
In fact, I predict that if a LLM summary like DR's does NOT soon provide references to the sources of the facts it relies on, in no time users will disregard such summaries to be yet more uselessly unreliable pfaff from yet another net disreputable -- as we do with search engine summaries now.
Oh Sweet summer child.
------
Hey, I've noticed a few things in your style that are both strengths and opportunities for improvement:
Strengths:
- You clearly have deep knowledge and back up your points with solid data and examples.
- Your confidence and detailed analysis make your arguments compelling.
Opportunities:
- At times, your tone can feel a bit combative, which might shut down conversation.
- Focusing on critiquing ideas rather than questioning someone's honesty can help keep the discussion constructive.
- A clearer structure in longer posts could make your points even more accessible.
Overall, your passion and expertise shine through—tweaking the tone a bit might help foster even more productive debates.
------
Just reply here if you want the full 500+ words analysis that goes into more detail.
Management Consulting was bad enough before the ability to mass produce these graphs and stats on a whim. At least there was some understanding behind the scenes of where the numbers came from, and sources would/could be provided.
The more powerful these tools become, the more prevelant this effect of seepage will become.