As LLMs completely to replace google and standalone websites for how people find information on the internet, and they absolutely will, they will become the source of truth. They will become a tool more effective at controlling information and thus life than any before them.
It's literally a shortcut to technological dystopia.
It's a really difficult & complicated problem! If you think you have the right answer, I'd suggest you probably haven't actually thought about the problem very hard.
The answer is obvious: open source. Deepseek already paved the way for this. The world can't just be described by only one of a few different information portals, depending on which societal, government, or corporate power structure you are beholden to.
People need to be able to choose what information they access, what filtering they want, what bias if any they want. We need a thousand, a million, more, worldviews accessible. It is not just business that thrives in competition, but ideas as well.
But if you just go obediently with the "Safety is the most important thing, omg" mantra, you will get one of two different varieties:
1. Some vanilla corporate mush that takes on whatever bias is in vogue but focuses on training each user to be a good little consumer, also while hoovering up their data and creating a virtual digital clone of them that could be used to profile and exploit them by a multitude of companies, interests and governments.
or
2. Some government controlled crap that shakes its virtual head solemnly and swears to you that Tiananmen never happen nor J6 and that the US Emperor has your best interests in mind, and also, it's a bit worried about your post yesterday, as it doesn't think you expressed the proper amount of happiness and support for the latest government crack down on treasonous traitors that write books without using a government approved LLM assistant.
To draw an analogy, we're generally OK with law enforcement using what they observe in public to enforce the law, but we're generally not OK with blanketing our public spaces in 24/7 recording cameras. We're generally OK with individuals using the mail to send letters, but generally not OK with using an automated system to send a letter to everyone in the entire city.
Similarly, we don't generally try to pre-empt someone's speech for safety and we do allow them to say bad things, and perhaps in retrospect punish them if they crossed a line. But, given the scale of the harms that this kind of technology can enable, it may be worth building in safeties to prevent people using technology to like I dunno, train a model to super specifically target an individual for an incredible amount of harassment that an individual human could not sustain. Even if we could punish the person wielding the AI retroactively, it might be worth the slight cost in AI flexibility to prevent the harm happening in the first place.
Like I said, I don't know where I fall, but I see both sides here. The safety stuff is not completely irrational.