This isn’t anything like the trolley problem. And yes, taking actions has consequences intended or otherwise. That’s not the trolley problem either
Gene Roddenbury knew this, and it's kinda why the original Trek was so entertaining. The juxtaposition of super-technology and interpersonal conflict was a lot more novel in the 60s than it is in a post-internet world, and therefore used to be easier to understand as a literary device. To a modern audience, a Tricorder is indistinguishable from an iPhone; the fancy "hailing channel" is indistinct from Skype or Facetime.
https://www.telegram.com/story/news/local/north/2010/06/22/8...
"Ms. Sackett, with the aid of film clips, said that "The Return of the Archons," from the original series, was a good example of how Mr. Roddenberry employed elements of humanism in his works.
In that episode, a planet's population follows, in a zombie-like manner, a mysterious cult-like leader, who allows no divergent viewpoints.
The society absorbs individuals into its collective body and the world is free of hate, conflict and crime but creativity, freedom and individualism are stifled.
Ms. Sackett said that "Archons," like other Star Trek storylines, warns how people can be controlled by religion. In the end, the viewer discovers the cult leader is actually a computer."
"[N]o divergent viewpoints" sounds like Stackoverflow and forums run by software developers in general. The behaviour of "developers" can be extremelly cult-like.
Creativity, i.e., new work that is not comprised of a recombination of old work, does not seem compatible with "AI". The later relies on patterns found in old work.
There were several original Star Trek episodes that explored this scenario. Not plausible. Actual.
"So how bad could it really get"
Watch Rodenberry's orginal Star Trek to get some ideas.