It's a really interesting, and difficult, problem. Progress is slow but still visible: we really are "slouching toward Xanadu", as in https://subconscious.substack.com/p/block-reference-mechanis....

I think Google, GPT-3, co-pilot, simple tricks like a 'wordcloud' and even regurgitation/review communities like HN, reddit and good reads show that the presentation format of the information in a given system can be transformed to many diverse representations.

Books and movies have long made plots off new ways to help humans learn things and form connections by either 1) changing the presentation format or 2) downloading all the information into the human brain to let it do the sorting. For example, overlying data on a map in 3d using glasses or wiring a human brain into a computer's serial ports.

The main problem moving forward is making the data available. I have a fear that in the future only governments and large companies in bed with governments will have access to this data and use it in unethical ways.

Meanwhile, startups and individuals will be bared from creating 'unlicensed' presentations or accessing the underlying data itself (i.e. running a search engine)

>I have a fear that in the future only governments and large companies in bed with governments will have access to this data and use it in unethical ways.

I'd look back to the Memex for the solution to that problem, it stored everything locally, thus links couldn't be broken. Output included all the linked items. It's possible to build a system that stores everything you've seen on the web locally, to prevent things from disappearing.

I think local storage is highly undervalued, and the pendulum will shift back.

>overlying data on a map in 3d using glasses

Yes. The opportunity to increase our functional representations of 2D knowledge bases is going to be a game changer that will compliment the Cambrian Explosion currently underway with the PKM/TfT use of symantec Graph connections & AI LLMs. Some efforts ongoing towards this [1] already and imho there is so much you can do to interact with data once you open up the presentation model to spatial computing inside a 3D embodied user interface like VR today, Passthrough AR tomorrow, and full AR a little down the line.

[1] https://substack.soft.space/p/softspacear-prototype04

Did you suggest running gpt3 on someone's email?

Autoencode it into the latent space of the collective conceptual understanding. (See OpenAI’s latest paper on concepts arising due to “multimodal neurons” [1]). Then use all of hacker news to “fine tune” GPT-x. That will preserve the knowledge in a highly flexible manner that can be queried at will to generate any representation you like.

[1] https://openai.com/blog/multimodal-neurons/

> The main limitation here is there's only one possible tree, and it's the privileged view.

Mainly because inferring a different "view" over gigabytes or even terabytes of data (current collections of publicly available "general purpose" datasets can run in the terabytes) is really, really hard and computationally expensive.

And our current data models don't even include satisfactory ways of representing the kind of "common sense" data that would be reflected in a HN comment, an e-mail thread or really any kind of textual source. You can kinda get there by marking "named entities" of sorts but that's the tip of the iceberg, all the structure and semantics is still missing. It's something that's very much being worked on, and has been for decades (with "frame structures" and the like). But it's still hard.

There is more knowledge in your local library. But it requires discipline and time to process. Generating discipline and time at large scale is non trivial.