The idea of actual entities and relationships defined like triples with some schema and appropriately resolved and linked can be useful for querying and building up the right context. It may even be time to start bringing back some ideas from the schema.org back the day to standardize across agents/assistants what entities and actions are represented in data fed to them.
This doesn’t have anything to do with AGI or brains. They are typically created or tuned by humans and then models fit/match/resolve entities to match the ontology.
To have an LLM construct a knowledge graph of where someone is (and I know this example is incredibly privacy invasive but its a simple concrete example not representative). Imagine giving an LLM access to all of your text messages. You can imagine giving it a prompt along the lines of "identify who is being discussed in this series of messages, if someone indicates where they are physically located report that as well" (you'd want to try harder than that, keeping it simple).
You could get an output that says something like `{"John Adams": "Philadelphia, PA, US"}`. If either the left or right side are missing create them. Then remove any LocatedAt edges for the left side and add one between these two entities. You have a simple knowledge graph.
Seems easy enough, but try to ask slightly harder questions... When did we know John Adams was in Philadelpha? Have they been there before? Where were they before Philadelphia? The ontology I just developed isn't capable of representing that data. You can of course solve these problems and there are common ontological patterns for representing it.
The point is, you kind of need to know the kind of questions you want to ask about your data when you're building your ontology and you're always going to miss something. Usually you find out the unknown questions you want to ask of the data only after you've already built your system and started asking it questions. It's the follow-ups that kill you.
There has been a lot of work on totally unstructured ontologies as well, but you're moving the hard problem elsewhere not solving it. Instead of having high quality data you can't answer every question with, you have arbitrary associations that may mean the same thing and thus any query you make is likely _missing_ relevant data and thus inaccurate.
Huge headache to go down, but honestly I think it is a worthwhile one. Previously if you changed your ontology to answer a new question, a human would have to go through and manually and painstakingly update your data to the new system. This is boring, tedious, easy-to-get-wrong-due-to-inattention kind of work. It's not complex, its not hard, its very easy to double check but it does require an understanding of language. LLMs are VERY capable of doing this kind of work, and likely more accurately.
- The effort needed for KGs has higher general potential value because RAG is useful
- The KG ontology quality-at-scale problem is now solvable by LLMs automating index-time data extraction, ontology design, & integration
This is an area we're actively looking at: Self-determining ontologies for optimizing RAG, such as during evolving events, news, logs, emails, customer conversations. We have active projects across areas here already like for emergency response, cyber security, news mining, etc. If folks are interested here, we're definitely looking for design partners with challenging problems on it. (And, looking to hire a principal cybersecurity researcher/engineer on it, and later in our other areas too!)
A bit more concretely, for the LLM era, we're especially oriented around the move from vanilla RAG chunking to graph RAG, hierarchical RAG, "auto-wiki" style projects, and continuous learning LLMs. Separately, we've been working on neurosymbolic query synthesis for accessing this and mixing in (privacy-aware) continuous learning from teams using it. I think the first public details on this were in my keynote at the 2024 Infosec Jupyterthon, and we'll be sharing more at graphtheplanet.com next week as well. We haven't said as much, but we're also looking at the problem that the data itself isn't to be trusted, e.g., blind men and the elephant reporting different things over time on the news/social media/IT incident tickets.
Right now we're just focusing on building and delivering great tech, customer problem by customer problem. There's a lot to do for a truly good end-to-end analyst/analytics experience!
I think the next frontier is a a wiki-style, collaborative site, deliberately purposed at storing information for LLMs
I think it could work if each entry has to be peer reviewed by 3 (more?) humans. Although, AIs are better than humans at captchas now, so I'm not exactly sure how that would work either way...
https://aclanthology.org/2023.newsum-1.10/
Automatically created knowledge graphs using embeddings is a massively powerful technique and it should start to be exploited.
You can blame peer review for letting the definition of knowledge graph get watered down.
Also note that my coauthor. David, is the author of the first and still the best open source package for creating or working with semantic graphs.
https://github.com/neuml/txtai/blob/master/examples/38_Intro...
Gotta say txtai seems like a useful tool to throw in my toolbox!
Searching through their sources, it looks like the problem came from Neo4j's blog post misclassifying "knowledge augmentation" from a Microsoft research paper with "knowledge graph" (because of course they had to add "graph" to the title).
This approach is fine, and probably useful but its not a knowledge graph in the sense that its structure isn't encoding anything about why or how different entities are actually related. A concrete example in a knowledge graph you might have an entity "Joe" and a separate entity "Paris". Joe is currently located in Paris so would have a typed edge between the two entities of something like "LocatedAt".
I didn't dive into the code but what I inferred from the description and referenced literature, it is instead storing complete responses as "entities" and simply doing RAG style similarity searches to other nodes. It's a graph structured search index for sure but not a knowledge graph by the standard definitions.