flyingsilverfin

Hi all, CTO of TypeDB here. The question I actually wanted to answer was "does a type system help agents with correctness?" Everyone's intuition says yes, especially as complexity grows, but evidence is thin and contested, and I haven't found much that isolates the effect for LLMs beyond constrained-decoding work.

A database is a smaller surface than a programming language, so we turned it into a benchmark that holds the model constant and swaps the language: SQL, Cypher, TypeQL, same questions, same data. It's not a clean apples-to-apples comparison and the post's limitations section outlines that further.

One result that points us in the direction of a strong type system is that SQL wins first-shot, but 93% of TypeQL's wrong queries fail with an error, versus 5% for SQL and 40% for Cypher. Add in a retry loop TypeQL ends up ahead, because it can only fix mistakes it can see.

Of course there's a lot of variables here, such model, skill, token usage, question selection, etc. I actually think it's quite a hard thing to study!

If anyone knows of work on LLMs driving systems with stronger vs weaker verifiers, with the model held fixed, I'd love to see it.

I run the development of TypeDB, which doesnt use Cypher but works really well as a graph database. Certainly it, and other graph databases like neo4j, are used in production at scale. However, a lot of oss databases are open core on some level, it just depends on where they draw the line. We draw it at clustering/high availability for the time being, the rest is in the CE version.

Hi HN, I'm CTO at TypeDB. We shipped an agent mode in our web Studio that lets you describe what you want in plain English and get executable database queries back. The blog calls it "vibe querying," which is a little tongue-in-cheek, but the workflow is what you expect: ask a question, get TypeQL, run it, iterate.

TypeDB's data model has a strict schema, higher-level abstractions (role-based interfaces, subtyping & inheritance), and hypergraph structures (n-ary relations & relations in relations) built in. This combination enables better AI query generation.

It's like giving the LLM a strongly typed language instead of a loosely typed one. With a relational db, the model has to infer relationships from foreign keys and naming conventions. With graph databases, there's no enforced schema. In TypeDB, the schema says "friendship is a relation with two friends, each played by a person" — the LLM gets a constraint space to navigate rather than guess at. It still gets things wrong — the post shows a syntax error it had to recover from — but the error surface is smaller and more correctable.

Would love to hear your thoughts. Happy to answer anything about TypeDB as well!

Just spotted this! We (I'm CTO at TypeDB) just released some early benchmarks: https://typedb.com/blog/first-look-at-typedb-3-benchmarks/

I wanted to jump in here and say that what we're working on at typedb.com, in our 3.0 version (coming soon in alpha!), is that we're taking our earlier database query language and making it much more Programming-like: functions, errors containing stack traces, more sophisticated type inference, queries as streams/pipelines... I think it's super exciting and has a huge horizon for where it could go by meshing more ideas from PL design :)

Incidentally I think it also addresses what a lot of the comments here are talking about: not learning JOINs, indexing, build-in relation cardinality constraints, etc, but that's a separate point!

We've started to formalise TypeQL (TypeDB's query language) using dependent type theory, which fit together very nicely. Interestingly, the formalisation shows us how to consistently and safely extend the language with new structures for higher levels of expressivity, which is a huge benefit of type theoretic formalisms of production languages IMO.

I work on TypeDB (https://vaticle.com/typedb), and it sits somewhere at this intersection. The exposed query language has elements of both logic programming constructs and graph-like structures. Both amount to a kind of "constraint" programming.

I've had these for about 12 years, they don't get mentioned enough!

Downsides were that in the US there were expensive, and need proper daily cleaning, but the tradeoff is worth it - only need 1 pair per year - less waste too!

Going to get Lasik this year because they don't play nicely with travel in developing countries, but they were absolutely fantastic for school and university.

I think that you're correct in your assessment of relational vs graph-like structures: it's closer to our data domains we model and think of, more flexible, etc. We may be seeing something similar in the ML world where things are moving from tabular-dominant data to being able to process graphs more natively. A table is just a very structured graph after all!

SQL is the standard because, as others have pointed out, it's so entrenched and also builds upon a solid theoretical foundation. And given its dominance, it has been optimised and performed extremely well until recently, where data complexity is catching up again.

Recent noSQL databases won't take over SQL because of the lack of schema/typing. They do scale nicely, but aren't as constrainable as SQL, which is a feature (compare building a large software in Python vs Rust or Java) that enforces safety and good abstractions. There are some newer DBs which are combining strict schemas with NoSQL, which is promising!

Disclaimer: I work on TypeDB (vaticle.com/typedb) which is a native ERA (entity-relation-attribute) model with strict typing via the schema.

Haha! I love reading comments like these - check out TypeDB (vaticle.com/typedb)

disclaimer: work there

Same question! As I understand it: CO2 pulled out of the ocean is replenished by atmospheric CO2, because limestone in the ocean dissolves too slowly to make up for the imbalance and it more readily comes in from the air. But if that's true, then the calcium will actually not be replenished quickly in the ocean (not sure what the significance of this is)! If it were true that the calcium is dissolved fast enough to replenish, then there must also be CO2 released from underwater limestone? Which means extracting Ca and CO2 will not remove any atmospheric CO2 really.

Alternatively, we do end up extracting Ca from the ocean that is not replenished (there's probably so much we don't care) and rely on the atmospheric CO2 to correct ph balance of the ocean?

Fantastic, thank you!

Hi yes please that would be amazing! Some of my family is in the US so it wouldn't be too hard to drop by on the way there :)

Hiya - I don't live in a dengue hotspot country but have had it (grandparent lives in a tropical country) and would like to get Dengvaxia to make it a bit less risky to go back... do you know if your country would allow flying in to get the vaccine? May I ask which country it is?

Thanks!!

You're not wrong - the other big difference between datalog and prolog is that datalog is a subset of prolog's functionality that enables forward-chaining to be safe (gets trickier with negation for example). I was mostly referring to the fact that Graql implements datalog semantics/capabilities, plus negation - but the execution strategy is backward-chaining.

We don't do this right now - optimising for incremental changes is actually better done by maintaining forward-chaining inferences (like the OP) rather than backward chaining, and then invalidating and updating just the inferences that have modified dependencies. Which you want depends on use case often: if you have often changing data backward chaining is generally better, but if it's largely static you can get large performance wins by materialising all the inferences, once, via forward-chaining.

The company I work for, Grakn Labs (grakn.ai), builds a database that does datalog + negation type rules! You can add/remove data as you go, and also add/remove rules in the schema over time. The terminology the post here uses is different from ours, but roughly we do "Backward-chaining", which starts from the query to the set of facts that are inserted, which the article calls "top-down".

It's always funny to see people mentioning hypergraphs in relation to knowledge graphs, this is exactly what we do at Grakn Labs (disclaimer: work there) https://grakn.ai

For others: we're also starting to look into ML on knowledge graphs, check out our initial work at https://github.com/graknlabs/kglib :D

If you look into Bazel (build system), you start getting to the point where everything including dependencies, build system, and deployments can be defined as "source" code and ideally should be treated as a first class software

Grakn (Grakn.ai) offers a strongly typed graph database that is open source :) (disclaimer: work there)