smiths1999

Answered elsewhere but the tl;dr is lock the db file. The design is multiple readers, single writer.

Why not use something else? I wanted to build something from scratch.

What you've built with ArcadeDB is definitely impressive, but much larger scale that what I was trying to accomplish with this. Like the repo says, think sqlite but for graph dbs :)

Not comparable. Similar to Neo4j vs LatticeDB. Lattice's goals are simplicity, single file, super easy to start using. Surreal supports embedded mode, but my understanding is it is not a single file. Surreal supports a huge number of data models, Lattice is more focused. Surreal also appears to have a company behind it. Lattice is just me.

If you are looking for something lightweight and simple, choose Lattice. If you are an Enterprise looking for hosting, Enterprise support, and maybe a database that can handle all of your needs instead of a specific one, pick SurrealDB.

I hadn't heard of SurrealDB until your comment though, so take what I have to say here with a grain of salt.

Just added file locking to avoid concurrent writers across processes in the latest release.

Thanks - appreciate the feedback! I wanted the README to be something you could quickly glance at, get a sense of the project and whether it's what you are looking for, and be done with it.

Thanks for sharing this. The lattice numbers are pretty close but my sqlite numbers are way off. I got a new computer since then so will remeasure.

I started thinking more deeply about how I would actually do this today and it won't be as straight forward as I initially thought. Will keep it on the radar and see if I can figure out a clean way to implement. Great suggestions!

I am in the final stages of adding this based on your comment. Just wrapping up doc updates and will push a new release with hot copy functionality tonight!

Within the same process it's enforced at the db level. After reviewing the code more I see we don't have a mechanism in place to manage safety across processes. Will add a file lock mechanism tonight. Great call out!

In general though, the goal for this was single writer multiple readers. That was a design decision to keep things simple.

The permissions question is interesting. I think the answer depends on context. One approach would be to create some edge types `hasAccessTo` and `accessibleBy` that connect a user to a node. Then I'd create an edge type `childOf`. The rest is business logic. Permission checks can just traverse up to the first root node with permissions. The downside is this is all business logic, so can't really look at the database and understand this is how it works. Depending on the database you could create a function that returns permissions for any node, that encapsulates this logic.

Anyways, thanks for checking it out! Really appreciate it. Good luck with your project!

very cool site!

Great question! Two reasons. One, I wanted to build something on my own from the ground up rather than contribute to an already established large project. For me, it's a better way to learn. Hopefully people find it cool and want to use it. But if the best that happens is it's just a fun project I built then that is just fine for me. Secondly, the data layouts are different. They are very similar in the single-file, graph db respect. But lattice is transactional and row oriented while kuzu is columnar.

Yes! This is something I've been thinking about quite a bit the past few weeks. We are going in this direction at work and I think graph storage is a natural way to think about this.

I used claude extensively (as well as codex, I finding myself switching between the two every few months). How I used claude varied by the stage. I spent a lot of time initially going back and forth with claude on the idea, figuring out what exists, what would make this interesting, key features I wanted as a user and how to build something around that that made sense as a product.

The initial phases of building I would build out piece by piece. For example, building out the file system interactions I would have claude build a feature and explain how it worked in an educational manner (e.g., like it was a section in a book on latticedb internals). I would then read through the code. This was a great way to learn and build, simultaneously.

In the later stages, where the features and work was more complex, I would spend more time discussing, instructing, and verifying, but less time understanding the actual implementation. I'll give you an example. It's been a long time since I've handwritten SIMD code. I could try and review claudes output, but I am certain I'd miss any subtle bugs that may exist. I found it more productive to assume the code was right and focus on thinking about how I would verify that. Benchmarking, playing with latticedb, etc. were my primary tools for verifying the work. I could run a benchmark and see performance was great. Then I'd explore the test vectors and realize they were trivial, completely invalidating the benchmark results. So we would go back to the drawing board, create a new benchmark set, see results weren't great, and evaluate what was wrong with the implementation. Sometimes features would take days to get out just because of the iteration loop.

Thank you!

duckpgq is great. I'd say the tl;dr is duckpgq if you have tables you want to traverse like a graph sometimes, latticedb when graph traversal is the primary mechanism of querying.

One of the motivating use cases for me was experimenting with agentic memory. I use latticedb as the backing data store. Finding related memories is traversing the graph (kind of like graph RAG).

I like the aesthetics of this page. Very clean and visually appealing

I did some perf benchmarking with 1M nodes but I'm mostly using it at smaller scales for another project exploring agentic memory.

Most interesting part is a tough one. From a learning perspective the beginning was incredibly interesting because I was spending a lot of time learning about how other DBs work. Even something as relatively simple as writing to disk had a lot more complexity to it than I initially anticipated.

I used LLMs extensively in building this, and the other interesting part was seeing how they failed. I've always been a proponent that tests are no guarantee of quality code, and working with LLMs has only reinforced it. They often write superficial tests. Sometimes a suite of tests would pass, but when I would actually play around with the feature it was clearly broken. LLMs certainly enabled me to build something of this scope, but it was far from "build a graph DB and notify me when you are done"

If the company does any kind of work with the US Government it's far easier to just say no to Chinese-origin models rather than deal with all the overhead on the government contracts. Since just about every major tech company has US contracts they all have to be careful about the use of Chinese models.

I apologize if this is a stupid question but why wouldn't everyone be investing in India with these returns being standard? In the US we would assume a conservative rate of 7%

I don't follow. Aren't employee salaries a deductible expense? The above makes me think more along the lines of no longer being able to deduct GPU costs incurred from deep learning research.

I would suggest coming up with a set plan for meals and eat the same thing every day for two out of three meals. For me, during the work week, I have a bowl of oatmeal (steel cut oats, nuts, and a banana) for breakfast, a salad with lots of veggies for lunch, and then dinner varies. Once you get this stable, you can try doing things like occasionally eating once a day (but eating a much larger, _healthy_ meal).

I would absolutely not use a tank of pure oxygen.

This has been my experience as well. The first time I tried it I thought I was going to be miserable all day and have no energy. It ended being totally fine. I have this dull emptiness in my stomach but nothing that affects me in any way. And you are right, as soon as dinner rolls around my hunger increases dramatically.

Zettlekasten is more about making connections between ideas and keeping your ideas organized. The primary use is for generating ideas and writing, not for information retrieval.

The flow is: 1. Read something and take notes 2. Review and distill your notes into key ideas 3. Create cards in your zettelkasten linking these ideas with other ideas already in your zettelkasten

When you are writing and get stuck, refer to the topic in your zettelkasten and follow the flow of ideas.

AWS going down means I've lost it or temporarily lost access to it? Those are two very different scenarios. Of course S3 could lose data - a quick Google search shows it has happened to at least one account. My guess is it is rare enough that it seems like a reasonable decision to not prioritize backing up your S3 data. I'm not syaing "never ever backup S3 data" only that it seems reasonable to argue it's not the most important thing our team should be working on at this moment.

I have my family photos on a RAIDed NAS. It took me years to get that setup simply because there were higher priority things in my life. I never once thought "ahh I don't need backups of our data" I just had more important things to do.

I suppose the caveat is you have to have customers to lose them :) We don't know what the data is or the size of the company.

Sure you can. As the experienced engineer in that setting it is a great opportunity to teach the less experienced engineers. For example, "I have seen data loss on S3 at my last job. If X, Y, or Z happen then we will lose data. Is this data we can lose? And actually, it is pretty easy to replicate - I think we could get this done in a day or two."

It's also possible the response was "That's an excellent point! I think we should put that on the backlog. Since this data is already a backup of our DB data, I think we should focus on getting the feature out rather than replicating to GCP."

Those are two plausible conversations. Instead, what we have is "these arrogant 25 year olds that have 1-2 years of experience and know it all." That's a red flag to me.

Yes - the point of that line was to be ridiculous. Age has nothing to do with it. Anyone at any age can have good ideas and bad ideas. There are some really incredibly _older_ and highly experienced engineers out there. But there are others that think that experience means they are never wrong. Age has nothing to do with this - what is important is your past experience, your understanding of the problem and then context of the problem, and how you work with your team.

And again, my point isn't that you never need backups. My point is that it is entirely plausible that at that point in time backups from S3 weren't a priority.

I'm not sure how you got "backups are for old people" from my post. My point is that there are two sides to this. Perhaps the data being stored on S3 data _was_ backup data and this engineer was proposing replicating the backup data to GCP. That's probably not the highest priority for most companies. Maybe the OP was right and the other engineers were wrong. Who knows.

In my experience, the kind of person that argues about "arrogant 25 year olds that know everything" is the kind of person that only sees their side of a discussion and refuses to understand the whole context. Maybe OP was in the right, maybe they weren't. But the fact that they are focusing on age and making ad hominem attacks is a red flag in my book.