News for hackers. Stuff that matters.
Re:The “Build Your Own Database” book is finished by mgaunard on Saturday April 22, 2023 @12:36PM (#35668294)
1. Memory map a file

2. Manage access to it through a server

There you go, you know have a database.

Re:The “Build Your Own Database” book is finished by orf on Saturday April 22, 2023 @12:46PM (#35668415)
1. Draw some circles

2. Draw the rest of the owl

https://www.pillarsofimpact.com/post/draw-the-owl

Re:The “Build Your Own Database” book is finished by mmphosis on Saturday April 22, 2023 @12:46PM (#35668418)
Good to now.
Re:The “Build Your Own Database” book is finished by remram on Saturday April 22, 2023 @12:46PM (#35668419)
> Manage access to it through a server

This has real "draw the rest of the owl" energy.

How to make a DBMS? 1. Get a file 2. Make a DBMS

Re:The “Build Your Own Database” book is finished by jandrewrogers on Saturday April 22, 2023 @12:52PM (#35668476)
Memory mapping is a poor way to make a database[0], not all storage can be memory mapped, and this doesn't provide any of the functionality you expect of a database like indexing and concurrency control.

[0] https://db.cs.cmu.edu/mmap-cidr2022/

Re:The “Build Your Own Database” book is finished by jfrisby on Sunday April 23, 2023 @01:22AM (#35673520)
I saw a headline about the book on Reddit and made a joke to a friend of mine: "How much you wanna bet it uses mmap?" Well...

It uses mmap.

Re:The “Build Your Own Database” book is finished by mgaunard on Sunday April 23, 2023 @05:03AM (#35674611)
The concurrency control is managed by the server that manages access.

All databases that are any different than these two points are just bad.

Re:The “Build Your Own Database” book is finished by mkl95 on Saturday April 22, 2023 @01:18PM (#35668809)
1. Build your own database

There you go!

Re:The “Build Your Own Database” book is finished by LAC-Tech on Saturday April 22, 2023 @04:46PM (#35670769)
You seem like a real go-getter who can cut through red tape and find simple solutions to complex problems :D
Re:The “Build Your Own Database” book is finished by mkl95 on Saturday April 22, 2023 @05:05PM (#35670926)
The power of abstraction!
Re:The “Build Your Own Database” book is finished by icedchai on Saturday April 22, 2023 @01:53PM (#35669186)
I worked on a commercial product that did just this, except it also had a "transaction log" component. All access through the server was logged, and the log could be used for replay/recovery. While it worked for them for thousands of simultaneous clients, it was not a general purpose solution. There were no real indexes: the "indexes" (hash tables, really) were built in memory at startup.
Re:The “Build Your Own Database” book is finished by akamaozu on Sunday April 23, 2023 @06:11AM (#35674925)
i don't understand what you mean by no real indexes

why does generating indexes at start-up not count as having indexes?

(asking because i do this all the time)

Re:The “Build Your Own Database” book is finished by icedchai on Sunday April 23, 2023 @10:30AM (#35676540)
I suppose I should've said "persistent" indexes. There was also only a single type supported: hash table. And it was hard coded to 2 fields. You had to change the code (which was C) to change the database.

This system was rarely restarted, so in practice it didn't matter what it did at startup, as long as it didn't take more than a few minutes, but it did place some limitations on data size. (This was a 32-bit system and everything was memory mapped.)

Re:The “Build Your Own Database” book is finished by mamcx on Saturday April 22, 2023 @04:22PM (#35670565)
> 1. Memory map a file

Complex when you want to

> 2. Manage access to it through a server

A database is only "easy" IF:

- Append only

- No real "delete" or "updates" just to reiterate the above.

- Only Sequential scan

- Only need simple iterator-per-row

- No maintain secondary stuff like indexes, so not need to coordinate changes

- No concurrency

- Fit in RAM, and I mean in few MB

- No need to deal with SQL, use his own DSL (sql is so bad! so much weird stuff!, but is ok to have something sql-ish like LINQ)

- No need to deal with recursive data types, only scalars

- Is only embebed

- No need auth or security validations

Ok, after making this list, I sure forgot some other tips to make this easy!

Re:The “Build Your Own Database” book is finished by mgaunard on Sunday April 23, 2023 @05:05AM (#35674623)
There is no requirement to have the file fit in RAM to mmap it, that's the point.

Obviously you'd have a file per column (or index), and use directories to represent tables.

This is the correct way of doing it and yet so few databases do it.

Re:The “Build Your Own Database” book is finished by mamcx on Monday April 24, 2023 @11:29PM (#35696220)
Yes, I understood, but "mmap a file" that should be a database is not "easy", because now you can deal with segfaults and other weird things that are out the normal playbook.

Of course, depending on the other things of the list this is or not a major issue. Is more about how combining several ideas leads to a easy or complex implementation.

Re:The “Build Your Own Database” book is finished by preseinger on Wednesday May 03, 2023 @06:16PM (#35808539)
"correct" is not an objective measure, it's a function of use case

mmap is not a panacea, it improves specific access patterns by incurring specific costs, it's definitely not true that mmap is the right choice for all databases

Re:The “Build Your Own Database” book is finished by akamaozu on Sunday April 23, 2023 @06:07AM (#35674910)
why is this so heavily down-voted?

it feels correct, even though it's not a complete guide

(reliably persisting changes to disk is a big part of what dbs do, but is missing here)