News for hackers. Stuff that matters.
Re:The Untold Story of SQLite by adamgordonbell on Friday July 02, 2021 @11:09PM (#27719052)
Host Here. Thanks for sharing.

Richard is a great story teller and adventures that SQLite has taken him on pretty interesting. I think he in on hn as well ( @SQLite).

One interesting thing I found about how SQLite is developed is the amount of testing that is done. Between unit tests and parameterized tests and fuzzer tests they are doing billions of tests per release and spending days running them before each release. I know I don't like to wait for a long CI build before I deploy a service but something like SQLite needs whole different category of resilience.

Re:The Untold Story of SQLite by bsaul on Saturday July 03, 2021 @06:37AM (#27720643)
That part really made me laugh. A few months ago, i developed a wrapper in swift for sqlite on a project. A coworker of mine had a bug on what sqlite returned, and i couldn't find anything wrong. He told me "it has to be a bug on sqlite", and i said "no, sqlite has no bugs… but i don't really understand what's going on". And so we posted a bug report on the sqlite forums. I told my friend "if you found a bug on sqlite, on your first try, i'll buy you a bottle of champaign".

Of course, it was a bug with my wrapper (included a \0 char at the end of strings).

I just forwarded him the link to your podcast, saying "we really had no chance"

Re:The Untold Story of SQLite by thechao on Saturday July 03, 2021 @08:42AM (#27721241)
SQLite is a great way to find bugs in your fuzzer: if your fuzzer finds bug, it’s time to debug your fuzzer!
Re:The Untold Story of SQLite by joeyh on Saturday July 03, 2021 @10:39AM (#27721883)
I enjoyed this episode. I knew about all the tests before, but what came clear in this interview is that Sqlite inc is using their tests as a unique way to keep contracts on Sqlite development flowing. Since they can test it so much better than anyone else, it would be crazy not to hire them, rather than hiring someone else to develop a patch.

While they're keeping those tests proprietary, and relatedly have a closed development model, this strikes me as still better than typical open core strategies for monetizing free software, or at least pointing in better directions.

Agbell, since you're here, could I ask you add a standard link to the RSS feed for your podcast?

<link rel="alternate" title="Corecursive" type="application/rss+xml" href="https://link.chtbl.com/corecursive?platform=rss" />

Re:The Untold Story of SQLite by zinekeller on Saturday July 03, 2021 @11:45AM (#27722377)
> Sqlite inc

Minor nitpick: technically, the primary company that supports SQLite is an incorporated partnership named Hipp [ed note: yes, that's Mr. Richard Hipp], Wyrick [ed note: Hipp's wife] & Company, Inc. dba Hwaci (https://www.hwaci.com/).

Re:The Untold Story of SQLite by adamgordonbell on Saturday July 03, 2021 @04:39PM (#27724709)
Will do!
Re:The Untold Story of SQLite by adamgordonbell on Monday July 05, 2021 @07:59AM (#27737034)
Done, joeyh.
Re:The Untold Story of SQLite by macintux on Friday July 02, 2021 @11:12PM (#27719064)
Thanks, that really is a great conversation.

I'm going to have to dig to see if he describes somewhere why Fossil is such a great fit for SQLite.

Re:The Untold Story of SQLite by bch on Saturday July 03, 2021 @01:19AM (#27719545)
Couple things:

* immutable

* built in first-class ticketing system

Happens to use SQLite as its storage engine too (SQLInception?), which brings at least a couple (off the top of my head) features:

* SQL is used to query things, which is very powerful

* the repository is a single SQLite file. Easy to store, copy, and…

* one can checkout multiple working copies into various directories. Or, in other words, I can have three discrete, independent workflows happening simultaneously, but only one copy of the repository backing them all.

It’s also got a built-in server that you can use locally or publish for remote people to access via their web browser that’s essentially GitHub-in-a-box.

Also… (kidding. I’m just going to stop here).

Edit: formatting

Re:The Untold Story of SQLite by krylon on Saturday July 03, 2021 @04:55AM (#27720289)
I've migrated to git for my personal projects, mainly because magit (emacs frontend for git) is awesome.

But fossil is a pleasure to work with, especially on Windows, because a) it comes as a single executable that one can just copy to another machine, and b) it does not require ssh or anything else to be installed.

The web interface is amazing, especially considering the size of the executable file.

Re:The Untold Story of SQLite by severak_cz on Saturday July 03, 2021 @09:44AM (#27721581)
Fossil have much better command line IMHO. I even started git wrapper to get rid of it's cryptic syntax.

[1]: https://github.com/severak/ginger

Re:The Untold Story of SQLite by Cyph0n on Saturday July 03, 2021 @01:42AM (#27719641)
Regarding the multiple working copies feature, sounds the same as Git worktrees.
Re:The Untold Story of SQLite by bch on Saturday July 03, 2021 @02:00AM (#27719704)
I took a quick scan and: it might be analogous, yes. Not sure if any subtlties apply.

The git workflow in the docs (https://git-scm.com/docs/git-worktree) looks really clunky.

Here’s what I do in fossil:

$ cd ~/work/project_x

$ fossil open ../fossils/projx.fsl (Edit, edit, edit)

$ fossil ci -m ‘descriptive msg’

(Get an idea, but for whatever reason do not want to disturb this directory)

$ mkdir ~/work/radical_idea

$ cd ~/work/radical_idea

$ fossil open ../fossils/projx.fsl <*note here there is no sense of this being an inferior or “sub” checkout*> (Edit, edit, edit)

$fossil ci -m “cool idea” —branch cool_idea # commit work to new branch

(Test..)

$ fossil co trunk #switch branch back to trunk

$ fossil merge cool_idea #merges our work from cool_idea branch into trunk branch

$ fossil ci -m ‘merge [cool_idea] for radical new feature’

$ fossil close # this working dir syncs everything necessary to the repo (../fossils/projx.fsl) and is now essentially “offline”

$ cd ..; rm -fr radical_idea

$ cd project_x (Note: this working directory now not a tip of trunk anymore…)

$ fossil update # now up to speed (Continue hacking…)

Re:The Untold Story of SQLite by LukeShu on Saturday July 03, 2021 @05:02AM (#27720317)
It looks to me like the main difference is that with Git you can't have multiple worktrees on the same branch. Apropos of your comment "Note: this working directory now not a tip of trunk anymore…", to Git being on the tip of a branch is a thing of note, so it doesn't ever want to accidentally invalidate another worktree's notes about being on the tip of a branch.

In Git, that'd be:

    $ cd ~/work/fossils/projx.git

    $ git worktree add ~/work/project_x master # add a worktree with the 'master' branch checked out

    $ cd ~/work/project_x
(Edit, edit, edit)

    $ git commit -m 'descriptive msg'
(Get an idea, but for whatever reason do not want to disturb this directory)

    $ git worktree add ~/work/radical_idea -b cool_idea # add a worktree with a new 'cool_idea' branch

    $ cd ~/work/radical_idea
(Edit, edit, edit)

    $ git commit -m "cool idea"
(Test..)

    $ git switch master # whoops, can't do this
    fatal: 'master' is already checked out at '~/work/project_x'

    $ # let's just do the merge in the other worktree, since we're about to switch anyway

    $ cd ~/work/project_x

    $ git worktree remove ~/work/radical_idea # or: rm -rf ~/work/radical_idea && git worktree prune

    $ git merge cool_idea
(Continue hacking…)
Re:The Untold Story of SQLite by masklinn on Saturday July 03, 2021 @05:18AM (#27720375)
> <note here there is no sense of this being an inferior or “sub” checkout>

There is not in worktrees either. You can use worktrees with a bare repository for instance. It's just that for backwards compatibility and the convenience of most users a non-bare clone creates a default worktree. That's exactly what you see if you `git clone` a repository then `git worktree list`, a worktree corresponding to the working copy.

Re:The Untold Story of SQLite by vagrantJin on Saturday July 03, 2021 @05:02AM (#27720315)
Seeing this makes me want to try out fossil now.
Re:The Untold Story of SQLite by smartmic on Saturday July 03, 2021 @03:12AM (#27719909)
Here is a summary of similarities and differences on Git and Fossil functionalities: https://fossil-scm.org/home/doc/trunk/www/gitusers.md

Also quite interesting the general comparison: https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki

Both sites are written from Fossil guys, so they may not be 100% neutral.

Re:The Untold Story of SQLite by radiator on Saturday July 03, 2021 @02:24AM (#27719777)
Very good podcast, thank you. What I personally missed is a question about the future plans for SQLite. Also about version 4.
Re:The Untold Story of SQLite by collaborative on Saturday July 03, 2021 @08:44AM (#27721251)
Really enjoyed this. Is it possible to subscribe using an email address? The subscribe page seems to ask for a bunch of apps (Apple Podcast etc) that I don't use. Would be great to just receive an email and listen on a browser
Re:The Untold Story of SQLite by agbell on Saturday July 03, 2021 @08:52AM (#27721298)
If you scroll to the bottom of the page and sign up for the newsletter, I will email you whenever there is a new episode out.

The newsletter will come out once a month when there is a new episode and has a bit of personal take on the episode and maybe some links to other things I've been up to.

If you just want a pure notification, I'm sure you could hook the RSS feed up to an RSS to email service. Thanks for listening!

Re:The Untold Story of SQLite by not-my-account on Friday July 02, 2021 @11:11PM (#27719063)
Great podcast! I have been following it for a while. Can I ask how ended up starting it?
Re:The Untold Story of SQLite by agbell on Friday July 02, 2021 @11:18PM (#27719094)
CoRecursive started because I did some episodes as a host on software engineering daily and it was lots of fun so I decided to start one of my own. The first 4 episodes were originally on software engineering daily.

Originally the idea was to give some platform to the areas of computing I was interested in, heavy use of types and functional programming but I think I really found my place as a podcast when I started to focus on peoples stories. So rather than tell me about SQLite, it is tell me about the story behind creating SQLite.

Re:The Untold Story of SQLite by iso8859-1 on Friday July 02, 2021 @11:41PM (#27719183)
You are so polite, I am impressed with your ability not to push back on the interviewees. In this episode I would have asked "would sqlite really be worse today if you hadn't done your own VCS.". But I think I wouldn't be half as successful as you, people don't wanna be implicitly criticised by their interviewer :P

You are doing so great, favorite podcast by far!

Re:The Untold Story of SQLite by SQLite on Saturday July 03, 2021 @07:16AM (#27720823)
> would sqlite really be worse today if you hadn't done your own VCS.

Yes. Fossil is not only the VCS for SQLite, Fossil is also built around SQLite. SQLite is a core component of Fossil. Thus, when I am working on Fossil, I am forced to interact with SQLite as a "user" instead of as a "developer". In geek-speak, it forces me to "eat my own dog food". This, in turn, prompts me to add needed features to SQLite and more generally to make the SQLite interfaces friendlier to application-developers.

One recent example: SQLite version 3.34.0 added the ability to include two or more recursive terms in a Recursive Common Table Expression. (See item 2 in https://www.sqlite.org/releaselog/3_34_0.html and subsequent links.) This feature was added specifically so that I could more easily write SQL statements that would walk the Fossil version history DAG, as described by the https://www.sqlite.org/lang_with.html#rcex3 link.

I did not develop Fossil with this "dogfooding" idea in mind. It was an unanticipated benefit of Fossil. But in the end, I think it might have been the most important benefit of using Fossil instead of some other VCS.

Re:The Untold Story of SQLite by agbell on Saturday July 03, 2021 @08:14AM (#27721096)
I'm glad you like the podcast! I do push back when something doesn't sound right. I agree though that I could have asked more about fossil. I really didn't look into fossil that much when I prepared for the interview so it was a bit of a blind spot for me.

Not included in the interview is a discussion we had about types. I really think SQLite should be stricter in enforcement of types and Richard doesn't buy it. I may include it in a future bonus episode but the reason I cut it was it didn't really include any details about story of SQLite, it was just two people disagreeing about types. I've done a lot of episodes about types in the past and I wanted this to be about build SQLite, and not about types.

Re:The Untold Story of SQLite by cstork12 on Sunday July 04, 2021 @11:44AM (#27730514)
I was wondering exactly about this. I, for one, would be highly interested in hearing this discussion!
Re:The Untold Story of SQLite by neolog on Friday July 02, 2021 @11:48PM (#27719206)
I quite disagree. It's boring when they just say their talking points. The guests are experts in their field, they can handle some pushback and it would be much more educational for the rest of us to hear the answers to the questions we're really wondering about.

It doesn't have to be "implicit criticism", but an open mind doesn't blindly accept stuff, ask for justification.

Re:The Untold Story of SQLite by sitkack on Saturday July 03, 2021 @03:24AM (#27719958)
The justification is where the story is, the tension and relief. This is where you really learn from the interview.
Re:The Untold Story of SQLite by candeira on Friday July 02, 2021 @11:48PM (#27719205)
I was wondering why the episodes on the corecursive feed start at number 5.

I'll check your hosted episodes on Software Engineering Daily now!

Re:The Untold Story of SQLite by agbell on Saturday July 03, 2021 @08:19AM (#27721124)
Thanks, I have interviews on software engineering radio as well: https://www.se-radio.net/team/adam-gordon-bell/
Re:The Untold Story of SQLite by sigzero on Friday July 02, 2021 @11:10PM (#27719057)
He is on HN. It was a great segment.
Re:The Untold Story of SQLite by agbell on Friday July 02, 2021 @11:14PM (#27719072)
Thanks for listening! I'm glad you enjoyed it.