News for hackers. Stuff that matters.
Re:Writing System Software [video] by antirez on Thursday March 15, 2018 @12:48PM (#16594426)
In case you are interested in providing feedbacks about the next episodes, my current list of potential ideas is:

1. Code snippets: show small self contained functions from random places inside Redis.

2. Redis Sentinel: the first design, the immediate redesign of v2, and how it is internally implemented.

3. The use of goto in Redis.

4. Redis protocol parsing and zero-copying of objects.

5. The odd Redis persistence model: fork() and fighting the copy-on-write.

6. Redis crash report. Generation and usefulness over the years.

7. Observability inside Redis: latency hooks, LATENCY DOCTOR and MEMORY DOCTOR.

8. Hybrid data structures: Streams macro nodes implementation.

9. RIO: Redis I/O abstraction and OOP in C.

10. Redis Cluster: how the Redis message bus is implemented.

11. Redis Cluster: hash slots tracking and redirections.

If you have any preference and/or other ideas, will be cool to hear.

About this experience itself, good things: shooting a video takes more or less just the time to record. Preparation is just a few minutes, and later post processing is an automatic process... So in 1 hour max everything is done. However I see that the reach of the videos is very low compared to a blog post, which requires more time to write sometimes, at least to have the same amount of info. So it's a tradeoff. Also consuming the video may be a problem for many, for instance for language barriers due to the speaker accent (in my case) and the listener ability to understand English without subtitles. Let's say I want to make 10 episodes and re-evaluate if it's a good idea to change style and do 5 minutes episodes instead.

Re:Writing System Software [video] by anselmio on Thursday March 15, 2018 @03:34PM (#16595882)
> language barriers due to the speaker accent (in my case) and the listener ability to understand English without subtitles.

It's usually easier for a non-native English speaker (at least below ILR[1] Level 3 or CEFR[2] Level B) to understand English spoken by another non-native, because the pace is slower, idioms and expressions are not used so regularly, etc. Your accent is much easier for non-natives than a strong Irish accent, for example. So I wouldn't worry about that too much.

[1] https://en.wikipedia.org/wiki/ILR_scale [2] https://en.wikipedia.org/wiki/Common_European_Framework_of_R...

Re:Writing System Software [video] by pfp on Thursday March 15, 2018 @09:44PM (#16597987)
Fully agreed. I tend to find non-native English speakers' accents more understandable than many native British or American accents, ironic as it is. Salvatore's accent is quite easy on the ears.

Same goes for a tendency of many native speakers to indulge in idioms, archaic vocabulary and convoluted syntax just for the heck of it. Non-natives just cut to the chase, which is exactly what you want in a technical discussion.

Re:Writing System Software [video] by antirez on Thursday March 15, 2018 @05:50PM (#16596803)
Cool, makes sense, thank you. I just want to talk slower in the next videos, so that it will be easier... at the cost to enlarge the duration a little bit.
Re:Writing System Software [video] by mdn0420 on Thursday March 15, 2018 @10:37PM (#16598162)
Playback speed is adjustable so I wouldn't stress too much about it.
Re:Writing System Software [video] by db8787cxg87cx on Friday March 16, 2018 @06:28AM (#16599713)
Don't worry about the language. You are very easy to understand and follow. Great work!!! Also I enjoyed seeing you in video. Best regards A random fan :-)
Re:Writing System Software [video] by llimllib on Thursday March 15, 2018 @02:06PM (#16595154)
What's interesting to me about the videos vs. the blog posts (I like both) is that only the video shows me the way you work. I'd love to see a video where you recreate finding a bug, how you worked to go from issue to repro to fix, just to see more of how you work.
Re:Writing System Software [video] by antirez on Thursday March 15, 2018 @05:50PM (#16596811)
Yes, this is definitely the power of the video, to explain the same in a blog post would be like a chapter, showing in this case is simpler. I'll consider doing bug hunting recording the whole session, and then editing accordingly (because it takes hours or days... for the interesting bugs).
Re:Writing System Software [video] by llimllib on Friday March 16, 2018 @09:51AM (#16600600)
Yeah, that is the hard bit of it for sure, that's why I thought maybe you could just sum up and demonstrate the steps you took after the bug hunt was over. But an edited version would be very cool too.
Re:Writing System Software [video] by CoolGuySteve on Thursday March 15, 2018 @01:50PM (#16595007)
As an HFT systems programmer myself, 5, 10, 11 are interesting to me. As in, using Unix primitives to persist state in a distributed system.

It seems like there are really 2 meta topics: 1: Using C and the OS to maximize performance 2: building distributed systems

Re:Writing System Software [video] by randormie on Thursday March 15, 2018 @01:52PM (#16595030)
One vote for #5. Also a general suggestion: these sessions could be livestreamed on twitch.tv for live interaction with viewers. Of course the VOD can still be uploaded to youtube later.
Re:Writing System Software [video] by stephen82 on Thursday March 15, 2018 @01:56PM (#16595069)
How about writing a book with all these potential ideas and name it "The Design and Implementation of Redis in-memory Database"?
Re:Writing System Software [video] by antirez on Thursday March 15, 2018 @05:55PM (#16596835)
I wish I had the time... In N years hopefully I'll switch into writing much more, now I've a book about C that will never end while I work constantly at Redis for instance. But for now me doing first-hand coding is what the project needs... eventually I may probably switch more to a design figure, but will take time. However soon or later I may take a few sabbaticals, so that I can continue working at Redis as I do now but pausing, for instance, for 1 month every 4 months or alike, doing this kind of projects.
Re:Writing System Software [video] by mmjaa on Friday March 16, 2018 @08:52AM (#16600270)
I've always loved your LOAD81 project and would encourage you to return to it and add some new features - so maybe how you made it, what it can be used for, how you can extend it by adding functionality .. ? EDIT: I know its not as sexy as your Redis work, but I think there are a lot of wise nuggets in this project that other developers could learn from.

http://github.com/antirez/load81.git

Re:Writing System Software [video] by antirez on Saturday March 17, 2018 @11:03AM (#16607259)
Thank you, I may indeed use some non-Redis part of code for the next episodes.
Re:Writing System Software [video] by minimaxir on Thursday March 15, 2018 @12:59PM (#16594545)
> However I see that the reach of the videos is very low compared to a blog post

That's the reason why in my other comment, the data screencast I created was made in addition to a blog post, where the blog post and the video explicitly link to each other as cross-promotion.

That does extend production time significantly though.

Re:Writing System Software [video] by WJW on Thursday March 15, 2018 @02:05PM (#16595142)
So cool! Redis is one of my favorite pieces of software and it's super interesting to see some of the internals explained more clearly. At the moment I'm only on the second episode, but I look forward to the rest of them. Perhaps replication could be a topic as well, if it's not already covered in the Sentinel and/or Cluster episodes. Another thing that would be interesting is how connections are handled and how concurrent commands from separate clients get executed in a sane way.

As an aside, I noticed several typos sneaked into comments here and there ('command gets replicated to the salves', etc). I'm half considering filing a pull request for them, but feel it might come across as overly pedantic. Would such a PR be appreciated?

Re:Writing System Software [video] by antirez on Thursday March 15, 2018 @05:45PM (#16596780)
Great ideas! Thanks. Sure the PR fixing the typos will be accepted (I'm very slow with PRs usually, but the typos are faster to merge if you follow my hints later). Better if they are small PRs with a few typos fixed because many files are continuously changing otherwise a big huge PR will end conflicting in many places.
Re:Writing System Software [video] by vvanders on Thursday March 15, 2018 @01:20PM (#16594734)
One vote for #4, not sure how related it is to Redis but I always have fun showing how much snappier flatbuffers is vs protobuf because they don't need to allocate/copy/etc when reading data off the wire.

I feel like that knowledge needs to be shared around a bit more.

Re:Writing System Software [video] by juicy-fruit on Thursday March 15, 2018 @04:57PM (#16596474)
Thanks for the videos, it's always a pleasure watching good engineers work. A lot to learn.

As a side note, can you share your Terminal color scheme? It's really nice

Re:Writing System Software [video] by antirez on Thursday March 15, 2018 @05:52PM (#16596823)
This is inspired to the commodore C64, I just picked a few random colors with the picker in the terminal. Unfortunately I lost the color scheme the previous week while in Tel Aviv, since this color scheme does not work well when projected so I had to switch to B/W and the simplest way was to just modify the current color scheme.
Re:Writing System Software [video] by Scarbutt on Thursday March 15, 2018 @07:42PM (#16597474)
Do you do all your coding on a laptop with no external monitor?
Re:Writing System Software [video] by edsiper2 on Thursday March 15, 2018 @03:21PM (#16595770)
Definitely:

5. The odd Redis persistence model: fork() and fighting the copy-on-write.

Re:Writing System Software [video] by MrLeap on Thursday March 15, 2018 @05:18PM (#16596596)
Only thing I have to suggest is changing your sound recording setup to reduce the echo.

Interesting stuff, I haven't had to write C++ in a few years, it's nice to have a bit of a refresher.

Re:Writing System Software [video] by tstrimple on Thursday March 15, 2018 @06:38PM (#16597074)
Just hanging fabric on the walls should make a world of difference.
Re:Writing System Software [video] by jakeogh on Thursday March 15, 2018 @09:15PM (#16597879)
I'm enjoying this very much too, but the bell is jarring.
Re:Writing System Software [video] by itamarhaber on Friday March 16, 2018 @10:33AM (#16600905)
I'm actually waiting for S2E01 :)
Re:Writing System Software [video] by bogomipz on Thursday March 15, 2018 @01:57PM (#16595074)
I would be very interested in 2. I know some other folks who be as well. Cheers.
Re:Writing System Software [video] by rjammala on Thursday March 15, 2018 @04:35PM (#16596308)
Please add 1. Code Snippets.
Re:Writing System Software [video] by xstartup on Thursday March 15, 2018 @02:02PM (#16595121)
Please show me how Redis rebalances data when more shards are added.