> The most important thing Antirez did, in my opinion, was to say "No" to things. No to new features that didn't make sense.

Redis has had tremendous mission creep over the years. It started of course mostly as a volatile cache but I've now seen it also used as a message bus (in three different ways: BLPOP, PUB/SUB and Streams), for service discovery and as a general purpose database - something that Redis Labs (in my opinion: wrongly) encourages.

Memcache existed in 2009 when Redis was first released, also as a volatile cache...and still is just a volatile cache. Memcache is what "saying no" looks like. Redis is what "saying yes" looks like - and there are a lot of gotchas.

Memcache is what saying no to changing the use-case looks like. Redis is what saying no to changing the architecture in order to implement features looks like.

Redis has stayed the same, architecturally, from the beginning: it’s a keyspace where the values are arbitrary in-memory objects owned by their keys, and where commands resolve to synchronous function-calls against the objects (or ref-cells) at those keys.

Anything that can be done without changing that architecture, is in-scope for Redis. (Though if a data structure doesn’t have wide use outside of a domain, it’s best left to a module.) Anything that cannot be done without changing the architecture, won’t be done.

Much of the “fun” I’ve personally had in watching Redis evolve, has been seeing how Antirez has managed to solve the puzzles of getting features that intuitively wouldn’t be a good fit for this architecture, to fit into it anyway: changing technologies that are implemented one way everywhere else, into something else for Redis, that still work, are still performant, and still solve isomorphic use-cases—if not with the same steps you’d use to solve them in other systems. (E.g. using Streams vs. using a regular MQ; using Redis Cluster vs. using regular RDBMS replication; etc.)

Memcache also is about saying no to changing the architecture. That chosen architecture is similar to Redis Cluster -which is a change of architecture Redis did undertake.

I personally have not enjoyed all the strange and wonderful ways people have found to use Redis. Most of them are pretty fragile and (most dangerously) they often put all Redis uses in the same instance...with eviction turned on.

Wow you totally got it. This is exactly the idea I was using, I even saw an opportunity when the Redis model was potentially able to solve a new use case.

Would you mind elaborating on some of those 'fun' items - part of the problem of being aware of certain technologies without really 'following' them is understanding why they change.

It didn't really start as a volatile cache, it started as an in memory data structures server. It's useful for all kinds of stuff memcached isn't (and has been since day 1).

This. To me, Redis was always about the rich set of operations you can atomically perform on data without going full ACID. It's very different from a pure cache (although it's probably a good choice if all you need is a cache).

Yeah. I do think that Redis has suffered from some mission creep, including a deprecated misadventure in to a mode where values could be stored on disk and only temporarily cached in memory.

Maybe that's what Cal and his colleagues were using it for at the time, but the claim that it started as a volatile cache is just not true. Interesting datastructures and the fork based persistence model were there from the start.

It started as a memcached with persistence to disk for fast startup.

No, it was always about data structures.

Lists, hashes, sets and such that you have available in your programming language, but available as a networked server so any language/app/process can access the same data using the same useful structures.

Well, kinda. Feature list does look like basically random assortment of vaguely related things (or "toolbox" if you want to be nice).

But... things that are also just useful often and for what you'd need to spin off a separate service for, or make suboptimally in whatever (No)SQL you use to store your data.

So it is perfect if you just want to prototype whether given approach is viable, and "good enough" for many apps.

> but I've now seen it also used as a message bus

It works excellent as a message bus, though. And you can add HSET/HDEL to your list as the fourth way.

Strongly, strongly, disagree that Redis works well as a message bus.

Lots of reasons why but here is just one because I'm short on time: all of these (four) ways of doing a message bus with Redis offer either a work queue mode or publish/subscribe - but not both (perhaps excepting streams which I've forgotten some of the details of). I have yet to see many real world messaging scenarios where you don't end up using both.

A "proper" message bus needs to have topics, exchanges and queues - like Rabbit. 50% of that functionality is usually not 50% of the value but 10% or less (especially when you consider all the other things about Redis you need to think about - like how eviction is configured).

This is a bit typical of Redis: for any X, you can normally do X in Redis, but is it really a good idea to do X in Redis? IMO, usually no.

There's a wide field of messaging from distributed logs to queuing systems to event sourcing to complex service buses. Kafka, Pulsar, NATS, RabbitMQ, Tibco, NServiceBus, ActiveMQ, Solace, AMPS, and dozens more. What's "proper" depends on your needs.

Redis Streams actually does exactly what you describe and is basically a single-node fast Kafka replacement with multiple independent consumer groups, per-message acknowledgements, and queue semantics. We use it successfully in many high-load scenarios.

Name of the hash is a topic. Keys are nanosecond timestamps, values are the packets. You put in the data, you send a pub notify to the listener, listener goes and drains the queue. It acts as a direct exchange queue, recovers fine from a restart and it works wonderfully. For fanout, you can simply use pub sub. For topical, you can keep a list of direct exchange queue hashes. I haven't needed it.

Redis Streams is built-in and does all that for you. https://redis.io/topics/streams-intro

You're probably right. I did this before streams were a thing.

> There are plenty of problems with BDFL-style projects, but I think there are a lot of advantages too.

BDFL governance is extremely underrated in the opensource world. People attribute the success of the Linux kernel to its open contribution style, but I will argue that Linux is successful because there's a dictator at the top that enforces a direction, a long term goal, and most importantly, says NO.

Community based governance is a direct cause of that jwz's Cascade of Attention-Deficit problem one can find all over the open source world, especially Linux desktop related.

Does FreeBSD have a BDFL?

We don't have enough data points to determine if what you are saying is correct. I have also seen lots of BDFL projects smoke out.

Is the lesson that actual dictatorships only survive long term if 1) the dictator is a nice person and 2) they eventually get managed by an open democratic process?

I think you are actually describing the two clocks problem. Lots of BDFL projects have their own Cascade of Attention-Deficit issues, the project only goes in the direction the BDFL actually cares about.

A (software) dictator is not necessarily good. But let me double down and say that a community cannot create a product as good as a single head at the top can do, for the single reason that people care only about their own garden, and integration needs someone to oversee and be responsible for all parts.

There is no Steve Jobs in a democratic committee.

As usual, Fred Brook's Mythical Man-Month had some valuable insights on this back in 1975:

> To make a user-friendly system, the system must have conceptual integrity, which can only be achieved by separating architecture from implementation. A single chief architect (or a small number of architects), acting on the user's behalf, decides what goes in the system and what stays out. The architect or team of architects should develop an idea of what the system should do and make sure that this vision is understood by the rest of the team. A novel idea by someone may not be included if it does not fit seamlessly with the overall system design. In fact, to ensure a user-friendly system, a system may deliberately provide fewer features than it is capable of. The point being, if a system is too complicated to use, many features will go unused because no one has time to learn them.

* (This is quoted from the Wikipedia page as I don't have an online copy of MMM to hand).

Cannot seems rather strong. I haven't been paying very much attention, but I was under the impression that Postgres doesn't have a dictator?

It's evolved. There was a point that Tom Lane was the last line of defense against anything that strayed too far.

Heck at one of Postgres conferences there was a shirt that said something like "Tom Lane reviewed my patch and all I got was this stupid shirt."

While he was not a BDFL style person, in the past, if he didn't like a change, it didn't make it in.

Or Rust, which - whatever you might think of it - does not feel very designed-by-committee.

Rust governance is more amazing than Rust the language and that is saying something.

Here’s the thing: you need both. You need breadth AND focus. Breadth tends to come from the community and focus from a small number of individuals. But both are important.

Keeping both of those balls in the air at once is why being a software coordinator like this is such an exhausting job.

The main differentiation (and risk) with BDFL projects is having (or not) a very competent (like one percenter) BDFL.

The structure of a committee project doesn't make it better or worse than BDFL. It just makes it more average. Any greatness present is toned down. And substandardness is brought up.

This also main reason the "led by greatness" BDFL projects are better than the committee run ones and the "led by less than great" BDFL projects "smoke out". The committee's tend towards the middle of the bell curve, BDFL's are all over the place.

>Does FreeBSD have a BDFL?

No, but it doesn't have as much success as Linux either

this is IMHO no different the real world - a 'good' despot/monarch/etc can be a good leader, but this is highly dependent on the situation (read: leader), and leadership transition process is always a risky one. Democracies/Republics work well so long as the community has a generally shared vision, but can be hampered when there is too much dissent, but generally avoid the succession problem

OT: Indeed, a good dictatorship is better than any democracy, but it doesn't work in practice because for every Augustus there's 10 Caligula.

Democracy is just the least worse political system.

Democracy is more of a principle (is just means that legitimacy of government comes from some aggregate choice of the governed), than a political system. Radically different political systems can be classified as democracies, all the way from direct vote (self-organized communes, also Switzerland) to complex federations with layers of indirect representation, such as the USA or the EU -- with a lot of other interesting alternatives such as sortition.

In a FOSS it is very easy to fork, so if the BDFL get's mad you can just say thanks and fork the project and continue. So that keeps BDFL in check.

IRL you own a plot of land, or a home. Or you have a small factory, and there are a lot of network dependencies, who does you sell the supplies, where you sell your products. It the local BDFL get's mad it is very difficult to move. (Specially if there are some cash flow controls were you can't get out with the money that you get selling your home or that you have saved.)

You can fork Linux if the dictator doesn’t suit you, but you can’t fork a nation. You also can’t switch to another nation that is more appealing, generally. Finally, Linux can’t imprison and torture you if you criticize it

If you're comparing FreeBSD to Linux then I think that says a lot about the merits of BDFLs, really.

Did FreeBSD not have the discussion about this years ago? Though they did not have a single person, but a 20 person core team, which were cut down to a 9 person core team that is voted for every 2 years.

http://freebsd.dk/sagas/freebsd/#id1

The core team doesn’t really make technical decisions. It’s more for administrative/social/governance type stuff.

Who would then make the technical decisions? The owners of the different sub systems?

There's no hard rule. The committers are expected to behave like a civilised human beings, ie talk to each other to reach consensus.

Let's say you have a fix for some driver used on ARM. You'll submit it via Phabricator to get ARM maintainers to review it. If they don't, because eg they don't have time, then you'll probably commit it anyway, but if someone points out you did something wrong, you'll be expected to fix it.

Sounds similar to debian, which is also a very successful and stable project.

Underrated in the open source world — whilst at the same time, feels obvious in the writing world?

Imagine an author has gotten started writing volumes 1 and 2, and those volumes became popular — maybe the Harry Potter books as an example. Now, adding more authors with equal "story-line decision making power", or a committee, for volumes 3, 4, 5, is a weird idea, right.

Look at what happened with Game Of Thrones! Total disaster at the end that ruined the whole thing for everybody.

> I will argue that Linux is successful because there's a dictator at the top that enforces a direction, a long term goal, and most importantly, says NO.

Linux is made because of many people all over the world, but it is successful because of the BDFL at the top. A consistent sense of mission absolutely benefits a project in the long term. There just isn't anyone that's going to get your idea better than you, when you're doing something that you know enough about to seriously get started on.

I think it's _just_ underrated; I think it's underrated in the business and government worlds too. A good leader should take different points of view into account, but should also be able to make a decision when needed.

I feel like Antirez has already done everything needed to ensure this doesn’t happen, by 1. building the module system into Redis, and then 2. working with companies like Redis Labs to establish that Redis is enhanced through these modules, and to ensure that modules offer everything required to enhance Redis in all the ways that matter.

In essence, Antirez has protected the core of Redis from needing to bloat, by making it possible for anyone who wants to build on top of Redis to literally do so—build on top, rather than inside. As such, I expect future PRs to Redis Core to look much like current PRs to Redis Core: just fixing bugs, adding stability, and resolving infrastructure-level interoperability concerns.

I 100% agree with you and I think it explains why most OSS UI projects are, well, terrible. Questionable features, inconsistent UX, millions of "options"... these are the hallmarks of a community model lacking strong leadership and a coherent vision.

As for the future of Redis I choose to be more optimistic. I think the Redis philosophy (thanks to Antirez) has become ingrained. The benefits of a minimal feature set are hopefully well-established.

In the very least I suspect this will last much longer than many expect before the bureaucrats take over.

BDFL: Benevolent dictator for life https://en.wikipedia.org/wiki/Benevolent_dictator_for_life

(This is also referenced and linked in the article.)

> The most important thing Antirez did, in my opinion, was to say "No" to things.

This is hard. It makes you feel bad inside to tell people "No", especially when they open PR and clearly did a lot of work adding some feature.

Thats something a have regarding PR and code reviews: much frustration can be avoided by discussing what to do and how to do it before showing the work done (avoiding also infinite bikeshedding). Often code review talk about the strategy to do X and not the way to do X

And some people take the 'No' badly. It's frustrating having to explain that the code in a PR needs to be maintained forever, and it's the repo maintainer that will have to do it, not the contributor. A simple proposal upfront makes all the difference, and saves human effort and feelings.

>Saying no is one of the most important jobs of a project maintainer. It's a thankless task that upsets a lot of people. But it's critical for a project to stay successful and achieve the leader's vision.

Reminds me of Brian Goetz: "We have to say no to almost anything. If we didn't, things would pretty quickly degenerate to the point where the system collapses on its own weight (...) if we did this, which of these eighteen possible features might we foreclose on, and might we want to leave those possibilities open, at the cost of saying no to this thing here." (https://www.infoq.com/presentations/java-history-present-fut..., 24m+)

The one drawback of the BDFL model is that you need a person, and a very specific kind of person at that, which tends to be irreplaceable - to sustain it.

Redis has succeeded because of antirez. I really wish it can continue doing so for the long run without him. More than anything it reminds me of Steve Jobs' death (although gladly this is in much happier circumstances and I wish antirez many more decades of hacking). It's true that a decade later Apple is still doing great, but you just know things would have been different if he was still around.

Instead of a single BDFL, you could also have a board of e.g. 3 BDFLs. Those 3 people would have the final word, but they rank equal, and have to come to a common conclusion, e.g. by discussing internally. That only works if the board is not too big, and the people are somewhat compatible to each other. But in that case, I think it works much better, because there is less pressure on a single person, also less work (they can distribute it somewhat), and probably more reasonable and consistent conclusions.

It seems that this model is not so common? At least the list on Wikipedia (https://en.wikipedia.org/wiki/Benevolent_dictator_for_life) only lists Django as an example with 2 BDFLs.

I was active in a smaller open source project (http://www.openlierox.net) where it mostly was like that, i.e. we were 3 main developers, and all major design questions were discussed among us. But this example is probably not so representative, as there rarely were further contributions by other people. But anyway, I was very happy with this model, that there is not a single leader, but 3 people.

I think the mixed historical record of consul republics might be relevant to why a single leader is often a better idea than 2-3.

I'm thinking that a project that starts out with 3 BDFLs (like yours, @albertzeyer) and works fine & builds sth people "love" — it'll continue working fine with those 3 BDFLs.

And a project that starts with just 1 BDFL, and works fine & builds sth people "love", it'll continue working fine with that 1 BDFL.

If, however, replacing one of those 3, or adding another BDLF to a one person project — I'd think that by default, that won't work.

... Unless the new person has worked next to the earlier BDFLs for years, so the previous BDFLs can be ok certain s/he is "the right one"?

Whatever works fine, works fine — the risky thing, is introducing a change (a new person)?

Thing is, with a consul or triumvirate system (you can’t have more than one “dictator”; all of these terms are from Ancient Rome anyway so let’s use them properly), if any disagreement arises between the 2 or 3 leaders, it will develop a new instability where there wasn’t one before. Or else, one of the 2-3 leaders will start to dominate the others. Even without turnover the dictator system is more stable. In fact the only benefit of a consul or triumvirate system might be the stability and continuity under turnover.

Do you have any tips for explaining to stakeholders that things in a Redis cache expire on async schedules?

> I'm sure many would love to see some change there and finally overrule Torvalds' decisions.

Is forking no longer a solution to this problem? Or are you just wanting to wrestle control away from Linus?

You already see this happening, on the IoT world there are plenty of competing FOSS OS clones, some of them POSIX based, common to all of them is the use of non-copyleft licenses.

How does copyleft relate to BDFL-style governance? Python, for example, had a BDFL until recently but isn't copyleft licensed.

Taking projects into new directions , possibly closed source, due to political disputes.

Without the legal obligation to upstream, the majority wins what happens afterwards.

> Or maybe you see an old white guy in charge of Linux and that can be considered a problem in itself

Why is that a problem?

>> "old white guy in charge of Linux"

Erhmmm, why does his age or colour matter in this context? Debate can happen about people and their leadership style, (I'm not a fan of Torvalds) but when the first criteria someone sees for criticizing another person is their age, colour, or race, that seems a bit shallow and biased.

I agree. Isn't this overt racism, sexism, ageism, etc?

> Racism: prejudice, discrimination, or antagonism directed against a person or people on the basis of their membership of a particular racial or ethnic group

Of course it is, but it's the kind of bigotry nowadays some people evangelize as good (by redefining racism), and what the twitter mob encourages.

Chain of events: I was tempted to try a respectful reply with substance, but then realized I wouldn't be allowed to discuss this here, due to moderator preference for "not talking politics".

I was then [regrettably] tempted to write something with a "mic drop" feel, just to express my disagreement in an acceptably pithy way.

And then it occurred to me that this whole sub-thread is Twitter-length comments. And I was about to add one more. And I realized that maybe the moderation of HN actually prevents us from going as deeply on this topic as we do on everything else, and so that moderator preference has essentially reproduced a pithy and unproductive Twitter dynamic for topics like this.

Pithy and shallow self-censored comments in reply to pithy self-censored comments.

EDIT: But also, moderation is really hard <3 Just wanted to try to put words to a small dynamic that's perhaps playing out here.

There was a huge thread yesterday on HN about this topic with many comments much longer than tweets. It is certainly a topic that can be discussed at length on HN. People were voted down and up, but no one was cancelled; we're all still here.

I'm not aware of any moderator preference against talking about this subject. Again: huge thread on it yesterday that persisted on the front page all day.

However, I don't think every single off-hand mention of race or gender needs to devolve into a subthread of 50 multi-paragraph comments. So I appreciate your decision to avoid it in this instance.

FWIW, I probably disagree with you, but I wish we could talk about these things in a civil way. I think the PC/cancel-culture prohibition about discussing this stuff is a real impediment toward progress. Further, I don't envy the moderators since it's hard to keep things civil--bad actors on both sides come out of the woodwork. Anyway, have an upvote as a small gesture of peace and kindness in a world where disagreements are supposed to make us bitter enemies.

Aw thanks!

Yeah, I was a moderator on one of the first university-run online forums, and it was so stressful and difficult...! All the first-year students just spoke off-the-cuff, but "monitoring" all that speech was SO challenging...!

> due to moderator preference for "not talking politics".

There's no such preference. What made you think there was?

The question of political topics on HN is a hard one because some political stories are on topic, some are off topic, and even the on-topic ones tend to turn into flamewars. I've written a lot about how we approach this: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu.... Some good threads to start with might be https://news.ycombinator.com/item?id=21607844 and https://news.ycombinator.com/item?id=22902396. Also https://news.ycombinator.com/item?id=17014869, which shows how far back political discussion goes on HN, as well as the argument about politics on HN.

If anyone reads those explanations and still has a question that isn't answered there, I'd like to know what it is.

thanks dang. Was thinking about y'all when I added that "<3" edit. I appreciate you.

> What made you think there was?

I recalled a post way back when trump was voted in, asking the community not to speak about politics. (I'm sure I'm misrecalling the specifics, but that's the fuzzy take-home that's nestled in my mind after all these years.)

---

Ah yes, found it. Apparently it was just one week, but I remembered it as a standing request :) https://news.ycombinator.com/item?id=13108404

"Racism" against white people in the US is completely devoid of systemic material consequence. Understanding definitions isn't about "mobs" or even ideology, it's about identifying what happens in incidents of racial bias and why those things happen.

What? We are not ants. Even if being racist to a person doesn't affect the "colony" still is being racist to that person.

And redefining a word so is "bad for everyone but X" doesn't make X innocent of doing it, it makes X guilty and a bully.

Also, the world is much bigger than the US.

Fair enough, but does that make it "good" or simply "less bad"? And if it's just "less bad", then why should we tolerate it when there's no dichotomy between "racism against whites" and "racism against minorities" (i.e., we can and should aim for a world with no racism at all)? Further still, once you've established a racial worldview with a racial hierarchy that puts whites on the bottom, you've established a framework that facilitates white supremacist arguments (both by making it easier to frame whites as victims of racism and by validating racial worldviews in general--a worldview that puts whites on the bottom is a lot closer to a worldview that puts whites on the top than an egalitarian worldview is to either). Liberalism (i.e., treating people as individuals instead of identical constituents of their race) has been the driving force behind the massive reduction of white supremacy and anti-minority racism; it's an enormous success and an important bulwark that protects minorities. Why should we erode this important bulwark by replacing it with a system that (1) perpetuates injustice against whites and (2) paves the way for white supremacy? Racism is lose-lose.

I agree with you but that’s not how a lot of people see it these days.

Precisely no one wants to replace Linus Torvalds with a woman or person of color just to have a woman or person of color maintaining the Linux kernel.

You’re begging me to go nutpicking with the phrase “precisely no one” but let’s grant that point. Many people have been outspoken about, in general, replacing white males in positions of power and importance with non-whites or non-males. The degree to which none of these people want to replace Linus probably has more to do with the degree to which Linus Torvalds is perceived to be in a position of power or importance.