thinkingfish

We also published it as a crate https://crates.io/crates/segcache

What’s your billing model?

Congrats! Although I look back fondly on the many years I spent on debugging cache incidents (https://danluu.com/cache-incidents/) I don’t think the rest of the world should go through the same bumpy road.

It’s truly interesting to see a new generation of applications not only doing away with using bare metal hardware, but continuing to be built on increasingly higher level abstractions.

Of course, the key is for those abstractions to be dependable in production as well as easy to use. I think Momento actually takes runtime predictability quite seriously.

Hello! I still have your cookie jar LOL

Hopefully we can open source our deployment "generator" sometime soon. It's not gonna work in AWS (Twitter isn't a cloud first company) but you will get the basic idea.

Yes, swappable storage backend is a big driver of the design. Segcache for TTL-centric workloads, maybe some simple allocator-backed data structures for many popular Redis features, tiered storage for large time series... these were all internally discussed and sketched out to various extent.

Failure handling is very, very context dependent, both in terms of what the product does (which drives the ROI analyses) and where it runs (which determines ways things could fail). Still figuring out how to talk about fishing not the fish. Will give this more thoughts.

It was supposed to be a bit tongue-in-cheek... Obviously nobody should create a pingserver just for the sake of with the specs we had. But, as a halfway marker to a production-ready cache service, it's good to have something that lets you test the parts that will account for well over 50% of the CPU time in production.

They are not quite the same thing. Pelikan has the building blocks for an RPC server, storage, wire protocols, data structures, and whatever code that is needed to glue these together in some form to provide functionalities as a service (which is loosely defined, it can be a proxy too).

CacheLib, in terms of the role it plays, is closer to the modules under src/storage, but obviously in itself a lot more sophisticated in design to handle tiered memory/SSD storage.

We actually have been in touch with CacheLib team since before their public release. And we remain interested in integrating CacheLib into Pelikan as a storage backend as soon as their Rust bindings become functional :P

I assume you meant this as a comparison against Segcache?

We touched upon the CLOCK-Pro a bit in related work of the paper (https://www.usenix.org/system/files/nsdi21-yang.pdf, Section 6.1). I haven't done a deep dive, but can see two general issues. 1. CLOCK-Pro is designed for fixed-size elements, and doesn't work naturally for mixed-size workloads. Naively evicting "cold keys" may not even meet the memory size and contingency requirement needed for the new key. This is why memcached's slab design focuses primarily on size, and using a secondary data structure (LRUq) to manage recency. Something like CLOCK-Pro may be modified to handle mixed size, but I suspect that significantly complicates the design. 2. An algorithm without the notion of a TTL obviously doesn't take the strong hint of TTL. Given the workloads we have, we concluded that TTL is an excellent signal to use to make memory related decisions, and hence went for a TTL centric design.

It's a SaaS startup offering managed cache (like Elastic Cache). They have their own protocol but uses Pelikan-segcache as one of their backends.

I'm a big fan of their fountain pens and the piston filling mechanism.

Project creator here, didn't expect to see the link on HN :P

Here's the latest code architecture diagram: https://raw.githubusercontent.com/pelikan-io/pelikan-io.gith...

And a slightly outdated but mostly accurate description of our motivation. https://www.pelikan.io/2019/why-pelikan.html

#AMA We've done a few research-y and go-to-production projects in the last few years.

Oh the link broke when I changed the permanent URL format after the post was published. Here is the correct link: https://twitter.github.io/pelikan/2016/separation-concerns.h...

I've fixed the post as well, thanks for finding this bug.

Disclosure: I'm maintainer/author of Pelikan.

I wrote about Pelikan, a unified cache framework, and its relationship to Memcached and Redis in this blog from 2019: https://twitter.github.io/pelikan/2019/why-pelikan.html

Specifically talked about what we would like to change about each.

Here’s a blog version of the paper for anybody who isn’t into 12 page double column pdf files

https://twitter.github.io/pelikan/2021/segcache.html

Internally I’ve written a generator to spit out a deploy config based on input vectors such as (qps, data size, connections). But it assumes using Twitter’s container environment.

Publicly I think I want to do two things: 1. write a blog about cache operations and how to config Pelikan properly in general; 2. create some templates for common deploy mechanisms. What do you use for deploying services? What are your cache requirements? I can produce an example and put that in the repo/doc

lol I'm glad you asked there since I am on HN about once every 3 months...

Based on his response on Twitter I'd guess not https://twitter.com/dormando/status/1381706098290225152

But also agree with you that in that high concurrency usually not needed when using distributed caching. As I said in my tweet response (https://twitter.com/thinkingfish/status/1382039915597164544) IO dominates. And there's failure domain to consider. However, I've been asked a few times now about having the storage module used by itself or via message over shared memory in a local setting. That may very well present different requirements on cache scalability.

The nature of tech work means employees generally have more flexibility with their time, and can afford such involvement without jeopardizing their productivity or job. It is thus hard to quantify how much money it costs for such engagement.

I'm not disagreeing with your concern, but the mechanisms are a little different. As a result my opinion is that giving money to improve civic engagement among minorities, the poor, and under-privileged population is better discussed as a separate initiative.

That said, anybody can focus on projects that improve political engagement (making voting easier, help people find information about local politics) without bringing their political opinion/preference to the table.

I don't work on GraphDB, but basically the situation is this: 1) yes, it is not a great use case for MySQL, but that's how it started, partly because Twitter needed join on its graphs; 2) legacy systems die hard, especially at scale- Twitter is working on a better solution, but for now Flock is still what's running in production.

In general, Twitter infrastructure didn't start by building solutions in house. Like most early stage companies, it started by using whatever it can find in the OSS repertoire. But a lot of problems that are non-existing at a smaller scale or at least tolerable become more acute when you scale up.

This is when the developers face a decision: to improve the existing solution, or develop a new one? There are many factors that impact the final decision. Maturity of the technology? Community support/inclusion? How big is the necessary change? What scale can existing solution handle by design? How does in-house talent compare to those maintaining the original? What's the focus/need of the in-house use cases compared to the broader objective of the OSS solution? It should not be a surprise that the answer differs from project to project. But if the decision is to use the existing solutions, it probably won't raise much eyebrow/question.

It is very common for larger-scale operations to write up their own solutions. Because a different scale can change the nature of the problem fundamentally. Google, Facebook and whoever has production fleet above the 10K range tend to build a lot of in-house solutions, often after similar considerations as mentioned above.

CSCI 3350: Debugging- Beyond Print

CSCI 4600: When Big-O Complexity Becomes a Lie and Why

CSCI 4035: Refactor Your Own Term Projects from Past Semesters

Would be nice to allow filters- for example, leave out any results that match patterns in .gitignore.

This is not an issue for fatcache. slab item chunk size introduces internal fragmentation (because items sizes usually don't match "chunk" sizes unless you use slab profile, -z option, to match them up). The kind of external fragmentation you described is due to the eviction strategy memcached uses, which can be avoided by using slab level eviction strategies (Twemcache and latest Memcached both have this support). fatcache does slab level eviction based on write timestamps, which is the equivalent of LRC eviction in Twemcache, and you can read about the mechanism here: https://github.com/twitter/twemcache/wiki/Eviction-Strategie...