kenrose

Authored by Tobi, Shopify CEO.

An on-device search engine for everything you need to remember. Index your markdown notes, meeting transcripts, documentation, and knowledge bases. Search with keywords or natural language. Ideal for your agentic flows.

At 10:25am ET, HN is more up-to-date than Wikipedia (article hasn't been updated yet to reflect his passing).

We did this at OpsLevel a few years back. Went from AWS managed NAT gateway to fck-nat (Option 1 in the article).

It’s a (small) moving part we now have to maintain. But it’s very much worth the massive cost savings in NATGateway-Bytes.

A big part of OpsLevel is we receive all kinds of event and payload data from prod systems, so as we grew, so did our network costs. fck-nat turned that growing variable cost into an adorably small fixed one.

When mosh came out back in 2013, it solved a pretty real problem of ssh crapping out when you changed networks (like moving from in-office to home). It solves it at the app layer and uses UDP and is designed to work in high loss / latency environments. Very cool.

At the same time, in recent years, I've found that ssh running on top of Wireguard / Tailscale is way more usable than 2013 days. Those latter tools address the roaming IP issues directly at the network layer.

So while there are still issues with ssh / TCP if you're on a really crappy network (heavy packet loss, satellite link, etc), those have been less common in my experience compared to IP changes.

The “killer use case” for Mosh feels a lot less killer now.

Totally fair to bring up IPv6 vs. IPv4. However, I think Tailscale’s approach might sidestep some of that pain.

Avery (Tailscale CEO) has actually written about IPv6 in the past:

    - https://apenwarr.ca/log/20170810 (2017)
    - https://tailscale.com/blog/two-internets-both-flakey (2020)

IPv6 has struggled in adoption not because it’s bad, but because it requires a full-stack cutover, from edge devices all the way to ISP infra. That’s a non-starter unless you’re doing greenfield deployments.

Tailscale, on the other hand, doesn’t need to wait for the Internet to upgrade. Their model sits on top of the existing stack, works through NATs, and focuses on "identity-first networking". They could evolve at the transport or app layer rather than rip and replacing at the network layer. That gives them way more flexibility to innovate without requiring global consensus.

Again, I don’t know what their specific plans are, but if they’re chasing something at that layer, it’s not crazy to think of it more like building a new abstraction on top of TCP/IP vs. trying to replace it.

Can likewise confirm dblohm7 is a real human too :)

You're not wrong to think Tailscale is primarily a software company, and yes, salaries are a big part of any software company's costs. But it's definitely more complex than just payroll.

A few other things:

1. Go-to-market costs

Even with Tailscale's amazing product-led growth, you eventually hit a ceiling. Scaling into enterprise means real sales and marketing spend—think field sales, events, paid acquisition, content, partnerships, etc. These aren't trivial line items.

2. Enterprise sales motion

Selling to large orgs is a different beast. Longer cycles, custom security reviews, procurement bureaucracy... it all requires dedicated teams. Those teams cost money and take time to ramp.

3. Product and infra

Though Tailscale uses a control-plane-only model (which helps with infra cost), there's still significant R&D investment. As the product footprint grows (ACLs, policy routing, audit logging, device management), you need more engineers, PMs, designers, QA, support. Growth adds complexity.

4. Strategic bets

Companies at this stage often use capital to fund moonshots (like rethinking what secure networking looks like when identity is the core primitive instead of IP addresses). I don't know how they're thinking about it, but it may mean building new standards on top of the duct-taped 1980s-era networking stack the modern Internet still runs on. It's not just product evolution, it's protocol-level reinvention. That kind of standardization and stewardship takes a lot of time and a lot of dollars.

$160M is a big number. But scaling a category-defining infrastructure company isn't cheap and it's about more than just paying engineers.

I’m the CTO at OpsLevel, where we’ve been running a Rails monolith for ~6 years. We started on Rails 5, upgraded to 7, and are currently moving to 8. Before this, I worked on Rails at PagerDuty (including splitting a monolith into microservices) and on Shopify’s “majestic” monolith.

The best thing about Rails is its strong, opinionated defaults for building web applications. It handles HTTP request routing, data marshalling, SQL interactions, authentication/authorization, database migrations, job processing, and more. That means you can focus on business logic instead of wiring up the basics.

Rails isn’t as fast or lightweight as Go, but they solve different problems. For most web apps, the bottleneck is I/O, not CPU. Rails optimizes for developer productivity, not raw performance, and that tradeoff is often worth it, especially when speed of iteration matters more than squeezing out every last cycle.

Fair enough, but the system isn’t set up to optimize for the happiness of founders and employees. It’s set up to maximize returns, which agreed ends up concentrated in a very few rich outcomes.

(btw, big fan of Lost and Founder).

Dilbert... or any place that practices continuous delivery?

Sounds like your current/previous companies have heavier weight release processes that reduces how often releases go out?

I thought Subversion was the answer to complaints about CVS? Then git followed afterwards.

/s

It's great to see jq get some releases again.

It's a big part of our check evaluation infra at OpsLevel.

I have to ask: how are you using this in prod?

BMath 2005 here.

I worked with a bunch of smarter-than-me UW grads after graduating.

My “how to write large systems” takeaway from that early point in my career was to focus on the interfaces between various parts. What I’d never thought about until now is that is a very data centric viewpoint.

- What system has what data?

- In what shape?

- What shape does the next system need its data in?

- Are the interfaces between these orthogonal? Shallow? Easy to grok? Tight (as opposed to leaky)?

Great article.

Same here re: opt out.

You tell the agent you want to opt out. They scream “Opt out!” at the top of their lungs.

Then another agent takes you through the plexiglass door beside the metal detector.

That agent recites a brief script about how they’ll pat you down and then do so.

Pat down complete, they check their gloves for residues in a machine. Once that’s done, you’re free to go.

It generally adds 5-10 mins to the security line. Sometimes, if they really backed up and you say opt out early enough, it can actually be faster.

You have the right to opt out and can exercise it.

In addition to the "show your dev work to the CEO" use case, tunnels (ahem, "funnels") like this are useful when you're building functionality that requires pointing webhooks at your devlocal environment.

e.g., if you're building an integration with any of the myriad of SaaS tools that fire webhooks, you can test in devlocal and have a URL of whatever.tunnel.com.

There are tools like ngrok and localtunnel that exist to do just this. I'm looking forward to replacing those with TS Funnels.

“You either die a hero or you live long enough to see yourself become the villain”

Thanks for that feedback re: pricing and demo.

It's always a tricky balance, but I definitely sympathize with the feeling of just getting started without having to talk to a salesperson.