pipe_connector
pipe_connector on Hacker News
It's fair to distrust something because you were burned by using it in the past. However, both the examples you named -- Postgres and FoundationDB -- have had similar concurrency and/or data loss bugs. I have personally seen FoundationDB lose a committed write. Writing databases is hard and it's easy to buy into marketing hype around safety.
I think you should reconsider your last paragraph. MongoDB has a massive community, and many large companies opt to use it for new applications every day. Many more people want to use that product than FoundationDB.
MongoDB (via WiredTiger) has used MVCC to solve this problem since transactions were introduced.
Yes, I have worked on an application that pushed enormous volumes of data through MongoDB's transactions.
Deadlocks are an application issue. If you built your application the same way with Postgres you would have the same problem. Automatic retries of failed transactions with specific error codes are a driver feature you can tune or turn off if you'd like. The same is true for some Postgres drivers.
If you're seeing frequent deadlocks, your transactions are too large. If you model your data differently, deadlocks can be eliminated completely (and this advice applies regardless of the database you're using). I would recommend you engage a third party to review your data access patterns before you migrate and experience the same issues with Postgres.
Jepsen found a more concerning consistency bug than the above results when Postgres 12 was evaluated [1]. Relevant text:
We [...] found that transactions executed with serializable isolation on a single PostgreSQL instance were not, in fact, serializable
I have run Postgres and MongoDB at petabyte scale. Both of them are solid databases that occasionally have bugs in their transaction logic. Any distributed database that is receiving significant development will have bugs like this. Yes, even FoundationDB.
I wouldn't not use Postgres because of this problem, just like I wouldn't not use MongoDB because they had bugs in a new feature. In fact, I'm more likely to trust a company that is paying to consistently have their work reviewed in public.
1. https://jepsen.io/analyses/postgresql-12.3
MongoDB has supported the equivalent of Postgres' serializable isolation for many years now. I'm not sure what "with strong consistency benefits" means.
Agreed that this property is useful for efficiency. The danger comes from users believing this property is a guarantee and making correctness decisions based on it. There is no such thing as a distributed lock or exclusive hold.
I agree with the characterization of applications you've laid out and think everyone should consider whether they're working on a "tall" (most users use a narrow band of functionality) or a "wide" (most users use a mostly non-overlapping band of functionality) application.
I also agree with your take that tall applications are generally easier to build engineering-wise.
Where I disagree is that I think in general wide applications are failures in product design, even if profitable for a period of time. I've worked on a ton of wide applications, and each of them eventually became loathed by users and really hard to design features for. I think my advice would be to strive to build a tall application for as long as you can muster, because it means you understand your customers' problems better than anyone else.
Sure, I understand how to authenticate to my remote machine with a smartcard (and already do use this setup). I'm wondering how to authenticate to resources (over HTTP) from my remote machine while using webauthn.
Doesn't this only solve the problem for resources I am accessing over SSH? What about if I wanted to access something over HTTP like my web browser does?
Maybe I'm just missing something, let me explain:
I've already ssh'd to my work machine. I want to send an HTTP request to my company's internal web API from that machine, but we only use webauthn credentials. I'm going to use curl to send the request to the web API. With basic username/password auth or totp it's easy for me to write a script that prompts me for my password/totp code and marshals in into the expected format. How do I do this with my FIDO2 private key in a way that doesn't completely undermine the whole process?
Sorry, I think I missed something because the article doesn't mention GPG at all. How can you make a webauthn client defer to gpg-agent?
How do you authenticate from a machine that isn't local to you? I don't do any work on my work-issued laptop, I use a powerful remote machine instead.
While this might make this situation more likely to occur, you can never prevent concurrent accesses from happening in a distributed system.
This Bubblewrap is much older than the one you've linked.
Lots of people being negative about this, but if you've ever implemented anything that works in near-real-time at wide scale, most of this design makes sense and it works great.
One thing interested me: Why the difference in pathing between events and messages? I think the event flow makes sense, but why not have messages also go through your gateway server instead of through webapp? Surely there is needless latency there when you already have an active websocket open to gateway? I thought perhaps it was because your gateway was for egress-only, but then the events section made it clear you also handle ingress there.
You can still use a dedicated diff tool for these situations if they come up regularly. In my experience they are better than vim anyhow.
kubernetes can use anything that conforms to the CRI interface, which in practice is either CRI-O (RedHat) or Containerd (Docker, Inc.). Podman and Docker are also consumers of both of those engines
500,000 requests/day is ~6 requests per second.
In general I agree with you, but Go's simple syntax means that there are extremely few (if any? I've never run into one) edge cases with checking that an error was at least considered. The guarantees are the same as what you get with e.g. Rust. The only difference is that you have to run a third party's code, which I don't like, either.
If I'm reading your feedback correctly it sounds like you want:
1) Syntactic sugar for if err != nil { return err }; similar to ? in Rust
2) The errcheck linter to be enforced (part of go vet, maybe?)
3) Exhaustive switch statements
The first point was looked at and no one has proposed a solution enough of the community could agree was an improvement while maintaining sufficient explicitness. If someone has a really good proposal for this I am sure it would be considered.
The second point could probably happen with enough lobbying. So, verbosity is still there but at least every project will do something with their errors.
The third point I don't believe is possible today because of the way interfaces work. If Go added sum types then this should be possible, but I could be mistaken about that.
Mongo has customers at some of the largest technology companies around and perhaps the best-in-class cloud-agnostic managed service. There's less hype than when the webscale video came out, but they've successfully converted a good bit of that hype into a solid platform with happy customers. I've seen a lot of Mongo new deployments even still.
Connections must have unique IP:Port pairs between client and server. You're limited to 65K concurrent connections for the same client. In practice, no one is opening that many connections from a single client.
Is simplified failover referring to the currently-in-beta async replicas? Or is there something else on the way that will make it easier to failover?
I was asked this question when I interviewed in 2020 and finished in about 35 minutes without any C experience. I can confirm that this question made me very interested in working for MemSQL/Singlestore: I loved the experience. The next step of the interview was to implement a pretty large concurrent program. I was given conflicting/very little information from the recruiter on how to complete the assignment, which I thought was odd given that in my estimation this was far too large for an interview question. I submitted my work after a couple hours of effort in an incomplete state because I decided I didn't really want to work for the company anymore. I never heard back. Really soured me on the company.
Very often if your load balancer is custom. For example, we have an edge service that fulfils the role you have nginx for, but it handles both websockets and raw tcp traffic. Our edge service is the gateway for authentication and authorization -- from that service we can connect users to chat rooms, matchmaking, or actual game instances. We could get away with just nginx + room ids and manual upgrades to the pool of game/matchmaking/chat services for internal traffic possibly, but we ship updates to our edge service all the time and that process needs to be painless, so I was curious how others have done this.
We're currently using systemd sockets with Accept=no, then multiple edge services can accept() from the same socket that's always open and bound to a known port. Once a new service has started, we can signal the old service to shutdown for however long it needs by no longer listening on the socket and letting connections drain naturally. We're thinking about changing to dynamically allocated ports/sockets which is pretty natural in the container orchestration world.
How do you handle deploying a new version of nginx without forcibly closing connections?