quentusrex
For a WebRTC SaaS, the right product would likely be GlobalTURN. Advantage would be a single TURN server that works globally, and a TURN server that improves the quality and latency of connections, often even better than direct peer to peer.
More info can be found: https://subspace.com/product/globalturn
Also Reddit AMA: https://old.reddit.com/r/IAmA/comments/qws3p9/i_am_william_k...
Join Me on Reddit for an AMA Session About the Internet
There are several major issues when trying to measure network paths in real time, but one in particular is how to avoid the assumption about path latency related to RTT/2. The latency of a path one direction rarely is half of the over all round trip latency, usually there are a combination of things that cause path asymmetry.
Anything you or others would like to know more about in terms of network performance measurements?
Gartner talks a bit about it here: https://blogs.gartner.com/avivah-litan/2021/04/25/think-you-...
There are some folks tracking the incorporation of non blockchain native assets into NFTs: https://www.coindesk.com/nfts-can-bring-the-real-world-on-ch...
Deep Dive: Why NFTs Are the Keys to the Inevitable Metaverse
Why Real-Time Internet Needs Precision Measurement
DDoS: How to Prevent and Mitigate Without Jeopardizing Latency
Why 5G Alone Does Not Solve Network Latency
Security is a pretty broad topic, with many risk vectors involved. Subspace is building new versions of existing protections that no longer include the latency cost, as well as providing new tools to Security Teams at game publishers to help better secure and protect the experience of the community.
I'm curious what game security topics come to mind for the HN community.
Which games did you have in mind? There is a non-zero chance that it's already enabled.
Why the Internet Today Isn’t Built for the Billions of Gamers of Tomorrow
DD for me was one of the final attempts to find something, anything, that could handle the requirements I was working with, because Spark, Flink, and others just couldn't reasonably get close to what I was looking for. The closest 2nd place was Apache Flink.
Over the last year I've read through the DD and TD codebases about 5-7 times fully. Even with that, I'm often in a position where I go back to my own applications to see how I had already solved a type of problem. I liken the project to taking someone use to NASCAR and dropping them into a Formula One vehicle. You've seen it work so much faster, and the tech and capabilities are clearly designed for so much more than you can make it do right now.
A few learning examples that I consider funny:
1. I had a graph that was on the order of about 1.2 trillion edges with about 90 million nodes. I was using serde derived structs for the edge and node structs(not simplified numerical types), which means I have to implement(or derive) a bunch of traits myself. I spent way more time than I'd like to admit trying to get .reduce() to work to remove 'surplus' edges that have already been processed from the graph to shrink the working dataset. Finally in frustration and reading through the DD codebase again, I 'rediscovered' .consolidate() which 'just worked' taking the 1.2 trillion edges down into the 300 million edges. For instance, some of the edge values I need to work with have histograms for the distributions, and some of the scoring of those histograms is custom. Not usually an issue, except having to figure out how to implement a bunch of the traits has been a significant hurdle.
2. I get to constantly dance between DD's runtime and trying to ergonomically connect the application into the tonic gRPC and tokio interfaces. Luckily I've found a nice pattern where I create my inter-thread communication constructs, then start up 2 rust threads, and start tokio based interfaces in one, and DD runtime and workers in the other. On bigger servers(packet.net has some great gen3 instances) I usually pin tokio to 2-8 cores, and leave the rest of the cores to DD.
3. Almost every new app I start, I run into the gotcha where I want to have a worker that runs only once 'globally' and it's usually the thread that I'd want to use to coordinate data ingestion. Super simple to just have a guard for if worker.index() == 0, but when deep in thought about an upcoming pipeline, it's often forgotten.
4. For diagnostics, there is: https://github.com/TimelyDataflow/diagnostics which has provided much needed insights when things have gotten complex. Usually it's been 'just enough' to point into the right direction, but only once was the output able to point exactly to the issue I was running into.
5. I have really high hopes for materialize.io That's really the type of system I'd want to use in 80% of the cases I'm using DD right now. I've been following them for about a year now, and the progress is incredible, but my use cases seem more likely to be supported in the 0.8->1.3 roadmap range.
6. I've wanted to have a way to express 'use no more than 250GB of ram' and have some way to get a compile time feedback that a fixed dataset won't be able to process the pipeline with that much resources. It'd be far better if the system could adjust its internal runtime approach in order to stay within the limits.
Wasn't intentional, but oops.
That's unfortunate. The system is suppose to prove his guilt, and assume innocence by default.