Re:How Netflix uses eBPF flow logs at scale for network insight
by chrissnell on Tuesday June 08, 2021 @11:51AM
(#27436768)
We had a poor man's version of this in 2006 at Backcountry.com. We ran OpenBSD firewalls on the edge and used pfflowd to translate the pfsync messages (the protocol used to synchronize two PF firewalls in a HA configuration) into Netflow datagrams, which we could then monitor in real time with top(1)-like tools. It was awesome. I miss having that kind of visibility.
[ Parent
| on HN ]
Re:How Netflix uses eBPF flow logs at scale for network insight
by eb0la on Tuesday June 08, 2021 @03:46PM
(#27439870)
Netflow! My first "big data" project was getting netflow data from 100s of routers in a internet backbone and start to learn where traffic came and if it was worthy to do a peereng agreement with them.
It stopped working the day Google activated their proxies on mobile networks. I saw Google traffic increase 1-2% week-over-week during several months. I quitted the job when it was >30% of the backbone traffic and don't know what happened at the end...
Re:How Netflix uses eBPF flow logs at scale for network insight
by flotwig on Wednesday June 09, 2021 @01:11AM
(#27444438)
> It stopped working the day Google activated their proxies on mobile networks.
What is this referring to? You've piqued my interest.
Re:How Netflix uses eBPF flow logs at scale for network insight
by eb0la on Wednesday June 09, 2021 @06:55AM
(#27446479)
When you opened a web using Chrome on your mobile phone, you could send the request to google directly.
They had a lot of caching proxies and it made browsing faster.
For us the problem was everybody was using that chache instead of going directly to the end host. And it showed like google traffic was growing _a_lot_ week over week and we weren't able to tell where people was going to optimize the network.