maxhou

I tried switching a buildroot based CI server to the 26.04

One Makefile statement triggered a bug in rust ln:

  src/%:
      @ln -sfn $(DIR)/foo src

In parallel build, we would get random failure:

  ln: Already exists

Rewrote that Makefile to work-around it, and ended up with the same kind of bugs with parallel $(INSTALL) -D ...

Tried latest ubuntu 26.10 which supposedly fixes a lot of TOCTOU races in rust coreutils, but no better.

Gave up and switched back to original coreutils.

implementation of continue from one luajit fork:

https://github.com/zewt/LuaJIT/commit/c0e38bacba15d0259c3b77...

Yes they do.

A read() syscall takes longer than a getpid() syscall because read() has more work to do, it actually does a data copy of len bytes, which takes some time (and will be faster/slower if data is cache hot)

What we call the "syscall overhead" is what happens before and after the actual data copy, switching between user and kernel mode.

You make that overhead negligible by calling read() with a large size.

How do you know that the dataset fits in L2 ?

Assuming pv uses splice(), there is one only copy in the workload: copy_from_user() from fixed source buffer to some kernel allocated page, then those pages are spliced to /dev/null.

If the pages are not "recycled" (through LRU scheme for allocation), the destination changes every time and the L2 cache is constantly trashed.

yes and pv processes are not scheduled on the same CPU core, so different L2 cache.

Does the protocol implement any kind of negotiation (ciphers, ...) ? if not, how would you handle future type of attacks against the then hardwired constructions ?

I fully agreed that being in-kernel is the right choice for performance, but the chosen constructs excludes the possibility of using any type of existing crypto hardware accelerator that shines in the IPSEC use-case (cache cold data == no cache flush overhead, fully async processing of packets with DMA chaining). Time to start lobbying SOC vendors :)

> what ORM to use (or not to use an ORM)

do you have something to recommend ?

used Rose::DB in the past, then I discovered SQLAlchemy and it's difficult to look back...

> Suppose, for example, that you go to the Bank of America site to transfer some funds or pay a bill. As with Google, and as would happen with any other secure site, it turns out their certificate gets replaced with the Avast certificate. I doubt anyone needs me to lecture them on the potential security issues involved in having a third-party watching their banking transactions without permission!

Antivirus software runs with the highest level of privileges, divert system calls...

They could theoretically log everything you type on the keyboard, no need to MITM SSL connections

> Avast is replacing certificates with its own without bothering to check the validity of those certificates!

This is a far bigger issue

It won't help. To serve its purpose, any AQM (active queue management) like fqcodel must be done at the congestion point.

The end-user computer has a faster link than its internet connection upload speed, typically a gig ethernet whereas a xDSL upload speed is a few megabits/s.

If you transfer data to a random website (upload) from the computer, packets will accumulate inside router/modem tx queue, because this is the slowest link between the two hops. This is where it's important to have an AQM running to reorder/drop packets in that queue.

from the article:

> Each core needs to generate a few thousand data packets per second, because Ethernet packets typically contain up to 1500 bytes. This gives the CPU around 100 microseconds to process each packet.

No it doesn't, not when using TCP Segmentation Offload (TSO)

This only works for a particular use-case: sending static data using TCP, but this is the most common use-case since a typical "video streaming server" is actually a simple HTTP server that serves static MP4/MPEG-TS data.

for each connected client this is what happens - nginx/apache does sendfile(file, sock, off, <large_number>) - kernel issue large (> 10kB) DMA read to the file storage backend into a set of memory pages and wait for completion - kernel allocates/clone a small IP/TCP header (40 bytes) - kernel gives that small header + set of memory pages to network card, which will segment and create those 1500 bytes packets and send them on wire

if you have a lot of RAM, the read from storage could even be skipped because the previously read data pages are kept in the page-cache with a LRU approach. (help if clients are requesting the same file).

you can easily saturate a 10G link with spare CPU cycles on cheap hardware with that approach, no need to bypass anything.

You forgot the cost of memory access.

The L3 layer checksum is useless because IP packet is small and the kernel has to read/write all the fields anyway.

The L4 checksum covers TCP/UDP packet data, which the kernel can avoid touching if necessary.

When a TCP sender uses sendfile(), the kernel does a DMA read from storage to a page if the data is not already in memory (in the so called page cache), and just ask the network card to send this page, prepended with a ETH/IP/TCP header. That only works if the NIC can checksum the TCP packet content and update the header.

If the network card can do TCP segmentation offload, the kernel does not have to repeat this operation for each 1500 bytes packets, it can fetch a large amount of data from disk, and the NIC will split the data in smaller packets by itself.

simple checksum computation and/or verification, indeed most cards can do (sometimes with restrictions: not for IPv6, not for VLAN...)

the other kind of offloading that the kernel can use is TCP Segmentation Offload (TSO), which is much more complex to implement in hardware, and you won't find it on cheap NIC (like Realtek)

I'd have expected a small easteregg for the word "bro" ;)

The most obvious is that they have multiple cores, and it's easy to completely disable a non functional one.

Now can Intel be more granular than the core level, like running a core with some defect ALU, I really don't know.

What's publicly known from the binning process is that it involves disabling core, reducing total cache size, and finding the maximum working frequency.

The bottom line is that it requires more effort to deal with defects in complex logic, for DRAM they would reduce the total memory size.

> And if one atom is in the wrong place, you have to throw away the chip

this is certainly the case for CPU

but DRAM & NAND ? this is the typical case of designs where you can add redundancy to accomodate for manufacturing defects.

if you use mmap(), there is virtualy no difference between the streaming/non streaming parsing code.

In the real world, the slow part of "parsing" a CSV file is IO: reading the file content from disk to memory, and from memory to CPU cache.

You would avoid reading the file content more than once if you had to parse it.

> The first line counts the number of lines in a buffer (assuming that file is read into memory and copied to gpu buffer d_readbuff).

but this is what is done here, first search to find all \n, then multi-core GPU stuff for each line content.

Except parsing all the Android.mk and computing the dependencies is slow as hell.

Try running make at the top of an AOSP tree, it takes about 60 seconds, and that part cannot be parallelized.

If you are looking for alternatives, have a look at http://martine.github.io/ninja/

OP says the system was swapping.

Once your hash table and its linked lists have been swapped to disk, each pointer dereference causes a random disk access, which are orders of magnitude slower than RAM.

The problem is that all NAT related issues are not because of NAT itself, but because of the required stateful firewalling for NAT.

To be able to NAT and un-NAT, you first need to classify traffic (that NEW, ESTABLISHED, RELATED,... stuff in Linux netfilter), changing the destination or source ip addresses is only the second part of that process.

Protocols don't break only because of NAT, but mostly because of stateful firewalling, you'll face the same problems with IPv6 if you enable it.

Or maybe the opposite ?

If you charged Netflix, they would consider investing more money on trying to reduce the bandwidth they use: - innovative video codec - better encoders - new streaming technology

ENOSPC ?

actually since none of its tests ask to sync the data onto the disk, he might just be measuring each method efficiency in creating dirty pages.

of course that depends on the amount of RAM the system has, and how the kernel VM parameters are tuned (sysctl vm.dirty_*)

just add a fdatasync() call and you will take into account the time it takes to flush all dirty pages into the disk.

> My theory would be the cost in terms of space and electricity to generate 10Gbe of video streams has dropped orders of magnitude

Spot on.

You can stream video with basically nginx serving static files, you just need a 10G network card with TSO.

the CPU power needed is an order of magnitude lower than what you need to run full stack server side frameworks.

Just make sure the content fits in your page cache (RAM is cheap), and you will fill the pipe.

In the upcoming month, you will be able to do this with an ARM SOC sitting on a 10cm^2 board, for 5W TDP.

How about asking for net neutrality on DDOS traffic ? I find it unfair that my ISP is shaping it ! :)

True, but people claiming: "I pay for XXX Mbit/s, I want my XXX Mbit/s 100% of time to 100% of the Internet" think/want the Internet to operate like that.

> Having enough capacity to meet the streaming demand of your current customers during peak times is a solvable problem that doesn't require a full gigabit for everybody, all the time.

With adaptive streaming technology, there is not such thing as "today streaming demand"

If whenever you raise the pipe size, it is automatically filled by an higher resolution/quality/bitrate stream, it is a never ending game.

What I mean is that, whatever the state/quality of ISP networks is or could have been, we are doomed to face that "Netflix problem". If networks were better, then Netflix would already offer 4k or dual/triple HD streams and so there would be saturation anyway.

> (Ironically, and despite its protestations, Netflix's ability to pay this rent is a barrier to entry for its own future competitors.)

Actually, moving all Netflix traffic to private pipes will free the shared ones, so a new freeloader can use them and cut Netflix prices ;)

at least my ISP term of sales is clear about it, the guaranteed bandwidth is close to 0.

I do agree though that nowhere in the commercials it is said that you are paying for a peak rate.

> As a cable subscriber, I expect that when paying for N Mbps of bandwidth, I'm entitled to N Mbps of bandwidth of the content of my choosing

100% of the time ? not going to happen

So let's say I'm an ISP and I bring a fiber to your home, and gives you a gigabit ethernet port. You cannot expect all current and future customers to be able to use 1 Gbit/s at the same time. You would need a big non blocking switch with as many ports as subscribers, the technology for this does not exist once you reach a large customer base.

Now do you want me to shape your link to 0.1Mbit/s, because that's the only thing I can guarantee if all customers uses their link at the same time ? Or you'd rather have the 1Gbit/s possible bandwidth ?

Which one is better:

1) guaranteed 0.1Mbit/s for 10$, 5Mbit/s for 100$, 1Gbit/s for 4000$ 2) possible 1Gbit/s for 20$ ?

If you take the globalized approach, you cannot have business like Netflix, they destabilize the equation.