Apologies for not making this clear, I was definitely assuming a LIFO buffer would drop the oldest message when full, since dropping the newest message has the issue you describe. In that case, for a prolonged period of ingress slightly over egress, the buffer will contain the N most recently received messages that haven't been transmitted, which seems optimal, but _also_ will have no queueing-induced latency for new messages. If there's a gap in the incoming data stream, a LIFO buffer would emit the most-recent (and most likely to be useful) data first. Most importantly, there's now no performance drawback for making the buffer bigger, so tuning it is way less dependent on your expected operating environment.

One other optimization that would probably make sense in a practical implementation is a "max age" limit on things popped from the stack, so that an old message can't sit around for several seconds if the ingress rate happens to exactly matches egress. This limit can be fairly long though (~1s is fine), since it's just trying to approximate when a higher-level protocol would already have completed a full roundtrip and requested retransmit.

Head drop, so long as it preserves a "round" so a malignant sender cannot force all other traffic out of the queue, is great. This is what fq-codel, fq-pie, and cake do.

Can you clarify what you mean by a "round" here, please? Do you mean a complete round-robin pass of clients (or whatever segmentation method is used) in the fair queue?

The "sparse" and "rounds" concepts also works really well as a per station wifi scheduler, where it is now the default for a plethora of chipsets in linux. Some plots of which I will forever be proud here:

https://arxiv.org/pdf/1703.00064.pdf

Before/After on an ath10k chip here:

https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/590...

Close. A major innovation in the fq-codel derived algorithms over former forms of FQ like DRR and SFQ is what we call the sparse flow optimization. Packets from flows that have an arrival rate lower than the departure rate of 1 quantums worth of packets from all other flows (a "round") observe no queueing, where in drr or sfq, new flows always go to the back of the fq'd flows. (still a huge win over fifo or pure aqm)

This among other things made codel's head drop aqm safe and stable enough to deploy.

Paper: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8469111

from: https://datatracker.ietf.org/doc/html/rfc8290

The step that moves an empty queue from the list of new queues to the end of the list of old queues before it is removed is crucial to prevent starvation. Otherwise, the queue could reappear (the next time a packet arrives for it) before the list of old queues is visited; this can go on indefinitely, even with a small number of active flows, if the flow providing packets to the queue in question transmits at just the right rate. This is prevented by first moving the queue to the end of the list of old queues, forcing the scheduler to service all old queues before the empty queue is removed and thus preventing starvation.

   The resulting migration of queues between the different states is
   summarised in the state diagram shown in Figure 1.  Note that both
   the new and old queue states can additionally have arrival and
   dequeue events that do not change the state; these are omitted in the
   figure.

   +-----------------+                +------------------+
   |                 |     Empty      |                  |
   |     Empty       |<---------------+       Old        +----+
   |                 |                |                  |    |
   +-------+---------+                +------------------+    |
           |                             ^            ^       |Credits
           |Arrival                      |            |       |Exhausted
           v                             |            |       |
   +-----------------+                   |            |       |
   |                 |      Empty or     |            |       |
   |      New        +-------------------+            +-------+
   |                 | Credits Exhausted
   +-----------------+

   Figure 1: Partial State Diagram for Queues between Different States