News for hackers. Stuff that matters.
Re:A 100LOC C impl of memset, that is faster than glibc's by gpderetta on Friday November 12, 2021 @07:21AM (#29198608)
What's the use of filling your ram with zeros when the data needs to be on L1, L2 or L3? Unless you are memsetting hundreds of MBs of memory, memset/memcpy in practice need to be handled by the cpu or something very close to it.

Zen has CLZERO which can clear a cacheline in one go, but not sure how good it is.

Re:A 100LOC C impl of memset, that is faster than glibc's by vlovich123 on Friday November 12, 2021 @07:35AM (#29198690)
This would be a CPU command that works with the RAM controller rather than something you control yourself (kernels to my knowledge don’t talk directly to the controller beyond maybe some basic power management, if that).

There is a definite need to do hundreds of MB - the Linux kernel has a background thread that does nothing but zero out pages. What do you think happens to the GBs of RAM freed by closing Chrome? Once it’s made available in one spot, no reason others could use it (eg a hardened malloc implementation, etc).

Re:A 100LOC C impl of memset, that is faster than glibc's by monocasa on Friday November 12, 2021 @03:30PM (#29204171)
If you find yourself doing this a lot, there's write combining memory to coalesce writes to be more friendly to the RAM controller.

Additionally, CLZERO ends up doing very similar work since the resulting cache flush os seen by the RAM controller as a block write.

Re:A 100LOC C impl of memset, that is faster than glibc's by gpderetta on Friday November 12, 2021 @07:38AM (#29198709)
Interesting that you mention linux, because Linus has very very strong opinions about this :)
Re:A 100LOC C impl of memset, that is faster than glibc's by xvmt on Friday November 12, 2021 @08:08AM (#29198904)
What is his opinion about this?
Re:A 100LOC C impl of memset, that is faster than glibc's by gpderetta on Friday November 12, 2021 @08:46AM (#29199180)
He strongly believes that something like rep stos/rep mov is the right interface for memset/memcpy and off-core accelerators (like DMA) are misguided.
Re:A 100LOC C impl of memset, that is faster than glibc's by Something1234 on Friday November 12, 2021 @03:10PM (#29203982)
His reasoning or rant for this?
Re:A 100LOC C impl of memset, that is faster than glibc's by monocasa on Friday November 12, 2021 @03:34PM (#29204210)
I'm not sure about Linus's objections, but I've found that DMA accelerators for time sharing systems with general workloads haven't reaped benefits, as the overhead of multiplexing and synchronizing with them kills most of their benefits. At that point it's easier to blit memory yourself.
Re:A 100LOC C impl of memset, that is faster than glibc's by kwertyoowiyop on Friday November 12, 2021 @08:09AM (#29198909)
What are they?
Re:A 100LOC C impl of memset, that is faster than glibc's by rasz on Friday November 12, 2021 @08:14AM (#29198945)
Such ram capability would result in implementing hybrid compressed caches. Why waste whole cache line for storing zeroes when you can have dedicated compressed representation.

On a similar note part of ATI 2000 https://en.wikipedia.org/wiki/HyperZ was fast Z clear, today a norm on every GPU.

Re:A 100LOC C impl of memset, that is faster than glibc's by convolvatron on Friday November 12, 2021 @01:44PM (#29203016)
PPC has an instruction to 'load' a line ignoring its previous contents (just set up the cache state). useful in any case when you know you're going to overwrite the whole thing.
Re:A 100LOC C impl of memset, that is faster than glibc's by xgkickt on Friday November 12, 2021 @05:50PM (#29205440)
I used dcbz extensively back on the Wii.
Re:A 100LOC C impl of memset, that is faster than glibc's by FpUser on Friday November 12, 2021 @08:07AM (#29198899)
>"Unless you are memsetting hundreds of MBs of memory"

Not hundreds but in one of my apps I do have 10th MB of continuous cache that has to be zeroed before use / reuse.

Re:A 100LOC C impl of memset, that is faster than glibc's by knome on Friday November 12, 2021 @01:17PM (#29202698)
I wonder if someone was zeroing enough memory, where the memory is a private anonymous mapping, they might use madvise() with MADV_DONTNEED, which in linux will effectively zero the pages.

It returns the memory to the OS, and will pagefault on later accesses remapping them as zero-filled. It works in pages. Sizes smaller than a page result in whatever else is in the same page getting nuked.

If you don't immediately reuse the whole cache, it might spread out the zeroing/remapping over time, rather than in a single large go. Imagine some testing would be in order to see if a syscall + mapping changes ( require reloading with TLB for the process ? ) would be smaller than a straight run of writing zeros at some point.

IIRC, the zeroing is not something you can expect from non-linux madvise implementations.