News for hackers. Stuff that matters.
Re:Avoiding Instruction Cache Misses by jandrewrogers on Saturday June 22, 2019 @01:01PM (#20250555)
This is, in a nutshell, why high-performance systems engineering is a rare skill set. It entails writing C++ (or whatever) with full understanding of the machine code that is likely to generate and how that machine code will interact with the incredibly complex internals of modern microarchitectures. It essentially requires de-abstracting two levels of abstraction below the programming language, which exist to reduce cognitive load, in your software design and implementation.

It is unfortunate that this is still so useful in practice, given what it implies about the magnitude of waste in typical software systems.

Re:Avoiding Instruction Cache Misses by toast0 on Saturday June 22, 2019 @02:55PM (#20251104)
That skill set is helpful, but you can go a long way with profiling tools to find what's your bottleneck is spending the most time on, and try to do less of that. And, in general, trying to work with the least amount of abstraction amd making sure the abstractions you use match the underlying reality as much as possible. Almost always, you can't add a layer to get performance, just like you can't usually add a layer to get code quality, and you can't usually add a layer to get security.

Obviously, abstractions are useful, but it can be very hard to dig out of wrong abstractions, when they've influenced the design of the whole project.

Re:Avoiding Instruction Cache Misses by d1zzy on Saturday June 22, 2019 @05:29PM (#20251813)
What's interesting is that with the death of Moore's Law there's a sudden increased interest in accelerators (software or hardware assisted), specialized pieces of software/hardware that can perform certain tasks really fast and then integrating these in a PC system to boost performance of certain common tasks. Thus I found myself having to spend a lot more time lately learning how cache affects the code I write, branch prediction, speculative execution, etc. It feels like starting out learning to program all over again.
Re:Avoiding Instruction Cache Misses by gumby on Saturday June 22, 2019 @04:13PM (#20251498)
One of the criticisms of the C++ standardization efforts is how many of the improvements are quitearcane. But those arcane features aren't necessarily for user code but to make library code efficient. Implementations of the standard library are often almost unreadable because of all the weird corner cases and performance optimizations they have to transparently handle.

This can go a long way to improving user code...but of course it's no magic bullet.

Re:Avoiding Instruction Cache Misses by adrianN on Sunday June 23, 2019 @01:56AM (#20253522)
I really dislike this mindset in the C++ community. The people writing libraries are normal programmers too. They're not some kind of supermen on whom you can load infinite complexity. And it's not like the STL is this amazing piece of software that mere mortals can't beat. Quite the contrary, any sufficiently large piece of software tends to replace the STL at least in parts with their own implementation.
Re:Avoiding Instruction Cache Misses by danbolt on Sunday June 23, 2019 @05:31PM (#20258799)
Herb Stutter mentioned once that an advantage of the STL is that it gets you good general-purpose performance initially, but tries to stay out of your way if you want to replace it for something more specialized.

I think you’re right about a codebase replacing parts of STL as necessary. One I see frequently is `std::array` being wrapped with iterators to be essentially a stack-allocated vector.

Re:Avoiding Instruction Cache Misses by gumby on Sunday June 23, 2019 @11:23AM (#20256149)
I don't see that that in any way contrasts with what I wrote. The same is true of the C standard library.
Re:Avoiding Instruction Cache Misses by MaxBarraclough on Sunday June 23, 2019 @02:28PM (#20257659)
Do you have an example of what you're thinking of?
Re:Avoiding Instruction Cache Misses by cbetti on Saturday June 22, 2019 @08:46PM (#20252583)
Are you talking about STL interfaces or implementations?
Re:Avoiding Instruction Cache Misses by gumby on Saturday June 22, 2019 @10:12PM (#20252864)
STL in particular, but it's hardly the only case.
Re:Avoiding Instruction Cache Misses by rmdashrfstar on Saturday June 22, 2019 @01:27PM (#20250689)
Any advice to learn systems engineering properly?
Re:Avoiding Instruction Cache Misses by jeffreygoesto on Saturday June 22, 2019 @01:57PM (#20250843)
Practice is a main ingredient. Build and debug (down to below assembler level with i.e. vtunes, learn about weak memory ordering, ...) increasingly complex systems. Learn how high level language code is transformed into "what the hardware shall do" to be able to predict, to some degree, whether that particular code you review will be optimized well or not (then measure and confirm/extend/correct your prediction). Working with FPGAs helped me learning a lot about how to get the right data to the right execution unit at the right point in time. Solving this "space-time-problem" is exactly what optimizing is about in CPUs as well. For the engineering part I'd say: being strict about separation of concerns and single source principles and not too religious with abstractions certainly helps. And finally: read, read, read. Books, Papers, other people's code...
Re:Avoiding Instruction Cache Misses by jeffreygoesto on Saturday June 22, 2019 @02:04PM (#20250876)
Oh, and one more thing (tm): try to be the worst, not the best team member. Make sure you always can learn from peers. You need discussions, especially those wandering up and down the abstraction stack, asking "why" on every level.
Re:Avoiding Instruction Cache Misses by loeg on Saturday June 22, 2019 @01:43PM (#20250768)
Get a job where you do it some or full time, and acquire lots and lots of practice.
Re:Avoiding Instruction Cache Misses by jeffreygoesto on Saturday June 22, 2019 @01:58PM (#20250847)
+1 took me too long to type, otherwise I would have just quotey you ;-).