News for hackers. Stuff that matters.
Re:Original Age of Empires 2 dev talks about its usage of as... by malkia on Saturday January 06, 2024 @08:22PM (#38897406)
I worked on port of a Konami game from PSX to PC, that was 1999-2000 - the code (C) had lots of #ifdefs like where inline assembly was placed, and the original "C" code was kept. It seemed all done by one specific programmer, and what really saved us (in porting) the game was the originallly kept "C" code. My mips knowledge was never as good as x86.

So yes, it was the norm back then. My second job (1998), was working on a team that was going to do some software for Intel for the upcoming back then Katmai processor (Pentium I was it). It had all the new fancy SIMD instructions. The software was supposed to be something like media composer - you slap images, rotate them, etc all in realtime using software rendering (GPUs were still relatively expensive).

I wrote a bilinear and bicubic texture mapper with marching squares for transparent areas. It was all in assembly, and I spent lots of time optimizing it. Back then we used Intel's VTune, and it was super-precise (for the processors back then) - how they are going to pipeline, how much (supposedly) instructions cycles would take, waits, etc. That helped a lot!

But the real lesson was, that me and my manager - both claiming to be super good at assembly (after our recent achievements), rewrote the sphere mapping code for a game another team was writting in assembly, but alas our assembly (but no katmai instructions) code was slower than what the compiler did ;) - TBH, if we did proper mipmapping and texture swizzling we would've fared both ways, but hey, demo coders were not always to be found so they had to rely on regular programmers like us!

flipcode keeps a lot of good articles, with lots of good assembly for that - https://www.flipcode.com/archives/articles.shtml - there were even better materials from earlier years, but can't find them.

Turbo/Borland Pascal were so awesome, because they allowed for very easy of inline assembly use (somehow) than C/C++ - though you had to know which registers you can touch or not.

Re:Original Age of Empires 2 dev talks about its usage of as... by akoboldfrying on Saturday January 06, 2024 @09:33PM (#38897838)
It was always so disappointing, spending hours coding up a tightly wound assembly language version of some inner loop that uses half the instructions generated by the C++ compiler, only to find that your slaved-over version is actually 5% slower. But OTOH... the thrill when it actually was faster!

This was back in the Pentium 4 era, where there were deep pipelines and oddities like some simple ALU instructions (ADD, SUB, etc.) taking 0.5 cycles(!), while others (ADC, SHR) took 4 cycles IIRC.

Re:Original Age of Empires 2 dev talks about its usage of as... by chii on Saturday January 06, 2024 @11:22PM (#38898403)
> uses half the instructions generated by the C++ compiler

is there a tool that could profile/predict ahead of time, so that one does not attempt to hand write assembly before knowing for sure it will beat the compiled version?

Re:Original Age of Empires 2 dev talks about its usage of as... by adrianN on Sunday January 07, 2024 @01:16AM (#38898969)
For small pieces of code I would try to use a superoptimizer like souper.
Re:Original Age of Empires 2 dev talks about its usage of as... by a1o on Sunday January 07, 2024 @06:05AM (#38900212)
https://github.com/google/souper

It looks like it only supports Linux and macOS - no Windows, but no other things too like mobile.

It seems it exists for ten years, I wonder what optimizations aren't still picked by the recent compilers.

Re:Original Age of Empires 2 dev talks about its usage of as... by adrianN on Sunday January 07, 2024 @06:16AM (#38900277)
Compilers need to balance compilation speed with optimization. SMT solvers are right out for speed reasons.
Re:Original Age of Empires 2 dev talks about its usage of as... by akoboldfrying on Sunday January 07, 2024 @05:32AM (#38900066)
There was Intel VTune, which I heard was good, though I haven't used it myself. One difficulty is that there are many non-obvious and hard-to-predict factors that interact to produce pipeline stalls. Instructions had specified throughputs and latencies (throughout being the number of cycles before another independent instruction of that type could be initiated; latency being the number of cycles before its output could be used by another instruction), but that was only part of the story. Was that memory read from L1 cache? L2? Main memory? Is this conditional branch predictable? Which of the several applicable execution units will this micro-op get sent to? There were also occasional performance cliffs (alternating memory reads that were exactly some particular power of 2 apart would alias in the cache, leading to worst-case cache behaviour; tight loops that did not begin on a 16-byte boundary would confuse the instruction prefetcher on some CPUs...)

I may be getting x86 CPU generations mixed up. But having wrestled with all this, I can certainly see the appeal of hand-optimising for older, simpler CPUs like the 6510 used in the C64, where things were a lot more deterministic.

Re:Original Age of Empires 2 dev talks about its usage of as... by formerly_proven on Sunday January 07, 2024 @07:25AM (#38900668)
VTune still exists and is free since a few years. Neat thing with VTune is that it has support for a few runtimes, so it understands for example CPython internals to the point that stack-traces can be a mixture of languages. That's something only becoming available just now outside of VTune, like Python 3.12 has some hooks for Linux perf.

A purely pen-and-paper tool was IACA; you simply inserted some macros around a bit of code in the binary and IACA simulated how these would/could be scheduled on a given core: https://stackoverflow.com/questions/26021337/what-is-iaca-an...

Re:Original Age of Empires 2 dev talks about its usage of as... by akoboldfrying on Sunday January 07, 2024 @08:02AM (#38900856)
Didn't know VTune is free now, nor had I ever heard of IACA which looks very nice (and would have saved me a lot of brow-sweat)! Thanks.
Re:Original Age of Empires 2 dev talks about its usage of as... by LegionMammal978 on Sunday January 07, 2024 @12:03PM (#38902886)
Note that there's also the open-source uiCA [0], which similarly predicts scheduling and overall throughput for a basic block. Their benchmarks claim it to be more accurate than IACA and other tools for newer Intel CPUs, but I wouldn't be qualified to judge those claims.

[0] https://uops.info/uiCA.html

Re:Original Age of Empires 2 dev talks about its usage of as... by malkia on Sunday January 07, 2024 @01:57PM (#38904044)
Awesome - I'll try it again!
Re:Original Age of Empires 2 dev talks about its usage of as... by malkia on Sunday January 07, 2024 @01:57PM (#38904039)
Yup. Back in the day VTune was useful and good, then I haven't used it for more than 20 years. It might be still good, but knowing how much more complicated the current CPU architecture is, and how much I've lost touch with low-level assembly I don't know if it's going to be useful to me. I'm relying now on profiling, and other programmers that have become way better in this than me to hear their opinion, and use that as basis (or others on the web).

Most of the time, some good optimized library would do pretty good.

Re:Original Age of Empires 2 dev talks about its usage of as... by JonChesterfield on Sunday January 07, 2024 @07:13AM (#38900589)
This varies from trivial to very hard to mostly data dependent with different architectures. llvm-mca might be of interest.

One should be able to do a best-case calculation, mostly assuming caches hit and branch prediction gets the answer right. Register renaming manages to stay out of the way.

Getting more dubious, there is a statistical representation of program performance on unknown (or partially known) data. One might be able to estimate that usefully, though I haven't seen it done.

Re:Original Age of Empires 2 dev talks about its usage of as... by whaleofatw2022 on Monday January 08, 2024 @11:46AM (#38914657)
P4 ALU (or at least significant parts of it) was double pumped... so parts of a 2GHz P4 were actually running as fast as 4Ghz even back then...

Shifts were bad however, I think they left out the barrel shifter for some ungodly reason.

Re:Original Age of Empires 2 dev talks about its usage of as... by qingcharles on Monday January 08, 2024 @04:51PM (#38918703)
This! I would spend hours unwrapping loops etc, optimizing register use and then profile my amazing x86 version only to find the compiler had found a better route. But you're right -- on the times when you code came out 2X, 5X, 10X quicker than the compiler.. that was what you lived for :)
Re:Original Age of Empires 2 dev talks about its usage of as... by philistine on Saturday January 06, 2024 @09:56PM (#38897987)
You worked on the port of Metal Gear Solid?! OH MY GOD THIS IS SO COOL!

Edit: Looking it up you might have worked on a larger amount of games than I thought. Very awesome nonetheless. https://www.mobygames.com/game/company:99/from:1998/platform...

Re:Original Age of Empires 2 dev talks about its usage of as... by malkia on Sunday January 07, 2024 @12:52AM (#38898858)
Thanks!!! Yes it was MGS - the deal was with our studio to do the port from PSX -> PC and since Microsoft was the publisher, part of the deal was for Konami to port Age Of Empires (1? 2?) to PSX.
Re:Original Age of Empires 2 dev talks about its usage of as... by easyThrowaway on Sunday January 07, 2024 @07:39AM (#38900751)
It was Age Of Empires 2, I had a EU copy. I bet there are some interesting stories regarding that port, too.

AFAIK the user interface for the win9x version was drawn using native GDI windows library, and yet the PS2 version, using a completely different rendering framework and architecture, sported the very same appearance, font glitches and all. I wonder if they actually wrote an emulation layer around that.

Re:Original Age of Empires 2 dev talks about its usage of as... by markus92 on Sunday January 07, 2024 @09:23AM (#38901402)
The AoE2 PS2 version actually had half-decent USB keyboard and mouse support. Back then USB keyboard/mice were very uncommon (it was all PS/2) but when we tried it, it actually worked.
Re:Original Age of Empires 2 dev talks about its usage of as... by dgfitz on Sunday January 07, 2024 @11:35PM (#38908317)
You’re also my hero. I remember being like… 13 and seeing a copy of MG:S at the store… for PC! I will freely admit it, I hid the copy behind the most boring games on the shelf, and came back and bought it a week later, it was still safely hidden. I distinctly remember the clerk that I paid was quite irritated, almost as if they couldn’t find where I hid it so they could buy it themselves.

In any event, that game, and that port, were a part of my childhood that I will always remember.

Re:Original Age of Empires 2 dev talks about its usage of as... by zeusk on Saturday January 06, 2024 @08:49PM (#38897559)
Are there any good resources on using mipmapping and swizzling effectively?
Re:Original Age of Empires 2 dev talks about its usage of as... by malkia on Saturday January 06, 2024 @09:21PM (#38897767)
I was actually trying to find it - there were lots of .txt files published back then - and there was one about texture mapping from 1992... 1994? - and it explained swizzling and why it was efficient with caches.
Re:Original Age of Empires 2 dev talks about its usage of as... by malkia on Saturday January 06, 2024 @09:38PM (#38897863)
Now I remember - back then these texture mappers were called roto zoomers, Alex Champandard did excellent series here ->

https://www.flipcode.com/archives/The_Art_of_Demomaking-Issu... - look for zip file with the source code.

I've also found Go version (and there were 2 java ones)

https://github.com/search?q=BlockTextureScreen

So it's not as neat as swizzling (quickly looking at it) - but essentially same goal - keep pixels that have to be drawn together at close (e.g. blocks). Mipmapping helps too, as then you don't "skip" too many pixels - and you both gain better quality and less cache misses.

Re:Original Age of Empires 2 dev talks about its usage of as... by royjacobs on Sunday January 07, 2024 @05:22AM (#38900031)
You might be thinking of the article written by Pascal of Cubic Team. I think the archive is called pasroto.zip and it explained why rotozoomer performance would tank when the texture was rotated by 90 degrees (since you're stepping in the Y direction and blowing out your cache for every pixel). Really interesting at the time.
Re:Original Age of Empires 2 dev talks about its usage of as... by bananaboy on Sunday January 07, 2024 @07:31AM (#38900699)
fatmap.txt discusses tiled textures a bit https://hornet.org/code/3d/trifill/texmap/fatmap.txt
Re:Original Age of Empires 2 dev talks about its usage of as... by malkia on Sunday January 07, 2024 @02:00PM (#38904078)
I think that's what I was looking at, and read back then - it was only txt files :)
Re:Original Age of Empires 2 dev talks about its usage of as... by bananaboy on Sunday January 07, 2024 @04:58PM (#38905586)
Oh I just realised who I’m talking to haha! I’m Sam from shg!
Re:Original Age of Empires 2 dev talks about its usage of as... by malkia on Sunday January 07, 2024 @08:00PM (#38906967)
Holy!!!! hahahaha - cheers!
Re:Original Age of Empires 2 dev talks about its usage of as... by takiyo on Saturday January 06, 2024 @09:03PM (#38897650)
This was a nice read, thanks for writing.
Re:Original Age of Empires 2 dev talks about its usage of as... by pjmlp on Sunday January 07, 2024 @03:58AM (#38899692)
Another Flipcode user!

I missed that other game forums of similar vintage are now gone.

The way to being able to do Assembly in Borland products, and to lesser extent on VC++, was great.

I always feel like vomiting when looking at the mess UNIX compilers have come up for it, instead of inferring how registers are being used, and the whole string concatenation stuff.

Re:Original Age of Empires 2 dev talks about its usage of as... by bananaboy on Sunday January 07, 2024 @04:57PM (#38905566)
It’s cool that gamedev.net is still around but I really miss flipcode!
Re:Original Age of Empires 2 dev talks about its usage of as... by whall6 on Sunday January 07, 2024 @12:51PM (#38903332)
> mipmapping and texture swizzling

Imagine reading this as a layman

Re:Original Age of Empires 2 dev talks about its usage of as... by malkia on Sunday January 07, 2024 @01:59PM (#38904065)
well I've now realized I've used the terms I now know, and looking at what I've read back then - it was roto zoomers (lol) and it looks like it was the "block" texture way instead of what's considered now the term - swizzling - https://www.khronos.org/opengl/wiki/Texture#Swizzle_mask
Re:Original Age of Empires 2 dev talks about its usage of as... by Ericson2314 on Monday January 08, 2024 @12:34AM (#38908673)
Before I really knew how to program, I tried to port http://advsys.net/ken/voxlap.htm to Linux: https://github.com/Ericson2314/voxlap

It was written in the same C / inline assembly style. Unfortunately seemed to me like the final ASM didn't quite do the same thing as the original C.

Re:Original Age of Empires 2 dev talks about its usage of as... by muro on Monday January 08, 2024 @06:50AM (#38910719)
I think Katmai was Pentium III
Re:Original Age of Empires 2 dev talks about its usage of as... by malkia on Monday January 08, 2024 @10:15PM (#38921825)
Oh righ! I messed it up, Pentiums were much earlier. Thanks for the correction!
Re:Original Age of Empires 2 dev talks about its usage of as... by fennecbutt on Tuesday January 09, 2024 @11:36AM (#38928137)
Isn't that a pretty common tactic? Write everything in C and then at some point in the dev cycle start identifying functions/features that can be asm'd for performance reasons?