A good portion of this stuff does still go on in Games Development.

I don't work in games any more, but on the last title I worked on (Forza Horizon, Xbox 360), one of my colleagues engaged in a very similar exercise in order to allow data for the open world to be streamed in quick enough to deal with car (travelling at potentially 150+mph) to drive through the world without having to wait for loads, whilst streaming direct from the DVD (we weren't allowed to install to the HDD).

Given that the world was open and you could drive in pretty much any direction, trying to pre-load the right tiles of the world was difficult, and seek times made it tough to bring stuff in if it wasn't all packed together. However we were almost at the limit of DVD capacity so we couldn't do much duplication of assets to lower the amount of seeking required.

My colleague wrote a system that took about 8 hours overnight (running on a grid) to compute a best attempt at some kind of optimized packing. It did work though!

I was amazed myself that you can drive around at high speed through a busy city in GTA 3 on the PS3 without any noticable stutter and hardly any popups - good work there, too.

Very creative. I love this kind of stories. The first interactive display, made by Sutherland IIRC, had no framebuffer (irrealistic price at that time) so the team had to stream everything to fit into the small buffer they could get. As I understand it, everything was lazy and in sync with the screen 'machinery' so there was no delay even with limited memory capacity. It felt like hardware haskell.

Some day I will write a blog post about the tricks we have to pull off to fit all code & assets - or get animations running at 30FPS - on Pebble.

Trust me when I say this: low level development is alive and well at hardware companies.

Why wait?

Blog posts serve as great marketing these days, from showing off your engineering team's technical prowess, to recruiting others who hope to participate in some really advanced problem solving.

The press also helps maintain your existence in people's minds. With Google entering the market with Android Wear, and of course Apple's entry; a couple of 'this is how cool a Pebble watch is inside' would do well to not let us forget you exist!

Yup, you see quite a few people from game dev make the transition to the embedded space when they decide that working 100hr weeks isn't their idea of fun.

I talked to a Pebble firmware engineer at Bitcamp who told me about the hoops they had to go through. Really neat stuff.

Yeah sure, not saying it's non existent, just a lot less common. A blog post about that would certainly be very interesting :)

Please write this post. I love my pebble time (but really you guys need to fix some stuff in the new OS)

Subscribed! :)

We also have pretty amazing hardware today that is surprisingly underused most of the time.

I've seen many Unity projects with the worst code you could imagine still run close to 60 FPS when shipped.

Having an easy entry point means you're also going to get a lot of mediocre programmers using it. They seem productive in the first few weeks of the project but then quickly slow down halt once they start changing the design and end up with massive overtime hours while trying to debug and optimize the resulting mess.

Its made even worse with managers trying pseudo-agile-learned-in-a-two-day-masterclass adding even more procedures and overhead.

So yeah, you can make games today with less development talent than yesterday, it's still going to cost you more than having skilled software engineers and the resulting product will be a fraction of its potential.

> I've seen many Unity projects with the worst code you could imagine still run close to 60 FPS when shipped.

On the other hand, a lot of the worst Unity dreck is horribly unoptimized and runs incredibly poorly, despite graphical simplicity and no real visual effects. If you tool around on Steam or YouTube you can find tons of examples - look up, for instance, Jim Sterling's "Squirty Play" series. Not every bad game on there has issues, but many do.

[1] https://www.youtube.com/playlist?list=PLlRceUcRZcK0zAt8sV33Z...

There are some abstractions, like TCP sockets, that I don't think anyone would leave behind. But language abstractions like "classes" or even "functions" are certainly worth digging into. (The new lambda feature in Java 8 is an interesting case - you don't have to get into C to see the complexity there, but rather the bytecode, which is a kind of abstract sort of assembly.)

I'm not terribly familiar with the scene, but there are a variety of competitions for fun and art that operate within highly constrained environments. The demoscene[0], and computer golf[1], come to mind (although ironically computer golf is in some sense has to be very high level). There's also the security scene, which is quite bit-fiddly.

It's also the case that Go and particularly Rust are quite low-level system languages at their heart, so are presumably amenable to running in constrained environments.

[0] https://en.wikipedia.org/wiki/Demoscene

[1] E.g. Rhoscript http://rhoscript.com/

Can you suggest some explanation of the lambda feature in byte code?

Sure - read this gist from the bottom up: https://gist.github.com/javajosh/55339742aad2f1a1b881

Be sure also to check out Brian Goetz's excellent "Lambdas under the covers" talk, linked in the gist.

" the other is sad because you hardly use this low level stuff in todays software development world."

I don't think there is a dichtomy. It's always about smartly leveraging available resources. The problem with modern development perhaps is then that there are these tempting high level orthodoxies that often obscure the core matter at hand. Ie. focusing on some pointless candied abstract object interface rather than focusing design efforts on data flow and the datastructures and algorithms used.

The need for low level optimization has most definetly not vanished. When program data fits into local ram the bottleneck moves into cache optimization.

Please write this post. I love my pebble time (but really you guys need to fix some stuff in the new OS)

Uh, sorry, did you accidentally reply to wrong message?

appears so. sorry!

Just try to code for mobile or embedded platforms.

Yeah they might get bigger storage every year, but the reason why Google, Apple and Microsoft do talks about package size on their developer conferences is that size is number one reason for people to avoid installing apps, or to chose which one to remove.

Also given how the app life cycle works on mobile platforms, big apps are being killed all the time they go into background.

That's not really the same though. You are not really limited by RAM in most cases, also you are not limited by slow storage media like CD/DVDs as it was/is with most of the consoles. The reality is, hardly anyone on mobile writes their own engines (a part from some of the big players) because it's just too much of an investment and Unity (or one of the other popular engines) is well optimized and allows much faster development.

It's the same.

All iOS devices except the iPad Air 2 have less than 2GB RAM (most 512 MB). Android 1-4 devices have often less than 1GB RAM. It's common that only 1-3 apps can stay in RAM depending on the platform and the apps memory usage (foreground apps, not background services).

Applications/games in the Win95/PS1/N64 era were coded a lot more efficient. Back than, a common Win95a PC had 4-8MB RAM, (highend was 32MB).

Win95 machines with 4 MB RAM were exceptions, not the rule. It was very painful to use such machine, as it was swapping all time time, otherwise doing nothing.

Any realistic setup had 8 MB RAM or more.

Application at that time also didn't support i18n, didn't anti alias fonts, had low-res, low-color assets, that were enough at 320x200(240)/640x480 resolution.

Windows 95 was painful to use even with 8 MB. 12 MB was minimum amount that didn't cause it to swap all the time when actually doing something. 16 MB was nice.

A Pentium 133 with 8MB and 800x600 (32bit colors) run fine in with Win95a and several open applications. Try that with Android, even with 2GB RAM and quad core CPU - the Java based system on top of Linux is quite resource hungry. Flagship Android 5 phones have at least twice the hardware spec (twice as much RAM and CPU) of the iPhone 6 and are comparable in performance and user experience (latency) and not faster. That's the difference of Object-C vs. Java. And old applications like Microsoft Office are all coded in C/C++ and parts of older applications in Assembler.

One icon in Win95 had 512 bytes (32x32, 16 color, 4 bitplanes). One icon in Android has 256 kB (256x256, true color). The 800x600, 16-bit hicolor (that's what I used at the time) framebuffer had a bit under 940 kB. The 1920x1200 truecolor has 8,8 MB, not counting the texture backing stores used by modern display servers.

The amount of RAM needed has to do with assets used by the code, not the code itself. The code itself is miniscule.

And no, Android phones do not have 4 GB RAM. Low end has 512 MB, with many phones in 1-1,5 GB range and the 2015 flagships have 3 GB. (Nexus 5 a 7 have 2 GB. Nexus 6 has 3 GB). All that without swap (where would you like to swap? To flash?). While most modern 32-bit ARM CPUs do come with LPAE, Android does not support that, so going above 4 GB will have to wait for ARMv8.

Android doesn't support LPAE? That's pretty surprising, any sources about that? LPAE doesn't need any usermode support to function. What specifically does Android do to prevent using LPAE in underlying Linux kernel?

Well, one thing is what Linux kernel supports by itself, other is, what does the board support package for your chipset. So maybe there is LPAE Android device somewhere, where the SoC provider did bother, but in general, nobody does.

my android phone does :-) asus ze551ml

You phone is also Intel based, not ARM. That opens another question - would Intel be able to make phone SOCs, if Android SDK would compile to native ARM code, as some advocates prefer?

Java is not the problem, rather Google's shitty compilers.

Lots of industrial applications run embedded Java with a few KBs and acceptable performance for their use cases.

Android used the Oracle javac compiler until this year, so blaming anything on Google's new compilers is a little strange.

That comment shows how much you understand about compilers and Android.

javac has nothing to do with Dalvik or ART.

That shows how little you understand about the Android build tools. Dalvik and ART are not compilers.

Prior to this year, javac compiled the Java code to .class files and then dx translated the Java bytecode in the .class files into Dalvik bytecode in a .dex file, with some simple dedupe optimizations.

Only this year did the Android build system switch to Google's own compiler.

Go take a degree in computer science, learn about intermediate code representation, compiler frontened, compiler backend, CPU instructions, JIT compiler, AOT compiler, register selection.

Then make little drawings about which piece of Android is converting intermediate code representation into native CPU instructions.

For brownie points compare the quality of generated Asssembly code between Hotspot, Dalvik and ART for the same unmodified jar file.

Already done and wrote a non-optimizing lisp compiler and an optimizing toy compiler with common subexpression elimination and fancy register allocation.

I gather from your response that you've realized you were wrong about Android not using javac but were too proud to admit it. Don't worry, we can fix your pride problem with these tasks below:

1. Dalvik and ART don't take jar files as input, so it is impossible to get your brownie points. Learn why.

2. Oracle's Hotspot targets x86 and x86-64, and Dalvik and ART are mostly focused on ARM. Learn the difference between ISAs.

3. Hotspot and Dalvik make different tradeoffs between CPU and memory both in their choices of garbage collectors and in their JIT strategies. Think about why that would be.

4. The word "compiler" by itself refers to a program that translates source code into object code. Notably, an assembler is not usually considered to be a compiler, and JIT "compilers" were originally called dynamic translators for three decades, with JIT compiler only appearing in the 90s. Given that terminology background, figure out why most people would call javac a compiler but not Hotspot or Apple's Rosetta.

> Already done and wrote a non-optimizing lisp compiler and an optimizing toy compiler with common subexpression elimination and fancy register allocation.

And yet failed to grasp the difference between frontend, backend and intermediate execution format.

> I gather from your response that you've realized you were wrong about Android not using javac but were too proud to admit it. Don't worry, we can fix your pride problem with these tasks below:

I don't have to acknowledge anything. Anyone knows that javac does not execute code on the Android platform. As such talking about whatever influence it might have on runtime performance, besides peephole optimizations, constant folding and similar AOT optimizations only reveals ignorance about the Android stack.

> 1. Dalvik and ART don't take jar files as input, so it is impossible to get your brownie points. Learn why.

Yes they do. Jar files get converted into dex files, which means the same file can be used as canonical input for both platforms.

Then again we are learning about Android aren't we?

> 2. Oracle's Hotspot targets x86 and x86-64, and Dalvik and ART are mostly focused on ARM. Learn the difference between ISAs.

Maybe you are the one that should inform yourself about Oracle and certified partners Java JIT and AOT compilers for ARM platforms.

Learn about the Java eco-system.

> 3. Hotspot and Dalvik make different tradeoffs between CPU and memory both in their choices of garbage collectors and in their JIT strategies. Think about why that would be.

Of course they do different tradeoffs. The ones made by Dalvik and ART are worse than approaches taken by other Java vendors, hence why they generate worse code, which leads to bad performance.

Learn about commercial embedded JVMs.

>4. The word "compiler" by itself refers to a program that translates source code into object code. Notably, an assembler is not usually considered to be a compiler, and JIT "compilers" were originally called dynamic translators for three decades, with JIT compiler only appearing in the 90s. Given that terminology background, figure out why most people would call javac a compiler but not Hotspot or Apple's Rosetta.

Learn about Xerox PARC documentation and its references JIT compilers.

Or better yet feel free to dive into OS/400 documentation about its kernel level JIT compiler.

All of which go back a little earlier than the 90's

Interesting... Fast 486 (120mhz) with 12mb at 640x480 was swapping my so much fun could watch the logo screen rendering slowly down the screen

I remember 386DX (40 MHz) with 4 MB to be unusable at all (yes, it was possible to install Win95, but that's all) and Pentium 120 with 16 MB and S3 card running 800x600 hicolor to be great. In 1996.

I actually worked on a game that was running on the iPad2 (some gameplay https://www.youtube.com/watch?v=uaq0Sfp3_5Q ) and while you had to optimize quite a bit to achieve a certain number of drawcalls and memory usage it was still developed in Unity and far from what devs did in the ps1/2 era.

Except unlike smartphones, the PS1/N64 did not support VM or even had any storage to page to. So no, it's not the same.

iOS documentation:

"Instead, if the amount of free memory drops below a certain threshold, the system asks the running applications to free up memory voluntarily to make room for new data. Applications that fail to free up enough memory are terminated."

That has nothing todo with what op is referring to. Even on iOS you have a virtual memory system, like a normal PC does. If your app runs out of ram it will page to storage and the OS handles this for you. The PS1/N64 did not have any storage beyond RAM and also no virtual memory management so you had to write all the paging from CD/Cartridge to RAM yourself. Quite a difference.

Sort of. iOS does have a virtual memory system, but it's not as forgiving as a full OS would be; applications that fail to free up memory when asked to (i.e. in a low memory situation) are killed by the OS. See also: https://developer.apple.com/library/mac/documentation/Perfor...

That should be much easier to optimize against vs having no paging at all though. Are Apps running in the background asked to free up storage or be killed before the active app needs to do the same or are they referring to the active app ?

Background apps are even worse, as by default they are suspended.

If the system requires memory the ones with higher memory footprint are the first ones to go. They aren't asked nicely, just killed.

This might have changed on newer versions though. I am typing this from memory.

Also on the Watch there are also time constraints. How quick an app is allowed to execute.

Windows Phone also has similar constraints.

Embedded yes, but modern mobile is miles away from this. ~1GHz dual core, triple issue out-of-order CPU with 512MB-1GB RAM and maybe 10-20GB storage? That was a reasonable desktop PC not that many years ago.

Yet we have basic apps like the phone app stuttering on such "high-end" hardware.

Is your point something interesting or are you just snarkily pointing out that perfection has not yet been attained?

I guess the OP is referring to the code quality of said apps.

Not so sure about this. Most mobile games are developed with Unity. And what appear to be the cutting edge titles are usually using the Unreal engine. There's not much "to the metal" coding going on.

Besides, most mobile games are being played by the casual crowd. Games don't need graphics that push hardware limits to sell.

> Most mobile games are developed with Unity. And what appear to be the cutting edge titles are usually using the Unreal engine.

Any source by mobile OS that you can point to?

I am quite sure there are other contenders like home grown engines, LibGDX, Marmalade, Cocos (all variants), SDL, MonoGame, DirectXTK, Project Anarchy , Apple own Scenekit and SpriteKit,...

> Besides, most mobile games are being played by the casual crowd. Games don't need graphics that push hardware limits to sell.

Why do you think then all major OS vendors are teaching the devs how to reduce their packages sizes? I can happily post the links of such presentations, just need to hunt them down again.

Game logic + Assets + Engine

Something got to give if one is required to push the size down.

> I am quite sure there are other contenders like home grown engines, LibGDX, Marmalade, Cocos (all variants), SDL, MonoGame, DirectXTK, Project Anarchy , Apple own Scenekit and SpriteKit,...

It doesn't matter how many frameworks are out there. If you hang around the game dev scene long enough, you'll see that most small devs are using Unity, and if not that, Cocos2DX. Just head over to Gamasutra, /r/gamedev, or browse Steam & itch.io and see the # of cross platform mobile ports. Talk to devs, they are using Unity.

> Why do you think then all major OS vendors are teaching the devs how to reduce their packages sizes?

Reducing package size isn't really comparable to writing portions of your game in assembler. I don't consider that low level coding or pushing hardware limits.

> It doesn't matter how many frameworks are out there. ....

Former IGDA member, Gamasutra subscriber and GDCE attendee here, hence why I asked for numbers.

> Reducing package size isn't really comparable to writing portions of your game in assembler. I don't consider that low level coding or pushing hardware limits.

It is not, but the goal of fitting as much code as possible in small packages is.

> Former IGDA member, Gamasutra subscriber and GDCE attendee here, hence why I asked for numbers.

Maybe "former" is why. There are no numbers published to confirm or deny. It's apparent if you keep up with the community and ask developers what they use.

> It is not, but the goal of fitting as much code as possible in small packages is.

You're really talking about reducing sizes of assets & included libs. That has more to do with optimizing DL time than hardware performance, and nothing to do with low level coding where you're writing machine instructions without touching a higher level of abstraction. Not the same at all.

> There are no numbers published to confirm or deny. It's apparent if you keep up with the community and ask developers what they use.

So just an anecdote, kind of "on my neighbourhood...".

Yes, the PS1's 2MB of RAM are pretty generous compared to some embedded platforms.

A typical engine controller ECU in a car might have 256KB of RAM (and maybe 2-4MB of flash).

but you usually don't stuff 3d models and textures in an engine controller ECU ;)

Not with that attitude you don't :)

There is a tremendous number of hardware products and industrial systems where the processing is performed on small and cheap components (microcontrollers, digital signal processors).

Of course there exist very complex components in the category of microcontrollers, some of them even offer enough resources to run Linux, but if you stick to the $1-$5 range the specs are very limited.

Here are two examples, the first one costs around $3 and the second one is less than $1.

http://www.ti.com/product/tms320f28027 http://www.atmel.com/devices/attiny85.aspx

I develop on such platforms and even though there is an interesting challenge in programming these tiny processors and optimizing CPU cycles and memory usage all the time, in the long run it becomes quite strenuous because there is only low-level stuff and I miss the expressiveness and flexibility of more abstract languages.

I feel a little sad every time someone says "We have much higher CPU power/memory now, those things are unnecessary". Maybe they are right but I feel like this is not the path we should take

100% agree. And this goes for everything from server side, to games, to mobile to client side web. Some of my favorite web sites have gone from a 1-2 second load time to well over 6 seconds. And then they're sluggish after they load. It's sad.

You even see this with Google.

A typical Google search from 2009:

"Meaning of Life: Approximately 72,000,000 Results (0,00000042 Seconds)"

A typical Google search today:

"Meaning of Life: Approximately 364,000,000 Results (0,62 Seconds)"

It's not, but what is true is that people can focus more on features than on hacks to squeeze the maximum performance out of their system of choice.

If we're free to not waste time on tons of little performance hacks anymore, why do them anyway?

Making an (state of the art) engine is no easier. Making a game is only easier because the engines are more available now than they were in the past.

>>the other is sad because you hardly use this low level stuff in todays software development world.

During my engineering days(Circa 2005) I programmed in 8085 and our professor would give us all kinds of crazy assignments and small projects. That was the first taste of any genuine programming challenge I faced in my life. Immediately post that, programming in C felt a little boring.

Recently I worked on a embedded systems project for which I relived these kind of days. I had to invent all kinds crazy tricks to work around with resource constraints.

Your true creativity skills are kindled when your resources are constrained in all manners. Time or other wise. Unfortunately you can't academically recreate the same experience.

The amount of unoptimized crap they generate in a few weeks (in some cases) is quite depressive too I'm afraid. Game developers are wasting more and more CPU cycles, this of course reduces development cost. But it would be nice if they put some effort in making things run fast. And this doesn't only count for games!

Making things fast is more important than making them run fast. And it's not about the cost — it's about iterations for the sake of game design.