recently saw someone used deep learning to create higher resolution textures for old games. might be something to add on top of this to get more details:

https://www.overclock3d.net/news/software/classic_doom_has_r...

https://www.resetera.com/threads/ai-neural-networks-being-us...

I think you are really on to something here. It would be amazing to see some of the retro Sega classic arcade games such as Space Harrier and Turbo Out Run in 4K!

Q2VKPT seems like a rich area of research. If the entire pipeline could be automated. Using efficient subdivision surfaces and deep learning to intelligently predict light-transport simulation. Then we would merely have to input the game assets of a classic game title. And the output would look like Bioware's Anthem ;)

What about.. AI-powered texture compression.

Train the game upscaling AI with high resolution textures, then ship the game with lower res textures. High quality rendering with smaller file sizes, though won't be as good as the original pictures.

Not very useful in reality, but would be a pretty neat proof of concept.

The question is: would the AI data be smaller than shipping the high quality textures in the first place?

Textures usually ship compressed in a format that is supported by the hardware. Hardware texture compression saves a lot of bandwidth and therefore improves runtime performance. That compression step is done offline and it's slow enough not to do it on the client's side, otherwise just using ordinary JPEG compression would reduce file sizes substantially.

Shipping size is usually not the limiting factor for texture size, GPU memory is. Texture compression must work on small blocks for random pixel addressing.

Which doesn't entirely rule out a deep learning based compressor, just makes it much harder :)

Why use repeated textures? Just provide a low resolution "pattern definition", say some blades of grass, and fresh unique textures could be generated by a network. probably pretty computationally cheap too

Fwiw, I have seen a NN texture compressor that is compatible with game engines and doesn’t require any AI data to decompress. I can’t give any details because it’s not my project and not published yet. I just wanted to say I know for a fact that your idea there is a good one. ;)

Check out Nvidia DLSS

It would be especially neat to see this done for 3d models.

There are plenty of texture upgrades available for Quake, but I haven't really seen any model updates.

Agree, if you add highres textures to old games you quickly reach a point where it just feels less consistent than before, because everything is pretty low tech and then you have these out of place photorealistic textures. It might just be me, considering how popular high res texture packs for Minecraft are. I always felt the stock ones or at least those that tried to mimic the 8 bit style looked best.

So yes, upgrading models and even the level geometry would be very cool IMO.

Depends how well it’s done. Check out https://www.moddb.com/mods/quake-15 for Quake (I), for example.

I guess spotting an upscaled image will soon be pretty much impossible without making your eyes sore or employing another NN. So much for trying to preserve originals on the web.

I think the method employed is called ESRGAN (Enhanced Super Resolution Generative Adversarial Networks)

You should checkout Battlefield 5 with RTX, it's probably the most impressive game available: https://www.youtube.com/watch?v=-1fdpybF-8Y

BF5 applies ray tracing extremely selectively: they only trace light for some glossy and specular surfaces. Most of what you see on screen is still rasterized traditinally in that game, meaning, using some 30 to 40 different hacky methods and approximations to make it look good.

Q2VKPT is different in that it does no traditional rasterization at all for the 3d graphics (only the HUD graphics are rasterized). The rest is path traced with multiple reflections even for the diffuse surfaces. Because of this purity in its approach it is quite unique.

QVKPT is from the guy who invented the postprocessing filter used to denoise the images in this implementation. The graphics would be way worse without that filter (there has to be a convar to disable it).

There is a comparison screenshot a bit down the page

no filter: http://brechpunkt.de/q2vkpt/images/path_tracer_720p.png denoised: http://brechpunkt.de/q2vkpt/images/denoised_720p.png

The denoise filter might as well be the main achievement of this project. Edit: note that if I understand correctly, the filter doesn't operate on single frames, but considers several previous frames in order to maintain temporal cohesion and to improve quality.

>the filter doesn't operate on single frames, but considers several previous frames in order to maintain temporal cohesion and to improve quality.

Do you happen to know how bad are the artifacts? I remember temporal antialiasing produced very good results with static scenes, but fell apart on dynamic ones.

I believe the point was that BF5+RTX is clearly much more realistic looking than this Quake2, even though it's more "truly" ray-traced. BF5 is extremely impressive, with certain locations being on the verge of photo-realism.

Honestly, I thought it was pretty disappointing. If it wouldn't have been pointed out to me where it does its magic I probably couldn't have told whether the effect is turned on. I'd like to see random people who never played it before sat in front of 5 minutes of random gameplay with rtx on and off without telling them which is which and then have them guess afterwards.

Agreed, it's easy to tell there are only a few scenarios where it's actually used.

I found the lighting from the explosions particularly convincing in a way that most games don't capture. Part of my brain was expecting to feel the heat.

Really? Very first explosion in first second shows tons of scattered light on the floor to the left, but very little on the wall, with sharp edge between the two.

The low detail really is due to the outdated Quake 2 models. You could go quite a bit higher on geometry and texture detail before performance becomes an issue. I actually don't know why Christoph picked Q2 specifically for this project.

The lighting in this version of Q2 is vastly more physically correct than in a lot of current generation modern engines. So it really should feel more real. I think this shows why hardware accelerated ray tracing is such a big deal for realtime graphics.

I'd like to see it with shadows. Especially the deep black ones that made The Chronicles of Riddick: Escape from Butcher Bay (2004) look so amazing for its generation, even better than Doom 3.

It could look way better with more detailed geometry and proper materials. Most of those textures have baked-in lightning and lack normal maps and proper materials, which take away a lot of the effect. Take a look at this video from 2013: https://www.youtube.com/watch?v=abqAanC2NZs