graffix
Simulations commonly target GPU, and a game is just an interactive simulation, so in some respect it's straightforward to write games for GPU. From a utilization perspective, you'll need minimally tens of thousands of dynamic game entities, which is one among many reasons why it's uncommon.
> 1. substantial numbers of calculations can be parallelized
> 2. the computations to be done are stable and arrive at regular intervals ('regularity')
> 3. relatively few memory accesses are needed for a given amount of computation ('locality')
Where (1) fails, any modern multicore + SIMD + ILP desktop/console/mobile CPU will run at a tiny fraction of its peak throughput. While sufficiently small serial tasks still complete in "good enough" time, the same could be said of running serial programs on GPU (in fact this is sometimes required in GPU programming). People routinely (and happily) use PL implementations which are ~100x slower than C. The acceptibility of ludicrous under-utilization factors depends on the tininess of your workload and amount of time to kill. Parallelism is used broadly for performance; it's about as un-specialized as you can get!
(2) and (3) are really extensions of (1), but both remain major issues for serial implementations too. There mostly aren't serial or parallel applications, rather it's a factor in algorithm selection and optimization. Almost anything can be made parallel. Naturally you specialize HW to extract high performance, which requires parallelism, for specialized HW as for everywhere else.
The authors somewhat gesture towards the faults of their definition of "specialized" later on. Truly specialized HW trades much (or all) programmability in favor of performance, a metric which excludes GPUs from the last ~15 years:
> [The] specialization with GPUs [still] benefited a broad range of applications... We also expect significant usage from those who were not the original designer of the specialized processor, but who re-design their algorithm to take advantage of new hardware, as deep learning users did with GPUs.
The new thing is fast regression of a real light field into Gaussian splats, which can then be rendered at reasonable rates. Dynamic lighting requires full inverse rendering as a preprocess, which is way beyond the scope of the technique. Technically Gaussian splats could form part of an inverse rendering pipeline, and also be the final target representation, but I'm not sure there would be any benefit over alternative rendering primitives.
I read Aras' quip as more narrowly technical. OFC it's ambiguous and you'd have to ask the man himself.
The gist of NeRF is to obtain an NN representation of a 5D light field (3D position + 2D direction) from samples (photographs) of the real-world light field. Alarm bells ring already- 5 dimensions isn't that many! Considering NeRF has always used a low-rank spherical harmonic representation of the directional domain, it's even more like 3D-and-change. To reconstruct a function of such low dimensionality, why choose an NN?
Then at inference time, for each pixel, you have sample the NN repeatedly over the view ray. This part is exceedingly silly, as compact representations of light fields are a solved bread-and-butter problem in graphics.
Later on Plenoxels explicitly took the "Ne" out of "NeRF", giving far higher training and inference performance (also mentioned ITT). To be fair, and later still, Nvidia somewhat redeemed NNs here with Instant NeRF: https://nvlabs.github.io/instant-ngp/assets/mueller2022insta... ...where the twist was to interpolate fancy input emeddings, which are run through a tiny NN. That tininess is important, as the need to fetch NN weights from VRAM would kick NNs right off the Pareto frontier.
Zooming out, NNs have only seen wide adoption in graphics engineering for reconstruction from sparse data (inc. denoising). Makes sense, as that's a high-dimensional problem. Still, beware that the NN solutions rarely blow handmade algorithms out of the water. I also think using tiny NNs for compression- closely related to reconstruction- has a future too. Beyond that, if NNs were to set the graphics world ablaze, it would've happened by now.
Lots of graphics engineering is just approximating functions, so it's natural NNs have some place here. However, our functions tend to be more understandable, tractable, malleable. It's not an application domain where it's virtually impossible to write an algorithmic solution by hand (let alone one that performs well), like natural language understanding.
This is on Turing. Nvidia would've been motivated to de-risk the introduction of RTX by making boring choices. You may well see different results on later archs.
Today's "mid-size" to AAA game development culture is ultra-conservative, both technically and creatively. Ballooning team sizes and budgets mandate this. Do not join a studio bigger than ~10 people. Smaller teams/studios imply better jobs, in my experience.
They say not to make your hobby your job- unfortunately, there's no other way to break in to this highly competetive industry (echoing others here). So start with hobby game dev in your free time. It's worth learning C++ while doing so. C# is another option, but one that mostly locks you in to studios using Unity.
Beware of relatively low compensation and "crunch time" (really a perpetual, cyclical issue in game dev). I largely escaped the latter, but most in the industry can share overtime horror stories.
On any real camera, stars always cover multiple pixels (the entire screen strictly speaking). This is described by a point-spread function (PSF), where the "least blurry" physically realizable PSF is the Airy disk. The bloom effect occurs due to the long tails of the PSF. Inasmuch as stars appear as perfect points to the eye, it's down to perceptual limits and (I suspect) heavy filtering in the visual cortex.
Not saying Carmack is wrong about VR and video game rendering here. Just that if you're trying to replicate real sensors, the right way is precisely what he says to avoid. You must somehow draw
> large, dim stars
During a recent effort to implement physically-accurate star rendering at work, I had access to a photograph of the night sky. Bright stars visibly and clearly covered 3x3 pixels- more in reality, due to the above.