Writing vertex and fragment shaders blew my mind the first time I did it. "Wait, this code will be run for EACH PIXEL ON THE SCREEN?" It's obvious in principle, but the actual implementation felt awesome to experience first-hand. Like the feeling of accomplishment that you get in re-deriving a famous theorem yourself. It's not groundbreaking or novel, but it unlocks a much deeper understanding for why we need GPUs at all.
You can take a look at the humble repo I used to both learn C++ and explore 3D rendering [1]. If by some chance you are interested in learning more about how this stuff works, I highly encourage you to check out TheCherno's youtube channel [2]. I'm not a huge fan of his newer stuff (Hazel engine), but his older series implemented 3D graphics from scratch in Java. Taught me a huge amount about how this stuff works and really ignited my interest in programming.
[1] https://github.com/TimelyToga/tlotk/tree/master/src/engine/g...
[2] https://www.youtube.com/watch?v=iH1xpfOBN6M&list=PL656DADE0D...
Funny story: following a video (ThinMatrix) about creating a 3D engine in Java (I was translating everything to JavaScript and working around the old OpenGL concepts_) I learned that you can't have an object as a index in a JavaScript Dictionary. Living and learning.
exit(FUCKED_UP_SHADER_COMPILATION);
Start with https://github.com/jkuhlmann/cgltf to handle loading the data. Check the sidebar of https://www.reddit.com/r/opengl/ for tutorials.
https://victorribeiro.com/3Dsphere/
https://www.youtube.com/watch?v=w13Q_KL1I9A
I just need to focus a little bit more in order to get more complex stuff done.
for instance: I don't know jack about animations / bones / skin deformation. =(
try this if you have time
I only know the basics, but as far as I know that stuff is generally pretty straightforward (at least compared to actual rendering)
Basically you have an invisible vertex floating inside the mesh, and vertices on the visible mesh are "weighted" to it (with a floating point weight), so that when the bone moves, it "pulls" vertices towards it based on the weight strength. A mesh vertex can be weighted to multiple bone vertices at once. Bones typically consist of two vertices, where one is treated as an anchor and the other rotates around it. They're then arranged in a hierarchy, so that a parent bone's transform (translate/rotate/scale) applies to its children, and so on. So for example, rotating an upper arm bone causes the forearm bone to rotate with it.
An animation is then a series of keyframes on those bones (translate/rotate/scale to a particular x/y/z at time t). Those keyframes are interpolated when the animation is applied, and the mesh vertices are dragged along accordingly.
Motion-capture is just taking position data from a live actor and mapping it to those bone keyframes.
Things get slightly more complicated when it comes to blending between multiple animations (for example, running + swinging a sword; you basically average the keyframes). And then there's IK or Inverse Kinematics, which basically means that instead of driving an animation purely from explicit keyframes, you procedurally place the "end" of the limb (a character's foot goes all the way to the ground, for example) and then procedurally work backwards adjusting the bones up the hierarchy to satisfy that position. This is pretty common these days and leads to dramatically more natural-looking animation for characters and creatures (you can see it in action by walking on some stairs in a recent game). I don't know exactly how it's implemented.
But that's about it from a technical perspective. Of course the artistry of getting all of this just right is much harder - from the bone-weighting ("rigging") to the keyframes to the motion capture data cleanup - but the technical aspect isn't nearly as complicated as rendering (to my understanding).
A static model I just copy it's vertex position to a buffer, right? So when there's a bone animation I just update the vertex in my models and the copy it again to the buffer, for every frame?
I was using ThinMatrix [1] videos as guidelines when I was creating my engine. I'll take a look in how he implemented animations. Thank you.
The transformation matrices are interpolated for each rendered frame using the hierarchial animation data and the computed matrices are uploaded to the gpu in a buffer. The matrices are accessed in the vertex shader by indexing into the matrix buffer to calculate the vertex position.
How do you reconcile noise in motion capture data to bone transformations? For example, isn't it possible that the motion capture data slightly elongates or shortens the forearm in a way that makes it impossible to maintain the relationship with the upper arm bone?
In my head, it feels like you would have to pick the closest point on the sphere that's created if you rotate the forearm all around the elbow joint, but that seems expensive to do everywhere all the time.
That said - and I haven't done this myself, I'm just speculating - if you're using motion capture data directly maybe it makes sense to ditch the hierarchy altogether and have each bone be independent? Though, that could be problematic if you wanted to blend some "artificial" animations together with the recorded ones. I'm not sure what the standard practice is.
Of course raytracers don't work like polygonal hardware rendering but a lot of the maths and concept do carry over I've found. If you understand how normal mapping and reflections work in a software Raytracer you shouldn't have too much trouble implementing the same algorithms in a hardware accelerated pixel shader for instance.
In general I think that implementing a raytracer is just one of these projects that's just too cool and simple not to do, so I don't think you'll be wasting your time anyway.
For the level you see in Cyberpunk, you need to learn modern, physically-based techniques. This requires a bit more math, though nothing you wouldn't see in a CS degree. My favorite reference on PBR techniques is the PBR book, which is free online. Applying some of these techniques in a real-time setting is of course another challenge.
However, all this probably takes a ton of time and math and low-level programming knowledge for a newcomer.
There's plenty of job openings in game development industry outside of the top of the field roles for the best AAA projects. These job openings require less expertise and experience, but yet will allow you to get your hands dirty with various areas, 3d rendering included — although in smaller studios working on smaller projects you most likely will not be a part of dedicated rendering team and will be involved in different areas of the project.
https://raytracing.github.io/books/RayTracingInOneWeekend.ht...
This is good. The rest is more of theory. I can dump you a lot of papers, talks, and reviews like the OP one.
How do you got into CG? Do you fiddle with other areas, like AI, ML...? I'm asking cause I think you really have to concentrate in one field to get to the level of the guys behind cyberpunk. They probably never had to learn react / redux in order to get a job.