News for hackers. Stuff that matters.
Re:Hallucinations re: the rendering of Cyberpunk 2077 by atum47 on Thursday December 17, 2020 @04:55PM (#25461020)
thanks. I was able to get the basics stuff out of the way:

https://victorribeiro.com/3Dsphere/

https://www.youtube.com/watch?v=w13Q_KL1I9A

I just need to focus a little bit more in order to get more complex stuff done.

for instance: I don't know jack about animations / bones / skin deformation. =(

Re:Hallucinations re: the rendering of Cyberpunk 2077 by ilays on Thursday December 17, 2020 @05:11PM (#25461190)
Re:Hallucinations re: the rendering of Cyberpunk 2077 by corysama on Thursday December 17, 2020 @05:35PM (#25461423)
That book is a highly-recommended intro. The site for the book https://www.realtimerendering.com/ also list tons more great material.
Re:Hallucinations re: the rendering of Cyberpunk 2077 by brundolf on Thursday December 17, 2020 @05:13PM (#25461208)
> I don't know jack about animations / bones / skin deformation

I only know the basics, but as far as I know that stuff is generally pretty straightforward (at least compared to actual rendering)

Basically you have an invisible vertex floating inside the mesh, and vertices on the visible mesh are "weighted" to it (with a floating point weight), so that when the bone moves, it "pulls" vertices towards it based on the weight strength. A mesh vertex can be weighted to multiple bone vertices at once. Bones typically consist of two vertices, where one is treated as an anchor and the other rotates around it. They're then arranged in a hierarchy, so that a parent bone's transform (translate/rotate/scale) applies to its children, and so on. So for example, rotating an upper arm bone causes the forearm bone to rotate with it.

An animation is then a series of keyframes on those bones (translate/rotate/scale to a particular x/y/z at time t). Those keyframes are interpolated when the animation is applied, and the mesh vertices are dragged along accordingly.

Motion-capture is just taking position data from a live actor and mapping it to those bone keyframes.

Things get slightly more complicated when it comes to blending between multiple animations (for example, running + swinging a sword; you basically average the keyframes). And then there's IK or Inverse Kinematics, which basically means that instead of driving an animation purely from explicit keyframes, you procedurally place the "end" of the limb (a character's foot goes all the way to the ground, for example) and then procedurally work backwards adjusting the bones up the hierarchy to satisfy that position. This is pretty common these days and leads to dramatically more natural-looking animation for characters and creatures (you can see it in action by walking on some stairs in a recent game). I don't know exactly how it's implemented.

But that's about it from a technical perspective. Of course the artistry of getting all of this just right is much harder - from the bone-weighting ("rigging") to the keyframes to the motion capture data cleanup - but the technical aspect isn't nearly as complicated as rendering (to my understanding).

Re:Hallucinations re: the rendering of Cyberpunk 2077 by atum47 on Thursday December 17, 2020 @05:21PM (#25461288)
I get the concept because I used to be a 3d artist... But when it comes to program this...

A static model I just copy it's vertex position to a buffer, right? So when there's a bone animation I just update the vertex in my models and the copy it again to the buffer, for every frame?

Re:Hallucinations re: the rendering of Cyberpunk 2077 by corysama on Thursday December 17, 2020 @05:34PM (#25461410)
That can work. But, it's slow. The way this has been done is to not modify the buffers and instead do the work in the vertex shader. The vertex shader sends the skinned verts straight to the triangle rasterizer and then then are never needed again. So, no need to read-modify-write the buffer just to have the rasterizer read it back again.
Re:Hallucinations re: the rendering of Cyberpunk 2077 by atum47 on Thursday December 17, 2020 @06:03PM (#25461674)
yeah, I figured that should be a better solution.

I was using ThinMatrix [1] videos as guidelines when I was creating my engine. I'll take a look in how he implemented animations. Thank you.

1 - https://www.youtube.com/user/ThinMatrix

Re:Hallucinations re: the rendering of Cyberpunk 2077 by 936936 on Thursday December 17, 2020 @05:42PM (#25461483)
Your vertex buffer should hold indices for which bones influence the position of each vertex and by what fraction. Each vertex could, for example, have 4 indices and 4 floats to indicate these values, and these only need to be uploaded to the gpu once.

The transformation matrices are interpolated for each rendered frame using the hierarchial animation data and the computed matrices are uploaded to the gpu in a buffer. The matrices are accessed in the vertex shader by indexing into the matrix buffer to calculate the vertex position.

Re:Hallucinations re: the rendering of Cyberpunk 2077 by BearOso on Thursday December 17, 2020 @05:38PM (#25461451)
No, you'll generally pass the relevant bone information and desired transforms to a vertex shader, and that computes the new positions.
Re:Hallucinations re: the rendering of Cyberpunk 2077 by brundolf on Thursday December 17, 2020 @05:26PM (#25461330)
Ah, I misunderstood. Yeah I've never actually implemented it myself :)
Re:Hallucinations re: the rendering of Cyberpunk 2077 by oplav on Thursday December 17, 2020 @05:35PM (#25461416)
> They're then arranged in a hierarchy, so that a parent bone's transform (translate/rotate/scale) applies to its children, and so on. So for example, rotating an upper arm bone causes the forearm bone to rotate with it.

How do you reconcile noise in motion capture data to bone transformations? For example, isn't it possible that the motion capture data slightly elongates or shortens the forearm in a way that makes it impossible to maintain the relationship with the upper arm bone?

In my head, it feels like you would have to pick the closest point on the sphere that's created if you rotate the forearm all around the elbow joint, but that seems expensive to do everywhere all the time.

Re:Hallucinations re: the rendering of Cyberpunk 2077 by brundolf on Thursday December 17, 2020 @05:44PM (#25461515)
You can have bones that are parented but don't have one of their vertices directly attached to one of the parent's vertices. It's really just a hierarchy of transformation (child's overall transform = its transform relative to parent + parent's transform). Put differently, the "attached at the joints" notion is often approximated just because that's how human skeletons work, but it's by no means required. And "bones" in this sense don't always even correspond to actual skeletal bones; they're really just a tool for moving around groups of vertices without having to mess with hundreds or thousands of them individually.

That said - and I haven't done this myself, I'm just speculating - if you're using motion capture data directly maybe it makes sense to ditch the hierarchy altogether and have each bone be independent? Though, that could be problematic if you wanted to blend some "artificial" animations together with the recorded ones. I'm not sure what the standard practice is.

Re:Hallucinations re: the rendering of Cyberpunk 2077 by Daub on Thursday December 17, 2020 @07:21PM (#25462296)
Raw motion capture data almost always requires some editing after it has been captured. This is done by the animator, and they can’t work without their rigs. Rigging is an entire craft... tried a enough to know that I suck at it. Rigging quadrupeds is a god level skill.