How many cats have you skinned?
Chances are, as a graphics programmer, you’ve spent some time implementing some engine and shader code that does vertex skinning. I don’t mean anything clever, none of your dual or more quaternion methods, just a bunch of weighted matrix * vertex transforms indexing some kind of palette. Like many “simple” graphics tasks, writing skinning from scratch is one of those things that despite its simplicity, is a complete bug factory – see also, writing a GL application from scratch, rendering offscreen the right way up, etc.
In the spirit of OpenGL: Why Is Your Code Producing a Black Window?, I thought it would be good to make a list of possible mistakes and how to recognize them. Even though you may have implemented it before, it’s possible with all of this spare compute power, that you may be re-implementing it again soon.
Personally I don’t think these kinds of mistakes are a sign of a bad graphics programmer – I’m sure it happens to the best. However, I think how you approach investigating the bug and how quickly you are able to fix the mistake is more indicative of experience with the GPU tools available and problem solving. I’d even say that it could make a good interview test (if you’re inclined to do those things) – something like “@InternDept is stuck implementing their own skinning – can you help them?” and before breaking out the tools, you could even show pictures of what the result currently looks like and have the interviewee guess.
The stages of
If you’re implementing skinning incrementally, I’d expect you to go through the following stages:
1) In the beginning, you just pass through the bind pose vertex positions. This sounds easy and you’re not likely to cock up the shader code, but this is where you start checking your bindings and make sure your input and output buffers are pointing in the right places. However, if/when you mess this part up, the rendering result can easily be:
- Forgot to set up the input buffer.
- Reading/writing float rather than float3.
- In compute, reading from or writing to ByteAddressBuffers and forgetting they take/give ints and you’ve introduced an implicit cast.
2) Next, you move to indexing and weighting against an identity palette. This is the best place to hide fails since all those identity matrices will give you back whatever you put in unless you see:
- Shrunken skin
- Your weights don’t add to 1.0.
- Explodey skin
- Your indices index outside of your palette.
- A unit volume of noise
- Reading weights instead of vertex positions.
- A non-unit pile of noise
- Reading indices converted to floats instead of vertex positions.
- Small ball of noise with spikes
- Reading the palette instead of vertex positions.
- Half a skin with an explodey component
- 1/4 size palette when you pass matrices as float4s and the matrix count as the float4 count.
- Reading indices with int data reinterpreted as floats.
- Zeros and not identity matrices in your palette.
- Zero sized palette.
3) Then you generate some real animation output and use it to populate your matrix palette. This is where you get what I believe is the most common skinning fail:
- John Carpenter’s The Thing (another classic example at igetyourfail)
- A classic image that typically means that you’ve missed a transpose/mixed up the matrix-vector multiply order.
- A similar effect comes from reskinning the skinned data – possible if you have compute skinning feeding the skinning vertex shader path.
- A bat like monstrosity
- This can happen when mismatching indices and weights, e.g. using weight[i] with index[N-i] – possibly due to the endianness of your packing. Similar to The Thing, but typically features like fingers are extruded when skinned by the wrong bones.
- Not animated
- Are you still writing the input vertex positions direct to the output, or not actually using the palette?
- Skin at wrong location
- Palette includes the world transform and so does the shader after skinning – double world transform.
- If packing indices as bytes in a 32 bit value, you could find you completely cobbled the decode of the indices – did you mask the index out of the indices or just mask the index to zero?
- Same as previous, but the output has moved offscreen – look around!
- Your palette is full of rubbish.
4) Once you’re happily animating, you want to get the rest of the lighting inputs so you want to skin the normals and maybe some bitangents. At this point you might see:
- Black sphere of noise
- Overwrote the input/output vertex positions with either the normals or bitangents.
- Correct mesh with lighting craziness
- Transformed the normals or bitangents with a W of 1.0, since you copied from the vertex positions.
- Used the vertex positions for the normals or bitangents – either bound the wrong view or reading everything from the vertex position view.
The point of all this is that you should be able to check everything here in a few minutes with the great graphics debugging tools we have available – praise be to Razor and RenderDoc!