[PATCH] COLLADA: Support special vertex weight index on skinned models

[historic_bruno]

For skinned models, the ​COLLADA 1.4.1 spec allows a special vertex weight index of -1, which indicates the weight applies to the bind-shape matrix* instead of a particular bone, as seen in this example:

<skin>
  <source id="joints"/>
  <source id="weights"/>
  <vertex_weights count="4">
    <input semantic="JOINT" source="#joints"/>
    <input semantic="WEIGHT" source="#weights"/>
    <vcount>3 2 2 3</vcount>
    <v>
      -1 0 0 1 1 2
      -1 3 1 4
      -1 3 2 4
      -1 0 3 1 2 2
    </v>
  </vertex_weights>
</skin>

(*The bind-shape matrix represents the transform of the bind-shape prior to skinning, it transforms the bind-shape from object space to bind-space. It's specified by a <bind_shape_matrix> element in the <skin> or assumed to be an identity transform. See pages 4.6-4.8 of the COLLADA spec for more notes.)

Currently if you try to add such a model to the game, it will fail to import since the PMD converter thinks there are more than 256 bones (the -1 index is typecast to an unsigned int). Apparently Blender 2.60a likes to export such models and seems to have made general improvements to animation exporting, so we should find a way to implement this.

As a workaround, it may be possible to export animations with Blender 2.6 and the corresponding models with an older version.

[historic_bruno]

6-wheeled ram exported from Blender 2.60a

[historic_bruno]

6-wheeled ram exported from Blender 2.59

[historic_bruno]

updated description with more info

[historic_bruno]

Here's how my current solution looks:

• In the special case of this ticket, a vertex can be influenced by the bind-shape matrix instead of a bone, what this means in the common case of only one influence per vertex is the vertex is not affected by the pose (i.e. it's not animated). To indicate this when converting the model, I set the influence's bone ID equal to the total number of bones, because this value is constant and serves as a useful index. CModel::ValidatePosition is called on the resulting model prior to skinning, which updates prop points, the array of posed bone matrices (CModel::m_BoneMatrices), etc. These bone matrices include the various transforms necessary for skinning in world space.
• I reserve one "pseudo" bone matrix in the array, which contains only the model's world space transform at the time of skinning. In this way, the skinning functions don't change to handle the special case, they treat the influence ID as if it's referring to a bone. I allocate one extra bone matrix on model loading and offset the indices by one when loading the PMD in CModelDef::Load and when computing the weighted blend matrices in CModelDef::BlendBoneMatrices.
• The resulting array of bone matrices is indexed like this:

  0, 1, ... numBones-1, *numBones* , numBones+1, ... numBones+numBlends+1
  

  where the first numBones entries are real bones, followed by the pseudo bone (model world transform), and finally numBlends distinct weighted blends.
• There's one other "special" case, which you notice in the attached files. Blender sometimes exports skinned models with influences like this:

  <vertex_weights count="2148">
    <input semantic="JOINT" source=.../>
    <input semantic="WEIGHT" source=.../>
    <vcount>0 0 0 0 ... 1 1 1 ... 0 0 0 ... </vcount>
    ...
  

  where some vertices have no influences at all (an entry of 0 in <vcount>)! I've found no details in the COLLADA spec about this case. Because these influences end up as default values during the PMD conversion, we catch that if it occurs on the first vertex and throw an error (without skinning the model). Maybe we should treat these models as errors if that's undefined behavior, but I've found that treating the no-influence vertices the same as the special case works: giving them a single bind-shape influence with weight 1.0.

[historic_bruno]

Replying to historic_bruno:

but I've found that treating the no-influence vertices the same as the special case works: giving them a single bind-shape influence with weight 1.0.

I should clarify that this is handled during the model import and is transparent to the loading and skinning processes.

[historic_bruno]

^{Added patch which does just that}

[historic_bruno]

I haven't bumped the PMD converter version because this shouldn't affect existing models (they would have failed to import in this case).

[Philip Taylor]

Since the normal bones are computed in CModel::ValidatePosition as the animated matrix (which is rotation * translation) multiplied by world-space transform, isn't the new special-case world-space-transform-only bone equivalent to a standard bone with 0 rotation and 0 translation? If so, it might be cleaner to have the Collada converter explicitly create a bone with 0 rotation/translation (in both the PSA and PMD files, I think) so that we don't need the special case in the file format or in the engine code.

If not:

"numBones + numBlends + 1" and "mdef->m_NumBones + j + 1" could perhaps put the +1 in the middle instead of the end, to mirror the array layout better.

PMD_File_Format should be updated with the file format changes.

[historic_bruno]

Replying to Philip:

Since the normal bones are computed in CModel::ValidatePosition as the animated matrix (which is rotation * translation) multiplied by world-space transform, isn't the new special-case world-space-transform-only bone equivalent to a standard bone with 0 rotation and 0 translation? If so, it might be cleaner to have the Collada converter explicitly create a bone with 0 rotation/translation (in both the PSA and PMD files, I think) so that we don't need the special case in the file format or in the engine code.

That solution works with my test models, I think it still requires special casing in the engine, because we have animations and models that are in PSA/PMD format and so can't be converted now (for instance a new COLLADA model might reference an old PSA animation and vice versa - the bones counts have to match). I'd have to bump the PSA/PMD version numbers and check for older versions, manually add the extra bone, in addition to bumping the converter version. Unless I'm missing something or overcomplicating things.

[historic_bruno]

After discussing this with Philip, I realized the advantage of handling this in the PSA/PMD converters: it keeps the formats easy to describe and handle, as all the bone data gets encoded directly instead of requiring special run-time handling by the engine. The disadvantage is that it breaks backwards compatibility with old PSA/PMD files, because we have to allow for someone creating a COLLADA model that reuses old animations. This means adding more code to CModelDef::Load and CSkeletonAnimDef::Load to ensure we always have the correct number of bones, but the silver lining is that if we ever do replace those old files, we can remove the version compatibility code.

[historic_bruno]

Postponing this a bit, until I know if we can get rid of all the remaining PSA/PMDs.

[ben]

In 11242:

Implements COLLADA -1 vertex weight support, using simpler initial solution. Fixes #1012.

[historic_bruno]

Went ahead and fixed this with the initial solution, since old PMD/PSAs are not likely to be replaced soon (e.g. there are issues exporting from the source .max files and the animation import process is still not very intuitive).

I will also attach a patch for the alternative solution, but it complicated the PMD/PSA loading process unnecessarily IMO. All old models would have been required to have an extra bone added explicitly to maintain compatibility, but this way the change is transparent and only used for models with -1 vertex weights. If we do replace all the old PMD/PSAs, then the alternative solution would be nice for reasons mentioned above, but still not a significant improvement.

[historic_bruno]

alternative solution - extra bone added by the DAE converters