get skinned vertices in real time

I need to get the vertices of a skinned mesh as they are deformed... problem is, the SkinnedMeshRenderer component only exposes the sharedMesh, which isn't updated (it's the asset itself, not the mesh instance).
Am I missing something?

The MeshFilter component exposes both the mesh and sharedMesh, but from what I see it's not in synch with the SkinnedMeshRenderer.
I found trace of a SkinnedMeshFilter, which sounds like it would be just what I need, but I fear it's an obsolete piece of code (doesn't compile, SkinnedMeshFilter is unknown... I never heard of it before either).

I'd be okay with reinventing the wheel, ie manually calculating the vertices positions using the bones, if anyone knows the algorithm... some sort of barycenter of the bones' positions balanced by their weights?

 

Okay... so apparently it is a simple barycenter or whatever it's called, script below will draw gizmos on each vertex in real time.

If someone knows a less crazy solution, I'm still interested though... I'll log a "gimme the mesh instance dammit" feature request when it's not 2am.

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3. using System.Collections.Generic;
4.  
5. public class SkinnedVertices : MonoBehaviour
6. {
7.     Mesh mesh;
8.  
9.     class Bone
10.     {
11.         internal Transform bone;
12.         internal float weight;
13.         internal Vector3 delta;
14.     }
15.     List<List<Bone>> allBones = new List<List<Bone>>();
16.  
17.     void Start()
18.     {
19.         SkinnedMeshRenderer skin = GetComponent(typeof(SkinnedMeshRenderer)) as SkinnedMeshRenderer;
20.         mesh = skin.sharedMesh;
21.  
22.         Debug.Log("{0} vertices, {1} weights, {2} bones",
23.             mesh.vertexCount, mesh.boneWeights.Length, skin.bones.Length);
24.  
25.         for (int i = 0; i < mesh.vertexCount; i++)
26.         {
27.             Vector3 position = mesh.vertices[i];
28.             position = transform.TransformPoint(position);
29.  
30.             BoneWeight weights = mesh.boneWeights[i];
31.             int[] boneIndices = new int[] { weights.boneIndex0, weights.boneIndex1, weights.boneIndex2, weights.boneIndex3 };
32.             float[] boneWeights = new float[] { weights.weight0, weights.weight1, weights.weight2, weights.weight3 };
33.  
34.             List<Bone> bones = new List<Bone>();
35.             allBones.Add(bones);
36.  
37.             for (int j = 0; j < 4; j++)
38.             {
39.                 if (boneWeights[j] > 0)
40.                 {
41.                     Bone bone = new Bone();
42.                     bones.Add(bone);
43.  
44.                     bone.bone = skin.bones[boneIndices[j]];
45.                     bone.weight = boneWeights[j];
46.                     bone.delta = bone.bone.InverseTransformPoint(position);
47.                 }
48.             }
49.  
50.             //if (bones.Count > 1)
51.             //{
52.             //    string msg = string.Format("vertex {0}, {1} bones", i, bones.Count);
53.  
54.             //    foreach (Bone bone in bones)
55.             //        msg += string.Format("\n\t{0} => {1} => {2}", bone.bone.name, bone.weight, bone.delta);
56.  
57.             //    Debug.Log(msg);
58.             //}
59.         }
60.     }
61.  
62.     void OnDrawGizmos()
63.     {
64.         if (Application.isPlaying  enabled)
65.         {
66.             for (int i = 0; i < mesh.vertexCount; i++)
67.             {
68.                 List<Bone> bones = allBones[i];
69.  
70.                 Vector3 position = Vector3.zero;
71.                 foreach (Bone bone in bones)
72.                     position += bone.bone.TransformPoint(bone.delta) * bone.weight;
73.  
74.                 int boneCount = bones.Count;
75.                 Gizmos.color = (boneCount == 4) ? Color.red :
76.                     (boneCount == 3) ? Color.blue :
77.                     (boneCount == 2) ? Color.green : Color.black;
78.  
79.                 Gizmos.DrawWireCube(position, boneCount * 0.05f * Vector3.one);
80.  
81.                 Vector3 normal = Vector3.zero;
82.                 foreach (Bone bone in bones)
83.                     normal += bone.bone.TransformDirection(mesh.normals[i]) * bone.weight;
84.  
85.                 Gizmos.DrawRay(position, normal);
86.             }
87.         }
88.     }
89. }

 

Damn... now how do I get the skinned normals?? sounds trickier...

 

I think the skinned mesh data is written to a special write-only portion of memory for performance, or something. I brought this up a while ago but can't find the thread. At any rate, it's like how it is for a reason.

Cheers,
-Jon

 

especially with the higher end cards and the pure shader engine, its likely not written anywhere outside the GPU RAM, so out of your reach.

it only works if it falls back to CPU animation mode where the data are required to be on RAM (basically only Intel CPU to DVI adapters, that do not have any acceptable hardware vertex transformation performance if any support at all *X3100+, the 900 / 950 don't have it*)

 

It shouldn't be. Normals should be exactly the same as vertices as far as the math is concerned.

 

Thanks Jon, it makes more sense... still sucks though.

To transform the vertex I do this

Code (csharp):

1. Vector3 position = Vector3.zero;
2. foreach (Bone bone in bones)
3.     position += bone.bone.TransformPoint(bone.delta) * bone.weight;

I just don't see what math to do to transform the normal... I'm probably just too tired though, it's 4am here

Edit: just played Night of the Cephalopods, pretty cool

 

It should be the same, just using TransformDirection instead of TransformPoint.

 

Unity compute skin on the cpu only.

 

Well... that defies my sense of space (which is pretty crappy to start with), but you're right it works, thanks a bunch!
FWIW, I edited the script above to gizmo normals.

So... which is it? UT, any chance to have a definitive answer on this? (David I know you're out there, stop hiding )

 

Thanks Aras... so why can't I get the deformed mesh? (dammit please)
Could you explain what Jon hinted at? and more importantly, will that ever change? (after Windows dammit)

Edit: because right now it kinda works but I seem to have a small offset between the rendered mesh and some vertices I compute, and some normals are way out... plus this reinventing of the wheel is really inelegant, and I like my code pretty.

 

Thanks! So I guess both parties were right, it's kind of in between, good to know it's doable in theory.

BTW, any idea why I have an offset (I'm working on LateUpdate)? does my algorithm seem correct?

Gee, this is surprising!
After all you only had 15000 bugs reported in the last month... slackers.

 

This would actually be useful even if it is slow. For example, you might want to emit particles from the surface of a skinned mesh object to simulate an explosion, disintegration or whatever. I would have found this useful on a couple of occasions.

 

This won't work for me, seems it's not possible to multiply a Matrix4x4 with a float?

/P

 

thanks

Adding the matrices together is also done on the element level then?

/Patrik

 

So in your pseudocode, result and finalMatrix are Matrix4x4, right? As for value, I don't get what it is :? !

And once I have that result, how do I get my vertex normal?

Code (csharp):

1. Vector3 position = result.MultiplyPoint3x4(mesh.vertices[i]);
2. Vector3 normal = result.MultiplyVector(mesh.normals[i]);
3.  

Sounds right? should I use MultiplyPoint3x4 or MultiplyPoint?

Also, pardon my ignorance but does currentBoneMatrix (which I assume is bone.localToWorldMatrix, bone being a Transform) change every frame? or can I do currentBoneMatrix * bindPose on Start?


I left this issue standing because it was good enough (just had that offset), but now I want to tidy things up... it was only a month ago, yet getting my head into this again is hell! Never do tomorrow what you could do today, right?

 

Mmmmokay, so assuming value is the unskinned vertex (mesh.vertices), then finalMatrix * value is a Vector4, which (apparently) can be cast to a Vector3, so I guess result would be the skinned vertex... right?

Unfortunately, none of my trials lead to anything. On the bright side, I've seen my model distorted in all sorts of funny ways today. I'm actually stunned that I such a good result at the beginning with that clumsy delta thing.

 

Hello, anyone?... pretty please?
metervara, did you get anywhere with this matrix stuff?

 

I got it working, Aras's pseudocode works like a charm

I omitted the variable value as I couldn't see why that was needed, and it works fine without it. I'm using MultiplyPoint3x4 MultiplyVector to get the position+normal (just like you), but since I'm not using value it's with finalMatrix instead of result.

Code looks like this:

Code (csharp):

1.  
2. weights=mesh.boneWeights;
3. for(int i=0;i<mesh.vertexCount;i++){
4.     BoneWeight weight = weights[i];
5.     Matrix4x4 m0 = matrices[weight.boneIndex0];
6.     Matrix4x4 m1 = matrices[weight.boneIndex1];
7.     Matrix4x4 m2 = matrices[weight.boneIndex2];
8.     Matrix4x4 m3 = matrices[weight.boneIndex3];
9.     finalMatrix[i] = Matrix4x4.identity;
10.     for(int n=0;n<16;n++){
11.         m0[n] *= weight.weight0;
12.         m1[n] *= weight.weight1;
13.         m2[n] *= weight.weight2;
14.         m3[n] *= weight.weight3;
15.         finalMatrix[i][n] = m0[n]+m1[n]+m2[n]+m3[n];
16.     }
17.     vertexData[i].pos = finalMatrix[i].MultiplyPoint3x4(verts[i]);
18.     vertexData[i].normal = finalMatrix[i].MultiplyVector(normals[i]);
19. }

And to generate the matrices[] Array:

currentBoneMatrix I'm creating using Matrix4x4.TRS() and bindPose comes from mesh.bindposes

/Patrik

 

Thanks man, works perfectly now, my stupid offset is finally gone!

Here's my final script... I think if one only needs the vertices and normals, it can transparently replace a static mesh.

Code (csharp):

1. using UnityEngine;
2.  
3. /// <summary>
4. /// Compute a skinned mesh's deformation.
5. ///
6. /// The script must be attached aside a SkinnedMeshRenderer,
7. /// which is only used to get the bone list and the mesh
8. /// (it doesn't even need to be enabled).
9. ///
10. /// Make sure the scripts accessing the results run after this one
11. /// (otherwise you'll have a 1-frame delay),
12. /// or use the OnResultsReady delegate.
13. /// </summary>
14. public class SkinnedMesh : CsBehaviour
15. {
16.     internal Mesh mesh;
17.     internal SkinnedMeshRenderer skin;
18.  
19.     internal int vertexCount;
20.     internal Vector3[] vertices;
21.     internal Vector3[] normals;
22.     public System.Action<SkinnedMesh> OnResultsReady;
23.  
24.     void Start()
25.     {
26.         skin = GetComponent<SkinnedMeshRenderer>();
27.         mesh = skin.sharedMesh;
28.  
29.         vertexCount = mesh.vertexCount;
30.         vertices = new Vector3[vertexCount];
31.         normals = new Vector3[vertexCount];
32.     }
33.    
34.     void LateUpdate()
35.     {
36.         Matrix4x4[] boneMatrices = new Matrix4x4[skin.bones.Length];
37.         for (int i = 0; i < boneMatrices.Length; i++)
38.             boneMatrices[i] = skin.bones[i].localToWorldMatrix * mesh.bindposes[i];
39.  
40.         for (int i = 0; i < mesh.vertexCount; i++)
41.         {
42.             BoneWeight weight = mesh.boneWeights[i];
43.  
44.             Matrix4x4 bm0 = boneMatrices[weight.boneIndex0];
45.             Matrix4x4 bm1 = boneMatrices[weight.boneIndex1];
46.             Matrix4x4 bm2 = boneMatrices[weight.boneIndex2];
47.             Matrix4x4 bm3 = boneMatrices[weight.boneIndex3];
48.  
49.             Matrix4x4 vertexMatrix = new Matrix4x4();
50.  
51.             for (int n = 0; n < 16; n++)
52.             {
53.                 vertexMatrix[n] =
54.                     bm0[n] * weight.weight0 +
55.                     bm1[n] * weight.weight1 +
56.                     bm2[n] * weight.weight2 +
57.                     bm3[n] * weight.weight3;
58.             }
59.  
60.             vertices[i] = vertexMatrix.MultiplyPoint3x4(mesh.vertices[i]);
61.             normals[i] = vertexMatrix.MultiplyVector(mesh.normals[i]);
62.         }
63.  
64.         if (OnResultsReady != null)
65.             OnResultsReady(this);
66.     }
67. }

And here's a little test script that draw the vertices and normals, turning your model into a porcupine:

Code (csharp):

1. using UnityEngine;
2.  
3. public class SkinnedMeshDebug : CsBehaviour
4. {
5.     void Start()
6.     {
7.         SkinnedMesh mesh = GetComponent<SkinnedMesh>();
8.         mesh.OnResultsReady += DrawVertices;
9.     }
10.     void DrawVertices(SkinnedMesh mesh)
11.     {
12.         for (int i = 0; i < mesh.vertexCount; i++)
13.         {
14.             Vector3 position = mesh.vertices[i];
15.             Vector3 normal = mesh.normals[i];
16.  
17.             Color color = Color.green;
18.             Debug.DrawWireCube(position, 0.1f * Vector3.one, color);
19.             Debug.DrawRay(position, normal, color);
20.         }
21.     }
22. }

Thanks a lot Aras metervara for introducing me to the wonderful craziness of matrices !

 

Thanks for all the code so far! It has been really helpful. I've (modified and) used these scripts to combine skinned meshes for characters but my animations on my characters stop after I combine.

Have you run into this problem? If so, how did you solve it?

I thought of saving the animation so I can recreate it on the parent node (the node with all of the combined meshes and materials) but there doesn't seem to be a way to access the keyframes from the Animation Curve.

How can I combine meshes/materials and not lose any animations on them?

 

Yes, CombineChildren is for static meshes only. I'm surprised it works at all because it needs a MeshFilter, and skinned meshes don't have that component by default, did you add it? Or are you not using CombineChildren?

My first maybe-not-so-useful advice would be to combine modeler-side, maybe use a 3DS/Maya script or whatever. I don't know that it would be a good idea to solve it Unity-side.

But it should probably be possible though, and definitely an interesting challenge! First you'd have to recreate the bone influences, I don't think CombineChildren does that. Second step would be the animations... are you sure they're lost? My guess is the transforms are still animated but they don't deform the mesh because the SkinnedMeshRenderer isn't linked to the combined mesh.
But if they are lost, you'd have to combine them too (good luck with that ). AnimationClip can SetCurve() but not get it (shame, that), but you can access the complete curve (keyframes and all), but only in the editor (shame, again), so maybe you could combine everything at edit-time? I've never tried to save a script-created mesh, don't know if the data gets serialized... if it doesn't, a) hint hint UT, and b) you'd have to store all the vertices, influences and curves on your own, then recreate everything on Start. This would seem so much trouble that again, I'd try to do it modeler-side.

Lastly, please note that I'm a big pompous ass because I honestly don't know crap about animation, I can barely open Cheetah (discovered 2 days ago that it's able to open .3ds files natively), so all the above comes from a scholar knowledge of the API, not practical experience.
That said, enjoy your pain and let me know how that went !

 

Yeah I took that script as well as the great code you've submitted and used it to merge all the meshes into 1 MeshFilter... but now I realized yesterday (and your post confirms) that I need to merge the meshes into a SkinnedMeshRenderer with all the bone influences... that would probably explain (as you mentioned) why it's not animating... nothing is really lost, it just wasn't done correctly.

Unfortunately, our design limits us so that we can't do it modeler-side (I don't think). We want to do it programmatically so we can create avatars for our game that have customizable clothing. Meaning, swap pieces in and out and then when the player is done, we'll merge it all down, for performance reasons. And since each clothing piece is animated separately we need those animations to work after we merge...

Double shame... Anyone from UT can explain why we don't have access at the Keyframes or curves?

No worries, mate. I appreciate the feedback and you've been incredibly helpful in the way of scripts so far!

 

Oh man that's brilliant, didn't think you were aiming at something that dynamic! In that case, you only need to extract the curves at import-time and serialize them, then you'll have all the ingredients to do your combine magic at runtime... which you still have to figure out. Oh, the fun .

I'd love to get my hands on that once you're done! One of our client has also asked us about an avatar S*** (ok, not surprising, who isn't doing/planning an MMO these days :?), but we don't really know what's even possible yet... I smell there's gonna be some dynamic skinning somewhere.
Like doing a sort of inflatable body à-la Mii/Xbox 360 seems doable, but putting say a coat on top of the guy seems more difficult already... and I hadn't even thought of combining the guy and its clothes!

 

Obviously, I wouldn't be able to share any finished work with everyone publicly but I'm not so selfish that I wouldn't want to share back with the community which has so graciously shared with me. I have seen from multiple different posts throughout the forums (as I was looking for answers to how to solve this) that there are many others trying to do this exact same thing. I think I've already given away the steps we are taking to do this and it is mostly based on code that is freely available in the forums, including a great deal of code you've submitted. I will be happy to follow up on this thread whether we are successful or not which should encourage others that the answers are out there (or in here, depending on how you look at it).

 

Sure I didn't mean the full-blown avatar system, just the bone-combine and animation-combine parts.
Although if you happen to have any good links to share regarding avatar systems, I'm all ears (white paper, postmortem, whatever).

 

This is funny, I only just worked out this same process a few months ago in a face matching setup.

I designed a script to deform a face mesh according to a set of control points (rigged as bones) and then dump the new vertex data into another mesh rigged and animated for facial expressions. My code is really similar though:

Code (csharp):

1. Vector3[] verts = new Vector3[deformMesh.vertices.Length];
2. for (int i=0;i<verts.Length;i++)
3. {
4.     //This applies the skeletal subsurface deformation algorithm V(f) = Sum (M*M^-1*V*w) where we've pre-calculated M*M^-1 for each bone.
5.     verts[i] = Vector3.zero;
6.     Vector3[] boneDeform = new Vector3[4];
7.            
8.     //Calculate the partial deform for each bone influencing the vertex
9.     boneDeform[0] = (boneTransform[deformMesh[0].boneWeights[i].boneIndex0].MultiplyPoint3x4(deformMesh.vertices[i]) * deformMesh.boneWeights[i].weight0);
10.     boneDeform[1] = (boneTransform[deformMesh.boneWeights[i].boneIndex1].MultiplyPoint3x4(deformMesh.vertices[i]) * deformMesh.boneWeights[i].weight1);
11.     boneDeform[2] = (boneTransform[deformMesh.boneWeights[i].boneIndex2].MultiplyPoint3x4(deformMesh.vertices[i]) * deformMesh.boneWeights[i].weight2);
12.     boneDeform[3] = (boneTransform[deformMesh.boneWeights[i].boneIndex3].MultiplyPoint3x4(deformMesh.vertices[i]) * deformMesh.boneWeights[i].weight3);
13.  
14.     //Now sum the partial deforms to find the new vertex position.
15.     for (int q=0;q<boneDeform.Length;q++)
16.     {
17.         verts[i] += boneDeform[q];
18.     }
19. }
20. faceMesh.vertices = verts;
21. faceMesh.RecalculateBounds();

It would be fantastic if we could read the skinned vertex positions directly from Unity, though. The above code is incredibly slow on a detailed mesh - in my case I had to break up the loop and multithread it to get acceptable performance on around 10k vertices.

Edit: I should have looked closer at the previous code examples. Is it generally faster to lump the matrices into one matrix before multiplying the vertex position? I'll have to try that out.

 

Hello there,


just wanted to add a potential solution to your animation saving problem, marblez.

Here is the workflow :

1) Just do the modeling+animation work on 3rd party tool side, as usual.
2) Export a animated version of bones only, without skins.
3) Export separate non-animated versions of all of your different skin parts, only keeping the effective bones in each file, and of course, the mesh.
4) under Unity, just combine your fresh imported unanimated skins to a one-piece Mesh type, as you like, with some code. There are plenty of them, and Benblo's one seems to be good at it.
5) Instantiate your animated bone under your target gameobject.
6) set your bone SkinnedMeshRenderer.sharedMesh to your recently code-generated mesh.

7) profit



That works for me.



Cheers, and thank you again, all of you, for your great sense of community sharing.

 

Just wanted to say thank you Aras and Benblo (and others).

This is indeed a feature I would like to have... (I'm trying to slice up a skinned mesh)

 

I'm having speed issues as well here. I assume that I'm going to have to break this up and multi-thread it to get appropriate performance, but I just wanted to check if anyone has had any success doing this quickly. Basically what I need to do is get the positions of all the vertices (over 10K of them), then move the bones, and get the positions again.
This is taking about 7-8 seconds each time I need to get the verts (for a total of about 15 seconds) using the code that benlo posted. With the current implementation it also totally locks up the application during those 15 seconds, but I'm sure I can fix that with some threading trickery.

Any suggestions?

Thanks,
Liron

 

Soooo... 2 1/2 years later, any updates? I've got this spikey thing animated by a bunch of bones and I need it to roll around as it deforms, so I'd love a mesh collider. Any thoughts? Did anyone manage to create a good bit of code as discussed above?

Cheers,

 

This was exactly what I needed!

If you don't mind I converted it to JavaScript.
I'l post it here for the people in need for the JavaScript version.

Code (csharp):

1.  
2. #pragma strict
3. /// <summary>
4.  
5. /// Compute a skinned mesh's deformation.
6.  
7. ///
8.  
9. /// The script must be attached aside a SkinnedMeshRenderer,
10.  
11. /// which is only used to get the bone list and the mesh
12.  
13. /// (it doesn't even need to be enabled).
14.  
15. ///
16.  
17. /// Make sure the scripts accessing the results run after this one
18.  
19. /// (otherwise you'll have a 1-frame delay),
20.  
21. /// </summary>
22.  
23.     @HideInInspector
24.     var mesh: Mesh;
25.     @HideInInspector
26.     var skin: SkinnedMeshRenderer;
27.  
28.  
29. @HideInInspector
30.     var vertexCount:int=0;
31. @HideInInspector
32.     var vertices:Vector3[];
33. //@HideInInspector
34.   //  var normals:Vector3[];
35.  
36.  
37.  
38.  
39.  
40.  
41.     function Start() {
42.  
43.         skin = GetComponent(SkinnedMeshRenderer);
44.  
45.         mesh = skin.sharedMesh;
46.  
47.  
48.  
49.         vertexCount = mesh.vertexCount;
50.  
51.         vertices = new Vector3[vertexCount];
52.  
53.       //  normals = new Vector3[vertexCount];
54.      
55.        //animation example
56.        for (var b:int= 0; b < mesh.vertexCount; b++){
57.             var cube : GameObject= new GameObject.CreatePrimitive(PrimitiveType.Cube);
58.             cube.name=b.ToString();
59.             cube.transform.localScale.x=0.1;
60.             cube.transform.localScale.y=0.1;
61.             cube.transform.localScale.z=0.1;
62. }
63.     }
64.  
65.    
66.  
67.     function LateUpdate(){
68.  
69.         var boneMatrices: Matrix4x4[]  = new Matrix4x4[skin.bones.Length];
70.  
71.         for (var i:int= 0; i < boneMatrices.Length; i++)
72.  
73.             boneMatrices[i] = skin.bones[i].localToWorldMatrix * mesh.bindposes[i];
74.          
75.  
76.  
77.         for (var b:int= 0; b < mesh.vertexCount; b++){
78.  
79.              var weight:BoneWeight = mesh.boneWeights[b];
80.  
81.  
82.  
83.               var bm0:Matrix4x4 = boneMatrices[weight.boneIndex0];
84.  
85.               var bm1:Matrix4x4 = boneMatrices[weight.boneIndex1];
86.  
87.               var bm2:Matrix4x4 = boneMatrices[weight.boneIndex2];
88.  
89.               var bm3:Matrix4x4 = boneMatrices[weight.boneIndex3];
90.  
91.  
92.  
93.              var vertexMatrix:Matrix4x4 = new Matrix4x4();
94.  
95.  
96.  
97.             for (var n:int= 0; n < 16; n++){
98.  
99.                 vertexMatrix[n] =
100.  
101.                     bm0[n] * weight.weight0 +
102.  
103.                     bm1[n] * weight.weight1 +
104.  
105.                     bm2[n] * weight.weight2 +
106.  
107.                     bm3[n] * weight.weight3;
108.  
109.             }
110.  
111.  
112.  
113.             vertices[b] = vertexMatrix.MultiplyPoint3x4(mesh.vertices[b]);
114.             //   normals[i] = vertexMatrix.MultiplyVector(mesh.normals[i]);
115.            
116.             //animation example
117.             var fetch= GameObject.Find( b.ToString());
118.             fetch.transform.position = vertices[b];
119.             }
120.  
121.  
122.            
123.        
124.  
125.     }
126.  
127.  
128.  

 

Unity has SkinnedMeshRenderer.BakeMesh now.

--Eric

 

Yes, but I used that before this script, the problem is that the vertex offset is always wrong, when my cubes follow the vertices in this script it is spot on, when I use bakemesh they are oriented over the wrong axis, creating the cubes to move in the wrong direction.

 

How My God you try kill me ! ? how i can understand that's !

 

using bakemesh... too slow for realtime

 

Hi guys !

I have a riddle for You

How to partially transform a point by matrix like: [ V = m.MultiplyPoint3x4(V) * weight ] and then back to it's original position ?? If there is no "weight" or weight = 1 - It's just m.inverse used to do this.

I need it to transform several meshes to their skeleton space, make some adjustments to normals and positions and then transform their back for normal animation proccesing.
Just can't figure it out - I feel, the solution is simple but maybe I'm just tired ..

Please help, Thank You

Stan

 

1. for (int n = 0; n < 16; n++)
2. {
3. vertexMatrix[n] =
4. bm0[n] * weight.weight0 +
5. bm1[n] * weight.weight1 +
6. bm2[n] * weight.weight2 +
7. bm3[n] * weight.weight3;
8. }

I got Matrix4x4 .get_Item() called over 100 thousands for less than 10k vertices. Is there anyway to avoid get_Item() call?
I've compared List.get_Item() with array[], the second one is much faster. (I really thought List was nothing but a wrap of array[] like vector in cpp, never thought calling List[] would cause me problem in a large loop).

 

Code (CSharp):

1. Matrix4x4 vm = new Matrix4x4();
2.                 vm.m00 = bm0.m00 * weight.weight0 + bm1.m00 * weight.weight1 + bm2.m00 * weight.weight2 + bm3.m00 * weight.weight3;
3.                 vm.m01 = bm0.m01 * weight.weight0 + bm1.m01 * weight.weight1 + bm2.m01 * weight.weight2 + bm3.m01 * weight.weight3;
4.                 vm.m02 = bm0.m02 * weight.weight0 + bm1.m02 * weight.weight1 + bm2.m02 * weight.weight2 + bm3.m02 * weight.weight3;
5.                 vm.m03 = bm0.m03 * weight.weight0 + bm1.m03 * weight.weight1 + bm2.m03 * weight.weight2 + bm3.m03 * weight.weight3;
6.  
7.                 vm.m10 = bm0.m10 * weight.weight0 + bm1.m10 * weight.weight1 + bm2.m10 * weight.weight2 + bm3.m10 * weight.weight3;
8.                 vm.m11 = bm0.m11 * weight.weight0 + bm1.m11 * weight.weight1 + bm2.m11 * weight.weight2 + bm3.m11 * weight.weight3;
9.                 vm.m12 = bm0.m12 * weight.weight0 + bm1.m12 * weight.weight1 + bm2.m12 * weight.weight2 + bm3.m12 * weight.weight3;
10.                 vm.m13 = bm0.m13 * weight.weight0 + bm1.m13 * weight.weight1 + bm2.m13 * weight.weight2 + bm3.m13 * weight.weight3;
11.                
12.                 vm.m20 = bm0.m20 * weight.weight0 + bm1.m20 * weight.weight1 + bm2.m20 * weight.weight2 + bm3.m20 * weight.weight3;
13.                 vm.m21 = bm0.m21 * weight.weight0 + bm1.m21 * weight.weight1 + bm2.m21 * weight.weight2 + bm3.m21 * weight.weight3;
14.                 vm.m22 = bm0.m22 * weight.weight0 + bm1.m22 * weight.weight1 + bm2.m22 * weight.weight2 + bm3.m22 * weight.weight3;
15.                 vm.m23 = bm0.m23 * weight.weight0 + bm1.m23 * weight.weight1 + bm2.m23 * weight.weight2 + bm3.m23 * weight.weight3;
16.                

Instead of using a for loop and [], directly access the m00 - m33 will avoid the indexer's method call. This will improve the performance quite a lot! And the m30 - m33 could be ignored, if you look into Matrix4x4::MultiplyPoint3x4 method, it never uses the m30 - m33.

 

I was trying to get the skinned vertices and found this post. I tried the scripts by @benblo using a mesh with 8k verts. The main problem is that getting the vertices, normals or bones weights creates a copy. This creates about 3GB of data per call and runs at about 0.2 fps on my machine.

Performance tests:
Standard skinned mesh: 1,400 fps
Orig code: 0.2fps
Store single copy of mesh data: 56 fps
Store data and matrix suggestion by @YoungXi: 120 fps
With bakemesh: 1,100 fps

 

Sorry to bring this thread back. I'm not sure If I follow the results from your test. What approach gave you the best results?

Thank you

 

I uploaded the scripts for reference. I forgot to get the data from the baked mesh and to transform the points to world space. My new stats are:

No animation: 2,000 fps
Standard animation 1,500 fps
Only calling bake mesh 1,300 fps
Baked mesh without transformed points but calling GetVertices() and GetNormals(): 540 fps
Baked mesh with transforms (could be improved): 230 fps
Optimised code: 100 fps
Orig code: 0.2 fps

So the optimised code is about 500 times faster than the original but bake mesh is about 2-5 times faster than that. If I needed the data in realtime I’d probably use a compute shader to transfrom the points after calling bake mesh.

 

Awesome! thanks, @High_Noon, I will check it out now.

 

@CodeKiwi this is a great asset, thanks. Works really fine for blendshapes, allowing me to spawn/transform objects at vertices of deformed SMR in realtime. Good job!

 

This really is a gem, thank you all so much.