Morph Target Script

Hi all,

Seems like folks ask fairly often about morph targets with Unity:

http://forum.unity3d.com/viewtopic.php?t=9103
http://forum.unity3d.com/viewtopic.php?t=6199
http://forum.unity3d.com/viewtopic.php?t=14235
http://forum.unity3d.com/viewtopic.php?t=9076
etc...

Well, inspired by Eric's MeshMorper script on the wiki, and with some help from a friend who works with morph targets for a living, I managed to create a simple script (attached) that handles mesh morphing between multiple weighted targets.

Here's a sample where I used it to set up some face customization using sliders to control the different morph targets: http://breccasfall.com/unity/createaface.html (It takes a few seconds to load. )



Like the MeshMorpher script, which was the starting point for mine, the target for each morph "attribute" comes from a separate mesh, and each target mesh must have the same number of vertices laid out in the same order as the source mesh.

This script could easily be used for basic facial animations. With some more work, it could also probably be extended to work with skeletal animations as well as mesh morphs.

Enjoy!

-B

Edit: Attached a package showing how my scene is set up.

 

Just want to say it's not my script, I only translated the C# script which I think was by Joachim.

--Eric

 

My bad; thanks for correcting me. Just saw your name next to the JavaScript version and assumed you wrote the original.

 

hi,
would you be able to publish the project file with your script inlcuded?
I couldnt figure out how it works and whats the big difference to erics/joachims scripts.
We have problems with the meshs, since they need to be sewed and after sewing the uv-texture doesnt fit anymore

cheers
d

 

I am at work now, but I can publish the project file later today.

The original MeshMorpher script lets you morph between several versions of an entire mesh. While there can be any number of targets, only one of them can be in play at a time. It won't let you combine the results of multiple morph targets.

My script registers which vertices in the mesh are affected by each morph target. So when you adjust how much weight a target has (like with a GUI slider), the script will morph the proper vertices for that target, but it will also take into account all the other targets that affect those vertices. As a result, you can create more complex morphing effects.

To get it to work, you need to set up the meshes and call the SetMorph() function from another script. In my example, I call SetMorph() from the GUI sliders when any of them are changed.

Hope that helps. I'll post the project file later.

 

Excellent work, this works fantastically (at least from your example). I'll have to give this a try sometimes.

 

Uploaded a package with my sample scene to the original post.

 

Ooh, thanks a heap for this! Doing facial anims with bones is just too much hassle for me so this is great news

 

hieeee guys
well i havent seen ur code yet but surly go through it.
i have manage to get the Morphing of Face on IPhone its working fine but it i still want to increase the speed for the Morphing.I have loaded a single Obj Mesh morphing some special parts of face such as chin, nose, lips eyes, ears forehead i have subdivided this parts im doing calculations on only one part which ever is selected. can anyone guide me to increase the speed of Morphing???

The model which im working on is Having approx 900 faces.
im doing calculations on per-vertex base so its obvious tht it will go slow when the face count is more

the thing i would like to know is tht is there any option for scaling / translating Set of Triangles 2gether rather than going through the set of Vertices present in tht Triangles Set?????

 

I'm trying to figure out how to set up my character in Max.

I can't read the Cheetah .jas file to see how you set up the mesh and there is only one .jas file in the meshes folder. I was rather surprised not to see several .jas files in the meshes folder.

How did you set up your mesh in your modeler where there is only one .jas file?

So in Max I need to create a mesh for each morph?

I have clothing which will be turned off and on, so those will be on the master mesh and the face change morphers will be as you set up?

 

The .jas file contains all the different meshes. When the Cheetah file imports, all the meshes import with it, and they can then be used by the morph script.

You do need one mesh for each morph -- it shouldn't matter whether they're in the same file or different ones. In my case I just created them all in the same file.

Hope that helps.

-B

 

I just wanted to add that I think this is a great project. Good work.

 

Finally we have been able to get this to work (from Max) thanks trypho for such a useful tool.

Although it is a bit slow to load, its effective.
In max we have a avatar and we made morphs from this original one, so all the verts are the same. We had vocal, eyebrows, emo, quite a bit of them.
Exported this file as an fbx.
In unity played around with the script,
Made sure (like the script requires) check "calculate normals" and set angle to 180.
some of the meshes do get a little deformed, not sure why but looks fixable (I hope)
Ran the script.(modified by our programmers )

Now on to blending this with custom layered animation....

 

I found if you export models with normals using fbx (from packages that support it) then unity won't need to recalculate them and won't add extra points to your model, which is very important when trying to morph them.

 

Hey, this project has been a huge help for me, but I wanted to ask if anyone who might have done something similar had a more elegant solution to scripting these to trigger off keyboard input. Here's the script I've got triggering them, there's 20 morphs but I've only bound the top row of keys to them, as this is more of just a proof of concept.

Code (csharp):

1. var speed = 0.1;
2. var faceSpeed = new Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
3. var morphObject : MorphTargets;
4. var morphspeed = 0.04;
5.  
6. function Update () {
7.  
8.    x= Input.GetAxis ("Horizontal") * speed;
9.    // Rotate around works in radians
10.    transform.RotateAround (Vector3.up, -x);
11.    if (Input.GetKey (KeyCode.Q))
12.    {
13.     faceSpeed[0] = morphspeed ;
14.    }
15.    else if (Input.GetKey (KeyCode.W))
16.    {
17.     faceSpeed[1] = morphspeed ;
18.    }
19.    else if (Input.GetKey (KeyCode.E))
20.    {
21.     faceSpeed[2] = morphspeed ;
22.    }
23.    else if (Input.GetKey (KeyCode.R))
24.    {
25.     faceSpeed[3] = morphspeed ;
26.    }
27.    else if (Input.GetKey (KeyCode.T))
28.    {
29.     faceSpeed[4] = morphspeed ;
30.    }
31.    else if (Input.GetKey (KeyCode.Y))
32.    {
33.     faceSpeed[5] = morphspeed ;
34.    }
35.    else if (Input.GetKey (KeyCode.U))
36.    {
37.     faceSpeed[6] = morphspeed ;
38.    }
39.    else if (Input.GetKey (KeyCode.I))
40.    {
41.     faceSpeed[7] = morphspeed ;
42.    }
43.    else if (Input.GetKey (KeyCode.O))
44.    {
45.     faceSpeed[8] = morphspeed ;
46.    }
47.    else if (Input.GetKey (KeyCode.P))
48.    {
49.     faceSpeed[9] = morphspeed ;
50.    }
51.     else if (Input.GetKey (KeyCode.Alpha2))
52.    {
53.     for(var i = 0; i<20; i++)
54.         {
55.             morphObject.attributeProgress[i] = 0;
56.         }
57.         morphObject.SetMorph();
58.     }
59.     else if (Input.GetKey (KeyCode.UpArrow)){
60.         faceSpeed += 0.01;
61.     }
62.     else if (Input.GetKey (KeyCode.DownArrow)){
63.         faceSpeed -= 0.01;
64.     }
65.     updateTalk();
66. }
67.  
68. function updateTalk()
69. {
70.     for(var i = 0; i<20; i++)
71.     {
72.         morphObject.attributeProgress[i] += faceSpeed[i];
73.         if(morphObject.attributeProgress[i] >= 1)
74.             faceSpeed[i] -= faceSpeed[i]*2;
75.         if(morphObject.attributeProgress[i] <= 0)
76.         {
77.             morphObject.attributeProgress[i] = 0;
78.             faceSpeed[i] = 0;
79.         }
80.     }
81.     morphObject.SetMorph();
82. }

EDIT: Also, I'm thinking I'm going to need to attach this to a skinned mesh renderer instead of just a mesh renderer, I'm pretty new to all of this, does this sound feasible?

 

Mostly we have been able to use this code to import "morphs" into unity with success. We are having an issue with some elements.
We turn all the normals to 180 before exporting the fbx from max.
Import into Unity.

We don't have to "calculate normals" on ALL models because some of them work on the fly.
We have to calculate normals when unity says that all the vertices do not match. Although all the morphs are taken from the main mesh. So they should be all the same amount of verts...

Anyone know of a solution?

 

Hi,

I somehow felt, that the performance of the JavaScript was to weak, so I wrote a C# version, which improves my fps from 7 to 400 (in my special case).

Code (csharp):

1.  
2. /**
3.  * Just a 20min port of trypho's JS script to C# by skahlert
4.  * In my experience the performance improved significantly (about 57x the speed of the JavaScript).
5.  * Have fun with it!!
6.  *
7.  * If you find a nicer way of compensating the lack of dynamic arrays (line 81 and following) it would be nice to hear!
8.  *
9.  */
10.  
11.  
12. using System;
13. using UnityEngine;
14.  
15. class MyMeshMorpher : MonoBehaviour
16. {
17.     internal class BlendShapeVertex
18.     {
19.         public int originalIndex;
20.         public Vector3 position;
21.         public Vector3 normal;
22.     }
23.  
24.     internal class BlendShape
25.     {
26.         public BlendShapeVertex[] vertices;// = new Array();
27.     }
28.  
29.     public String[] attributes; //Names for the attributes to be morphed
30.     public Mesh sourceMesh; //The original mesh
31.     public Mesh[] attributeMeshes; //The destination meshes for each attribute.
32.     public float[] attributeProgress;
33.  
34.     private BlendShape[] blendShapes;
35.     private Mesh workingMesh;
36.  
37.     void Awake()
38.     {
39.         for (int i = 0; i < attributeMeshes.Length; i++)
40.         {
41.             if (attributeMeshes[i] == null)
42.             {
43.                 Debug.Log("Attribute " + i + " has not been assigned.");
44.                 return;
45.             }
46.         }
47.  
48.         //Populate the working mesh
49.         MeshFilter filter = gameObject.GetComponent(typeof(MeshFilter)) as MeshFilter;
50.         filter.sharedMesh = sourceMesh;
51.         workingMesh = filter.mesh;
52.  
53.         //Check attribute meshes to be sure vertex count is the same.
54.         int vertexCount = sourceMesh.vertexCount;
55.         //Debug.Log("Vertex Count Source:"+vertexCount);
56.         for (int i = 0; i < attributeMeshes.Length; i++)
57.         {
58.             //Debug.Log("Vertex Mesh "+i+":"+attributeMeshes[i].vertexCount);
59.             if (attributeMeshes[i].vertexCount != vertexCount)
60.             {
61.  
62.                 Debug.Log("Mesh " + i + " doesn't have the same number of vertices as the first mesh");
63.                 return;
64.             }
65.         }
66.  
67.         //Build blend shapes
68.         BuildBlendShapes();
69.     }
70.  
71.     void BuildBlendShapes()
72.     {
73.  
74.         blendShapes = new BlendShape[attributes.Length];
75.  
76.         //For each attribute figure out which vertices are affected, then store their info in the blend shape object.
77.         for (int i = 0; i < attributes.Length; i++)
78.         {
79.             //Populate blendShapes array with new blend shapes
80.             blendShapes[i] = new BlendShape();
81.  
82.             /** TODO: Make this a little more stylish!
83.              *  UGLY hack to compensate the lack of dynamic arrays in C#. Feel free to improve!
84.              */
85.             int blendShapeCounter = 0;
86.             for (int j = 0; j < workingMesh.vertexCount; j++)
87.             {
88.                
89.                 if (workingMesh.vertices[j] != attributeMeshes[i].vertices[j])
90.                 {
91.                     blendShapeCounter++;
92.                 }
93.             }
94.            
95.             blendShapes[i].vertices = new BlendShapeVertex[blendShapeCounter];
96.             blendShapeCounter = 0;
97.             for (int j = 0; j < workingMesh.vertexCount; j++)
98.             {
99.                 //If the vertex is affected, populate a blend shape vertex with that info
100.                 if (workingMesh.vertices[j] != attributeMeshes[i].vertices[j])
101.                 {
102.                     //Create a blend shape vertex and populate its data.
103.                    
104.                     BlendShapeVertex blendShapeVertex = new BlendShapeVertex();
105.                     blendShapeVertex.originalIndex = j;
106.                     blendShapeVertex.position = attributeMeshes[i].vertices[j] - workingMesh.vertices[j];
107.                     blendShapeVertex.normal = attributeMeshes[i].normals[j] - workingMesh.normals[j];
108.  
109.                     //Add new blend shape vertex to blendShape object.
110.                     blendShapes[i].vertices[blendShapeCounter]=blendShapeVertex;
111.                     blendShapeCounter++;
112.                 }
113.             }
114.  
115.             //Convert blendShapes.vertices to builtin array
116.             //blendShapes[i].vertices = blendShapes[i].vertices.ToBuiltin(BlendShapeVertex);
117.         }
118.     }
119.  
120.  
121.     public void SetMorph()
122.     {
123.  
124.         //Set up working data to store mesh offset information.
125.         Vector3[] morphedVertices = sourceMesh.vertices;
126.         Vector3[] morphedNormals = sourceMesh.normals;
127.  
128.         //For each attribute...
129.         for (int j = 0; j < attributes.Length; j++)
130.         {
131.             //If the weight of this attribute isn't 0  
132.             if (!Mathf.Approximately(attributeProgress[j], 0))
133.             {
134.                 //For each vertex in this attribute's blend shape...
135.                 for (int i = 0; i < blendShapes[j].vertices.Length; i++)
136.                 {
137.                     //...adjust the mesh according to the offset value and weight
138.                     morphedVertices[blendShapes[j].vertices[i].originalIndex] += blendShapes[j].vertices[i].position * attributeProgress[j];
139.                     //Adjust normals as well
140.                     morphedNormals[blendShapes[j].vertices[i].originalIndex] += blendShapes[j].vertices[i].normal * attributeProgress[j];
141.                 }
142.             }
143.         }
144.  
145.         //Update the actual mesh with new vertex and normal information, then recalculate the mesh bounds.     
146.         workingMesh.vertices = morphedVertices;
147.         workingMesh.normals = morphedNormals;
148.         workingMesh.RecalculateBounds();
149.     }
150.  
151.  
152.  
153. }
154.  
155.  
156.  

Hope it helps!

Cheers,
Stefan

 

Oops,

I forgot the RequireComponents... Here's the updated script.

 

I downloaded the file : morphtargets.unityPackage but when i try to import it into Unity 3D (2.5 Pro) i get this error message : Error while importing Package: Couldn't decompress package.

Any ideas ?

 

i have used this script of yours and it works perfectly for my needs.


i have also made an editor script that allows you to better control the scene where you use the morphs (specially in crowded scenes that need model updates)

find it here:

http://forum.unity3d.com/viewtopic.php?p=241386#241386






and... thankz!

 

I just noticed a problem... I tried hte blendshape script. It works great! One problem though. As soon as i apply a material with a normal map... well the normal map material doesnt work... it looks all glitchy.. anyone have any idea why and how I could fix this?
-----------------
My mistake, it was another issue.

Though I do have another problem now, basically it doesnt see it as the same vertex count or mesh or whatever if i dont use calculate normals at 180... even though they should all be soft already. And when I do do calculate normals it creates a rather ugly lookn distortion down the center of my characters face where his texture is mirrored over.

It is doing this for the normal map.. though the version of the model without recalculation of normals looks fine. The normals along the center are soft in both versions... rather strange issue.

 

I've added a couple of blendshapes to the script, but the buildBlendShapes() function takes allmost a minute to complete when added 6 blendshapes. Is there any way to speed this up?

great script by the way

 

can someone post a working example? I downloaded the zip but for some reason the example does not play anymations when I press play

 

Can this work with a Skinned Mesh Renderer? I'm trying different things but no luck.

 

Anyone? :?

 

@jak: I think it's possible to pass the mesh in your skinned mesh renderer to the script and back.

 

I tried several things like that but wasn't able to get it working so I ended up not using morphing. If anyone does have a working example of this with a skinned mesh renderer it would be helpful.

 

There's no problem in using morphing with SkinnedMeshRenderer, you just need to use SkinnedMeshRenderer instead of MeshFilter and .sharedMesh intead of .mesh .

 

How come when I use MorphTargets in my scripts, I get an error message saying 'MorphTargets' does not denote a valid type?

 

I want to prepare my model in maya for use with this script. I currently use blend shapes in maya, so I need to export 'simple' FBXs with my target morphs for each target.

Does anyone know how to do this in maya? I assume people using this script encountered the same need...

(Assume I can manually alter the blend shapes to get the model to the target state)

Thanks!

 

This fix the problem:

PHP:

using System;
using System.Collections.Generic;
using UnityEngine;

[RequireComponent(typeof(MeshFilter))]
[RequireComponent(typeof(MeshRenderer))]

class MorphTargets : MonoBehaviour
{
    internal class BlendShapeVertex
    {
        public int originalIndex;
        public Vector3 position;
        public Vector3 normal;
    }

    internal class BlendShape
    {
        public BlendShapeVertex[] vertices;// = new Array();
    }

    public String[] attributes; //Names for the attributes to be morphed
    public Mesh sourceMesh; //The original mesh
    public Mesh[] attributeMeshes; //The destination meshes for each attribute.
    public float[] attributeProgress;

    private BlendShape[] _blendShapes;
    private Mesh _workingMesh;

    void Awake()
    {
        for (int i = 0; i < attributeMeshes.Length; i++)
        {
            if (attributeMeshes[i] == null)
            {
                Debug.Log("Attribute " + i + " has not been assigned.");
                return;
            }
        }

        //Populate the working mesh
        MeshFilter filter = (MeshFilter)gameObject.GetComponent(typeof(MeshFilter));
        filter.sharedMesh = sourceMesh;
        _workingMesh = filter.mesh;

        //Check attribute meshes to be sure vertex count is the same.
        int vertexCount = sourceMesh.vertexCount;
        //Debug.Log("Vertex Count Source:"+vertexCount);
        for (int i = 0; i < attributeMeshes.Length; i++)
        {
            //Debug.Log("Vertex Mesh "+i+":"+attributeMeshes[i].vertexCount); 
            if (attributeMeshes[i].vertexCount != vertexCount)
            {

                Debug.Log("Mesh " + i + " doesn't have the same number of vertices as the first mesh");
                return;
            }
        }

        //Build blend shapes
        BuildBlendShapes();
    }

    void BuildBlendShapes()
    {

        _blendShapes = new BlendShape[attributes.Length];

        //For each attribute figure out which vertices are affected, then store their info in the blend shape object.
        for (int i = 0; i < attributes.Length; i++)
        {
            //Populate blendShapes array with new blend shapes
            _blendShapes[i] = new BlendShape();

            Vector3[] attributeMeshVertices = attributeMeshes[i].vertices;
            Vector3[] attributeMeshNormals = attributeMeshes[i].normals;
            Vector3[] workMeshVertices = _workingMesh.vertices;
            Vector3[] workMeshNormals = _workingMesh.normals;

            List<BlendShapeVertex> vertices = new List<BlendShapeVertex>(_workingMesh.vertexCount);

            for (int j = 0; j < _workingMesh.vertexCount; j++)
            {
                //If the vertex is affected, populate a blend shape vertex with that info
                if (workMeshVertices[j] != attributeMeshVertices[j])
                {
                    //Create a blend shape vertex and populate its data.
                    
                    BlendShapeVertex blendShapeVertex = new BlendShapeVertex();
                    blendShapeVertex.originalIndex = j;
                    blendShapeVertex.position = attributeMeshVertices[j] - workMeshVertices[j];
                    blendShapeVertex.normal = attributeMeshNormals[j] - workMeshNormals[j];
                    vertices.Add(blendShapeVertex);

                }
            }

            _blendShapes[i].vertices = vertices.ToArray();
        }
    }


    public void SetMorph()
    {

        //Set up working data to store mesh offset information.
        Vector3[] morphedVertices = sourceMesh.vertices;
        Vector3[] morphedNormals = sourceMesh.normals;

        //For each attribute...
        for (int j = 0; j < attributes.Length; j++)
        {
            //If the weight of this attribute isn't 0    
            if (!Mathf.Approximately(attributeProgress[j], 0))
            {
                BlendShape shape = _blendShapes[j];
                //For each vertex in this attribute's blend shape...
                for (int i = 0; i < shape.vertices.Length; i++)
                {
                    //...adjust the mesh according to the offset value and weight
                    morphedVertices[shape.vertices[i].originalIndex] += shape.vertices[i].position * attributeProgress[j];
                    //Adjust normals as well
                    morphedNormals[shape.vertices[i].originalIndex] += shape.vertices[i].normal * attributeProgress[j];
                }
            }
        }

        //Update the actual mesh with new vertex and normal information, then recalculate the mesh bounds.        
        _workingMesh.vertices = morphedVertices;
        _workingMesh.normals = morphedNormals;
        _workingMesh.RecalculateBounds();
    }

}


Access to Mesh.vertices and Mesh.normals every time copy this arrays. Do it once for all access to it elements.

 

Wow men excellent great aport

 

I am having trouble with this.
In unity the faces just fracture, which is cool but undesirable.

I am producing targets in max using the Morpher modifier.
Before export I collapse the stacks/convert to mesh.

any suggestions would be appreciated.

 

Thanks for your share trypho.

I have two .obj files directly exported from FaceGen tool, one for base mesh and the second one as the target mesh and both have the same vertex and triangle number, i am pretty sure about it. But still the verts are scattering around and my beautiful face model turning into pretty much the same with spawn, subzero, scorpion etc. has.

Do you guys have any idea about it? Have you guys had any difficulties when working on the .obj files directly exported from FaceGen tool? Do you guys think that the FaceGen tool might mix up some of the vertices orders?

Your answers will be appreciated. Thanks in advance.

 

Same problem related to morph target exporting correctly from Max.

My targets have different number of vertices / number of triangles once they get imported into Unity.
Although I did set Import Normals to None in Unity Importer and also disabled Tangents calculation.

Can anyone post a tutorial to how to do this properly in 3DS Max?
I'm sure it will help a lot of people.

 

I think you will need to look at another system you wont get it to work too well with script as the orders get totally messed up when importing into Unity. You need a proper morphing system, ideally one with custom exporter for max, I use MegaFiers and it works perfectly, has an exporter doesnt get ugly smoothing like the other morphers I have seen, is super fast (and multi threaded) and you get a load of extras. http://forum.unity3d.com/threads/89750-Mega-Fiers.-A-mesh-deformation-system-RELEASE

 

Did someone try to use the morphing with an animated character?
Because I tryed to morphe my character while moving, but the morphed mesh appears in t-pose and didnt react to the morphing.

 

If you are using the script on this page then you will need to change it a little to work with skinned meshes, also if you are using bump maps or normal maps then this script doesn't deal with tangents and if you have a mesh with any kind of special smoothing or complex sub meshes then again this script is not going to give great results. Search the forum for other, although not free, complete morphing solutions.

 

Thank your script very much. I am very happy to see your name in your code.please....