[RELEASED] Realistic Eye Movements

I think it has just been my confusion over these settings.... I may have been setting them at odds to one another.

On the bright side I now realize I can trigger a "look at" by resetting the distance to 0 then a high distance (with Playmaker).

Thank you for advice!

 

That's an interesting hack to get your character to look at the player ;-), although you have more direct control over the look behavior if you call LookTargetController's API functions like LookAtPlayer or LookatPoiDirectly. I use them from code, but it should be no problem to call them with PlayMaker.

 

This asset is amazing- it breathes so much life into my characters, its stunning. I have one quick question, the head following component; does that only apply to Mecanim characters, or will it work with Makehuman MHX2 format? Thanks and keep up the incredible work!

 

@Fletcher Studios

Thanks, I'm glad you find the asset helpful.

Could you elaborate on what you mean with the format? As far as I understand, MHX2 is a format for MakeHuman for importing into Blender. To import MakeHuman characters into Unity you need the fbx format. Are you maybe asking about whether the rig needs to be Humanoid or can be generic? For Mecanim IK the rig needs to be Humanoid, for Final IK it can be generic as well.

 

Hi everyone, a new version is out! Here are the changes:

• Fixed slow blinking for blendshapes (blinking when eyes sweep more than 25 degrees)
  
  
• Fixed changing look target when nervousness is set to higher value.
  
  
• Added support for Final IK.
  
  
• Added setting „Keep Target Even When Lost“
  
  
• Split „Max Eye Horiz Angle“ into two angles, one towards the nose and one away from it.
  
  
• Added support for 3rd person perspective games.
  
  
• Blendshapes configurations are now saved by blendshape name, not blendshape index, so you don't have to reconfigure when you modify the number of blendshapes in the 3D modelling software.

Enjoy!

 

Awesome, it seems like all my little issues are taken care off.

I hope Realistic Eye Movements is doing well for you, it's really great.

 

Thanks!

 

Sound interesting. Could you please explain more?

 

@montyfi The player-related functionality like "notice player" or "stare back" can now be given a gameobject as reference point. Before, it always used the main camera as the player, so it worked for first-person perspective games only. All you need to do now for a 3rd person perspective game is to have a gameobject parented to the player character's head so that it stays in between the eyes and assign it to the LookTargetController's slot "Player Eye Center".

 

Amazing! Thank you!

 

I have been messing with this for a couple of days and can't solve the problem, so I am asking here if anyone has any suggestions. I have eye control working with blinks, all just fine, but I cannot for the life of me get the character's head to move as the eyes move. One complication is that I have a generic skeletal rig on a Mixamo/Fuse model, so I figured that the EyeAndHeadAnimation.cs script is not finding the Head and Spine bones. So, I modified that script and forced it to find the bones as they are named in my rig and verified that the correct bones were found. Still no luck. I can manually change the Head bone transform rotation in the editor and the head moves fine. I have a fairly simple Animator with a blend tree between an idle and a walking animation, which are working fine. I mention this because the idle animation does mess with the Head. I followed wetcircuits advice and made sure that my Animator Base Layer property has the IK pass-through set. I am out of ideas. And yet I am so close! I'm also using the Brows Up blend shape to good effect when the eyes are looking upwards. This also raises the upper eyelids. I love REM so far!

Thanks

 

@knchaffin
Unfortunately Unity does not yet support inverse kinematics on non-humanoid rigs. Can you switch your Mixamo rig from generic to humanoid? Alternatively, you could use another ik solution like Final-IK.

 

I think you might need to change the figure's animation type to Mechanim (in it's import settings)... not sure if it works with Generic.

 

Thanks for your advice Faikus and wetcircuit. Either I am confused with Mixamo/Fuse, or something is not working correctly. I just went through the Mixamo rigging and animation steps again and downloaded the Unity format FBX's. I created a new character in Unity based on these FBX files and yet in the Scene Hierarchy view the bones are still labelled as Mixamorig:Head, etc. Should these show up with the Humanoid rig bone names at this point? I reimported the model FBX and made sure it was set to Humanoid, as it was defaulting to Generic. This is where I went awry a couple of weeks ago when I was first working with Mixamo/Fuse and used the default rig import setting (generic). I'm still not sure I am doing this correctly. At least now the FBX import rig type shows Humanoid and the "configuration" shows the correct Mixamorig to Humanoid rig bone names. When REM calls Animator.GetBoneTransform(HumanBodyBones.Head) , does the rig bone names get translated through the Avatar? Obviously, I'm a bit hazy about how Unity handles rigs and bone names. I still have a few hours of work to do to get this Humanoid character to where my Generic character was. So, I can't fully test yet.

Thanks.

 

Ugh, no. I had to do this process 100x when I used Fuse for my spacecamp kids.

1. In the project folder, select the Fuse figure.
2. In the inspector go to RIG and change the dropdown to "Humanoid". That makes it Mecanim-able.
3. go to ANIMATIONS and deselect "Import Animation" (not sure this step is necessary)



If the figure has animations attached, you might need to download them separately to use in the Mecanim Animator window....

 

Okay, I have my new Fuse/Mixamo character working properly with REM now. The key is to set ALLLL of the FBX assets to the Humanoid rig, as the default is Generic. Since I did not do that initially with my first character, I was never able to successfully redo the skeletal type later without getting a lot of errors. It could have been that I had not set all of the animations to Humanoid though, so the animations were not matching the model rig.

I still have some things to work on to get my new character back to where my old one was.

Thanks for your help!

BTW, I have to extensively play with materials and shaders and textures to get the Fuse/Mixamo characters to look good. For some reason Mixamo defaults to setting a lot of the materials to use Standard Transparent shader, which causes all sorts of disappearing mesh problems. The eyelashes are particularly difficult to get right as I have to create a new material for that and then re-import the Body Diffuse texture to set the alpha transparency correctly to work properly with the body and the eyelashes.

 

@knchaffin The bone names in the scene view keep their original names like "mixamorig:Head". Code like Animator.GetBoneTransform(HumanBodyBones.Head) returns the bone that is assigned as head in the Mecanim rig configuration window in the Mapping tab. To import your Mixamo character you don't have to do anything else other than what wetcircuit wrote: just set the rig to Humanoid. Unity finds the correct mapping itself. No need to map bones manually or rename them. Then assign an animator controller with IK pass activated to the animator, drag the LookTargetController and EyeAndHeadAnimator scripts onto the character, and load the Mixamo preset in the EyeAndHeadAnimator component. The whole process takes just a few clicks. You can have a look at the Mixamo character in the demo scene to see one in action.

 

Yeah, mine is working fantastically now! I did have to change all of the Mixamo animation FBX imports to Humanoid, one at a time, as I had to do the model FBX. Then I did set the Animator to IK pass-through and dropped in your scripts, designated the eye bones, specified blend shapes for the eyelids control and the magic worked. I still have to go back and set up the blend shapes correctly for the eyelids, eyelashes and brows.

Thanks for your suggestions.

So, I was able to start from scratch and in 4 hours I had the new Fuse/Mixamo character working with REM. Most of that time was spent on fixing things that do not work correctly with the Unity shaders by default. I would not want to guess how many weeks or months I have taken in the past to create a fully rigged and animated character with eye and head control! This (Fuse, Mixamo, REM is a fantastic productivity tool suite!

 

Ah, hadn't seen you already had solved the problem when I wrote my last reply, glad it works for you now!
One question: you didn't seem to have used the existing preset for Mixamo and instead assigned eye bones and blendshapes manually. Did the presets not have good settings for you?

 

No, I used the Mixamo presets for blend shapes and rig. I do have to change quite a lot in the Mixamo produced materials and shaders. I did try some different things to try to get the Generic rigged character working, but that is unnecessary with the Humanoid rig.

 

I'm working with anime-style characters where the eyes are much bigger than real life.

​

The eye bones here are set way back in the head — roughly halfway back through the skull. The mesh attached to these is just the iris. So when you rotate these bones a little bit, the iris appear to move much more than you would expect. To give you an idea of how it works, here's what happens when I rotate the right eye bone by 90°:

​

That's not an earring... that's the iris of her eye!

As a result, normal methods that would essentially have the eye bone LookAt a target do not work here at all.

I'm wondering whether/how REM could deal with this situation?

 

Hi Joe,
REM would rotate the iris such that it's on the line from the bone origin to the target, so I think it should work.

 

No, if it does that, then it'll end up rotating it much too far — that's the whole point I was trying to make. So I think it's not going to work.

But thank you very much for the reply anyway!

 

Ah, I thought you were asking about whether the irises would look in the correct direction at all, but you meant the maximum angles. The maximum angles are adjustable. The maximum vertical angles are set in the normal setup process (you rotate eyes and eyelids to their maximum looking-up and looking-down positions and save these), and the maximum horizontal angle is just a number that is a parameter of the component. For the anime character a much lower number than the default 35 degrees would have to be set.

 

No, it's not about the maximum angles. It's about moving too much. With this rig, if you're just rotating the bone, the irises move too much and point in the wrong direction, because the bone is set too far back in the head for that approach to work.

I don't know how else to explain it — but I got the answer I needed, so I won't trouble you any further.

 

Sorry for having misunderstood you. I had another look at your screenshot and you're right, the asset doesn't work for this model.

 

Hi,

I am trying to get this to work with a Genesis 3 figure from Daz as in the reviews someone said it would work.

I have followed the instructions but nothing happens and I get this error :

NullReferenceException: Object reference not set to an instance of an object
RealisticEyeMovements.LookTargetController.LateUpdate () (at Assets/RealisticEyeMovements/Scripts/LookTargetController.cs:268)

The scripts I have attached to the highlighted node, perhaps it is not the right one ?

Please could you help.

 

So I tried with using mechanim and it gets weirder.

I have the test model standing on a plane and she goes like this - she sinks into the ground, her arms are clenched and her mouth wide open.

 

Okay I am getting somewhere
I started a new project, I think I was causing problems with having two avatars perhaps.

So, new project, single avatar. Mechanim bones appear to be working now and also game objects. So far I have done nothing with the eyelids.

Set up a fresh new animator and did the IK thing to let her head move.

The concept is working fine, but she is still stuck in this devilish pose and is still sinking into the ground with mouth open. It's pretty freaky to watch, like a zombie trying to get out of the ground !

Any ideas on how I can stop her going into this strange pose please ?

 

And finally after some digging, I found if I add an animation to her then she isn't a zombie any more !
Sorry for the noobiness, but maybe my comments will help someone else
I really like the effect now I have seen it in action - very cool.

 

I thought Faikus might want to know that I really like this library, enough to flat-out recommend it in a video:

 

Thanks a lot Craig, that's a great video! I'll add in checks for incorrect eyelid bone assignment in a future version. Btw, if your character is a MakeHuman character I think you can make him stand less spread out by correcting the T-Pose in Mecanim: the legs should go straight down in the T-Pose, such that the feet are close together.

Looking forward to your future videos!

 

Hm! I'll look into that, it's a good tip. The pose is just a neutral stand, but I noticed MakeHuman models are bowlegged when using mocap. That's probably why.

 

Hi recently bought the asset and learning to use it. I have some basic Mixamo characters. But I didn't see any bones for the eye lids? If I choose "Mecanim eye bones" and it seems they are not found. Any help?

 

Hi Nadan, Mecanim doesn't have bones for the eyelids, the eyelids are controlled by blendshapes. You don't actually have to configure that yourself, just import the Mecanim blendshape configuration via the Import button in the EyeAndHeadAnimator component, the preset is in the Presets folder.

 

Hi Faikus, and thanks for the support!

I have a free mixamo character. The rig animation type is set to Humanoid. When I choose Eye Control: Mecanim eye bones. It says "Eye bones not found; is the Mecahim rig set up correctly?". Then I choose "Import" and pick the "Mixamo" file from /presets but it says "Settings don't match target model".

I have Unity 5.3.4p2 and Realistic Eye Movements 1.9.2.

Here is the FBX if you want to try: www.ihmeellinen.net/temp/Goblin_D_Shareyko.fbx

 

Non-Mecanim Setup

When using the EyeAndHeadAnimator/LookAtTargetController scripts I'm only seeing motion with the eyes and eyelids. The eyes look great, the blinking looks great, but I don't get any head movement at all.

For a non-mecanim body, the eye and head animator script has you specify bones for the eyes and eyelids. Looking through the code, it looks like it is trying to infer the bones for headParentXform and eyesRootXform (and maybe some others).

Should these be exposed to script and setup in the same way as the eyes/eyelids, or should I be placing these components onto one of the character's joints rather than on the root of the character or am I missing something?

 

Ah, that model does indeed not have bones for the eyes, and the eyes are not separate meshes either. If you want to use that model, you would have to modify it with a 3D program like Blender or 3Ds Max to cut out the eyes into a separate mesh. Most other Mixamo models (like the ones you can create with Mixamo Fuse) have eye bones and separate eye meshes.

 

For head movement you need either to declare the rig in Mecanim as Humanoid or use Final-IK. Is either of that possible in your setup?

 

Yeah, I'll use Final-IK, thanks. I'm working on getting that integrated.

With this setup, it seems that headParentXform does need to be exposed or I get lookatpoints down at my character's feet. My first shot at Final-IK had my characters always staring at the ground. With headParentXform being set, I'm getting closer.

 

Hi Bob, you're right, there needs to be additional code for the case of non-Mecanim setup to tell the asset where the head is. I'll add it to the next update. In the meantime, you can modify the code yourself: In the Initialize function, replace this code:

Code (CSharp):

1.  
2.    Transform headXform=null;
3.    if (animator != null)
4.      headXform = animator.GetBoneTransform(HumanBodyBones.Head);
5.    if ( headXform == null )
6.      headXform = transform;
7.  

with this:

Code (CSharp):

1.  
2.    Transform headXform=null;
3.    if (animator != null)
4.      headXform = animator.GetBoneTransform(HumanBodyBones.Head);
5.    #if USE_FINAL_IK
6.      if ( headXform == null && lookAtIK != null )
7.        headXform = lookAtIK.solver.head.transform;
8.    #endif
9.    if ( headXform == null )
10.      headXform = transform;
11.  

Please let me know if that works for you.

(Btw, with Final-IK, don't forget to set the script execution order as descriped in the docu.)

 

Great, thanks. Beats the way I was doing it. I forgot that the Final-IK setup already knew about the head. Everything is looking good now, thanks.

 

Good to hear, thanks for the info!

 

Really nice asset, thanks!! I'm having a bug where the character's spine "spins in circles" when using Final IK. Here's a video showing the problem. Any tips?

 

I'm using "Look AT IK" and and "Realistic Eye movement" together on at least 2 characters and never had problems. But I saw in the vid you assigned a lot of spines in the "Look AT IK" component. Maybe reduce them?

 

Hi Selzier, did you change the script execution order to position the REM scripts after the FinalIK scripts and can you check whether they are still in this order? Also, you can position a dummy object at the place FinalIK is told to look at and see what's happening with it.

In LateUpdate, after the line

Code (csharp):

1. lookAtIK.solver.IKPosition = headTargetPivotXform.TransformPoint( eyeDistanceScale * Vector3.forward );

place this line:

Code (csharp):

1. Utils.PlaceDummyObject("HeadTargetDummy", lookAtIK.solver.IKPosition);

And see where the created head target object is moving.

 

Script Execution Order appears to be correct:


Here's the Dummy Object:

 

Thanks for your videos, they are very helpful in understanding the issue. Slowbud is right, there is a problem with the current implementation of REM when the FinalIK chain goes down very deep, there is a feedback mechanism due to where the angles are computed from. I will change that in the next update. For now you can fix it by replacing this line (line number 688):

Code (CSharp):

1. headParentXform.parent = spineXform;

with this line:

Code (CSharp):

1. headParentXform.parent = transform;

You can also let her look more directly at you by setting head weight to 1 in LookAtIk (it was set to 0.5, that's why she didn't look at you when you only had LookAtIk active) and maybe reducing the body weight.

 

Fixed now thanks! Great support Faikus much appreciated. Slowbud nice catch

 

The picture doesn't do justice to the feeling when wearing an oculus rift and getting up close to her.
Thanks for an awesome asset.

Feature request
I'd really like to request if this could be implemented in a future update:
- Just as there is a "list' of elements that can be created for "Points of interest"... it would be awesome to have a list of "actions" to perform if you get close to the personal space parameter. Sort of a random list of actions. It's not always that the person shys away.

Maybe he/she "smiles" (can be done via activating blend shape) or a list of preset animations such as wink, lift hand up etc..
Being able to build a list and populate it with actions / audio clips would really take this asset a notch up!
I suppose it can already be done with playmaker and triggers, but the personal space trigger already there just begs for some extra pizzazz

Thanks again.