Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Still no way to get fine-grained masking with humanoid animations?

Discussion in 'Animation' started by JoeStrout, Feb 14, 2016.

  1. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    9,859
    I'm experimenting with combining animations with masking for the first time today. I'm trying to use some imported animations that don't apparently include jaw or eye motion. So when I apply this to my model, she deals cards as she should, but also stops blinking and holds her mouth open — she looks like a gaping fish!

    Also, I want her to deal the cards sitting down, while the dealing animation I have is standing up.

    It should be "animation masking to the rescue" as I let the sit/idle animation that came with the model continue to play on her face and lower body, but override this with the dealing animation on her upper body, arms/hands, and neck.

    But I can't get it to work properly, and I think it's because of the problem described here — the "Transform" part of an animation mask is ignored for humanoid avatars, and only the crude "Humanoid" portion of the mask is used. So I can't allow the dealing animation to control her neck and head, but not her face. Either I turn off the whole head/neck portion of the body map, in which case she stares idle at the ceiling while her hands deal cards; or I turn it on, and get the gaping fish.

    Is there really no way to get Transform-level control over animation masks when using humanoid animations?

    (Note: I'm comfortable with coding, and so if there is a solution that involves doing it from code, please give me a shove in that direction!)
     
  2. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    9,859
    I've seen some suggestions that AddMixingTransform can be used to get finer control over animation mixing. But it looks like that would mean throwing Animator and AnimationController out the window, and using the legacy animation system — is that correct?

    I'm fine with that if it works, but my main concern in that case is retargeting of animations from one model to another (for example, using mixamo animations, or other mecanim animations from humanoid models from the Asset Store). Would I still be able to do that with this approach?

    Also, it seems so heavy-handed, I feel I must be missing something... the whole point of Mecanim is to allow retargeting of animations, but if doing so causes your model to gape like a fish, and there is no way to adjust the animation mask to avoid it, that seems like a major limitation. What am I missing?
     
  3. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    1,035
    I remember testing this and got it working, but can't remember how.
    Did you try disabling the whole humanoid mask and selecting each bone manually in the transform section?
    I seem to remember the humanoid was overriding the transform.

    EDIT
    I found the test scene, here is how I have it setup.
    1-On base layer I have the regular animation.
    2-On top of that I have an override layer with a mask with all the humanoid parts red and then I selected only the head and tail bones in the transform section( I had to test tail animation).
    3-The override animation is made on the same skeleton of the humanoid rig, so I don't know if it's working when using a different skeleton.
     
    Last edited: Feb 14, 2016
    theANMATOR2b likes this.
  4. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    9,859
    Hmm, I can't get this to work. In my test, I have a base layer with the regular animation; and then an override layer, using a mask pretty much as you describe — all the humanoid parts are red, and I selected just the spine joints, arms, and head (but not the face).

    But when I run it, I don't see any of the override animation at all — I'm seeing just the animation from the base layer. But when I use the crude humanoid mask, I do see the override animation in whatever parts I select.

    So it's working with the humanoid mask; the only problem is that the resolution is much too crude. I want to animate the face independent of the head, and there appears to be no way to do that, at least in my hands.


    I also tried a different approach: masking the animation itself, in the animation import settings. I thought perhaps I could make a series of facial animations for various expressions, and then just apply those as an override layer. But no — when I do this, all the joints which I masked out go to the "default" (relaxed fetal) position, and stay stuck there. It appears that the animator controller applies all joints, even those not included in the animation! Argh!

    This is really frustrating. I'm having serious thoughts of throwing Unity's whole animation pipeline out the window and rolling my own. Or, having several invisible avatars playing the different animations, and then mixing these myself onto a visible model in LateUpdate. But either one seems stupid. Or I could switch from Humanoid to Generic, but then I lose the ability to use IK (unless perhaps I use a 3rd-party IK package), and will probably have a hard time making use of animations from different sources (such as Mixamo). Surely I'm missing something? Is there really no way to override just the face transforms (for example) when using humanoid animations?
     
  5. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Huge mistake, as mecanim is one of the absolutely brilliant spot on AAA parts of Unity, it just needs to be handled right. There's 2 kinds of animation: Generic and Humanoid. If you want to blend an animation on top you need another "layer" and this layer receives a special avatar mask, that will mask out the bones you want to effect.
     
  6. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    1,035
    I think the transform mask doesn't work with normal retargeting, so you must have the exact same bone hierarchy for the masked bones. Probably yours is different.
     
  7. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    9,859
    Thanks for jumping in, but please read the rest of the thread. The whole point is that, with Humanoid animation, the masking available is woefully inadequate.

    In this test I used the very same avatar to create the mask as for the model itself, and it still didn't work. It only worked when I switched from Humanoid to Generic — could that be what you did in your test, too?
     
  8. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    1,035
    No, I am using humanoid.
    Anyway the point is not what you use to create the mask, but what is the skeleton of the animation you are playing in the override layer.
    You should check the skeleton of that animation and see if it match with the skeleton of your character. It must match 1:1 at least until the point where the bone you need is.
    For example if you need to override the spine you can have :
    hips-spine-chest in your character
    and
    hips-spine-spine2 in your animation and you will have the hips and spine overridden (if selected in the mask) but not the chest.
    if in the animation you have hips-spine1-chest, only the hips are matching, because chest is child of spine1 and not spine as your character(not 100% sure of this second case,I never verified, but should be like this).
     
  9. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    9,859
    I'm virtually certain that both animations (and the mask, for that matter) were all made from the same model. But if it's working for you, then it's worth triple-checking. I'll make an even smaller test project that only has one model in it, and see if I can get it to work there. I would love to find some conditions under which what you're describing actually works, because so far, everything I've seen indicates that the Transform mask is simply ignored when using humanoid animation.
     
  10. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    9,859
    OK. Here's what appears to be happening:

    Any bone which is mapped to a Humanoid slot can be controlled only by the "Humanoid" section of an animation mask. The Transform part of the mask is irrelevant to these bones.

    Any bone which is not mapped to a Humanoid slot is controlled by the Transform part of the animation mask.

    I took great care in this test; I actually dragged an avatar out from the model, to another folder, and named it TestAvatar (it was written out as a model, which contained TestAvatarAvatar, but whatever). Then I made sure that the model itself, the animation I'm testing with, and the mask were all using TestAvatarAvatar, and all set to Humanoid.

    The result: when I turn off everything under Humanoid in the mask, the model is frozen in the relaxed-fetal position... except for the eyelids. The eyelids continue to blink, as long as those transforms are checked in the Transform part of the mask. I believe this is because the eyelid joints are not mapped to Humanoid slots. When I uncheck the eyelid bones in the Transform section of the mask, then the blinks stop too.

    Conversely, for any bone that is slotted into the Humanoid part of the avatar config, checking or unchecking it in Transform makes no difference whatsoever. These only respond (in groups I have no control over) to the Humanoid portion of the mask.

    I've been thinking of "Humanoid" and "Generic" animations as two separate things, but I guess really they're not — every Mecanim animation has 0 or more bones slotted into the Humanoid framework, and 0 or more bones which are not. "Generic" is just the extreme where there are no Humanoid bones at all, but in a Humanoid animation, there may nonetheless be some bones (like the eyelids in my test) which are not slotted, and so don't suffer from all the limitations (or benefits) of Humanoid animation.

    So I guess this could work fine for something like a humanoid character with a tail or wings... these joints would simply be ignored by the Humanoid system, and so you could mask them as you please at the Transform level. The problem only arises when you are trying to have finer control over bones which are also part of the Humanoid system, e.g., layering control of the jaw. But I may be able to just unslot the jaw and other face joints, and deal with those in a Generic way, while continuing to use Humanoid animation for the body and limbs (for which Humanoid's crude masking is probably good enough).

    Still not what I would call an AAA solution... but possibly an adequate work-around, now that I understand what's going on.
     
    maewionn and Whatever560 like this.
  11. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    1,035
    You are right, I just tested and it's working only for non humanoid bones. I was also testing head scaling for some cartoon effect and I thought it was working on humanoid bones, but that's probably because humanoid doesn't do scale animation so it wasn't overriding that.

    I noticed you talk about the character stopping moving, that shouldn't happen. It is better if you apply the mask only on the upper layer, and not on the base layer, it's easier to work this way.

    Few more tips:
    1-You can control the base pose when defining the avatar, if you close the mouth when setting the t-pose on the avatar it will default to mouth closed(no fish effect).

    2-Since all you need is some simple blinking, why don't you do it by script?
    It will be even more realistic. On lateupdate start a rotation every tot seconds on the eyes bones.

    3-For the sitting, On base layer put the dealing animation, on the override layer put the sit animation with a mask with green chest and legs only. It should do the trick. You can also try using additive and experiment with layer weights.
     
  12. theANMATOR2b

    theANMATOR2b

    Joined:
    Jul 12, 2014
    Posts:
    7,790
    I remember @TonyLi detailing a proper workflow for this setup - humanoid rigs (with bone controlled expressions)
    So a complete character can be retargeted and still have facial rigs. He has also explained in several posts what is needed to go about 'retargeting' on generic rigs.
    I looked for a while but couldn't find the exact thread. Might help to search his posts.
    An alternate process could be to use morphs/blend shapes instead of bone controlled facial expressions. This would remove the need to mask the facial bones.
     
  13. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,670
    I use FaceFX. The Unity - FaceFX integration scripts use legacy animation. I use Mecanim Humanoid for the body and head (including IK), and legacy for the face. So ultimately I'm doing what Joe described -- controlling bones that are outside the Humanoid rig's definition. I don't assign the Jaw to Mecanim; I leave it under legacy control.
     
    theANMATOR2b likes this.
  14. Skyunity

    Skyunity

    Joined:
    Nov 15, 2014
    Posts:
    5
    Sorry to necro this post, but it's high in the search results for people with this issue and never came to a satisfactory conclusion.

    If you want to use a seperate facial animation from your body animation, here is an easy solution:
    1. Open the Avatar your Animator is using. (click it in project folder > Configure Avatar in the Inspector)
    2. Click the Head button (found in the left corner of the main image of the avatar)
    3. Set the joints you want to use for facial animation to None
    4. Create a separate animator for your facial rig

    The downside of this method is you have to use 2 animators, which means calling animation state changes in two locations as well as I assume more overhead. However, its the only solution I've been able to find. If someone has a better method, please chime in.
     
    jeromeWork likes this.
  15. TSC_Lindy

    TSC_Lindy

    Joined:
    Mar 6, 2017
    Posts:
    6
    Unity, please give us the ability to add custom bones checkboxes to a Humanoid Avatar Mask.
     
  16. JohnSmith1915

    JohnSmith1915

    Joined:
    Apr 21, 2016
    Posts:
    143
    Any news about this request?
     
  17. UnitedBluff

    UnitedBluff

    Joined:
    Apr 19, 2015
    Posts:
    15
    2011 and now 2018..come one Unity, give us the features we really need. We don't need to be able to deploy to a billion devices, we need fine grained control over the engine, which we just dont have.
     
    JoeStrout likes this.
  18. chgeorgiadis

    chgeorgiadis

    Joined:
    Jan 30, 2018
    Posts:
    51
    also interested what would be the best solution. In my side i use faceware and i cannot support head tilt or movement
     
  19. Ne0mega

    Ne0mega

    Joined:
    Feb 18, 2018
    Posts:
    745
    Oy, so this is the limitation of mecanim, eh?
    Yup, generic rig and my faces animations show up.
    If I use humanoid rig, I have to do some sort of magic and layers and masks, which I am trying to avoid because those hurt performance, and I am making a RTS.
    Bummer.
    Maybe I will just do generic, as I find most pre-made animations form the asset store to not be what I need, anyway.
     
  20. XRA

    XRA

    Joined:
    Aug 26, 2010
    Posts:
    265
    is this ever fixed?
     
    JoeStrout likes this.
  21. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    9,859
    I'm not sure — I've moved on from the project where I needed this — but I might suggest looking at the Animancer plugin. I use that in most of my projects nowadays, and it does a good job of fixing or working around most of the boneheaded limitations in Unity's animation system. It performs well, and I also find it substantially easier to use.
     
    Kybernetik likes this.
  22. Kybernetik

    Kybernetik

    Joined:
    Jan 3, 2013
    Posts:
    2,555
    Animancer won't solve this problem out of the box since it uses the same AvatarMasks as the Animator Controller system, but it supports animation jobs which should make it possible to implement. One of the Animation Jobs Samples is a weighted mask mixer which would let you not only mask individual bones but give them individual weights as well. I haven't tried using it myself, but I'd be happy to work with anyone who has a real use case for it.

    Edit: this Github Issue has a downloadable package demonstrating how you can use the Weighted Mask Mixer sample in Animancer. You can ignore the code changes in my first post because they're all in the latest version now.
     
    Last edited: Feb 19, 2022
    JoeStrout likes this.
  23. netpost

    netpost

    Joined:
    May 6, 2018
    Posts:
    388
    I can't believe that we are in 2021 and such a simple thing can not be done in Unity. How can you make a serious production if you can't have a mask for the face bones only (without having to mask the head too). Does someone from unity read these post? 8 years and still no solutions. Really?
     
  24. netpost

    netpost

    Joined:
    May 6, 2018
    Posts:
    388
    In the meantime I use UMotion Pro to copy blend shapes from one animation to the other but revised scripts is a common thing in the animation chain. replacing one facial motion capture with another revised one should be an easy process. Not rocket science... :confused:
     
    jeromeWork likes this.
  25. o1o101

    o1o101

    Joined:
    Jan 19, 2014
    Posts:
    639
    Still wanting this in 2022 :(
     
    chgeorgiadis likes this.
  26. Whatever560

    Whatever560

    Joined:
    Jan 5, 2016
    Posts:
    505
    That is such a pain ... we can't have a humanoid mask that exclude the body and shoulders transform, but that keep the rest of the arms ...

    The control there is on humanoid mask is very limited. That is such a shame
     
    Last edited: Feb 10, 2022
  27. bennes4

    bennes4

    Joined:
    Jan 22, 2013
    Posts:
    6
    I need this too...
     
  28. qvp

    qvp

    Joined:
    Dec 9, 2017
    Posts:
    2
    No one at unity gives a damn.
     
    JoeStrout likes this.
  29. Whatever560

    Whatever560

    Joined:
    Jan 5, 2016
    Posts:
    505
    Or under the radar. Don't know who we should ping for this.

    Though, it strange that no solution exists, I would guess someone at least found a workaround, this looks like a common thing you'd like to have in any game relying heavily on animations and there are a bunch released.
     
  30. maewionn

    maewionn

    Joined:
    Jan 18, 2016
    Posts:
    37
    So, another half years has passed, it's over 9 years since the thread started and no one at unity even said a thing? Wow...
     
    JoeStrout likes this.
  31. alicja121

    alicja121

    Joined:
    May 26, 2023
    Posts:
    5
    I need this please anyone guide me
     
  32. wshjle

    wshjle

    Joined:
    Oct 4, 2019
    Posts:
    1
    It's 2024 and Unity still hasn't solved this fundamental problem. Unity can't claim to be able to do AAA games