[RELEASED] LipSync Pro and Eye Controller - Lipsyncing and Facial Animation Tools

Potentially, yes. Can you try (re)installing the Visual C++ redist (you can get the latest from here: https://support.microsoft.com/en-gb/kb/2977003). If that fixes the problem, I will include the redistributable in with the package from the next patch.

Thanks for letting me know. I'll look into it.

I think it's pretty great, but then I'm a bit biased

 

Hi! Great news. Installing those Redistutables worked.. I downloaded both 32bit and 64bit.
Now that i actually get to use this for my own audio files, i gotta say its very impresive.

 

Ran into a bug with the editor. If I add an emotion to the timeline, right click on it, go to marker settings and set the intensity of the emotion to anything less than one, then save the clip. I playback the scene and the emotion seems to play at the correct intensity. Next I reopen that clip in the lip sync editor, and save, all emotions will then play at max intensity, the editor seems to reset the emotion intensity values when opening a clip, or something.

 

Great, glad that worked!

Thanks, I've added it to the tracker - I'll put the fix in with the next patch.

 

Any news on the 5.4 patch?
We'd like to ship a video of in-game conversation of our new game (featured on madewithunity)
These are the characters:

 

It's not completely ready, but I can send you it in its current (working) state if you want. Send us an email at contact@rogodigital.com and I'll reply with the download.

 

Good news if you've picked up Flux in the 24hr sale today - a package of Flux Events for LipSync will be going live later today! Look out for it in the Extensions Window. Events for USequencer will also be available in the next few days.

 

The new tutorial series has started! The first two tutorials are available now, and there are another two going live tomorrow.
There's also going to be some more over the next couple of weeks on specific features like bone animation, gestures, and some of the third party asset integrations.

 

That's curious. I installed those as well and it's still giving me the same error as before. Maybe I need to install a specific version of it? The one I installed was for Visual Studio 2015 (14.0.23026) for both 32bit and 64bit and it's a no go for either. I might try and see if the other ones work for me and I'll list the one that I manage to get Autosync to work with.

 

yea i also tried installing those files and it did not fix the issue v_v still no autosync for me *cries in the corner*

 

Running into this error every time I set up a sprite based lipsync component:

ArgumentException: An element with the same key already exists in the dictionary.
System.Collections.Generic.Dictionary`2[System.String,RogoDigital.Lipsync.EmotionShape].Add (System.String key, RogoDigital.Lipsync.EmotionShape value) (at /Users/builduser/buildslave/mono-runtime-and-classlibs/build/mcs/class/corlib/System.Collections.Generic/Dictionary.cs:404)
RogoDigital.Lipsync.LipSync.Awake () (at Assets/Rogo Digital/LipSync/Components/LipSync.cs:208)

All Phonemes are set up, no emotions or gestures are connected, it doesn't seem to appear at any point during setup until I connect an audio listener.

Update: Rtyper helped me resolve this:I had a duplicate emotion In the project settings (2 emotions named "Fear")

 

There seem to be a bug across the entire plugin: the component setup resets quite often when reopening a scene, it's as though the values on the plugin don't get saved after it has been setup. For example, setting up gesture animations, saving, then reopening to find that the gesture slots are all empty, or that the animation clips are still there but viewing it in debug mode reveals that the triggers are now blank. In the sprite manager for sprite based lip syncing, I can populate the list of sprites and layers that I want to use, and half the time, the list just resets breaking everything upon reopening the scene.

My editor is set to force text mode and it seems as though the component isn't marking my changes as "dirty" thus when I try to save the scene it doesn't register any changes.
I tested a workaround involving forcing a "dirty" change to the scene by moving a transform in the scene, this forced the scene to save updates.

 

Damn, sorry about this. Leave it with me for a bit, I'll have to keep looking into this. As far as I can tell, the VC++ DLL is the only dependency of either PocketSphinx.dll or SphinxBase.dll that isn't pre-installed with Windows.

Are you both getting the same errors after installing VC++ as you were getting before, or are they different?

That looks like you have multiple emotions with the same name in your project settings. I'll make this more obvious in the update, but each emotion needs to have a unique name.

I've been trying, but I can't recreate this. I've got my editor set to Force Text mode, but everything is still serializing as expected... Are there any plugins/scripts in your project that could be affecting the scene saving process maybe?

 

I should have also explained that despite the nature of the error, all of my emotions do have unique names so I was surprised to see it, and I haven't run into this error for blendshape based lipsync systems so far, only on my sprite base systems did it start showing up.

Edit: Or so I had thought, turns out I have two emotions named fear in this list.

The only other plugins I'm using on this project currently are the cinema director, I've also downloaded the Lipsync scripts for cinema director.
And Shaderforge.

Also if it helps I'm using Unity 5.3.2p3 on windows 7

 

You have two emotions named fear in the list there - number 4 and number 20 The reason you didn't get the error with another character could be that the other component was added when there was only one fear emotion in the list (or none), so the second didn't get added to the character later, but the character using the sprite blend system was created with both of those emotions in the project settings so both fear emotions got added to the character, causing the error.

This is a bug though, it shouldn't be possible for two identical emotions to be added in the first place, so I'll put a fix for that in the update.

This problem's more confusing though... Does it seem like there's any logic to when things fail to save? If so, if you could type out step by step how to recreate the problem, that'd be a huge help. Otherwise I'm not sure what else I can do to track this down.

 

Wow, I don't know how I didn't catch that, sorry about that, still hopefully this will help make it easier to use going forward, I'll look into the other issue a bit more today, I've also updated Unity to 5.3.4f1 to see if that makes any difference.

 

It seems I was able to download the 2015 version without needing service pack 1 but i think that may be what the issue is, i'll install service pack 1 and play with a few things, then get back to you.

On another note, I have made use of the system via another computer and transferred the files over to help create this new trailer for my game Alone Without Her

 

Hi!

I keep all my plugins in a "Plugins/" subfolder to compile them separately (for performance and clarity), but this throws errors at multiple locations in your code where the location for ProjectSettings is hardcoded to "Assets/Rogo Digital/LipSync/ProjectSettings.asset". Could you make this path detection more dynamic to fix it?

EDIT: In fact, the whole plugin is unusable and throws lots of errors if I put it in a "Plugins" subfolder. This is a major issue for us because of our project's size. Can you please fix this ASAP?

Thanks!

 

Hey guys! First of all, awesome plugin, really enjoying using it!

I'm afraid I have a bit of a problem, I am trying to use SetEmotion function to adjust emotions during runtime. I understood that this can only be done when no animation is playing, which is fine. However, when I switch the emotion when no animation is playing the emotion on the face adjusts just fine but after that it won't properly animate the lipsync anymore. It would play the audio, but the animation would not play, instead freezing the face to the before-last emotion and slowly transitioning to the emotion I just changed it to.

Anyone familiar with this or what I might be doing wrong? I wasn't able to find anything like this in the guide or the thread.

Cheers!

 

Can't get the autosync to work at all after all, even when not putting it in a Plugins/ subfolder.

• OS: Windows 10
• LipSync Pro: 1.0
• Unity: 5.3.4f1
• SoX: 14.4.2 (Verifies)
• Language Model: en-US

Stack:

Code (CSharp):

1. [Error] [AutoSync] ERROR: Invalid parameters.
2. 0. RogoDigital.Lipsync.<>c__DisplayClass8_0.<RecognizeProcess>b__0() at Assets/Plugins/Editor/Rogo Digital/LipSync/SphinxWrapper.cs:61
3. 1. RogoDigital.Lipsync.SphinxWrapper.psRun()
4. 2. RogoDigital.Lipsync.SphinxWrapper.RecognizeProcess() at Assets/Plugins/Editor/Rogo Digital/LipSync/SphinxWrapper.cs:111
5. 3. RogoDigital.Lipsync.SphinxWrapper.Recognize() at Assets/Plugins/Editor/Rogo Digital/LipSync/SphinxWrapper.cs:33
6. 4. AutoSync.ProcessAudio() at Assets/Rogo Digital/LipSync/AutoSync/Editor/AutoSync.cs:114
7. 5. AutoSync.ProcessAudio() at Assets/Rogo Digital/LipSync/AutoSync/Editor/AutoSync.cs:143
8. 6. LipSyncClipSetup.StartAutoSync() at Assets/Rogo Digital/LipSync/Editor/LipSyncClipSetup.cs:2438
9. 7. UnityEditor.GenericMenu.CatchMenu()

 

Sure, some parts of the plugin need to be in certain root directories (Editor Default Resources and Gizmos) but I'll sort out a fix for the rest of it. Could you email contact@rogodigital.com so I've got an address I can send the patch to? Afterwards I'll merge it into the next update as well.

That's not the error I expected! That implies there's something going wrong with the DLL wrapper... What format is your audio clip, and what's the full path of it? I think it could be an issue with certain characters in the path messing something up, but I haven't come across this happening before.

Looking into this, I discovered a bug with the SetEmotion method where you can't set another emotion after a clip finishes playing if you already had an emotion set when it started playing (and I'll add this fix to the update), but I couldn't recreate your issue exactly. Does the clip you're trying to play have phonemes in it, or is it just emotion(s)? Do you get the same issue if you try playing one of the example clips instead?

 

That was it! I had accented characters (É, é and è) in the path.

 

Now, when I try autosync with MP3 files, I get the following errors:

Code (CSharp):

1. AutoSync: SoX Conversion Failed: C:/Program Files (x86)/sox-14-4-2/sox.exe FAIL util: Unable to load MAD decoder library (libmad).
2. AutoSync: SoX Conversion Failed: C:/Program Files (x86)/sox-14-4-2/sox.exe FAIL formats: can't open input file `C:/Users/.../Projects/.../Assets/Audio/Voice/Alphabet.mp3':

I figured this is because LibMad / LAME are not installed with SOX. I found this S/O questions which links to some DLLs, but they're all dead: http://stackoverflow.com/questions/...ad-decoder-library-libmad-function-mad-stream

EDIT: Found the DLLs here.

 

I'm not sure if that is the same LAME but I always need a LAME dll as Audacity plug-in.
Here it is.
http://lame.buanzo.org/#lamewindl

Hope this helps.

 

Ah, good. I'll add a check for this so the error's a bit more descriptive.

Brilliant, I only found out a couple of days ago that the LibMad link in the Using SoX pdf was dead, and I hadn't got around to finding a replacement yet. @gurayg I'll have to check if the standard LAME dll works instead of LibMad, if it does that link will be best (being the original source). (Edit: It doesn't.) Otherwise I'll see if I can host it myself.

 

The clips I am trying to play do contain phonemes, but no emotions in the clip editor. My idea was to adjust the emotions while playing a clip, but since that would not be possible according to the documentation, I figured it would also be fine to just switch to the corresponding emotion at the beginning of each clip / ending of the old one.

I tried to play the example audio clips Gettysburg and Tempest by setting them as a default clip and having the avatar play it on awake, but both refuse to play. I am getting the following error as well:

"KeyNotFoundException: The given key was not present in the dictionary.
System.Collections.Generic.Dictionary`2[System.String,RogoDigital.Lipsync.EmotionShape].get_Item (System.String key) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Collections.Generic/Dictionary.cs:150)
RogoDigital.Lipsync.LipSync.ProcessData () (at Assets/Rogo Digital/LipSync/Components/LipSync.cs:802)
RogoDigital.Lipsync.LipSync.Play (RogoDigital.Lipsync.LipSyncData dataFile, Single delay) (at Assets/Rogo Digital/LipSync/Components/LipSync.cs:411)
RogoDigital.Lipsync.LipSync.Awake () (at Assets/Rogo Digital/LipSync/Components/LipSync.cs:218)"

I recall having a dictionary-related issue before, but right now I don't remember precisely how I fixed that. If I'm not mistaken, that was because I didn't have a language model yet, so I downloaded the English one. Could it be related to that??

 

Alright Autosync now completely works with Windows 7 Service Pack 1 installed and a repair of the 2015 C++

I am a very very happy camper! starting to enjoy the asset now, whoooooooo!

 

Dos it work on w10 Unity5.4? does it have blend shape preset for daz g2?

 

Getting random null reference exceptions when adding a LipSync component. Last time it was:

UPD: this one is fixed by redownloading the package several times

When deleting/readding Lipsync component to the same gameObject, it remembers the settings for some reason.

When I try to import an Adobe Fuse preset, it is not imported and sometimes even the list of existing phonemes disappears from UI. Exception is:

Both standard and exaggerated presets have 0 phoneme/emotion shapes if I look at them in inspector.
UPD: this one is fixed by redownloading the package several times

Finally, Unity crashes when I try to process a large file (about 3 minutes of stereo sound in wav) on 'recognizing phonemes'.

Win 10 x64
Unity 5.3.4f1
LipSync Pro 1.0 (clean install)
VS redists installed, both x86 and x64

 

I've tracked down the cause of this now, it was due to the cache system not taking custom set emotions into account when determining if the animation needs recalculating or not. It's fixed in 1.1. There's only 3 or so things left on the to-do list for 1.1 now, so it'll be out sooner rather than later.

Yay! I'll see if I can legally bundle this dependency in with the DLL, and if I not I'll just include the redist installer in with the next version.

It does work in Windows 10, but Unity 5.4 support is currently limited. The 1.1 update will have a Unity 5.4 version, but (like 5.4 itself) it's still in beta. There also isn't currently a preset for Daz Genesis, but it is compatible with it, and once you've set up one character you can save a preset yourself to use later.

Sorry I didn't get to your post earlier! I can't say I know what was causing those earlier errors, but if they've gone away that's fine.
The settings that are remembered are part of a separate hidden component called a Blend System that LipSync and Eye Controller use as an interface with your character models/sprites/etc. They get deleted when you press play if there's no other component using them.

As for the crash though, I'll need to look into that. I haven't personally tried it with clips that long before, so it may be a limitation of the PocketSphinx system. You could try splitting your clip into multiple shorter ones, and running them through the batch processor instead to save time.

 

I purchased the unitypackage (Lipsync pro 1.0) but cannot actually download the ''AutoSync Language Model - En-US'' extension.
I've got this error message:

 

Hmm.. I'm able to download it just fine here - sounds like it must be a network issue of some kind. I've uploaded the file to dropbox here, let me know if you still can't get it from there.

 

LipSync Pro 1.1 is nearly finished! Just some more testing and updating the documentation left to do now, so it'll be available to download from the LipSync website from Monday the 6th of June, and the Asset Store a short time after that.

Here's the changelog for this version:

1.1

Features
- Phoneme Marker Sustain - join two phoneme markers together to maintain the pose for longer.
- Improved LipSyncData inspector with audio and animation preview area.
- Added options for animation timing (audio playback, custom timer or fixed-framerate).
- Emotion marker labels now show as tooltips if they don't fit on the marker.
- Emotion marker intensity is now visible without opening the marker settings window.
- Added option to allow emotions to persist after a clip has finished.
- Animations now blend in from the custom emotion if one is set when Play is called.
- Some improvements to how Blend Systems are managed.

Eye Controller
- Added ability to use LipSync Poses for blinking and looking instead of raw blendshapes and bones.

Changes
- Realtime Preview targets are now selected from a drop-down menu.
- Moved Pose Editor code out of LipSyncEditor.cs for re-usability.
- Included VC++ redistributable with package.

Fixes
- Fixed bug where Save as and Save buttons don't always function as intended.
- Fixed re-loading of marker intensities in the Clip Editor.
- Fixed errors with playing LipSync animations after setting custom emotions.
- Added specific error for invalid characters in an audio path in AutoSync.
- Removed hard-coded paths.
- Added correct undo support to pose editors.
- Added warning when multiple identical emotions exist on a LipSync component.
- Prevented multiple identical emotions being added to the Project Settings.
- All modal windows are now consistently not dockable.

 

Great! especially this:
"- Phoneme Marker Sustain - join two phoneme markers together to maintain the pose for longer."

This will work well when trying to lipsync to lyrics for songs.

Hoping you'll also change the Playmaker plugin's "Data Clip" field to be able to use variables so opening up the possibility to play random emotions/lipsync clips.

On another note an interesting discussion took place in the comments section of a video I'd uploaded.
That of timing with Lipsync and recording / rendering of video from within Unity.
If you've got time.. have a look: http://bit.ly/1P4Kj1Z (using bit.ly because this forum tries to embed the video from youtube)

Copying the main comment from there, below
"Your problems is actually a problem of LipSync Pro. Unity can use different timing methods - realtime (counting seconds from a scene start) and scaled time (Time.timeScale). Time scale method gives you correct time scaling, and possibility to correctly do slowmotion, stop and even reverse playing of animations, particle systems and time based shaders. VR Panorama respects this system of scaled timing because it has to render everything in non realtime (actually, all esisting integrated capture systems does this, as it is the only correct way of managing time) and all animation plugins should implement scaled time based timing. LipSync on the other side works in realtime, and it's internal time clicks at a realtime rate. As VRPanorama renders at slower rate, you are getting fast movements. I'm suggesting you to contact a developer of LipSync, I know that they lately implemented a function for correcting this issue - but I'm not sure if this latest version is available yet on assetstore."

 

Yeah - the current version uses the audio playback position for animation, the idea being that you usually want the animation to line up with the audio. It breaks in non-realtime rendering though because Unity doesn't slow down audio to match the framerate. This is fixed in 1.1 by the new animation timing options though - fixed framerate is usually the best option when you know the output framerate of the video.

 

Same problem here. Splitting the clips isn't really a solution for everyone who just want a clip over 59.9 seconds to play on Start. adding the Lupsync.Play(LipsysncData) to OnFinishedPlaying parameter works, but will continuously loop. Adding Lipsync.Stop() to the OnFinishedPlaying is called at the same time as the second clip, so that's not a solution either. Hope to see support for clips longer than 59.9 seconds

 

Yeah, I understand. Longer clips can be played back and created in the editor without any problems, so it's just AutoSync that runs into this problem. I'm looking into it now, but I suspect it's an issue with PocketSphinx itself (the library that handles the phoneme recognition). I should be able to add a feature to automatically split clips when processing, then join the markers together, similarly to how the audio conversion works at the moment.

 

LipSync 1.1 is now available from the LipSync website.
Click the Downloads button in the top-left and enter your Asset Store invoice number and email address to access your downloads.
​

At the moment, only the version for Unity 5.1 to 5.3 is online. I'll be putting the 4.6, 5.0 and 5.4 (beta) versions live tomorrow. Asset Store version coming soon.

Quick note regarding Visual C++: The redistributables required for AutoSync on Windows are not included in this version, to avoid increasing the download size for customers who don't need them. Instead, they will be available to download from both the LipSync website and the Extensions Window in the next couple of days.

Here's the (slightly amended) change list:

1.1

Features
- Phoneme Marker Sustain - join two phoneme markers together to maintain the pose for longer.
- Improved LipSyncData inspector with audio and animation preview area.
- Added options for animation timing (audio playback, custom timer or fixed-framerate).
- Emotion marker labels now show as tooltips if they don't fit on the marker.
- Emotion marker intensity is now visible without opening the marker settings window.
- Added option to allow emotions to persist after a clip has finished.
- Animations now blend in from the custom emotion if one is set when Play is called.
- Some improvements to how Blend Systems are managed.

Eye Controller
- Added ability to use LipSync Poses for blinking and looking instead of raw blendshapes and bones.
- Eye angles are now correctly constrained to the shared rotation ranges when in look at target mode.

Changes
- Realtime Preview targets are now selected from a drop-down menu.
- Moved Pose Editor code out of LipSyncEditor.cs for re-usability.

Fixes
- Fixed bug where Save as and Save buttons don't always function as intended.
- Fixed re-loading of marker intensities in the Clip Editor.
- Fixed errors with playing LipSync animations after setting custom emotions.
- Added specific error for invalid characters in an audio path in AutoSync.
- Removed hard-coded paths.
- Added correct undo support to pose editors.
- Added warning when multiple identical emotions exist on a LipSync component.
- Prevented multiple identical emotions being added to the Project Settings.
- All modal windows are now consistently not dockable.

 

Just signed in to say you have made a great asset here and hope you continue to have success with it. It didn't take long (would have been faster if I Windows 10 Voice Recorder saved to .wav in the first place) before I had a talking model with emotions and eye blinking (Morph3D uses a single mesh for both eyes so I will look at replacing them later to take more advantage of EyeController).

Thanks!

 

Nice to see the update! I tried multiple times to download but got server 500 errors on the website. Managed to download about 5 mins ago, though.

Eagerly waiting to know if the Playmaker plugin has been updated to supporting a variable for the field for "Data Clip".
Really need this to be able to trigger random expressions.

Best Regards

 

Great! Glad you like it!

Not sure what the server problem was, it's all working fine on my end. The Playmaker plugin hasn't been updated yet, that and the other plugins will be updated over the next few days. I'll make a post here when it's live.

 

Heads up to anyone who's downloaded the 1.1 update already: The version that was uploaded yesterday contained a small bug that caused the Pose Editor in the LipSync component to not reset when a pose was closed. It can't cause any damage to your project, and the poses would reset correctly if the game object was deselected. This has now been fixed and the new version has been uploaded.

If you download the .unitypackage from the site again and import it into the project as if it were a normal update (removing the previous installation first) everything should be fixed. Sorry about that!

 

Hi! Just wanted to thank you for an amazing product, it works right out of the box with ease!

I did have one quick question - then setting up gestures, I do get a message saying 'Gestures need regenerating - run the Gesture Setup Wizard' which I do, (providing its the 'Begin Setup' button right above the message) which asks me to pick a layer to add the gestures and if I add a new layer or keep the current one, it usually does nothing, but if I flip back and try to use the original layer, I get this error:

ArgumentOutOfRangeException: Argument is out of range.
Parameter name: index
System.Collections.Generic.List`1[GestureInstance].get_Item (Int32 index) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Collections.Generic/List.cs:633)
GestureSetupWizard.OnContinuePressed () (at Assets/Rogo Digital/LipSync/Editor/GestureSetupWizard.cs:117)
RogoDigital.WizardWindow.OnGUI () (at Assets/Rogo Digital/Shared/Editor/WizardWindow.cs:91)
System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)
Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:232)
System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MethodBase.cs:115)
UnityEditor.HostView.Invoke (System.String methodName, System.Object obj) (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:187)
UnityEditor.HostView.Invoke (System.String methodName) (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:180)
UnityEditor.HostView.OnGUI () (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:76)

thanks for the amazing asset!

 

Haven't seen that one before! It looks like there are more gestures defined in the project settings than on the character, which is odd - they should have been copied over when the you clicked the gestures tab on the character. Do you think you could send the scene that produces the error to contact@rogodigital.com, so I can see what's going on? You can leave out character models and things.

Thanks.

 

LipSync 1.1 is now available from the Asset Store!
This update brings a number of new features and fixes, some GUI improvements and better Unity 5.4 support out of the box. 5.4 is still in beta, and as such isn't supported to the same degree as 5.0 to 5.3. As of Unity 5.4.0b20 LipSync Pro should be fully functional, but I still don't advise upgrading a project using LipSync unless you absolutely need to.

The full changelist can be found a few posts up.

The corresponding update for Eye Controller on store has also been submitted and is currently awaiting approval along with the first release of LipSync Lite! For those of you who don't own LipSync Pro but want to try it out, LipSync Lite gives you all the basic features of LipSync Pro, excluding Emotions, Gestures and AutoSync, for free!
I'll update here when it's available.

 

Hi,
We bought lip sync pro, and we are trying to make lip sync to a non english language, we would like to add our own phonemes and attach them to the right poses, is it possible to add new phonemes instead of just changing the poses of existing phonemes?

 

I am having the dll issue on AutoSync also, but nothing from this forum is fixing it. Just bought LipSync Pro 1.1 from Asset Store today, and am using it on a new project in Unity 5.3.5f1 on OSX 10.

 

LipSync Lite is now available on the Asset Store.
​

Good news for anyone wanting to try LipSync but doesn't want to take a risk on buying Pro - you can now try out all the same core lipsyncing features for free in LipSync Live! It's not a demo or a trial, so you're also free to use it in shipped projects, but if you decide you want any of the extra features like emotion and gesture tracks, or AutoSync, you can easily transfer any animations, characters and presets you create over to LipSync Pro.

We've also updated the LipSync Pro PlayMaker integration. It now supports using PlayMaker's FSM Variables with the LipSyncPlay action, and contains a new action for setting emotions outside of animations. You get download it now from the extensions window.
​

​

Not at the moment. It is on the roadmap though, possibly for version 1.2. There'll also be new language packs for AutoSync to support other languages.

Sorry about this, It's being looked into! I've had a couple of other reports of this on OS X, and I've been able to recreate it on my own macbook so hopefully a fix shouldn't be too far off! I'll let you know when I've found the fix.

 

First, I wanted to say this is an amazing asset! :3
I'm a potential buyer of LipSync Pro but I did have some questions before I buy.

None of us are programmers, we are all artists primarily.

Me and my team have some unique ideas for a game we are creating, an explicit adult one, and have been scouring the Asset Store for months looking for the right thing.

We intend to create one to a few base meshes that can morph into other species and originally opted to use Mega-Fiers, Skele Animation or even Adventure Creator to do the morphing for us... but we found Adventure Creator is the source of where everything should go.

Does LipSync Pro offer more than just LipSync such as altering body shapes independent of mesh topology or would another asset do that for us?
We intend to use LipSync Pro for facial animation without a doubt.
The goal is to allow for morphing of a mesh, saving a particular morph and even randomizing it.
We want to take from UMA2 / Morph3D and make it our own unique base model that morphs into animal species and builds including accessories and attachables similar to UMA2.
The player character would be able to choose pre-existing characters, a generated morphed character or a random one.
We also found ICE Creature Control for crowds and wanted to utilize that and use random morphs for them.

Due to the explicit nature of the game we are creating, we need dynamic penetration of orifices and haven't found any asset that provides some sort of... intimate solution.

For example, when male genitalia goes inside female genitalia, we want it to be dynamic without pre-made animation.
The female genitalia vertices should deform based on male genitalia to make for believable penetration.

We are exploring ways to achieve realistic penetration, mesh on mesh deformation collision and more but we haven't found a solution yet.

Adventure Creator is the primary tool we are using for this but we have a ton of assets we are looking into.
Due to the amount of compatibility with other assets, we would like to know if LipSync Pro could work for our needs.

Thank you if you respond, we look forward to it :3

 

I know the question isn't directed at me but from a person who's been using LipSync Pro for awhile, all it does is change the values of blend shapes and bone transforms assigned to either a phoneme or emotion on the target mesh using LipSync component. Unless you plan on overhauling the scripts that it comes with, the only method that these phonemes and emotes get triggered is when the LipSync component is asked to play a Lipsync data file created using the Lipsync editor.

If you're planning on allowing for customization that involves morphing the character mesh, as long as the blend shapes and bones take into account the morphing applied, you should be able to use it for LipSync without issue. As for your case where you're needing to deform the mesh that isn't triggered by audio then I feel that you may need a different solution for that. That's just my take on it though.