Voice command

Has anybody been able to implement some sort of Voice Command in Unity?

 

Do you mean voice recognition?

 

Yes. It should be limited to simple commands ("shoot", "walk", "open", "run", etc), I don't need to recognize a full text, for example.

 

I'm also interested in this.
For now I tried different approaches - mostly using external (C, C++) libraries and integrating them into Unity XCode project, but most of the time I have serious issues with that.

Btw I use PocketSphinx open source project for voice recognition part - both using VocalKit wrapper for that and my custom one.

With my custom light weight wrapper which works fine on it's own, ie. runs and recognizes voice commands, when I'm trying to run it from inside of Unity XCode project I keep getting
AudioQueue error -66681 "kAudioQueueErr_CannotStart" which is described as "audioqueue is having some kind of trouble and cannot start " -not very helpful

i.e. it even not getting to the voice recognition part, fails just trying to initialize an audioqueue (I'm trying to create and record a WAV audio file to use voice recognizer on that later).

Has anybody had any experience with integrating external audio handlers with Unity project in XCode?

I think Unity somehow prevents me from using AudioTools externally, is that so, can anybody confirm?

Oh.. I forgot to mention that I mean integration in XCode project only. From Unity I just trigger external method in my custom Obj-C class (using standard ways for that) - i.e. in Unity project I have my own OBJCPlugin class with only this inside:

Code (csharp):

1.  
2.  
3. [DllImport("__Internal")]
4.     private static extern void _activateVoiceCommand( bool _act );
5.  
6.     public static void activateVoiceCommand( bool _act )
7.     {
8.         // Call plugin only when running on real device
9.         if( Application.platform != RuntimePlatform.OSXEditor )
10.         {
11.             _activateVoiceCommand( _act );
12.         }
13.     }
14.  

activateVoiceCommand is called by pressing a GUIButton in Unity scene.

and in my Objc-C part I have a corresponding class UnityBinding, which implements this only method:

Code (csharp):

1.  
2. void _activateVoiceCommand( bool _act )
3. {
4.     run my voice recognition code...
5. }
6.  

And in that "run my voice recognition code" part I get that error.

So as you can see this code is not really related to Unity in any way - if I run it alone - it works fine on iPhone! So I can't imagine how Unity might affect it.. but it does!
[/code]

 

Why I didn't use VocalKit - is because with that I constantly had app termination because of low memory error... very strange also. Because it runs just fine on it's own too. So I switched to a lighter my own wrapper for PocketSphinx library. But now I get different kind of error I mentioned above.

 

Thanks olegklk, I'll try something like this, and I'll be able to run, I'll reply here.

Bests

 

Hey, I just figured out the problem with it.
Unity seems to change the AVAudioSession category, so I had to change that to PlayAndRecord just before initializing the audio recorder (this was inspired by another thread on this forum considering usage of microphone in Unity).

This is what I do for that:

Code (csharp):

1.  
2. AVAudioSession* avSession = [AVAudioSession sharedInstance];
3.     NSError * error;
4.     [avSession setCategory: AVAudioSessionCategoryPlayAndRecord error: &error];
5.  

And it now works just fine.

So you basically need any kind of audio recording code that would work on iPhone, saving it into a file (I used a simple tutorial from TrailsintheSand.com go for Audio tag from the tag cloud ) and then the recorded audio file gets processed with PocketSphinx, using its API as usual.

 

It is possible to play a 3d animation by voice activation, instead of touch button? when said (PITOCO) the animation start.

 

Yes it is doable, but you may want to ensure the command words that you choose are both phonetically simple and different enough to not confuse the software and frustrate the user.

If it helps the microsoft speech recognition library is pretty simple, free to use and should work find with mono C#
(as least it was free when I last looked)

 

Could you help me with the script?

I want the command word PITOCO start the animation (PLAY), and the command word JUMP start the animation (JUMP).

it is possible!

 

im using unity, and the aplication is for Iphone and Android.

 

can u help me, do that with a simple project?

 

i'm waiting to. help me

 

Hi @olegklk , do you have any full example on what you are doing ? I understand you have Unity C# script that calls external ObjecitiveC code that uses with the Sphinx framework. But I still dont understand the AVAudioSession hack you are doing there, and also, how are you sending the resulting Sphinx, speech to text result back to Unity.

I pretend to use www.api.ai for the A.I part, using Unity HTTP requests to their system, but I need to send the speech text, result of the Sphinx (Speech to Text) functionality. Since apparently there is NO Unity Library for Speech Recognition, and Sphinx works in iOS so as long as I can tell Unity to communicate with iOS then it should be fine.

Could you please add a screenshot or post your iOS Class that communicates with Unity ?
I understand that first there most be a objc code that records audio, saves to file and then use Sphinx to process that audio and bring up the resulting TEXT, then send that back to Unity so I can make a HTTP request to API.ai get a response and then execute my game logic.