Word Detection - Verbal Commands

Yes. There's a detection event with a score. Some of the examples pause the detection for a little bit after each detected word. Using the callback you can do anything.

 

Awesome, that's what I needed. Will buy

P.S - Any special discounts for students?

 

few queries..

• does it support unity 5?
• how many languages and accents does it support ?

 

Hi theylovegames,

Thanks for your great job. This asset is the best one I can find for voice recognition for Unity!

I am working on a project that involves using voice commands to control a camera. Is this possible using your tools? Do you have any similar demos?

Many thanks in advance!

Bluesyhfang

 

Would you be able to give any example on the word detection thing? I'm not quite sure how to work it out. And the example scene for word detection is a little bit different from what I want.

 

I put together a Getting Started Guide.
https://github.com/tgraupmann/TAGENIGMA-Docs/blob/master/Word-Detection.md#getting-started

Be sure to ask any questions so I can document all the features.

 

Hello,

I am interested in your package, one thing I am not clear about is if this detects words that I prerecorded so other people can use it as soon as they open the app and it will be recognized when they say a word. What I am trying to say is this something like the OpenEars plugin but a different way of doing it?

Thanks,
jrDev

 

Try the demo - http://theylovegames.com/WordDetection_1_17.html

Each user has to record before using and you can save and load word profiles.

Check out the getting started guide.
https://github.com/tgraupmann/TAGENIGMA-Docs/blob/master/Word-Detection.md#getting-started

 

is this verbal commands support mobile?

 

Yes this package supports mobile iOS and Android.

I've also expanded the package to work with the Emotiv brain headset.

 

Hi there,

I'm interesting in this package. I did try web samples but it doesn't work as it should i think. First I recorded all the commands with set. And after that when I tried to use it, it doesn't understand me at all. Every word that I said was randomly understand as completely different. So for example if I said "left" it understands "right", "up", "down".

To be short, if this cannot be extremely optimized than it is not useful for us. Is there a option to get a demo version of asset just to test it if suit our needs?

Thank you in advance...

Regards,
Igor

 

Hi Foriero,

A community contribution! That's great!

The trouble with waveform matching is that it's a sum of the containing frequencies which makes frequency detection difficult to extract. FFT gives you the probability for a set of frequencies.

In the end it's all data that can contribute to the scoring and detection. I'm looking into machine learning algorithms to improve the matching. As with all things it's a learning process. A nice feature of the package is that it's full source to extend.

I will gladly take contributions and add to the next beta.

+1 (community contribution award)

Thanks,

~Tim Graupmann

 

Hi Igor.

The package is full source. So there's not a demo. I sent you a voucher over PM so you can try it out.

 

i would like to test your plug in verbal commands but my friend already bought it . and he find a problem which is when you hide the gui (game object Bool false and put it in a button) the command stucks and wont trigger the command . but when the gui visible is true , that time the command will execute with out saying the command ?. . is it possible to hide the gui text and still make the command work ? i really need this asset .

 

Hi Jaibe, check your inbox as I've sent a PM with a voucher! Let us know how it goes!

 

Hi, I am new to this asset.
Just first testing it out after purchase, want to do some tests with VR in the near future.
One thing that so far nobody has mentioned: seems Unity 5.3 breaks all the demos, the interfaces do not show up (oh how I hate Unity' s indifference towards backwards compatibilty with even point releases ).
There are also errors regarding the goat mp3 files.
Installed 5.2 back and mostly ran fine technically, though I sometimes find the demos to be very hard to train and there is problems with the play button of the audio---

Main question:
What I have not yet understood properly yet is:
how do you use the "set" buttons correctly?

Do you activate them before talking or after talking or while talking?
My results vary in such a manner that I cannot really tell what is the correct way yet.

Also in Unity 5.2 while everything else mostly seems to be working, the audio clip play functions does not seem to work at all (in the web demo everything worked, so I assume that the package needs an overhaul for the newer versions.)
Do you have any schedule for when Unity 5.3 compatibility will be done?

Oh, and the goat scene seems to be missing stuff, the demo does not work even in 5.2.


Keep up the good work, this is an fascinating asset with lots of potential.

 

HI OlliIllustrator,

I moved the goat media files to a different package to reduce the size. So if you really need them, there's a link in the docs.
https://github.com/tgraupmann/TAGENIGMA-Docs/blob/master/Word-Detection.md

https://github.com/tgraupmann/TAGEN...ata/WordDetection_GoatMediaClips.unitypackage

I have a 5.3 beta that I can send to you. Just shoot me a link. support at theylovegames.com

The webplayer demos need to be reexported for WebGL since the web player is being retired.

The demo buttons are push to talk. So you press the button and hold, say the word, and then release.

Thanks,

~Tim Graupmann

 

hi there,
i'm totaly newbie at unity, i had already bought your plugin. in my case, i want to make an android application which can use a voice command to show the information of the model. so for example, i said chair, and there will an information about the chair in text. can u help me?

i already running your example 17, and theres no save and load for any voice i already input. and how my aplication automaticaly choose the mic from android.

thanks before.

 

The same examples should work on Android. Your Android app has a data folder which you can save the profiles in. Microphone selection is the same as PC. However, if Unity can't detect your Microphone it might not be listed. What device are you using?

 

i'm using nexus 5, and i didnt built into apk yet. just try it at unity.

can i simply put code from example 4 to example 17, the code is for save and load profile?

 

Yes the code from each example should be interchangeable, given that you match the field and parameter names.

 

hi there,
i've been try to built an apk app, but my saved didn't include to the apk package.
so i must set the voice command from the begining if i install the apk to the new phone.

theres a way that if i built an apk and my saved profile can be loaded automatically?
and how auto select the mic? so if i start the app, i just can use the voice command without any setting about noise etc.
so if someone use my apk he didnt must set any voice command.

thanks

 

You can load your profile in the Start event of a MonoBehaviour script. You can put the profile into a Resources or SteamingAssets folder.
http://docs.unity3d.com/Manual/StreamingAssets.html

You could put the profile into an AssetBundle.

The profile is just an AudioClip and getting the data from the AudioClip.

You could even put the profile in "Assets/Plugins/assets" so it gets bundled into the APK.

You should be able to unzip the APK and see your profile in the "assets" folder.

 

i'll try to put it into assets folder, but how about auto select microphone?
so i didn't have to click the microphone list at the begining.

thanks b4

 

ah, i already found it. its in examples.cs, just comment the gui script.

 

Microphones are selected by using the string name.
http://docs.unity3d.com/ScriptReference/Microphone-devices.html

The API uses the string Microphone name to start recording.
http://docs.unity3d.com/ScriptReference/Microphone.Start.html

I wish Unity used an identifier or something instead for the case that the Microphone has an empty name or special characters.

 

did my code is right to use assets stream

load profile:
try
{
string spath = "jar:file://" + Application.dataPath + "!/assets/";
string path = string.Format(spath + "{0}_{1}.profile", GetType().Name, command);
//string path = string.Format("Assets/Plugins/Assets/{0}_{1}.profile", GetType().Name, command);
if (File.Exists(path))
{
using (
FileStream fs = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
)
{
using (BinaryReader br = new BinaryReader(fs))
{
AudioWordDetection.LoadWord(br, details);
//Debug.Log(string.Format("Loaded profile: {0}", path));
details.Label = command;
}
}
}
else
{
Debug.Log(string.Format("Profile not available for: {0}", path));
}
}

save profile:
string fpath = string.Format("{0}_{1}.profile", GetType().Name, details.Label);
string path = System.IO.Path.Combine(Application.streamingAssetsPath + "/" + fpath);
string result = "";
IEnumerator Example() {
if (filePath.Contains("://")) {
WWW www = new WWW(path);
yield return www;
result = www.text;
}
}

in the load profile no code error, but in save profile there's an error CS1525, CS8025, and CS 1027 at IEnumerator.

NB : i have using system.Collection;

 

Reading via WWW is going to work. I don't think you'll be able to write to that path on iOS? You'll have to find the iOS documentation of how to get write access to the file system or data folder for iOS?

 

no, i didn't wrote on iOS, i wrote it on android, in the url u gave, there's a code to make a steramingAssets used:
path = "jar:file://" + Application.dataPath + "!/assets/";

and to access it using streamingAssetsPath which code is:
using UnityEngine;
using System.Collections;

public class ExampleClass : MonoBehaviour {
public string filePath = System.IO.Path.Combine(Application.streamingAssetsPath, "MyFile");
public string result = "";
IEnumerator Example() {
if (filePath.Contains("://")) {
WWW www = new WWW(filePath);
yield return www;
result = www.text;
} else
result = System.IO.File.ReadAllText(filePath);
}
}

and when i implemented the streamingAssetsPath there's alaways error.
in my opinion, the streamingAssetsPath can't be used at filestream proccess. i'm i right?

 

It's likely you can write to `Application.dataPath` but you will need to add write permissions to your android manifest.

 

Hey I have a problem, when I run an example the mic selection it says count=1 the microphones doesn't appear on the selection only the message "count=1"

Please help

 

I bet this has to do with your Microphone name being reported as an empty string. If you email support@theylovegames.com I'll send you the latest beta which fixes this issue.

 

Is there any way that I can use Word Detection to drive a character controller for free in my game?

 

Email support@theylovegames.com and I'll grant you a voucher code. I have a limit of 12 vouchers that I can generate per year. So there is a limited supply.

 

Hi, great asset. i modified the current release to try and get it working with the latest version of Unity (5.3.4), but I can see here in the forums that you have a Beta of the next release. Is it possible to get a copy of that so i can check it out, we've just started moving onto Voice Detection on our game and I really need to check if this asset we bought will hold up for our needs.

Cheers,
-Andy.

 

Hi Andy. Email support@theylovegames.com to get a copy of the beta.

I've been refactoring for a simple API and redoing all the examples in the new Unity GUI and examples in WebGL. I'll have a new beta available in the next couple weeks in Unity 5.3.4.

 

I just realized that Unity 5.3.4 still lacks Microphone support for WebGL.
http://forum.unity3d.com/threads/webgl-and-microphone.308197/

I don't see Microphone/WebGL support on the Unity roadmap.
https://unity3d.com/unity/roadmap

So a 3rd party workaround has to be used in the meantime.

If you think Microphone is important because you want to use word detection in WebGL, be sure to let Unity know in the forums and on social media!

 

In the meantime, I can update each example to use the new Unity GUI and rebuild for Web Player. Because it's Web Player, it probably only works in Firefox.

WordDetection - Example 1 - Detection Mic
http://theylovegames.com/WordDetection_1_1.html

This example captures raw wave data from the Microphone and displays the Spectrum.

 

Here's a preview of the example code with the new API.



WordDetection - Example 1 - Detection Mic
http://theylovegames.com/WordDetection_1_1.html

Code (CSharp):

1. using UnityEngine;
2.  
3. namespace UnityWordDetection
4. {
5.     /// <summary>
6.     /// Display two graphs
7.     /// 1) Display the microphone wave data
8.     /// 2) Display the spectrum analysis of the wave data
9.     /// </summary>
10.     public class Example : MonoBehaviour
11.     {
12.         /// <summary>
13.         /// The number of cubes to use in each graph
14.         /// </summary>
15.         const int PLOT_SIZE = 512;
16.  
17.         /// <summary>
18.         /// The Y position of the wave graph
19.         /// </summary>
20.         const int POSITION_GRAPH_WAVE_Y = -100;
21.  
22.         /// <summary>
23.         /// The Y position of the spectrum graph
24.         /// </summary>
25.         const int POSITION_GRAPH_SPECTRUM_Y = 200;
26.  
27.         /// <summary>
28.         /// The scale for cubes of the wave graph
29.         /// </summary>
30.         const float SCALE_GRAPH_WAVE = 500f;
31.  
32.         /// <summary>
33.         /// The scale for cubes of the spectrum graph
34.         /// </summary>
35.         const float SCALE_GRAPH_SPECTRUM = 10f;
36.  
37.         /// <summary>
38.         /// The color bias for the wave graph
39.         /// </summary>
40.         const float COLOR_BIAS_GRAPH_WAVE = 10f;
41.  
42.         /// <summary>
43.         /// The color bias for the spectrum graph
44.         /// </summary>
45.         const float COLOR_BIAS_GRAPH_SPECTRUM = 0.1f;
46.  
47.         /// <summary>
48.         /// A graph of wave data
49.         /// </summary>
50.         private Graph _mGraphWavePlot = new Graph();
51.  
52.         /// <summary>
53.         /// A graph of spectrum data
54.         /// </summary>
55.         private Graph _mGraphSpectrum = new Graph();
56.  
57.         /// <summary>
58.         // The data set holds the wave and spectrum data
59.         /// </summary>
60.         private MicrophoneDataSet _mMicrophoneDataSet = new MicrophoneDataSet();
61.  
62.         /// <summary>
63.         /// Set up the graphs to display wave and spectrum data
64.         /// </summary>
65.         void Awake()
66.         {
67.             // create the wave graph
68.             _mGraphWavePlot.Init(transform, PLOT_SIZE, POSITION_GRAPH_WAVE_Y);
69.  
70.             // create the spectrum graph
71.             _mGraphSpectrum.Init(transform, PLOT_SIZE, POSITION_GRAPH_SPECTRUM_Y);
72.         }
73.  
74.         /// <summary>
75.         /// Display microphone data on a fixed update
76.         /// </summary>
77.         void FixedUpdate()
78.         {
79.             if (null == DetectionMicrophone.Instance._mWordDetectionMicrophone)
80.             {
81.                 // microphone hasn't been selected
82.                 return;
83.             }
84.  
85.             // read the microphone data
86.             DetectionMicrophone.Instance.GetData(_mMicrophoneDataSet);
87.  
88.             // draw the wave
89.             _mGraphWavePlot.DrawWave(_mMicrophoneDataSet, SCALE_GRAPH_WAVE, Color.red, COLOR_BIAS_GRAPH_WAVE, false);
90.  
91.             // draw the spectrum
92.             _mGraphSpectrum.DrawSpectrum(_mMicrophoneDataSet, SCALE_GRAPH_SPECTRUM, Color.green, COLOR_BIAS_GRAPH_SPECTRUM, true);
93.         }
94.     }
95. }
96.  

 

Hi, I just bought the plugin and I'm having problems with the demos. Step-by-step for the demos (atleast one) would be good, for example mentioning how the buttons work.
Anyway, I'm running the demo "example 7" and figured out you have to push and hold when teaching the words. The problem is, if other words has been recorded, and you record new command holding the button, the previously recorded buttons activate as well for a very short time (presumably assuming I'm saying those words also). This makes it impossible to run the demo with own words. I'm running Unity 5.3.4p1, windows 10.

Is there something I'm doing wrong?

 

Thanks for the suggestion! I'm in the process of refactoring and redoing all the examples in the new GUI. Adding pitch detection (ignoring octaves) sounds like an excellent feature.

 

In the later examples, there is push to talk where detection is suspended while recording a new word. I'm in the process of refactoring everything to make the API easier and to add step by step tutorials.

 

please make a detail demo on how its works. got stuck at select the microphone count=0;

 

can you give me your email?

 

Feel free to email support@theylovegames.com and I'll send you the update beta which might fix your issue. Sometimes Microphones have a blank name.

 

I'm a bit confused over the use case of this - does this work out of the box for new end users or does it require that the user do a voice training mode to record their own voice saying each command?

 

The end user needs to record a word for each voice command (training). The package is designed to work offline without an Internet connection.

 

Has this been tested to work on iOS and Android?

 

Both are supported.

 

Can you write a document to explain the parameters,for example "CaptureTime","SampleRate","Threshold" and so on.If we change the value,what will happen.
Thank you very much