Unity Community


Page 1 of 5 123 ... LastLast
Results 1 to 20 of 98

  1. Location
    Washington, USA
    Posts
    79

    Word Detection - Verbal Commands

    Word Detection 1.6
    http://u3d.as/3pP

    Assign verbal commands to your gameplay on Standalone, Web Player, Android and iOS. Control your game characters just by talking. Detection is standalone and does not require a network connection. Add verbal commands at runtime.



    Controlling Mechanim with Word Detection -


    Preview Video -


    Instructions -

    1. Label the first item as Noise (as this will just capture background noise most of the time)

    2. Label each word that you add to make your life easier

    3. Wait 1 second before speaking to let only the noise fill in

    4. Speak the word you aim to detect

    5. Immediately after speaking the word, click the button next to the label, as that will build a profile

    6. Always try to speak the word the same way in the same tone and in the same loudness

    7. If you have trouble matching a word, just repeat the word and click the valid button to update the word details

    8. Try to use dissimilar words (not like: go, go go, gos) and (like: attack, defend, run, escape, affirmative, acknowledged, zzzz)

    9. You can remove a word by clicking the remove button

    Demo 1.1 - http://theylovegames.com/WordDetection_1_1.html
    Demo 1.2 - http://theylovegames.com/WordDetection_1_2.html
    Demo 1.3 - http://theylovegames.com/WordDetection_1_3.html
    Demo 1.4 - http://theylovegames.com/WordDetection_1_4.html
    Demo 1.6 - http://theylovegames.com/WordDetection_1_6.html

    API Overview - Word Detection Example:
    Last edited by theylovegames; 02-16-2014 at 05:35 PM.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  2. Location
    Italy
    Posts
    648
    The idea is cool ...but honestly I was not able to understand if your demo works or not.


  3. Location
    Washington, USA
    Posts
    79
    I'll post the how to video today.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  4. Location
    Washington, USA
    Posts
    79
    The video added includes the details about each example scene.

    The last scene is the word detection scene and the video explains how to use it.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  5. Location
    Washington, USA
    Posts
    79
    A user wrote in about not detecting a few words, here we do word detection for: "Test, Weasel, and Rutabega".


    Make sure your mic volume is set high enough to hear yourself. If you have the gain too low, it would be tough to distinguish words from background noise.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  6. Location
    Washington, USA
    Posts
    79
    I'm not doing any noise removal on the profiles, which I bet would increase the accuracy.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  7. Location
    Washington, USA
    Posts
    79
    The example word detection of clicking on the set button after saying the word is a bit counter intuitive. I'll change that to a push to record button.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  8. Location
    Washington, USA
    Posts
    79
    I've improved demo 1.2 by adding noise filtering, push to talk, and trimming. You can record profiles faster, and you can play back the sample. It helps if you forgot how you said the word last time.

    http://theylovegames.com/WordDetection_1_2.html
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  9. Location
    Washington, USA
    Posts
    79
    In demo 1.3 you can use voice commands to move a cube around.
    http://theylovegames.com/WordDetection_1_3.html
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.

  10. cel cel is online now

    Posts
    24
    is it me or this doesn't work that well in the webplayer... is accuracy better in standalone?


  11. Location
    Washington, USA
    Posts
    79
    Quote Originally Posted by cel View Post
    is it me or this doesn't work that well in the webplayer... is accuracy better in standalone?
    The accuracy is the same on all platforms.

    I am in the process of improving the noise filtering algorithm.

    My current theory as that samples are similar because they all have the same background noise.

    Audacity has great noise filtering, I'll review how that works.
    http://audacity.sourceforge.net/download/source
    src/effects/NoiseRemoval.h
    src/effects/NoiseRemoval.cpp

    In the meantime make your audio commands distinct by annunciating as I improve the accuracy.
    Last edited by theylovegames; 09-25-2012 at 01:49 PM.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.

  12. cel cel is online now

    Posts
    24
    ok... hope you can improve it... it's a genial idea and i'm eager to try out... a few ideas pop to mind... any idea about the price?
    keep up the good work!!!


  13. Location
    Washington, USA
    Posts
    79
    Quote Originally Posted by cel View Post
    ok... hope you can improve it... it's a genial idea and I'm eager to try out... a few ideas pop to mind... any idea about the price?
    keep up the good work!!!
    This is going to be an alternative inexpensive solution to an expert system. The price is going to start out at $30, where I'll integrate any accuracy tips from the community and make this a stellar multi-platform product. The 1.0 product should be available any day as it's under review currently.
    Last edited by theylovegames; 09-25-2012 at 03:18 PM.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  14. Location
    Dubai
    Posts
    24
    does tool support different languages? i have interest
    I am very excite about Unity 4
    sorry for english


  15. Location
    Washington, USA
    Posts
    79
    Quote Originally Posted by fpspro9001 View Post
    does tool support different languages? i have interest
    Any audio would be supported. It's not specific to languages. Instruments would work as well. As long as the samples are unique.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  16. Location
    Dubai
    Posts
    24
    Quote Originally Posted by theylovegames View Post
    Any audio would be supported. It's not specific to languages. Instruments would work as well. As long as the samples are unique.
    thanks for info could game for music...
    I am very excite about Unity 4
    sorry for english


  17. Location
    Washington, USA
    Posts
    79
    Quote Originally Posted by fpspro9001 View Post
    thanks for info could game for music...
    Music is trickier. I bet the RIAA or MPAA has some algorithms for that. The thought did cross my mind, whether it could detect Skrillex music based on a profile. The intent here is more to use word detection as a controller, to fire an event.

    Although there's no reason why you couldn't train it with common words to do dictation.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  18. Location
    Washington, USA
    Posts
    79
    One tiny snag where I need to rewrite the FFT algorithm in C#. The asset store only supports licenses for MIT, Creative Commons, and Simplified BSD(BSD3). The Fourier transform that I was using was under LGPL which is too restrictive.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  19. Location
    Washington, USA
    Posts
    79
    Quote Originally Posted by theylovegames View Post
    One tiny snag where I need to rewrite the FFT algorithm in C#. The asset store only supports licenses for MIT, Creative Commons, and Simplified BSD(BSD3). The Fourier transform that I was using was under LGPL which is too restrictive.
    The FFT algorithm has been rewritten and replaced.

    This package has been accepted in the Asset Store:
    http://u3d.as/3pP

    Now that the package has been accepted, I can move forward with better noise removal logic.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.


  20. Location
    Washington, USA
    Posts
    79
    I still have plans to implement the Audacity noise removal algorithm which has great noise matching. And I figure that if it can accurately match a noise profile, it might also have better pattern matching.

    The accuracy could increase if you push to talk on the voice command. That way the length of the word would be a better match. Right now it's matching on the first few syllables which generally matches all words.

    Another demo could be dictation if we train using a bunch of letter combinations. St, Th, Ch, ABC. This will need a wave analysis algorithm in addition to spectrum analysis.

    The fingerprinting process, takes the original wave and runs a FFT algorithm. Wave is Amplitude over time. The length of the wave is the frequency. FFT counts each frequency. The math is tricky because a wave is the sum of frequencies, so you get a probability table.

    The fingerprint is all the frequencies of the word over the length of the word. Which is not as accurate as if we split up the word into small chunks and compute the frequency of each segment. In that way we can have a more accurate fingerprint.

    The fingerprint process happens in Example4.cs as SetProfile. And then the selecting algorithm happens in WordDetection.cs.

    WordDetails.cs holds the Wave, and the Spectrum. Although I should add a List<Spectrum> for the profile chunks.

    And then FourierTransform.cs I would need to modify to work over the range of the chunks, without making a bunch of array copies.
    They Love Games
    http://theylovegames.com

    Generate native plugins with matching C#/C++ signatures.

    Bake light mapping using remote computers locally or over the Internet.

    Add verbal commands to your game.

    Merge changes in your scene with ease.

Page 1 of 5 123 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •