Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Word Detection - Verbal Commands

Discussion in 'Made With Unity' started by theylovegames, Sep 23, 2012.

  1. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
    CaptureTime needs to stay 1 second. That's the detection window.

    I'm phasing out SampleRate and Threshold in the new system. Profiles will retain the original AudioClip/Wav/MP3 and auto-resample as needed.

    Threshold affects the scoring during matching so that scores beyond the threshold are treated as noise.

    SampleRate is frequency times channels, IIRC.
     
  2. Nefisco

    Nefisco

    Joined:
    Jul 25, 2016
    Posts:
    1
    Hi, first of all, congratulations for your work, that is awsome!
    i'm testing the demos in firefox, but does not show the input of the microphone, i use firefox 45.0.1 in a MSI laptop, you have any idea of why this happen?
     
  3. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
    It could be browser permissions that you need to permit access to the microphone. It could be that Unity doesn't support your microphone. The Unity player is deprecated and I need to provide WebGL support for the plugin. Currently Unity WebGL doesn't provide Microphone in the API, so I'll have to write a WebGL plugin to make it work in the browser.
     
  4. Ascensi

    Ascensi

    Joined:
    Sep 7, 2013
    Posts:
    579
    @theylovegames I'm very interested in purchasing your asset and have been looking for something that can trigger events/animations objects etc. I thought it through how such a system could store keywords/audio files but this may be big and many.. then I thought what about a library of words that are typed in and if spoken could trigger things.. I started searching more and I found this http://leadingones.com/articles/voice-recognition.html (How to Instruction)
    It apparently uses text for keywords then when spoken the demo spawns a cube etc. I'd like if possible if you could add this kind of functionality as well as event/object triggering.

    If you do add these I think you should advertise yourself as an essential VR asset.. you can add voice commands to open menus, prompt text or spawn AI for help/backup etc
     
    Last edited: Nov 29, 2016
  5. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
    This package works by sound matching. This package does not have text to speech or speech to text features. The package will match sounds recorded by the user. The package will not match sounds recorded by a different person.
     
  6. Ascensi

    Ascensi

    Joined:
    Sep 7, 2013
    Posts:
    579
    That's unfortunate, I can't use it if it only matches the person that recorded it.. that's also why I suggested if you have the time and interest that maybe you can implement the windows speech recognition with a list of text keywords -save on storage space and make it recognize everyone's voice.
     
  7. Gunhi

    Gunhi

    Joined:
    Apr 18, 2012
    Posts:
    300
    Please make a step by step tutorial. I don't really know how to use it and how it work.
    Let's say I just want it ca recognize a simple command like when user say "ABC" then the abc model pop up!
    I don't really know we should hold the button to speak or just press to speak.
     
  8. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
    I think this is what you are looking for.
    https://github.com/tgraupmann/TAGEN...ord-Detection.md#user-content-getting-started

    In the next version, things will be very simplified.

    It might be a couple years though. Unity WebGL needs to add Microphone support which is very distant if it's even on the roadmap. I can get the feature earlier making a custom WebGL plugin, which I've done. But getting the same functionality is going to take a lot of work.
     
  9. Gunhi

    Gunhi

    Joined:
    Apr 18, 2012
    Posts:
    300
    I got it working very close to what can be expected.
    When I build and run on iOS the volume is very quite, is that from Mic Volume while recording or Audio player?
     
  10. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  11. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
    Last edited: Jan 13, 2017
  12. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  13. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  14. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  15. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  16. HeyJoeLang

    HeyJoeLang

    Joined:
    Jun 14, 2013
    Posts:
    4

    This seems great! I purchased version 1.11 and am having trouble abstracting out the audio recording code from Legacy GUI to work with 4.6 Canvas. It seems with these new updates, you have done just that!

    Is there a possibility you may supply the updated source code? I can provide proof of purchase of the asset.
     
  17. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
    Send an email to support@theylovegames.com thanks!
     
  18. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  19. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  20. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  21. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  22. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  23. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
    Now with Mac support!

     
  24. dttngan91

    dttngan91

    Joined:
    Nov 21, 2013
    Posts:
    80
    I intend to buy it but a bit confused, which language does it support? Or I can talk any words in any language and the algorithm can detect the matching word? May I ask which algorithm are you using? (MFCC and DTW?) Is it possible to customize your algorithm to match my requirements?
     
  25. kapaakea2

    kapaakea2

    Joined:
    Sep 4, 2013
    Posts:
    7
    Hi,

    Any possibility for adding support so you can recognize mouth shapes via the camera for assisting to correctly identifying the word being said with greater accuracy based on the phonemes to mouth shapes to audio? Perhaps you could get this working via an expansion/integration.

    Could phonemes be the base of the detection..having a real hard time with it activating when it shouldn't.

    I have a handful of words and its activating regularly even when no words, or other words are being said. Threshhold doesnʻt seem to fix it and I want it to work without training for a specific user if possible.

    Kalani

    Update: I just realized the record buttons don't work...it records 100 ms sounds...how do I use them..tried click and hold, click, etc., when I click play I just here 100 ms sounds (1/10 of a second). Something is not working with the audio recording and detection if its doing that for the recording - its doing that for the detection as well...I was able to through about 20 minutes of effort get 8 words recorded so I could hear the word when clicking play. However when sounding the word back it was still very inaccurate (though much better)...whats going on with the mic and detection...anything I can do to improve it. (My mic is fine for recording, even did a headset..same thing).
     
    Last edited: Oct 12, 2017
  26. lzt120

    lzt120

    Joined:
    Apr 13, 2010
    Posts:
    93
    Does this plugin support Chinese ?
     
  27. theylovegames

    theylovegames

    Joined:
    Aug 18, 2012
    Posts:
    176
  28. Sambastard71

    Sambastard71

    Joined:
    Nov 25, 2018
    Posts:
    1
    hi, i'm courious to know if it function on oculus go?