Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Spatializer with generated sounds?

Discussion in 'Audio & Video' started by HemiMG, Dec 21, 2015.

  1. HemiMG

    HemiMG

    Joined:
    Jan 17, 2014
    Posts:
    911
    I'd like to use the Oculus spatializer (or some other spatializer that uses the native plugin functionality) but I'd also like to use the native audio sdk to generate sounds. I'm curious whether or not this is possible. I figured I could loop a silent audiosource and have my native plugin mix in the additional sounds in order to get my sounds associated with an audiosource, but it seems like the native spatializer code runs before the mixer gets to it. Am I missing something or would this break my idea entirely? I don't see anyway to pass anything but an audio source to the spatializer in the Oculus documentation, or in the Unity documentation.

    This doesn't really just effect my idea, one of the native audio samples introduces white noise into an audio source. Wouldn't the white noise also need to be spatialized in order to sound correct? I'd imagine there are a lot of instances where you'd want to do something to the audio that changes its sound.
     
  2. richtea

    richtea

    Joined:
    Oct 22, 2015
    Posts:
    5
    There's a demo routing plugin in the Audio SDK examples that loops audio back from the mixer to an Audio Source via a 'Speaker Routing' script.

    The routing plugin copies its input to a block of shared memory, the routing script reads that back to the audio source using OnAudioFilterRead.

    You can place this plugin after your generator then route the audio source to a different mixer channel - if you check the spatialize box in the source it will then be positioned appropriately.

    That's the only way I've found to spatialize audio generated using a plugin other than directly copying to a spatializer & bypassing unity altogether (which of course limits you to a spatializer you have the source code for).

    Unfortunately this means the marshalling overhead in moving your audio back to C# is still present.

    For generating audio that needs to be spatialized but doesn't need pre-processing by other audio sdk plugins, I suspect it's better to use a standard native plugin and go directly to OnAudioFilterRead. I'm basing this on the assumption that the copy to shared memory will add a block of latency to guard against indeterminate ordering of processing calls from the audio thread. I've not actually checked, so maybe it doesn't.

    I'm hoping audio sources get plugin slots or (preferably) the whole audio routing system gets replaced with some sort of dynamic, scriptable node graph.

    If I'm wrong I'd love to know as it's the exact problem I hit as soon as I started trying to use the Audio SDK.
     
    Last edited: Jan 5, 2016
  3. Matexyu

    Matexyu

    Joined:
    Feb 22, 2017
    Posts:
    13
    I've also tried to add Unity Audio Effects "pre-spatializer" but it looks like the only spatializer allowing it is the recently released Steam Audio for Unity, however I stopped using it due to a not really convincing room acoustics simulation.

    I've been trying with the Google VR SDK (seems one of the best spatializers I've tried so far) but they "bury" a dynamically created AudioSource ar runtime into their custom GvrAudioSource, so no Audio Filters can be attached... hacking time ahead? :/

    Is this supposed to be solved in the latest Unity?

    @richtea : I've tried to search for "Audio SDK examples" but I could not find them. Could you provide a link?

    Thanks!
     
    Last edited: Mar 10, 2017
  4. mtytel

    mtytel

    Joined:
    Feb 28, 2014
    Posts:
    84
    @HemiMG I'm also trying to do this and am running into the same issue where I can only route generated sounds in *after* the spatializer.

    @richtea
    Unfortunately in the speaker routing demo the spatializer doesn't actually change the audio. It doesn't even use the default left/right panning built into Unity. The only thing it does is set the gain based on distance. If you've gotten more to work please let me know cause it's pretty frustrating.

    The best I've come up with is putting in an audio clip with a 1 value DC offset as an AudioClip in an AudioSource and routing that to the AudioMixerGroup my plugin is running. The spatializer runs on that DC signal and then when it gets routed to my plugin I multiply my generated signal by the input signal. The result is spatialized, but you're not going to get the HRTF or any other filter benefits. You only get left/right panning.

    Here's what I'm doing for the DC Offset:
    Code (CSharp):
    1. AudioClip one = AudioClip.Create("one", 1, 1, AudioSettings.outputSampleRate, false);
    2. one.SetData(new float[] { 1 }, 0);
    3.  
    4. GetComponent<AudioSource>().clip = one;
    5. GetComponent<AudioSource>().loop = true;
    Would love to get in contact with the Unity Audio team. I have a lot of questions for them and I'm not sure they are very active on this forum.
     
    Vea_Arthur likes this.
  5. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,303
    re: the generated/supplied audio via OnAudioFilterRead is not run fully thru the spatializer

    I've run into this too, the positional information is indeed ignored, except the weird leftover in the form of gain change when the sound is in 3D.
    I've managed to send a repro, QA managed to reproduce it, now it remains to be seen what else can be done with it
    ( In my opinion it is indeed a bug since e.g. even all UI controls values except stereo pan on an AudioSource are applied properly to the OAFR buffer. )

    @mtytel It's true that forum voice is sometimes hard to be heard, but I'm sure that fine folk at unity such as @superpig or @Erik-Juhl does occasionally stroll around these parts, and may be able to find out what will happen to poor generated audio in 3d space.

    The bug I've submitted for this should be #887455 and I think all three people in this thread would be glad to know if it's fixable, or can be worked around.
     
  6. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,303
    This is very nicely sounding idea, I'll make sure to test it out, would be nice to have at least stereo pan working.
    Cheers!
     
  7. Vea_Arthur

    Vea_Arthur

    Joined:
    Oct 5, 2017
    Posts:
    2
    @mtytel How can I multiply the generated signal by the input signal? Does that happen in OnAudioFilterRead? I would love an example. I am very new would appreciate any help you can give.