onAudioFilterRead OAFR function can only run one instance in Unity?

I have two scripts generating a sine wave through onAudioFilterRead on different notes, on two different game objects.

When i run the scene, only one of the OAFR can be heard. if i uncheck that script, suddenly the other one starts playing, and when i restart it, it stops the other one again.

Does that mean that OAFR is a monophonic function?

I presume that it is, because to have many running you just have to add the audio together into one function, and there is no reason why they would include a mixer on unity's sound card function.

 

Hi,

Can you post your code?

The behaviour you're describing is either a bug ( which version of Unity are you using? ), or the result of some mistake in your code.

Cheers,

Gregzo

 

I've had that happen when I tried multiple audio sources on the same object. I expected them to be additive.

Other wierdnesses you might encounter are panning issues and one that I found slightly surprising. OAFR callbacks are indeed on a different thread, what I found surprising was that each callback is on a *new* different thread, not just *a* different thread.

It'd be nice if we had an instanced callback with each audio source given it's own dedicated thread at min latency consitantly ala a mix of the audio clip callback and OAFR.

 

I'm using the latest version of unity, Here is the simplest possible code using the OnAudioReadFunction that streams a sawtooth, you can change the tone of the audio to different frequencies, and can place the script twice on the same object or many times on many game objects, the code only runs once, only one tone and one amplitude of sound can be heard.

I don't think it's possible that the monoscript compiler is not making the same compiliation as in .cs, you can try i think the bug still happens.

Do you think that it's becaue i am using unity free version? perhaps there are audio limitations like only one filter per audio source or something?

Code (csharp):

1.  
2. var tone : float = 2048; //change this to hear different sounds polyphonically
3.   function OnAudioFilterRead(data:float[] , channels:int )
4.   {
5.     for (var i = 0; i < data.Length; i = i +channels)
6.     {
7.         data[i] = (i/tone)%1;
8.         data[i+1] = (i/tone)%1;
9.     }
10.   }
11.  

 

Hey marionette, thankyou, how can you see that OAFR callbacks are on many different threads? Does that mean that the code runs many times but that it is only heard once by the soundcard?

As it's a soundcard buffer input code, i would expect it to be just raw, write data to DirectxSoundBuffer32 implementation in unity or something of that type, i don't know why the code would sound mixdown capabilities.

What do you mean by function callbacks? i don't understand the idea of callbacks very well, i just thought that the function was called to run async from other code and to input an array. Thankyou

 

Another absolute clarification needed is: Is the OAFR callback function *itself* a pro only feature or not. I understand that the *filters* are, but need a definitive answer regarding OAFR is needed imo.

 

To check the threading: if you are using c#, add this to the OAFR callback. I also use visual studio to write my code as well as the Unity's 2013 tools to attach to unity. You can see the threads spawning like crazy.

Debug.log (Thread.currentthread.managedthreadid);

You'll need to reformat that of course and check for spelling. I'm on my mobile atm.

Think of Function callbacks as events with ref parameters in this case.

OAFR is more static while the audio clip creation allows you to define an instanced method for it's callbacks

 

One reason the do the OAFR the way they do is to 'stack' the DSP with filters. What we *need* is a way to populate data similar to the way that portaudio does it. A single dedicated instanced callback per source.

That way I could implement ISyncronizationContext and be able to send events back to the main thread if I wanted.

 

Code (CSharp):

1. void OnAudioFilterRead(float[] data, int channels)
2.     {
3.         Debug.Log(Thread.CurrentThread.ManagedThreadId);
4.     }

 

Hi,

OAFR is not a pro only feature.

The problem with the OP's code is that each component will overwrite data from the previous one. Either have only one OAFR implementation per audiosource, or mix additively to the data array.

 

Ahhh gotcha, and thanks for the heads up on OAFR.

Since I have you, and not to hijack the op's post, but may I send you a pm about a separate issue I'm having?

 

How do you assign only one OAFR implementation per audiosource. I found that it runs without an audio source. I tried putting an audio source on every OAFR making gameobject and audio source was a sample and not interacting with the OAFR.

The reason why it generates 1000 of threadId's must be because it is interacting with a different processor that the thread processor. it perhaps has to change thread all the time so that if the thread hangs the sound doesnt stop.

 

@Marionette
Feel free to pm, but do link to a forum thread you could start for the issue: nice to share info with all!

@drudiverse
Afaik you need an audiosource for oafr to fire. It's clip can be null. Maybe this changed lately and I wasn't aware, but I doubt it.

About thread id: I also doubt these strange results. The data array is recycled between all audiosources, and it would make little sense to start a new thread for each... Try comparing threads by reference, see what happens.

 

or they might be delegates for the callback. shrug. all i know is when i wanted to fire an event back to the main thread, i started implementing a synchronization context and that's when i found that different threads were being spawned per callback..

theoretically, i would've thought that there should be only 1 different thread id each time it's called, but it's not, as you see.

 

btw, I also requested something in feedback. If you agree, please vote

http://feedback.unity3d.com/suggestions/instanced-pre-spatialization-audio-callback

 

@Marionette
About getting pre-spatialization data: you could hack around it by implementing your own AudioClippish container, pipe audio data via OnAudioFilterRead, and piggyback Unity's spatialization calculations by using a tiny AudioClip of buffer length filled with only 1.0f values.

You then multiply your own audio data by the spatialized array of 1.0f, et voilà: 0re-spatialization callback and / or spatialization of your own generative/pre rendered audio data.

Not elegant, but worth a try. Of course, any roll off should be applied after, and not on the dummy AudioClip.

 

Hmmm.. i'll have to check it out. thanks

i just hate having to do it this way, especially since UT already have what we need ;(

 

I'm toying with the idea of using an in line ambisonic solution... we'll see, the journey begins tonite

 

Sorry Gregzo, 1/It doesn't need an audio source 2/It can't run polyphonically even in the simplest case scenario. please only affirm that it can work polyphonically if you have a clear memory of running it so in the past, as i have kept going back and forth in unity to test what you say and it seems that unity has been updated/OAFR never had an integrated mixer function. Unless you know a trick to run many OARF in parallel we can conclude that it doesnt work polyphonically, it doesnt need an audio source, it does need a listener, and the OAFR function does produce a new thread ID every time it runs, 521,523,524,525,526...1211,1212,1213..., which is something to do with the sound API.

What would be great is if there was a Execution Order of Sound Events unity reference page, so that we can see how audioclips, filters, and OAFR's are arranged.

Unless proven otherwise, the answer to this question appears to be No!

 

Hi,

I think we might live in parallel universes.

In mine, OnAudioFilterRead requires an AudioSource to be called.
Edit: It just occurred to me that you might be adding your scripts to the listener, in which case OAFR will run too, but will overwrite all audio data. most probably that's your issue.

In mine, I attach the following script to any reasonable number of game object and it works.
I've been working with OAFR since the method appeared in Unity 3.5, and I can assure you all I write here is backed up by recent tests, on Unity 4.3, 4.5, 4.6 and 5.0 beta 13 and 16.

Try this component on empty, virgin game objects:

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public class SineTest : MonoBehaviour
5. {
6.     public float frequency = 440f;
7.     public float gain = .1f;
8.     public bool  addAudioSource = true; //Set to false = OAFR won't be called
9.     AudioSource _source;
10.  
11.     float _increment;
12.     float _phase;
13.     float _cachedFrequency;
14.  
15.     int _sampleRate;
16.  
17.     void Awake()
18.     {
19.         _sampleRate = AudioSettings.outputSampleRate; //Can't call from audio thread, so cache here
20.      
21.         if( addAudioSource )
22.         {
23.             _source = this.gameObject.AddComponent< AudioSource >();
24.         }
25.     }
26.  
27.     // Simple sine gen, phase not properly updated on freq change.
28.     void OnAudioFilterRead( float[] data, int channels )
29.     {
30.         int i;
31.         float sinVal;
32.      
33.         if( channels != 2 ) //Quick test supports stereo only
34.         {
35.             return;
36.         }
37.      
38.         if( _cachedFrequency != frequency ) //recompute increment only if freq changes
39.         {
40.             _cachedFrequency = frequency;
41.             _increment = frequency * 2 * Mathf.PI / _sampleRate; //set increment
42.         }
43.      
44.         for( i = 0; i < data.Length; i += channels )
45.         {
46.             sinVal = Mathf.Sin( _phase ) * gain;
47.             data[ i ] = sinVal;
48.             data[ i + 1 ] = sinVal;
49.             _phase += _increment;
50.             if( _phase > ( 2f * Mathf.PI ) )
51.             {
52.                 _phase -= ( 2f * Mathf.PI );
53.             }
54.         }
55.     }
56. }
57.  

Edit: Corrected phase resetting.

 

About audio thread ID:

It seems that System.Threading.CurrentThread.managedThreadID is misbehaving.
The deprecated AppDomain.GetCurrentThreadID method returns a consistent ID for the audio thread:

Code (CSharp):

1.  int id = System.AppDomain.GetCurrentThreadId();
2. Debug.Log( id );

There is one and only one audio thread, as there should be.

Cheers,

Gregzo

 

Full testing code for the thread ID issue, place on virgin go:

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public class AudioThreadID : MonoBehaviour {
5.  
6.     static int __audioThreadID = -1;
7.  
8.     void Awake()
9.     {
10.         this.gameObject.AddComponent< AudioSource >();
11.     }
12.  
13.     void OnAudioFilterRead( float[] data, int channels )
14.     {
15.         int id = System.AppDomain.GetCurrentThreadId();
16.  
17.         if( __audioThreadID == -1 ) // cache if uninitialized
18.         {
19.             __audioThreadID = id;
20.             Debug.Log( "Audio thread id: " + id );
21.         }
22.         else if( __audioThreadID != id ) //crash if wrong
23.         {
24.             while( true )
25.             {
26.                 Debug.Log( "I'm infinitely wrong." );
27.             }
28.         }
29.     }
30. }

 

@drudiverse

It just occurred to me that you might be adding your scripts to the listener, in which case OAFR will run too, but will overwrite all audio data. Most probably that's your issue.

 

Everywhere I search, folks use thread.currentthread.managedid. I also read that the appdomain thread call is deprecated? Why would the script be in a separate domain anyway? I understand for external dlls or mebbe www stuff etc, but scripts? How is it misbehaving? Is there a link to what should or shouldn't be used? I also searched for unity3d appdomain and not much came up other than folks trying to load/unload their dlls in a new appdomain.

Not trying to be argumentative, just trying to understand.

 

@Marionette

I don't understand myself. Just researched other ways to get current thread ID because it is ABSOLUTELY IMPOSSIBLE that a new thread would be spawned at every audio frame. Found this stack overflow post, third answer, decided to give it a shot even if AppDomain.GetCurrentThreadId is deprecated.

The results I got were consistent, with different and persistent Ids for the main thread, the audio thread and custom spawned threads.

Cheers,

Gregzo

Edit: re-reading the SO posts, the 4th answer explains why we're seeing these discrepancies:

An operating-system ThreadId has no fixed relationship to a managed thread, because an unmanaged host can control the relationship between managed and unmanaged threads. Specifically, a sophisticated host can use the CLR Hosting API to schedule many managed threads against the same operating system thread, or to move a managed thread between different operating system threads.

This explains why while we are observing managed thread ID being incremented once for every call to OnAudioFilterRead, the OS thread is persistent and there really is only one.

 

It's actually not impossible, for example if I used a new thread delegate to fire off an event then joined the other thread. Not saying that's what they're doing, or why, just that it could be done. I think this would be an excellent question to pose to soren or UT in general. To be honest, I think both methods are suspect now. If you change an experiment to suit the results you want, are the results still valid? To say the other call is misbehaving without knowing why calls this into question. Which then further brings other things into question. Such as, what if they actually are spawning separate threads each call? I know it seems unlikely, but I don't have the code to parse. Shrug.

I suppose I could break out my old black ice and try to step through the registers..

Additionally, I read the stack overflow link, and he's right. Appdomain thread id returns not the managed thread but the pinvoked os thread, which is 1 reason it was deprecated. It was causing too much confusion. Since we're in the .net host process for our scripts, it *should* be managedid. Now, that being said, I'm not sure what mono does or the new 'transcription' layer UT was talking about when dealing with not updating mono and xamarin with licensing.

 

Good point.

The word thread is also ambiguous.
The way I interpret the results, the OS thread is unique and is very much THE audio thread. Managed threading could occur on top of that, all managed threads being handled by the unique OS thread. Just a guess...

Soren isn't around anymore btw. Wayne and Jan hold the keys to the audio temple now.

Cheers,

Gregzo

Edit: simultaneous edits causing confusion...

 

Ahhh yeah that's right, I forgot soren split ; (

The clarification needed I think is if are we actually using a managed thread like we think we are or not. The managed thread id's are recycled etc, os threads are not.

 

One thing I'm empirically sure of is that all OAFR calls are made sequentially.

The data[] is recycled, each call occurring just before the data is mixed and cleared for the next audio source's output.

You can observe that by grabbing a reference to the data[] and plotting it on the main thread: you'll observe "leaks" of audio data from other audio sources contaminating your graph. Soren also confirmed here somewhere that indeed, the buffer is shared between multiple audio sources.

Cheers,

Gregzo

 

Sure, I'll agree to that

What I'm not sure about is the *how* of what they're doing. To maintain latency, they might have some sort of ring buffer internally populated by different threads. Shrug. Dunno. All I know is that I get different id's and to be honest, I agree, it doesn't make much sense, but then what about the results then? I can't just throw them away because I don't like them or they don't make sense to how I would do it

 

Thanks! I still have different results from you.

Did you manage to get two SineTest OAFR objects playing at the same time and making a gong sound? i couldnt with your code.

If i add the SineTest with AudioSource to an empty game object, it doesnt make any sound.

If i add the SineTest to an object with AudioListener, it makes sound

If i add the SineTest to an object with AudioListener and delete the AudioSource it makes sound...

I'm adding the script to the same object as the audio listener, because i tried everything else and it doesn't work.

 

@drudiverse

My script creates it's own AudioSource, don't add it to a go which already has one!

And yes, I've been able to generate 20+ sines simultaneously.

Cheers,

Gregzo

 

And don't do generative audio on the audio listener, as I wrote above, it will overwrite all audio in your scene!

 

Another note: make sure there isn't a clip of any kind associated with the audio source. If there's no sound make sure to move the listener close enough to hear it. Other than that, I'm not sure what to add that gregzo hasn't already suggested. I use sources added in my stuff and it works as well.

 

Thanks, very good information. I tried for ages to get multiples ones working in the previous scenes, and so just now, i started a scene from scratch with empty gameobjects, and it worked. If i add an OAFR to an audio listener, it stops the audioSources from working even if i delete it again, and then you have to restart unity for the audioSources to work again. nice one. Thanks.

 

Have found some very weird behaviour with AOFR and .JS code. .JS doesnt compile in time to associate an audio source with the soundcard thread.

If i use Gregzo's SineTest C code next to a .JS synth code, it works.

If i tell the .JS synth code to Awake an AudioSource, the OAFR doesnt run!

If i start unity next to an AudioSource with a .JS code, again OAFR function is initialised prior to AudioSource!!!!

And to make things even more confusing, if i run the scene one time with a JS code that had a OAFR that didnt initialise, then i can fix it but it doesnt make a sound until i restart the project.

I'm just using .JS because it's faster for me and it will be easy to translate the project into C# afterwards. It's the first time that i find an error with .JS code compilation in Unity, other than when .JS has to be put in Plugins to compile early. It doesnt make a difference when i compile the .JS... so it is a new bug. can't find any way to make it work other than including a .CS script with Awake {make audiosource} next to the .JS

thanks Gregzo.

 

@drudiverse

I just tried a JS version of the same script, no issues.

You need to understand the following:

1) OAFR is a mixing callback. When implemented on the same GO as an AudioSource, you get spatialized AudioClip data as the clip is playing, or zeroed data( but a chance to generate your own data ) if AudioSource.clip is null.

2) If you implement OAFR on the same GO as the listener, do NOT add an audio source. The data you will get is the full, final mix: you can add effects to it, pipe it to a plugin or a file, whatever suits your needs. But if you generate audio there, you'll overwrite the whole mix, unless you additively mix yourself.

3) You can chain OAFR components on the same AudioSource, but you must be aware that you'll need to additively mix subsequent OAFR implementations in order not to overwrite.

Here's the ported to JS script, with additional additive bool param and corrected mistake in phase reset handling:

Code (CSharp):

1. #pragma strict
2.  
3. @Range( 0, 1 )
4. public var gain         : float = .1;
5. public var frequency     : float = 440;
6. public var additive     : boolean = false;
7.  
8. public var addAudioSource : boolean = true;
9.  
10.  
11. private var _increment             : float;
12. private var _phase                 : float;
13. private var _cachedFrequency     : float;  
14. private var _sampleRate         : int;
15.  
16. private static var __PI2 : float;
17.  
18. function Awake()
19. {
20.     if( __PI2 == 0 )
21.     {
22.         __PI2 = Mathf.PI * 2;
23.     }
24.     _sampleRate = AudioSettings.outputSampleRate; //Can't call from audio thread, so cache here
25.        
26.     if( addAudioSource )
27.     {
28.         this.gameObject.AddComponent( typeof( AudioSource ) );
29.     }
30. }
31.    
32. // Simple sine gen, phase not properly updated on freq change.
33. function OnAudioFilterRead( data : float[], channels : int )
34. {
35.     var i : int;
36.     var sinVal : float;
37.        
38.     if( channels != 2 ) //Quick test supports stereo only
39.     {
40.         return;
41.     }
42.  
43.     if( _cachedFrequency != frequency ) //recompute increment only if freq changes
44.     {
45.         _cachedFrequency = frequency;
46.         _increment = ( frequency * __PI2 ) / _sampleRate; //set increment
47.     }
48.    
49.     if( additive )
50.     {
51.         for( i = 0; i < data.Length; i += channels )
52.         {
53.             sinVal = Mathf.Sin( _phase ) * gain;
54.            
55.             data[ i ] += sinVal;
56.             data[ i + 1 ] += sinVal;
57.            
58.             _phase += _increment;
59.            
60.             if( _phase > ( __PI2 ) )
61.             {
62.                 _phase -= __PI2;
63.             }
64.         }
65.     }
66.     else
67.     {
68.         for( i = 0; i < data.Length; i += channels )
69.         {
70.             sinVal = Mathf.Sin( _phase ) * gain;
71.            
72.             data[ i ] = sinVal;
73.             data[ i + 1 ] = sinVal;
74.            
75.             _phase += _increment;
76.            
77.             if( _phase > ( __PI2 ) )
78.             {
79.                 _phase -= __PI2;
80.             }
81.         }
82.     }
83. }

 

Oh right brilliant. i had a soundcard crash and 20 times when a scene wasnt working before and was working after, and simple codes that didnt work in one scene and did in the next, always not putting any scripts on the audio listener, and then i changed the awake call to not change the buffer size while it's installing the audio clip and i had less errors. Dont know how i manage so many confusing unity sounds