AudioStream - {local|remote media} → AudioSource|AudioMixer → {outputs}

Hi @hzq_hassan,

it should be - I've just tried an internet radio on Android device with both Mono and IL2CPP backends using the above example from Wednesday just continuosly printing the sum of the buffer via OnGUI and left it runing for some time - the sound didn't exhibit any distortions or irregularities.

What Unity version are you using ? Also, please try to check DSP Buffer Size - it's under Edit - Project Settings - Audio and should be set to Default for now.

I will try local file streaming meanwhile.

Thanks!

 

I've just tried local file playback with GetSpectrumData on an Andorid phone and all seems to be working alright, @hzq_hassan

If you want DM me, or send an email (at the asset store page), I can take a look at your project since nothing obvious which might cause this is occuring to me - except the above mentioned Default DSP Buffer Size ( which is a known limitation at the moment ).

I am not sure what you mean by laggy sound, too.

Hope this helps!

 

Thank you @hzq_hassan for the email once more !

the problem was active AudioSourceOutputDevice on the AudioStream game object in the demo scene, which has apparently performance problems on mobiles.
As a workaround if anybody stumbles upon this meanwhile it is enough to just remove the AudioSourceOutputDevice component from AudioStream game object in the demo scene - it has to be removed, not just disabled.

I hope to correct it in next update, at least separate demo for AudioSourceOutputDevice into its own scene (duh).

Sorry for the troubles! I guess @JonDadley would like to be notified of this, too.

 

Hello! Taking a look at this plugin and I have a question.

I'm working on a project where I have 2 audio sources, 1 is for a microphone & 1 is for video playback audio - I want the video's audio to play out of the speakers, but I do not want the microphone to play out of it. Currently I can't do that as it's all "Unity Audio", would this plugin allow me to assign just that audio source to those output speakers?

Is there any trial version available that I could test with my setup?

Edit- I read earlier in the thread that the audio output device component silences the unity audio and reroutes it to other device, any way to have both? basically keep the original unity audio then also have it play to speakers? I need the Unity audio as I will be recording a video of the whole thing with mic audio and video audio.

Thanks!

 

Hi @Adkaros ,

yes - the AudioSourceOutputDevice reroutes output of an AudioSource to selected output device, so if you e.g. leave the video to be played on default output and redirect mic to play its output somewhere else, that's perfectly fine.

Yes, it depends on just one line to be commented out, or not.
I might maybe expose this for the inspector, but was reluctant so far since it's not very common functionality and very easy to tweak since you have full source code at your disposal. Everything was in a single demo scene so far and things would not be easily accessible for all users, too.

In your case I'd determine which outputs can be fed by Unity on default system audio output and silence / redirect / duplicate .. the rest appropriately.

As for trial: I can refund you should you choose so via the Asset Store ( but I _hope_ it would never come to that .) -
I don't have any trials, unfortunately - as you can see also from your example, it's maybe not even possible to capture every possible user scenario in a single test package.

Sorry for late reply - I've just pushed update to AssetStore which btw brings streaming of audio input from any available input/recording source to the Unity AudioSources.(*)

I hope that cleared things a bit
Cheers!




(*) unfortunate spatialization conditions still apply.

 

update on this - a.k.a. capturing currently playing sound on a desktop computer in an Unity application / @AveryBartlett

In latest update - not yet on the asset store - I've added ability to record from arbitrary available input driver and with sound card driver setup as described above by pointing the plugin to configured device it is indeed possible to record any playing audio by system even in the editor.

note: by record I mean you get actual audio data on/in an AudioSource. You still have to process / save them by yourself if you want to, for now.

 

V 1.4 seems to made it and is live

- new AudioSourceInput component - allows recording audio from any available recording device.
- update for FMOD Version 1.09.03 ( it was not completely necessary, you can stay on any previous version ) /*
- added new demo scenes for each respective functionality type
- few 'quality of life' improvements and usability fixes based on users requests

/* FMOD added support for GoogleVR audio in 1.09 release - there are some quirks with compiling on iOS with this version which should be documented in the readme; other platform should not be affected

 

It looks like the sale continues with AudioStream still in, it being 30% OFF.
I thought the sale ended yesterday, but apparently the assets from previous week are still included (▀ ͜ʖ▀)

I wanted to go for 50% discount, but 30% was maximum selectable so I chose that.

/ cheers!

 

Currently there are few gotchas on iOS for some users:

- the newly added libgvraudio to FMOD 1.09 requires non Bitcode enabled build ( unfortunately ). This is probably an oversight on FMOD part and should be fixed in time, but meanwhile it is necessary either remove libgvraudio from the build altogether if you don't need Google VR support ( just delete it from Plugins/iOS ), or build the whole application with Bitcode disabled.

- for recording on iOS to work some setup in Unity and Xcode is needed - I am mentioning the steps here since these are not included in the package's README file yet:

• 'Prepare iOS for Recording' must be checked in the Editor Player settings:

• on recent iOS versions you need to explicitly request microphone usage from the user due to privacy concerns. The permission request will be displayed by iOS if NSMicrophoneUsageDescription key and its value ( message for the user ) is present in the application Info.plist ( a.k.a. Target Info Settings in Xcode )
  ( recent Unity versions might add this key automatically )
  
  
• last, but not the very least - DSP Buffer Size in Edit -> Project settings -> Audio must be set to Best latency:

this is currently needed for mic input to work

- for some reason the AudioStreamMinimal component works ( meaning after acquiring the connection to network audio source it actually starts the playback ) only if used exclusively in the scene - if at any time during scene lifetime AudioStream playback is invoked, too, any subsequent AudioStreamMinimal playback request only connects and stops without streaming and playing the audio. This only happens on iOS 10+ and with newest FMOD package, on my testing iOS 6.1.6 (yes) device (with the newest FMOD package) both components work as intended ¯\_(ツ)_/¯ .

It's hard to tell which one of iOS and FMOD needs to be fixed.

 

Product Enquiry.

Can this plugin load a microphone recorded raw PCM stereo file from a directory ( has access) and assigned to the AudioSource for playback ?

can dropbox a sample file for you to test.

Max

 

Hello Max,
that's perfectly valid use case for AudioStream, and the format shouldn't be a problem. Feel free to PM me a link to be sure.

 

Based on previous inquiry I've added support for RAW audio format with user specified sound parameters for the next version.

OP updated with Tizen notice, which apparently has unresolvable problems for now.

 

Hi r618,

I'm wondering, is this asset have background playing functionality in mobile devices? Can it run continously in the backgroud while the streams keeps playing?

Take care

 

The radio streaming works out of the box? Ios and android?

 

No, not out of the box -

( the OP notice concerns normal streaming while the app is running normally.
I haven't tested background streaming, but it might be enough to just add the required background mode for iOS to keep the audio playing ( but you won't have e.g. the controls on home screen and the app won't be updating while in the background automatically ).
I've less experience with Android, but on some the app was even playing / streaming when the device was locked without modification IIRC, on some not. )

edited initial reply since I've misread the question. The background audio limitation applies though. and was aimed for @otnemem

 

now as I'm reading my reply, I've probably misread your post @gringofxs - I thought about background audio mode, for reasons unknown

So to clarify:
yes, you can pass any reasonable url such as network radio playlist to AudioStream and it will play the radio station - on all tested and supported platforms without limitation


Sorry for misinterpretation if that was the case

Cheers!

 

Given your disclaimer, do you think it's possible to use this asset in combination with the GVR audio spatializer?

Edit: By the way I guess it'd work as long as I can grab the audioclip(s) and use them with their component which is similar to an AudioSource.

 

It might @sandman3 - when you utilize the full FMOD Studio plugin in Unity ( from which I'm using in the package only tiny part which wraps FMOD low level API - the rest is simply not used and can be removed from unity project ), you should be able to use gvr spatializer added in 1.09 in the FMOD Studio project imported into Unity.

but note that I've never worked with FMOD Studio extensively, so I have no good advice of its interoperability with Unity's AudioSources - it would be great if it worked, though.

 

yet one more addendum to the previous: it also seems that gvr functionality is not present / exposed in the low level API which I am using - so, unfortunately, there is no way to access it directly, i.e. without FMOD Studio

 

Hello, I'm working on a project which need to play game audio from mobile speaker while it connect to a headphone.
In your description of the plugin, you said "meaning these do not work on mobile platform out of the box".
So does your plugin can play audio from mobile speaker while it connects to a headphone or it won't work, I'm abit confuess right now.

 

@Michelangeloapple it won't work -
from application point of view there is still one output device available on iOS, only os/hw switches between headphones / speaker/s transparently - you cant distinguish where the actual output ends up in your application when using this package
moreover, iOS 10 and latest fmod have various issues when using output device - i dont recommend using it for this purpose, at all for now, sorry.

- it might be possible to delve into CoreAudio some more and find out whether its possible to tell iOS where would you like the audio to end up, but its not part of this plugin, and I'm not sure if its possible at all

 

Thanks, but I still want to know if it works with android devices?

 

Hi, it's the same situation - the OS handles primarily the presence of headphones and dis/activates outputs appropriately
In AudioStream there's no way to distinguish between them, i.e. these are all available interfaces as reported by FMOD regardless of headphones being connected or not ( the available outputs are listed on the top ( there is only one ) ):


It might be possible to request speaker via plugin on Android: http://stackoverflow.com/questions/...-internal-speaker-while-headphones-plugged-in

 

While resolving an issue with recording on the iOS I was able to find a way to direct the audio on the speaker when the headset is connected; but once requested, this setup is valid for all playing audio in the application ( until 'called off' )
I wonder why/how do you want to use it @Michelangeloapple ? Setup like this will get convoluted pretty quickly - an user wearing headphones expects the audio to be played on them, you'd have to switch between outputs while the app will be running and all audio will be either on speaker, or on headphones ( unfortunately? )

Also, no Android for now; although the plugin itself is not complicated ( probably ), I've no spare resources to integrate native Android plugin for now. Thanks for understanding.

I can add this for iOS, but it will take a while to expose everything properly should I choose to.

 

Thank you, in our project we use 3.5mm mic input as a sensor input. We want to hear audio from the speaker while sensor is active. We have tried alot of plugin but there is no luck.

 

@Michelangeloapple that makes sense, but do you need speaker to be active _while_ actively recording ?
- Having just earpiece active while recording is deliberate choice in order to prevent feedback -

The way it works now with fixes in 1.4.1 is that I request speakers for audio playback and earpiece for recording on iOS automatically (so if you have 'Prepare iOS for Recording' checked you still get audio playback on speakers as usual ).

For your use I'd have to expose a setting which would set the output on speakers explicitly ( but again, I find very low value in this since you'd get feedback while recording from sensor which is i.e. not far enough from speakers )

Note: if you need speaker while not recording only (and having 'Prepare iOS for Recording' checked) that should work with fixes already done for 1.4.1 -
It will take couple of days before I submit this new version.

 

Thank you very much, the sensor we use has nothing to do with sound, its more like a magnetic sensor, so it won't create feedback loop with speaker sound.

 

OK, I'll try to squeeze in some advanced setting for iOS in 1.4.1 update

 

I've spent some time on this @Michelangeloapple , but according to https://developer.apple.com/referen...sionproperty_overrideaudioroute?language=objc
"the system changes the audio routing for input as well as for output"

the input and output are paired - meaning they are either from/on receiver + earspeaker, or headset mic + headphones - , with the best option I could come up with being having input from receiver and output to main speaker ( using kAudioSessionOverrideAudioRoute_Speaker override )

I've also tried AVAudioSessionModeVoiceChat mode leading to similar results.

It seems people had more luck by using external bluetooth device where in iOS 10 there was added new category which preserves external mic.

[I haven't tried multirouting which requires more elaborate setup of AVAudioSession, which I am not sure is compatible with Unity]

So, not gonna happen for now, sorry.

 

I'm sorry to hear that, I'll try something else.
Still, thank you for your effort.

 

New update 1.4.1 is live - containing mainly fixes for recording on iOS and fix for stream sample rate conversion (was doing weird things previously if channel count of the stream and Unity audio settings did not match)

V 1.4.1 052017
- RAW format support also in the Editor
- iOS recording guide and fixes
- readme updated with general remarks on usability of FMOD and Unity versions
- startup sync to better synchronise user scripts with FMOD initialisation
- Unity audio sample rate compensation for stream sample rate bugfix for AudioStream component
- added AvailableOutputs() also to AudioSourceOutpuDevice allowing its better independent usage

 

Spoke too soon with the stream sample rate bugfix *grin*
But hold on, 1.5 update is coming (already submitted) and it will contain very positive (hopefully) surprise.

 

Update finally just went live, in which the 3D limitation is finally gone (yay) :

V 1.5 052017
- finally support for Unity spatialisation
- simple minimal 3D demo scene
- few inconsistencies and limitations were resolved in connection to the above

^
the last sentence primarily concerns sometimes wrong sample rate conversion, and more reliable playback and recording for all components, mainly on mobiles.

However, as probably all things in life, the 3D spatialisation support doesn't come for free : -
everything is alright for playback, but the recording / mic latency went up significantly, unfortunately ;/ -

- I'm not yet sure how to make it better, tried to set all the critical buffers on c# side as small as possible until it still worked, but so far I was not able to overcome this. It seems to be showing 0.5s and more depending on machine/OS.

- I might be able to introduce new ( in reality old ) rec/input component just for 2D where the latency is much better, if it will make sense.

Cheers

 

Was contemplating adding saving streamed audio as .wav file last night, so I'll probably add it.

 

Hey I am new to Audio Stream. JUst wanted to use the AudioSourceOutputDevice. But i have this problem when i build. In the editor it works fine but when i build the project to windows. The build doesn't work and It makes the AudioSourceOutputDevice not work anymore. I am required to remove the component and add the component again. Could this be because of the fmod build settings?

 

@TimNguyenCreates Adding/removing - or more precisely - enabling/disabling AudioSourceOutputDevice is not supported, and I'm not sure how to do it properly, yet.

If possible, use StartRedirectWithSampleRate and StopRedirect methods when appropriate - have a look at the OutputDeviceDemo scene and script where those are used - and leave the component enabled all the time; or when you add it manually calling StartRedirectWithSampleRate manually should work too.

Feel free to send me an email/PM if you get stuck, I can have a look at your project.

Hope this helps!

 

Please, someone can prove if with this plugin you can do streaming of the following radio:

"http://endirecte.amposta.cat:8000/radio"

I would like to know before buying it, thank you very much.

Regards

 

yes @pescAPPs it streams ok

Cheers

 

Just submitted hotfix release for AudioSourceOutputDevice only -

V 1.5.1 062017
- AudioSourceOutputDevice hotfixes release:
- fixed sample rate mismatch and consequent memory leak leading to sound degradation over time (regression from 1.4)
- fix for Best latency DSP buffer size in Audio Settings
- much smoother runtime device change without noticeable audio popping
- error message instead of plain exception when AudioSourceOutputDevice is not initialized / startup have to be synchronized since 1.5 /

 

changes are live on the store

 

Does this support streaming from another application on mobile (for example, on Android, from another music player that supports M4A)?

 

Hi @greay ,
no, for interprocess/interapp audio on mobiles you have to go different route, which in the case of Android I have no idea where to even begin

 

Thanks to our thorough users we've caught an issue preventing streaming files with tags of large size - typically e.g. mp3s with embedded artwork.
Fix will be submitted soon, if you have problems streaming some files just PM/let me know.

 

I have Audiostream working but a couple questions.

my use:

Audio Visualization, using AudioStreamIn to get the windows audio so user can play spotify or whatever they want and use my unity-made toy to visualize whatever is playing.

I noticed a large latency when using just AudioStreamIn... not sure if the latency is fmod getting the windows audio, AudioStream wrapper of FMOD, sound driver, OS, etc. I need it as close to realtime as possible since the unity app is visualizing the audio in realtime. my entire per frame for visualization is sub 10ms... for getting the audio spectrum data from the audio source, to processing and gpu rendering the visualization. The audio data I get seems at least 1/3 to a full second behind what is being heard from the speakers.

The hack/workaround I am using is setting windows to use a non-plugged-in digital output as default, then using AudioStreamIn to grab it, and AudioStreamOut to the speakers. Then there is 0 lag since Unity is what is sending the audio to the speakers.

So.... I recall latency issues for GuitarRig and other analog/realtime effects software use ASIO4All to get latency down to ~10ms or less. Is there anything built into FOD/AudioStream I can set/use to get rid of the latency? I can't really expect users to rig their system audio to a dummy output like my work around does.

Second question... is there a way to have AudioStreamIn automatically use whatever windows default output device is?

 

hi @blaher ,
the latency issue is known; - it's because i'm using full unity's processing of the audio in order to have full 3D spatialisation;
- I'm working on more lightweight component for recording/input, which should be 2D only, but have reasonable latency ( although not 0 ) - i'm looking into two main options right now and would like it to be released in 1.5.2 - i can't promise it will be there though;

as for your second question: the current system default device - either input, or output - is always the first one as reported to FMOD, i.e. with index 0

 

if you mean the actual recorded audio - i've added a 'listen to' checkbox to AudioStreamInput component in 1.5.2, so it can be turned on/off, but it itself should be always playing on default - unless you have redirect on the same game object - or I didn't fully understood the question

 

Thanks for the quick response. Appreciated even though not what I hoped to hear since this was the only reason I bought the component (realtime low latency grabbing of windows audio from external app so I could visualize in realtime). Know of any other solution that can do this in unity with as low of latency as possible?

 

latency is a notorious problem when dealing with microphone input and since probably all other relevant assets are based on it i would think it would be unfortunately the same

I should be able to send you 2D version of input capture with lower latency in a course of a day or so, PM me if interested still
- but based on your description I think 2D component would work for you

 

I'm interested in the 2d version for sure. Thanks.

Oh and I really just want the spectrum data from the windows default output, looks like FMOD has built in DSP, any chance of just exposing that w your wrapper and bypass using the unity audiosource?

http://www.fmod.org/documentation/#content/generated/FMOD_DSP_PARAMETER_FFT.html

 

hmm I would have to look into the whole DSP chain, but once I have fmod's record data fed into an AudioSource as I intend to, the difference between this and what I want to send you should be negligible, if any ( Unity uses the same FMOD functionality internally btw )
I'll see what I can get from testing.