Loading Streaming Audio from the Resources Folder?

I have a lot of audio which I need to load in each of my scenes. Each scene can easily contain a few hundred moderate length audio files, but it only ever needs one or two at a time. Precisely which clips are needed is not possible to predetermine, so I've dismissed AssetBundles as impractical. Resources.Load would be ideal, but it seems to load them into memory even if they're marked as streaming.

So at the moment, I'm doing:

Code (csharp):

1.  
2. VoiceClip = Resources.Load(Voice) as AudioClip;
3. Resources.UnloadAsset(VoiceClip);
4.  

Amazingly, this actually works. Unloading the asset doesn't make it unusable - it plays correctly when I attempt to play it, and the maximum audio memory usage is never higher than the size of the largest single clip.

This cannot be the most efficient way of doing this though, surely? There has got to be more efficient way to dynamically load streaming audio. If I referenced the audioclip in the inspector, I imagine that would achieve this precisely, so it must be possible to achieve the same effect. It's just that I can't reference the clips as they're determined at runtime.

 

Anyone?

 

Thanks for the reply. It would definitely be nice if the default behaviour was changed, but I can see that it would be difficult to change without breaking anything. As I know well, API changes which break existing code are incredibly frustrating, so I can see that you guys need to tread lightly with this.

The two suggestions seem promising. I didn't really think of using asset bundles on a 1-to-1 basis with clips, but that does sound doable, since I can automate things with an editor script. The WWW object and the streaming assets folder sounds even better, since it would probably require no changes to the data I've already created. I'll give them both a go and see whether either one looks as though it suits my needs perfectly.

Thanks very much for the assistance.

 

I've tried implementing suggestion #1 but I can't set streaming to true. If I set streaming to true, the start of the sound is skipped and we begin a fraction of a second into the audio. If I set the streaming flag to false, it's fine. I am, of course, waiting for isReadyToPlay to be true. I've tried both OGG and WAV, but the results are always the same. I tested to see whether isReadyToPlay was returning true too soon, but it's irrelevant. No matter how long I wait, setting the Streaming flag to true always causes the audio to skip the first half a second or so of the clip. This seems to be a bug in Unity 3.5? Is it solved in 4? I see nothing in the release notes to suggest that it is.

 

I've also had time to try your other suggestion now, but I'm stumped. I don't see any way to access the content of the asset bundle before it's fully loaded. If I can't access the content of the bundle before it's fully loaded, I can't see any point in it. The whole point of streaming audio is to begin playing it before it's fully loaded.

 

I've submitted a bug report with the streaming audio clip issue. In response, I got an email telling me that the list of bugs is huge and it probably won't be fixed any time soon. If I'd like to pay $1,500 for a three month premium support subscription, they'll take a look at and make sure it's really a bug as a matter of priority. As far as I can see, I wouldn't actually stand any better chance of getting it fixed, despite that $1,500 investment.

Is this now official Unity policy? Call me odd, but most people would be ashamed to admit their bug list was so large it was unmanageable. Actually turning that into a pitch for why I should pay another $1,500 for you to determine whether one of your major features you've been advertising for age is actually completely broken took me by surprise.

I emailed back for clarification if I had understood correctly, but have received no reply. So I guess you do have at least a little shame.

EDIT: In case it matters, the case number is 522051.

 

I totally agree with Phila.

This is a huge, gargantuan problem in Unity, for any project other than a toy project.

Consider sound effects. Every game has this. In Resources/SFX you'll have say 40 folders (the different category of sound effects)

In each of those folders you'll have a random number of SFX for that category (whether 1, 5, 10 etc)

YES -- you can tediously make an AudioSource for every single one of these in the Editor (!)

And manually drag every one in the Editor

THEN - you can correctly use stream-from-disc

But just as Phila says if you try to simply avoid using the childish drag-in-editor situation ....... you cannot.

Loading of audio assets is broken because it doesn't understand stream-from-disk if you load them from a Resources folder.

I wish there was a fix for this.

it's a catastrophe.

 

Streaming audio from asset bundles is not possible, only loading the entire file out of the bundle into memory and decompressing first. The code for streaming .ogg files from the StreamingAssets folder is included a few posts below.

 

+1

It's a real shame.

Unfortunately Unity won't work with fmod for integration. They really should. This form of audio engine is primitive and becomes more primitive every day.

 

Many thanks for the suggestion, Games Foundry. I've attempted to implement it, and it seems to be working fine, except that there is a huge pause before the first WWW request returns true for isDone when I build to a standalone. From the editor, it works fine. I've timed each section of the process: Loading the data, asynchronously loading the bundle and then streaming the audio from the bundle.

From the editor, the times are something like

0.05 seconds
0.2 seconds
0.05 seconds

From the Standalone Windows build, the times are something like:

2.2 seconds
0.2 seconds
0.05 seconds

I added code to check the progress each frame. Progress is zero every single frame until the frame it loads, when it ticks to one. Which seems to suggest that loading is not really taking place for the first two seconds. Very odd.

Note that this ONLY seems to happen on the first request. So there is a two second gap before the first line of a conversation begins playing, but all subsequent lines load at normal speed. Am I doing something wrong? Do I have to do something to initialize use of the WWW class or something?

 

I believe we witnessed this too. As a quick fix I preload all the AssetBundles but I haven't gotten around to looking at the memory impact of doing this. I'm assuming that if all the audio files in the bundle are set to streaming, pre-loading the Asset Bundle should have minimal memory impact. But then you know what they say about assumptions. I've just been working on our AssetBundle code so it's a perfect time for me to run some tests and post the results.

UPDATE 1:
So it looks like using this method ends up loading the entire audio file into memory and decompressing, even though it's set as "stream from disc" and in ogg format within the bundle. Audio Memory jumps from a few hundred K to 29MB ( the size of a single uncompressed soundtrack file ) from an ogg file of approx 7MB. This explains the delay while the background thread loads and decompresses and entire file. Next I'll try www.audioClip to see if that works.

UPDATE 2:
So I tried not using asset bundles and instead adding a .ogg file marked as "stream from disc" to the StreamingAssets folder and used www.audioClip. Alas, it still loads the entire file into memory.

UPDATE 3:
Tried audioSource.clip = www.GetAudioClip ( false, true ); and that does stream, taking 130ms before actually playing and adding about 350K to memory. However, it does mean your .ogg file is available to everyone in the StreamingAssets folder. It doesn't look like it's possible to stream from asset bundles so I've updated the original post.

I can also confirm the bug in clip.length for streaming clips, and audioSource.isPlaying not reseting to false when the clip has finished playing. The temporary workaround solution is to pass in the lengths manually. Strangely, pausing the editor causes isPlaying to reset.

Code (csharp):

1.  
2.         if ( Application.platform == RuntimePlatform.IPhonePlayer )
3.             filePath = "file:///" + Application.dataPath + "/Raw/";
4.         else if ( Application.platform == RuntimePlatform.OSXPlayer )
5.             filePath = "file:///" + Application.dataPath + "/Data/StreamingAssets/";
6.         else
7.             filePath = "file:///" + Application.dataPath + "/StreamingAssets/";
8.  
9.         // The above can now be replaced with Application.streamingAssetsPath ( old code )
10.        
11.         filePath += oggFilename; // including .ogg file extension
12.  
13.         WWW www = new WWW ( filePath );
14.         yield return www;
15.            
16.         if ( www.error != null )
17.             Debug.Log ( www.error );
18.            
19.         audioSource.clip = www.GetAudioClip ( false, true );
20.        
21.                
22.         if ( audioSource.clip != null )
23.         {
24.             if ( audioSource.clip.isReadyToPlay )
25.             {
26.             ... continuation of original code
27.  
28.  

 

I was actually going to ask you whether loading assetbundles was actually streaming or not once I had it working properly, so thanks for answer that. Now that you've gone back to GetAudioClip(false, true) though, isn't that exactly what I posted above? When I use GetAudioClip(false, true) the beginning of the sound gets cut off.

 

We've encountered a streaming sound having the first bit of it being skipped in Unity 4.0 as you described. Happens during cut scene dialogue for us, when streaming the 30th sentence audio file in the dialogue ( we play a sentence, then clean up both the audio clip and the www about 0.5s after; profiler shows everything behaving as expected ). The bug happens every time at exactly the same spot, no idea why. In our case we are able to work around it by combining audio files.

 

I've received an email from support telling me that the bug has been confirmed and telling me that they are now passing it over to be fixed. I don't know if it's going to be a fix for 3.x, 4.x or both, but it's definitely good news.

 

Resources.Load on an AudioClip currently seems to always load/decompress the whole thing in to memory. The only work around I've found so far is to create a prefab for each AudioClip (which just contains a script which references the audio clip), and do the Resources.Load on this.
I've described it in this other thread, here !

 

Hi,

I don't quite understand when is the audio streamed and when not (provided I have selected StreamFromDisc as LoadType on my music).

It might not be entirely according to the thread title since I'm not using Resources folder now, but I guess you here will have the most info about it so I'll ask here, hope it's ok.

Let's say I have a folder Music, with some, well, music... Pretty big files, pretty long, StreamFromDisc on them. In scene, I have an 'Ambient' object (simple object with AudioSource on it). If I simply assign my music file to this AudioSource, will it stream the music without loading it whole into memory?

Second case - I want to change the ambient music. I have some trigger for example, with script attached. To this script, I assign another music from the inspector (to an AudioClip field). When I enter the trigger, I assign this AudioClip to the audio.clip in my 'Ambient' object. In this case, when is the music loaded (StreamFromDisc selected, of course)? When the scene loads, the trigger is entered (-> the clip is assigned to the audioSource), or is it streamed without previous loading as it should be?

Thanks, I'm really confused about this loading behavior since many people write about bad loading even with StreamFromDisc selected, I'm just not sure if it's always weired or only while using Resources.Load etc?

Edit: Btw I use Unity 3.5 Indie (+Android/iOS) if it matters

 

(i'm not a pro with unity and I don't know this for a fact, this is just my opinion)

when you load a scene, the objects within the scene that have audio clips attached to them will decide whether or not to load the audio to ram. if the setting is 'stream from disk', no loading will occur until the moment of playback.

So when you run a script (your trigger) to change the audio.clip field, no audio is loaded, still, until the very moment you wish to PLAY that audio file, then the audio is streamed.

as long as you work with gameobjects that have the playback setting set to streamfromdisk, you can change that audio.clip as many times as you wish, and nothing will pass through the ram / cpu until you push play on that object.

this is how it typically works - but again, i'm no Unity guru, and the unity audio engine is quite terrible.

 

@Rob Mills Audio

"Unfortunately Unity won't work with fmod for integration." <Forgot to Quote >

http://www.squaretangle.com/FMODUnity.html

 

It's such a shame that this issue has been ignored for so long. I've contacted support several times but the only responses I've had so far were to confirm that they haven't fixed it and an offer that I could pay $1,500 USD to have them confirm it as a bug. (But not fix it). They have subsequently confirmed it as a bug despite me not paying the ransom. I tried contacting Graham, since he seems to be the main man on these issues, but I've had no reply at all from him.

All hope is not lost, however. Unity were announcing on Facebook that the IssueTracker votes are used to prioritize which bugs they fix. This seems like a very unwise way of prioritizing your resources, but at least now I know why the issue was being ignored. I didn't even know I *could* vote for a bugfix, so it was on zero votes. A few kind Unity users on FB have already added some of their votes to it, so if you would like to get this issue resolved, please add all the votes you can spare here :

http://issuetracker.unity3d.com/issues/streaming-audio-is-broken

Maybe, with luck, we can get something done about this. Creating 10,000 prefabs doesn't seem like a solution to me, as kind as it was of wickedworx to share that solution.

 

Just ran into this very specific issue. I'm porting to Xbox right now so memory is tight!

 

I'm afraid I never found a satisfactory workaround for this problem. You can add each separate audio file to a bundle and stream the bundle into memory, but you can't begin playing it until the entire bundle is loaded. So it would be entirely worthless for long streams. I tried contacting support again and I've tried to contact Graham personally, but I just get ignored. To be fair, if the decision makers have decided that this isn't going to be fixed, there isn't anything they can say which wouldn't just inflame the situation.

Given that Unity are due to announce that Unity 5.0 will come with a vastly redesigned audio system tomorrow, my guess is that - at some point - we'll be told to pay for an upgrade to 5.0 if we want the features they've been advertising since 3.x.

 

I tried the WWW method mentioned on the previous page to load Resource files acynchronously, and I could play them fine. I couldn't figure out how to release the memory after I played them though, which makes it useless. I couldn't use Resources.UnloadAsset, it fails. Any ideas?

 

Does anyone know if this has been

fixed by Unity now that it is 2016?

 

Does anyone know how to get this www method to work for Android and iOS. It seems to work fine on PC but on nothing else. PArticularly how to setup the url for Android to be used with WWW for .ogg files storede in the streamingassets folder.

 

Each streaming AudioClip seems to allocate around 100KB (a big problem when you have thousands of clips to choose from) even if I leave the preload option unchecked. This seems broken. What am I missing?

 

You don't have to do it tediously, just use OnValidate() method in UNITY_EDITOR mode:

Code (CSharp):

1. [SerializeField]
2. private AudioClip[] _audioClips;
3.  
4. #if UNITY_EDITOR
5.  
6. private void OnValidate()
7. {
8.     _audioClips = Resources.LoadAll<AudioClip>(Path);
9. }
10.  
11. #endif

since _audioClips are loaded in the EDITOR mode they will be used in runtime the way as if they were created manually.