Importing audio files at runtime

Hello! I made a topic about this before, but after weeks of bumping and looking for the answer I came up empty-handed. I hope this does not happen again because this is getting really important.

So my team and I are working on a game like AudioSurf which reads audio files and does things according to the audio output. Everything's good except that it can only read files that have been imported in the editor before. We need to enable the players to pick their own audio files from their computers and play them.

We had several approaches for this but none of them worked. I am aware that Unity can only read ogg files but even with ogg files we failed to make this feature.

I am not going to post code this time because I would like to start from the zero and collect more information about how such mechanics would work.

Can anybody point me in the right direction ? Any help will be really, really appreciated!

Cheers!

 

I'll do anything to be able to import audio at runtime so there's no problem for me to use third-party libraries. However I have no clue what to search for. Can you point me what exactly to search for ?

 

http://unity3d.com/support/documentation/ScriptReference/WWW-oggVorbis.html
http://unity3d.com/support/documentation/ScriptReference/WWW.GetAudioClip

 

Oh no, this wouldn't work. I've tried it many times and I never got it to work. I've tried both the oggVorbis variable and the GetAudioClip function. Here's my previous topic: http://forum.unity3d.com/threads/134361-Does-not-load-the-audio-clip.

Thanks for the answers so far!

 

Well we need the players to be able to load the most common files - mp3, wav and ogg (at least to my knowledge).

I've been looking up the .NET framework class library trying to find something audio related but I couldn't find such thing. I don't have a lot of experience in dealing with external libraries and finding ones yet so please excuse my ignorance. My problem is that I don't really know what to look for so I will greatly appreciate it if you point me into the right direction!

Thanks for the help so far!

 

Oh sweet! Thank you (how did you find this)!

I'm not at work so currently I'm not able to start working on it, but just to make things clear:
I open an audio file, use one of the decoder classes (don't know which one because I haven't found any documentation yet) in the decoder folder to read the file and turn it into an array of floats, then I use AudioClip.SetData() to import the information into an audio clip and play it, right ? If it's like that, that should be fairly easy to achieve. I hope it doesn't take too long for the process to read the file and convert it to an AudioClip.

 

I'll stick to the first method for now. In our case we need to have an array of audio clips which need to be accessed fast so maybe I'll just put a loading screen at the begining while the whole process takes place.

Anyway, I starded to import the library into Unity but I ran into an issue. The console said I was missing assembly references to some namespaces (system.data, system.drawing, system.windows) so I started to search for those files in the net and found System.Data and System.Drawing and imported them into Unity. Then I found System.Windows.dll in Silverlight's directory on the computer and also imported it.All the assembly references disappeared but left me with another error:

I've never seen such an error before. I searched for it and I couldn't find a solution that works. It seems to me as something's incompatible. I switched from .NET Framework 2.0 Subset to .NET Framework 2.0 and still this error doesn't budge.

Have you seen this before and do you know what causes it ?

 

With the use of WWW class u can read external audio (I tried .wav .ogg) file in unity3d. Its works fine. For playing external audio file u dont need to build ur project again. Just replace ur audio file at ur datapath.

 

So you're saying that either one of the dll files I've imported is built agains .NET 4.0 ? How can I know which one is it and where can I find one built against 2.0 ? If I've understood you correct, I think it might be the System.Windows dll which I got from my computer (I have .NET 4.0 installed) however I couldn't find this speciffic library anywhere unlike the other two.

Thanks for joining Ram Thakkar
I gave a link to my previous topic about this above:
http://forum.unity3d.com/threads/134361-Does-not-load-the-audio-clip.. I couldn't get this to work after weeks of trying...it just didn't load the audio clip into the audio source. After lots of debugging I couldn't find the reason why it wasn't working. Maybe you can tell me what I did wrong ?

 

Hello Dark protocol,

With the below code i successfully load external audio into my unity exe:

var www = new WWW ("file://" + Application.dataPath.Substring (0, Application.dataPath.LastIndexOf ("/")) + "/result.wav");

AudioClip myAudioClip= www.audioClip;
while (!myAudioClip.isReadyToPlay)
yield return www;
gameObject.GetComponent<AudioSource> ().audio.clip = myAudioClip;

audio.Play ();

 

you just forgot to write "file://" before ur Application.datapath or may be u dont know about this.

 

Yes, I tried your method in Javascript and it worked for me. It's a bummer that I can't stream several files, put them in an array, change the scene and still have them as playable audio clips (because that's what we need in the game) but I can keep an array with file links and as soon a the new scene starts I can start loading them one by one while playing the first loaded one.

Well okay, that's awesome so far and many thanks for the help, Ram Thakkar

I'm currently having difficulties finding out how to use the decoder classes from mp3sharp. I couldn't find any documentation on it so I guess I'll just have to read through all the classes and find out how they work.

 

But in mycase, i played multiple audio files from external source with same method and its work fine for me.

 

It was my bad. There was a problem with something else that prevented the whole thing from working. It works as a charm now after I fxed it.
I'm having another issue now. I need the imported songs to play in 2D, not in 3D so I wrote something like this:

Code (csharp):

1.  
2. var samples = new float[myAudioClip.samples * myAudioClip.channels];
3.                     myAudioClip.GetData(samples, 0);
4.                 var newClip : AudioClip = AudioClip.Create("song", myAudioClip.samples, myAudioClip.channels, myAudioClip.frequency, false, true);
5.                     newClip.SetData(samples, 0);
6.                     songList.Add(myAudioClip);
7.  

This is supposed to create a new audio clip which is 2D, extract the data from the imported one and put it into the new one. However, after I play the new clip, I don't hear anything. The audio clip is playing but it has no sound information in itself. I couldn't find other topics for this so I need to ask for this too. What could be the cause of this ?

Thanks both of you for the help. It is much appreciated!

 

I got the error. The console says "Cannot get data from streamed sample". So does that mean that there isn't any way to get the samples? That would be bad

 

Search in this forum theme http://duc.avid.com/forumdisplay.php?f=91, i remember someone post the solution for the similar problem ...

 

hi, i copy paste this code into javascript file and it don't work :/
could you help me please?

i need to access to a folder and open the wav file

 

Hey Dark,
I'm wondering if you managed to use the GetData function on mp3 in order to make some postprocess in your Audiosurf-like game!
When I do, my float[] is full of 0s... Any idea on how to fix it ?

Btw, if it still helps anyone : the way to get a non-stream, 2d file is with the function .GetAudioClip on your WWW var.

 

Hi,

Sorry to have to confirm that indeed, GetData does not work with streamed clips.
In other words, if you get an audioclip through WWW.GetAudioClip, you canonly play the file, not access the samples.

The super slow solution I can think of is to decode mp3 to wav, then convert the wav to floats, then load the samples into an empty AudioClip. This can be done in blocks too, and the AudioClip can be quite small and act as a ring buffer.

 

Hello, guys, I did not manage to get the GetData function to work, neither import mp3 files with Unity's built-in methods. I moved to Fmod Ex instead. Fmod requires a license to be paid, as well as the mp3 file format, but that is for commercial games.

 

Tried this, it wouldn't work. but AudioClip myAudioClip= www.GetAudioClip(false,true); should work.

 

I realize this is may be late but heres what i did, i've tested it and it works with .ogg, .wav, and (most importantly) .mp3

using UnityEngine;
using System.Collections;
using System.Windows.Forms;
using System.IO;
using NAudio;
using NAudio.Wave;

public class FFTAnalyzer : MonoBehaviour {

public FFTWindow FFTMode;
public int sampleCount;
public Transform[] cubies;
float[] samples;
public AudioSource source;
bool isReady = false;

IEnumerator Start () {
samples = new float[sampleCount];
OpenFileDialog file = new OpenFileDialog();

file.Filter = "Ogg Vorbis files (.ogg)|*.ogg|Wave files (.wav)|*.wav|Mp3 files (.mp3)|*.mp3";
file.FilterIndex = 3;
file.Title = "Song Selection";
file.ShowDialog();

char[] chars = new char[3] {file.FileName[file.FileName.Length - 3], file.FileName[file.FileName.Length - 2], file.FileName[file.FileName.Length - 1]};

string ext = new string(chars);

if(file.FileName[file.FileName.Length - 3] == "mp3"[0])
{
Directory.CreateDirectory(System.IO.Path.GetTempPath() + @"\MusicalDefense");
Mp3ToWav(file.FileName, System.IO.Path.GetTempPath() + @"\MusicalDefense\currentsong.wav");
ext = "wav";
}
else
{
Directory.CreateDirectory(System.IO.Path.GetTempPath() + @"\MusicalDefense");
File.WriteAllBytes(System.IO.Path.GetTempPath() + @"\MusicalDefense\currentsong." + ext, File.ReadAllBytes(file.FileName));
}

WWW www = new WWW("file://" + System.IO.Path.GetTempPath() + @"\MusicalDefense\currentsong." + ext);
AudioClip a = www.audioClip;

while(!a.isReadyToPlay)
{
Debug.Log("still in loop");
yield return www;
}


source.clip = a;
source.Play();
isReady = true;
}

public static void Mp3ToWav(string mp3File, string outputFile)
{
using (Mp3FileReader reader = new Mp3FileReader(mp3File))
{
WaveFileWriter.CreateWaveFile(outputFile, reader);
}
}

this uses the Naudio library which is free at http://naudio.codeplex.com/releases/view/114106

it gets a file (you just need to get the file path, i used the open file dialog but you can use any method you prefer) and checks the extension. If its an mp3 it converts it to a .wav. then it writes the file to the temporary directory of the pc and retrieves it via the WWW method. its a bit of a round and about way of doing things but works fairly fast nonetheless, about 2 seconds or so on my intel 3770k

 

Hey! I appreciate your response! I've been stuck with this issue for ages. For some reason when I try importing the dlls, I get the following error message, is there something you did or a specific download you used?

Internal compiler error. See the console log for more information. output was:
Unhandled Exception: System.Reflection.ReflectionTypeLoadException: The classes in the module cannot be loaded.
at (wrapper managed-to-native) System.Reflection.Assembly:GetTypes (bool)
at System.Reflection.Assembly.GetTypes () [0x00000] in <filename unknown>:0
at Mono.CSharp.RootNamespace.ComputeNamespaces (System.Reflection.Assembly assembly, System.Type extensionType) [0x00000] in <filename unknown>:0
at Mono.CSharp.RootNamespace.ComputeNamespace (Mono.CSharp.CompilerContext ctx, System.Type extensionType) [0x00000] in <filename unknown>:0
at Mono.CSharp.GlobalRootNamespace.ComputeNamespaces (Mono.CSharp.CompilerContext ctx) [0x00000] in <filename unknown>:0
at Mono.CSharp.Driver.LoadReferences () [0x00000] in <filename unknown>:0
at Mono.CSharp.Driver.Compile () [0x00000] in <filename unknown>:0
at Mono.CSharp.Driver.Main (System.String[] args) [0x00000] in <filename unknown>:0

 

Hey mdbzdmvp, you need to change the optimization settings for API compatibility to .NET 2.0 instead of .NET 2.0 Subset under player settings.
Check out this tutorial that shows it http://denis-potapenko.blogspot.com/2013/04/task-6-loading-mp3-audio-via-www-class.html

 

Just released asset that allows you to load AudioClip from waw, mp3, aiff and ogg files at runtime:
http://u3d.as/content/neitri-industries/runtime-audio-clip-loader

 

How fast is it on android? Unitys WWW class can handle mp3, but it takes 5 seconds to get the audio clip.
Is it possible to get the length of the audio file faster?

 

thank's i will look later
at this time i request information about frequenzy
when get AudioClip.GetData and saveWav
and
AudioClip result = AudioClip.Create("Combine",(int)( bluff_lenght_sound ), 2, 44100, false, false);

sound is sometime make like pitch

so for exemple if i got
float[] buffer = new float[(clips.samples) * clips.channels];
clips.GetData(buffer, 0);

and i want fix speed anyone can help for say me where scearch

 

It's an old thread but I used your code and when I build the game, it crashes (when IEnumerator Start called). Any clue?

 

for my part i use only wav

 

Use DebugView.exe and watch Unity's log messages. My exception was blank but in the log it said FMOD said the file was an invalid format. Many times when you can't find the error in the editor log itself. You can find more or different information by using debugview. Also this program can trap debug messages even from a built version or messages coming from any program on your PC.

 

the best way that i achieved to it is this...

Code (CSharp):

1. public class PlayList : MonoBehaviour {
2.  
3. string path;
4. List<AudioClip> Cliplist;
5.  
6. List<string> audioname;
7.  
8. void Start ()
9. {
10.  
11.  
12. path = Path.Combine(Application.persistentDataPath, "Audios");
13.  
14.  
15.  
16. if (Directory.Exists(path))
17. {
18. DirectoryInfo info = new DirectoryInfo(path);
19.  
20. foreach (FileInfo item in info.GetFiles("*.wav"))
21. {
22.  
23. audioname.Add(item.Name);
24.  
25.  
26. }
27.  
28. }
29. StartCoroutine(LoadAudioFile());
30. }
31.  
32. IEnumerator LoadAudioFile()
33. {
34. for (int i = 0; i <audioname.Count; i++)
35. {
36. UnityWebRequest AudioFiles = UnityWebRequestMultimedia.GetAudioClip(path + string.Format("{0}", audioname[i]),AudioType.WAV);
37. yield return AudioFiles.SendWebRequest();
38. if(AudioFiles.isNetworkError)
39. {
40. Debug.Log(AudioFiles.error);
41. Debug.Log(path + string.Format("{0}", audioname[i]));
42. }
43. else
44. {
45. AudioClip clip = DownloadHandlerAudioClip.GetContent(AudioFiles);
46. clip.name = audioname[i];
47. Cliplist.Add(clip);
48. Debug.Log(path + string.Format("{0}", audioname[i]));
49. }
50. }
51.  
52. }
53. }