Microphone Network Test

Morning all,

I've been playing around with the new Microphone class is Unity 3.5 and managed to get it working in 3D over a network connection. If the player prefab is local then it starts receiving from the Microphone and send the samples each frame via RPC to the other clients. As it is, a new AudioClip is created and played each frame at the client end, which I'm not sure is the best way to do this.

I'm posting for two reasons:
1. To share the code if other people are trying to do the same.
2. If anyone can suggest anything I can do to improve the sound quality. It's currently a bit crackly and unclear.

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public class MicTest : MonoBehaviour {
5.    
6.     int lastSample;
7.     AudioClip c;
8.     int FREQUENCY = 44100;
9.    
10.     void Start () {
11.         if (networkView.isMine) {
12.             c = Microphone.Start(null, true, 100, FREQUENCY);
13.             while(Microphone.GetPosition(null) < 0) {} // HACK from Riro
14.         }
15.     }
16.    
17.     void Update () {
18.         if (networkView.isMine) {
19.             int pos = Microphone.GetPosition(null);
20.             int diff = pos-lastSample;
21.             if (diff > 0) {
22.                 float[] samples = new float[diff * c.channels];
23.                 c.GetData(samples, lastSample);
24.                 byte[] ba = ToByteArray(samples);
25.                 networkView.RPC("Send", RPCMode.Others, ba, c.channels);
26.             }
27.             lastSample = pos;
28.         }
29.     }
30.  
31.     [RPC]
32.     public void Send(byte[] ba, int chan) {
33.         float[] f = ToFloatArray(ba);
34.         audio.clip = AudioClip.Create("test", f.Length, chan, FREQUENCY,true,false);
35.         audio.clip.SetData(f, 0);
36.         if (!audio.isPlaying) audio.Play();
37.     }
38.    
39.     public byte[] ToByteArray(float[] floatArray) {
40.         int len = floatArray.Length * 4;
41.         byte[] byteArray = new byte[len];
42.         int pos = 0;
43.         foreach (float f in floatArray) {
44.                 byte[] data = System.BitConverter.GetBytes(f);
45.                 System.Array.Copy(data, 0, byteArray, pos, 4);
46.                 pos += 4;
47.         }
48.         return byteArray;
49.     }
50.    
51.     public float[] ToFloatArray(byte[] byteArray) {
52.         int len = byteArray.Length / 4;
53.         float[] floatArray = new float[len];
54.         for (int i = 0; i < byteArray.Length; i+=4) {
55.                 floatArray[i/4] = System.BitConverter.ToSingle(byteArray, i);
56.         }
57.         return floatArray;
58.     }
59. }

Cheers
Rooch

 

Many thnks

 

thanks! but it sending 1 megabyte in 5 seconds, how i can compress data?

 

Well.. for voice you don't need to record at 44100 you can get by with as low as 8000. I wrote a little loop to change it down to 8k

Also.. floats are 32 bits. You could make a loop also to pack them into shorts (2 byte long integers) or even bytes.

I had this working pretty well today but all of a sudden Microphone.devices is giving me nothing

 

could you post your improved version of the script?

thanks,
Slav

 

Slav,
My script is not totally perfect... still working it out. But here is some code/pseudo code.

If you record at 44,100 and you want to downsample to 22,050 you just take every other element in the array.

So first.. fill up a buffer with what the Microphone recorded...

float[] buffer = new float[audioClip.samples * audioClip.channels];
audioClip.GetData(buffer, 0);


Then get the length of our new buffer.

float modifier = (float)audioClip.frequency / (float)newFrequency;

float length = audioClip.length;
int newNumberofSamples = Mathf.FloorToInt(length * newFrequency);
int newBuffLen = newNumberofSamples * audioClip.channels;

float[] newBuffer = new float[newBuffLen];

Then fill up the new buffer...

for (int i = 0; i < newBuffLen; i++)
{
newBuffer = buffer[Mathf.FloorToInt((float)i * modifier)];
}

Done... we have a downsampled buffer...

You can make a new audio clip and tell it to play this new buffer at a lower frequency...

-Carmine

 

Thanks!

I tried setting frequency to 8000 and it *seems* to work ok without any extra downsampling code.

 

Be sure to check DeviceCaps... My USB headset can only do 44,100 .. meanwhile my built-in mic can do as low as 8000.

 

i had apply to an seperate game object but when i am starting the server i am not able to test microphone on network view can u plz hlp me

 

What do you mean by "if the player prefab is local?" Right now, I have a network connection, and I can see that an audio file called "test" is being created on the player's end. However, it is only a 20 ms clip and isn't playing any sound. The microphones and speakers are working on both computers. Any help is greatly appreciated!

 

Hi, roach
I'm trying your code, and got some problems. I streamed the audio data from server to client. But on the client side, it gives an error:
"cannot set data on streamed sample".

Do you no why this happened?

Thank you

 

If you want better audio quality best thing what you could do is to build in a delay. At the moment you are sending the sound files every frame. I did some test if you wait for like .1 seconds the quality is way better. hope it still helps someone.

 

@Ralph1989

Do you mean like?

timeval += Time.deltaTime;

if(timeval > 0.1f) {

GetComponent<NetworkView>().RPC("Send", RPCMode.Others, ba, c.channels);

timeval = 0f;

}

 

@kinghack

Correct, this is how my update looks like.

Code (CSharp):

1.     void Update()
2.     {
3.         timer += Time.deltaTime;
4.         if (networkView.isMine && timer > .1f)
5.         {
6.             timer = 0;
7.             int pos = Microphone.GetPosition(null);
8.             int diff = pos - lastSample;
9.             if (diff > 0)
10.             {
11.                 float[] samples = new float[diff * c.channels];
12.                 c.GetData(samples, lastSample);
13.                 byte[] ba = ToByteArray(samples);
14.                 networkView.RPC("Send", RPCMode.Others, ba, c.channels);
15.             }
16.             lastSample = pos;
17.         }
18.     }

 

@Ralph1980 Thanks this works perfectly. Any ideas on how you could have a system that means the mic only records & plays when it gets an input like if(Input.GetKeyDown(KeyCode.V)) { ?

 

I haven't tested any of this or even using anything like this in any of my games.. but if this is to be used as network communication.. just a thought.. to keep size over seconds down.. continuously send recorded stream every half second or second or something like that instead of recording a 10 second long recording then sending the whole thing.. so while record.. if recording is longer than 0.5f, end recording and send, then start a new recording and continue recording. instead of 1 stream being megabytes large you'd have several (20) chunks sent over a 10 second time frame each at a 10th of the original size.

On the receiving end you could either.. capture all of the recording and play it back when its complete or play the chunks individually as a instant voice chat.

No idea how this will work, just an idea as a dev

 

For some reason other people can hear me but I can hear them speak with this script. Would this be due to Unity5?

 

Using FixedUpdate (instead of Update) with a time interval of 0.5 (as suggested here) improved the audio quality significantly, although it introduced some (manageable) latency on Unity5.

 

btw, error CS0619: 'RPC' is obsolete: 'The legacy networking system has been removed in Unity 2018.2. Use Unity Multiplayer and NetworkIdentity instead.'

so, about "if(Input.GetKeyDown(KeyCode.V)) {" - just add that "if" to update() to start play when press V down and stop play when press V up