Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Microphone Network Test

Discussion in 'Multiplayer' started by rooch84, Feb 17, 2012.

  1. rooch84

    rooch84

    Joined:
    Feb 18, 2011
    Posts:
    25
    Morning all,

    I've been playing around with the new Microphone class is Unity 3.5 and managed to get it working in 3D over a network connection. If the player prefab is local then it starts receiving from the Microphone and send the samples each frame via RPC to the other clients. As it is, a new AudioClip is created and played each frame at the client end, which I'm not sure is the best way to do this.

    I'm posting for two reasons:
    1. To share the code if other people are trying to do the same.
    2. If anyone can suggest anything I can do to improve the sound quality. It's currently a bit crackly and unclear.

    Code (csharp):
    1. using UnityEngine;
    2. using System.Collections;
    3.  
    4. public class MicTest : MonoBehaviour {
    5.    
    6.     int lastSample;
    7.     AudioClip c;
    8.     int FREQUENCY = 44100;
    9.    
    10.     void Start () {
    11.         if (networkView.isMine) {
    12.             c = Microphone.Start(null, true, 100, FREQUENCY);
    13.             while(Microphone.GetPosition(null) < 0) {} // HACK from Riro
    14.         }
    15.     }
    16.    
    17.     void Update () {
    18.         if (networkView.isMine) {
    19.             int pos = Microphone.GetPosition(null);
    20.             int diff = pos-lastSample;
    21.             if (diff > 0) {
    22.                 float[] samples = new float[diff * c.channels];
    23.                 c.GetData(samples, lastSample);
    24.                 byte[] ba = ToByteArray(samples);
    25.                 networkView.RPC("Send", RPCMode.Others, ba, c.channels);
    26.             }
    27.             lastSample = pos;
    28.         }
    29.     }
    30.  
    31.     [RPC]
    32.     public void Send(byte[] ba, int chan) {
    33.         float[] f = ToFloatArray(ba);
    34.         audio.clip = AudioClip.Create("test", f.Length, chan, FREQUENCY,true,false);
    35.         audio.clip.SetData(f, 0);
    36.         if (!audio.isPlaying) audio.Play();
    37.     }
    38.    
    39.     public byte[] ToByteArray(float[] floatArray) {
    40.         int len = floatArray.Length * 4;
    41.         byte[] byteArray = new byte[len];
    42.         int pos = 0;
    43.         foreach (float f in floatArray) {
    44.                 byte[] data = System.BitConverter.GetBytes(f);
    45.                 System.Array.Copy(data, 0, byteArray, pos, 4);
    46.                 pos += 4;
    47.         }
    48.         return byteArray;
    49.     }
    50.    
    51.     public float[] ToFloatArray(byte[] byteArray) {
    52.         int len = byteArray.Length / 4;
    53.         float[] floatArray = new float[len];
    54.         for (int i = 0; i < byteArray.Length; i+=4) {
    55.                 floatArray[i/4] = System.BitConverter.ToSingle(byteArray, i);
    56.         }
    57.         return floatArray;
    58.     }
    59. }
    Cheers
    Rooch
     
  2. durak007

    durak007

    Joined:
    Feb 9, 2012
    Posts:
    10
    Many thnks
     
  3. friuns3

    friuns3

    Joined:
    Oct 30, 2009
    Posts:
    307
    thanks! but it sending 1 megabyte in 5 seconds, how i can compress data?
     
  4. carmine

    carmine

    Joined:
    Jan 4, 2012
    Posts:
    394
    Well.. for voice you don't need to record at 44100 you can get by with as low as 8000. I wrote a little loop to change it down to 8k

    Also.. floats are 32 bits. You could make a loop also to pack them into shorts (2 byte long integers) or even bytes.

    I had this working pretty well today but all of a sudden Microphone.devices is giving me nothing :(
     
  5. the_gnoblin

    the_gnoblin

    Joined:
    Jan 10, 2009
    Posts:
    722
    could you post your improved version of the script?

    thanks,
    Slav
     
  6. carmine

    carmine

    Joined:
    Jan 4, 2012
    Posts:
    394
    Slav,
    My script is not totally perfect... still working it out. But here is some code/pseudo code.

    If you record at 44,100 and you want to downsample to 22,050 you just take every other element in the array.

    So first.. fill up a buffer with what the Microphone recorded...

    float[] buffer = new float[audioClip.samples * audioClip.channels];
    audioClip.GetData(buffer, 0);


    Then get the length of our new buffer.

    float modifier = (float)audioClip.frequency / (float)newFrequency;

    float length = audioClip.length;
    int newNumberofSamples = Mathf.FloorToInt(length * newFrequency);
    int newBuffLen = newNumberofSamples * audioClip.channels;

    float[] newBuffer = new float[newBuffLen];

    Then fill up the new buffer...

    for (int i = 0; i < newBuffLen; i++)
    {
    newBuffer = buffer[Mathf.FloorToInt((float)i * modifier)];
    }

    Done... we have a downsampled buffer...

    You can make a new audio clip and tell it to play this new buffer at a lower frequency...

    -Carmine
     
  7. the_gnoblin

    the_gnoblin

    Joined:
    Jan 10, 2009
    Posts:
    722
    Thanks!

    I tried setting frequency to 8000 and it *seems* to work ok without any extra downsampling code.
     
  8. carmine

    carmine

    Joined:
    Jan 4, 2012
    Posts:
    394
    Be sure to check DeviceCaps... My USB headset can only do 44,100 .. meanwhile my built-in mic can do as low as 8000.
     
  9. santosh810666

    santosh810666

    Joined:
    Sep 18, 2012
    Posts:
    5

    i had apply to an seperate game object but when i am starting the server i am not able to test microphone on network view can u plz hlp me
     
  10. barthkc

    barthkc

    Joined:
    May 17, 2013
    Posts:
    2
    What do you mean by "if the player prefab is local?" Right now, I have a network connection, and I can see that an audio file called "test" is being created on the player's end. However, it is only a 20 ms clip and isn't playing any sound. The microphones and speakers are working on both computers. Any help is greatly appreciated!
     
  11. xuxuflyaway

    xuxuflyaway

    Joined:
    Jun 10, 2013
    Posts:
    5
    Hi, roach
    I'm trying your code, and got some problems. I streamed the audio data from server to client. But on the client side, it gives an error:
    "cannot set data on streamed sample".

    Do you no why this happened?

    Thank you
     
  12. Ralph1989

    Ralph1989

    Joined:
    Sep 1, 2013
    Posts:
    13
    If you want better audio quality best thing what you could do is to build in a delay. At the moment you are sending the sound files every frame. I did some test if you wait for like .1 seconds the quality is way better. hope it still helps someone.
     
  13. kinghack

    kinghack

    Joined:
    Nov 30, 2013
    Posts:
    54
    @Ralph1989

    Do you mean like?

    timeval += Time.deltaTime;

    if(timeval > 0.1f) {

    GetComponent<NetworkView>().RPC("Send", RPCMode.Others, ba, c.channels);

    timeval = 0f;

    }
     
  14. Ralph1989

    Ralph1989

    Joined:
    Sep 1, 2013
    Posts:
    13
    @kinghack

    Correct, this is how my update looks like.

    Code (CSharp):
    1.     void Update()
    2.     {
    3.         timer += Time.deltaTime;
    4.         if (networkView.isMine && timer > .1f)
    5.         {
    6.             timer = 0;
    7.             int pos = Microphone.GetPosition(null);
    8.             int diff = pos - lastSample;
    9.             if (diff > 0)
    10.             {
    11.                 float[] samples = new float[diff * c.channels];
    12.                 c.GetData(samples, lastSample);
    13.                 byte[] ba = ToByteArray(samples);
    14.                 networkView.RPC("Send", RPCMode.Others, ba, c.channels);
    15.             }
    16.             lastSample = pos;
    17.         }
    18.     }
     
  15. kinghack

    kinghack

    Joined:
    Nov 30, 2013
    Posts:
    54
    @Ralph1980 Thanks this works perfectly. Any ideas on how you could have a system that means the mic only records & plays when it gets an input like if(Input.GetKeyDown(KeyCode.V)) { ?
     
  16. ExDeaDguY

    ExDeaDguY

    Joined:
    Aug 25, 2009
    Posts:
    503
    I haven't tested any of this or even using anything like this in any of my games.. but if this is to be used as network communication.. just a thought.. to keep size over seconds down.. continuously send recorded stream every half second or second or something like that instead of recording a 10 second long recording then sending the whole thing.. so while record.. if recording is longer than 0.5f, end recording and send, then start a new recording and continue recording. instead of 1 stream being megabytes large you'd have several (20) chunks sent over a 10 second time frame each at a 10th of the original size.

    On the receiving end you could either.. capture all of the recording and play it back when its complete or play the chunks individually as a instant voice chat.

    No idea how this will work, just an idea as a dev :p
     
  17. colorcraft

    colorcraft

    Joined:
    Jun 16, 2013
    Posts:
    41
    For some reason other people can hear me but I can hear them speak with this script. Would this be due to Unity5?
     
  18. Adeola

    Adeola

    Joined:
    Feb 6, 2015
    Posts:
    1
    Using FixedUpdate (instead of Update) with a time interval of 0.5 (as suggested here) improved the audio quality significantly, although it introduced some (manageable) latency on Unity5.
     
  19. zarex7

    zarex7

    Joined:
    Jan 25, 2016
    Posts:
    21
    btw, error CS0619: 'RPC' is obsolete: 'The legacy networking system has been removed in Unity 2018.2. Use Unity Multiplayer and NetworkIdentity instead.'

    so, about "if(Input.GetKeyDown(KeyCode.V)) {" - just add that "if" to update() to start play when press V down and stop play when press V up