Showcase [EXAMPLE] VoiceChat with UNET and Steamworks

Code (CSharp):

1.  
2. using Steamworks;
3. using UnityEngine;
4. using UnityEngine.Networking;
5.  
6. public class VoiceChat : NetworkBehaviour
7. {
8.     public LayerMask PlayerMask;
9.     public AudioSource audioSource;
10.  
11.     void Update ()
12.     {
13.         if (isLocalPlayer && Input.GetKeyUp(KeyCode.V))
14.             SteamUser.StartVoiceRecording();
15.         else if (isLocalPlayer && Input.GetKeyDown(KeyCode.V))
16.             SteamUser.StopVoiceRecording();
17.  
18.  
19.         if(isLocalPlayer)
20.         {
21.             uint Compressed;
22.             uint Uncompressed;
23.             EVoiceResult ret = SteamUser.GetAvailableVoice(out Compressed, out Uncompressed, 0);
24.             if(ret == EVoiceResult.k_EVoiceResultOK && Compressed > 1024)
25.             {
26.                 Debug.Log(Compressed);
27.                 byte[] DestBuffer = new byte[1024];
28.                 uint BytesWritten;
29.                 uint uncompressedBytesWritten;
30.                 ret = SteamUser.GetVoice(true, DestBuffer, 1024, out BytesWritten, false, new byte[0], 0, out uncompressedBytesWritten, 22050);
31.                 if(ret == EVoiceResult.k_EVoiceResultOK && BytesWritten > 0)
32.                 {
33.                     Cmd_SendData(DestBuffer, BytesWritten);
34.                 }
35.             }
36.         }
37.     }
38.  
39.     [Command (channel = 2)]
40.     void Cmd_SendData(byte[] data, uint size)
41.     {
42.         Debug.Log("Command");
43.         Collider[] cols = Physics.OverlapSphere(transform.position, 50, PlayerMask);
44.         for (int i = 0; i < cols.Length; i++)
45.         {
46.             if(cols[i].GetComponent<NetworkIdentity>())
47.             {
48.                 Target_PlaySound(cols[i].GetComponent<NetworkIdentity>().connectionToClient, data, size);
49.             }
50.         }
51.     }
52.  
53.     [TargetRpc (channel = 2)]
54.     void Target_PlaySound(NetworkConnection connection, byte[] DestBuffer, uint BytesWritten)
55.     {
56.         Debug.Log("TARGET");
57.         byte[] DestBuffer2 = new byte[22050 * 2];
58.         uint BytesWritten2;
59.         EVoiceResult ret = SteamUser.DecompressVoice(DestBuffer, BytesWritten, DestBuffer2, (uint)DestBuffer2.Length, out BytesWritten2, 22050);
60.         if(ret == EVoiceResult.k_EVoiceResultOK && BytesWritten2 > 0)
61.         {
62.             audioSource.clip = AudioClip.Create(UnityEngine.Random.Range(100,1000000).ToString(), 22050, 1, 22050, false);
63.  
64.             float[] test = new float[22050];
65.             for (int i = 0; i < test.Length; ++i)
66.             {
67.                 test[i] = (short)(DestBuffer2[i * 2] | DestBuffer2[i * 2 + 1] << 8) / 32768.0f;
68.             }
69.             audioSource.clip.SetData(test, 0);
70.             audioSource.Play();
71.         }
72.     }
73. }
74.  
75.  

This code will only send the voice data to close players. Might not be what you desire. If not, you can simply change that bit out.
It's NOT flawless. But it might help you get started.

This was created using the following documentation:
https://partner.steamgames.com/doc/features/voice

 

hey thanks for this! in what ways is it flawed?

i hooked up something very similar with steamworks a few months ago, but the playback was a little garbled. i haven't had a chance to revisit it. i had a pretty hard time ensuring that it was writing the data to the audio buffer in the exact correct spot.

 

For some reason there is a split second of silence making a stutter between the audioclips. Have been trying to remove it. Some sort of backlog of AudioClips was my inital idea. But it seems that simply switching clip has a stutter and I am yet to figure out a smooth way to do it. Try it if you like. Would love to know how/if you get it working

 

i think the approach i was taking was to have a single AudioClip with a bigger buffer than your typical audio sample that is sent down the wire. like say 2 seconds of worth of audio. from there -

when you start collecting samples, you paste them in the correct spot into the buffer (being careful when pasting a sample near the end of the buffer of course). after a small amount of time collecting samples (to ensure the playhead always has some audio to play), start looping playback. you also need to ensure that you're filling any 'holes' with silence so that you don't accidentally hear old data from prevoius playbacks. i'd love to figure out an easier way because this approach was pretty complicated, i thought!

 

That might be the answer. I'll probably try get that working when I have time.
So basically you will never reach the end of the clip. You could have a clip x times the size of one clip. Then fill it, start playing. And once the first part is played. You could replace that with a new one. So you allways have a clip with a few clips in.

 

i'll probably be revisiting this within a week.

one other thing that's worth noting here - if you're just using unet out of the box, i'm not sure that it's feasible to send voice data like this. passing it through the relay server might blow out your bandwidth usage.

i'm personally sending this data via steamworks' p2p layer to avoid all that mess

 

Glad you pointed that out!
Personally I am running dedicated only without relay. So for me it's fine. But feel free to let me know your results if you get back to it

 

hey i've been looking at this voicechat stuff again today and i got it sounding much better thanks to a tip on the steamworks forum. it's not perfect, but it sounds way better than my other approach.

i tried to clean up the code a bit into something that's easy to follow. it probably won't compile for you - this is more to demonstrate the basic playback approach.

you're going to need to connect some dots to get this working on your end though. notes to follow...

Code (CSharp):

1. public class SteamVoiceChatPeer : MonoBehaviour
2. {
3.     public SortedList<ulong, VoiceChat.VoiceChatPacket> packetsToPlay = new SortedList<ulong, VoiceChatPacket>();
4.  
5.     private AudioSource m_audioSource;
6.  
7.     int position = 0;
8.  
9.     VoiceChat.VoiceChatPacket currentPacket;
10.     int currentPacketSampleIndex = 0;
11.  
12.     void Start()
13.     {
14.         int size = VoiceChat.VoiceChatSettings.Instance.Frequency * 10;// bigger size seems to help with popping a little, but i might be making that up.
15.  
16.         m_audioSource = GetComponent<AudioSource>();
17.         m_audioSource.loop = true;
18.         m_audioSource.clip = AudioClip.Create ("VoiceChat", size, 1, VoiceChatSettings.Instance.Frequency, true, OnAudioRead, OnAudioSetPosition);
19.         m_audioSource.Play ();
20.     }
21.  
22.     void OnAudioRead(float[] data)
23.     {
24.         // fresh start?
25.         if (currentPacket == null) {
26.             currentPacket = NextPacket ();
27.             currentPacketSampleIndex = 0;
28.             if (currentPacket != null) {
29.                 packetsToPlay.Remove (currentPacket.PacketId);
30.             }
31.         }
32.  
33.         int count = 0;
34.         while (count < data.Length) {
35.             // copy the right data over.
36.             float sample = 0;
37.             if (currentPacket != null) {
38.                 sample = currentPacket.DecodedData [currentPacketSampleIndex];
39.  
40.                 currentPacketSampleIndex++;
41.                 if (currentPacketSampleIndex >= currentPacket.DecodedData.Length) {
42.                     currentPacket = NextPacket ();
43.                     currentPacketSampleIndex = 0;
44.  
45.                     if (currentPacket != null) {
46.                         packetsToPlay.Remove (currentPacket.PacketId);
47.                     }
48.                 }
49.             }
50.             data [count] = sample;
51.             position++;
52.             count++;
53.         }
54.     }
55.  
56.     void OnAudioSetPosition(int newPosition)
57.     {
58.         position = newPosition;
59.     }
60.  
61.     private VoiceChat.VoiceChatPacket NextPacket()
62.     {
63.         if (packetsToPlay.Count > 0) {
64.             var pair = packetsToPlay.First ();
65.             VoiceChat.VoiceChatPacket packet = pair.Value;
66.             if (packet != null) {
67.                 return packet;
68.             }
69.         }
70.         return null;
71.     }
72.      
73.     public void OnNewSample(VoiceChatPacket newPacket)
74.     {
75.         if (packetsToPlay.ContainsKey (newPacket.PacketId)) {
76.             Debug.LogError ("already have packet " + newPacket.PacketId + ". abort");
77.             return;
78.         }
79.  
80.         // convert immediately.
81.         newPacket.Decode ();
82.  
83.         // throw out silence for now just to see how it acts
84.         if (!newPacket.IsSilence) {
85.             packetsToPlay.Add (newPacket.PacketId, newPacket);
86.         }
87.     }
88. }
89.  
90. // this class was originally part of another voice chat library i found on github. i've since added to and butchered partsof it. you probably only need:
91. //  length, data, decodedData, and packetId for this example to work. and the Decode function for converting steam's decompressed data into a unity friendly format
92. public class VoiceChatPacket
93. {
94.     public VoiceChatCompression Compression;
95.     public int Length;
96.     public byte[] Data;// <<<<< this is your steam uncompressed voice
97.     public float[] DecodedData = null;
98.     public int NetworkId;
99.     public ulong PacketId;
100.     public double Timestamp;
101.     public int LengthInSamples;
102.  
103.     public bool IsSilence = false;
104.  
105.     public void Decode()
106.     {
107.         DecodedData = new float[Length / 2];// todo :: pool this array?
108.         for (int i = 0; i < DecodedData.Length; i++) {
109.             float value = (float) System.BitConverter.ToInt16 (Data, i * 2);
110.             DecodedData [i] = value / (float)short.MaxValue;
111.         }
112.         LengthInSamples = DecodedData.Length;
113.     }
114. }

so the main change was switching my AudioClip's constructor to be streaming, and using the PCM reader callback to write my audio data into the buffer object it gives you.

note that i left out all the stuff about capturing & decompressing voices, as that happens in another class in my game. the only thing you need to know about that class is that it decompresses the voice into the VoiceChatPacket's Data array, then passes it to the corresponding SteamVoiceChatPeer's OnNewSample function.

also note that this class uses a SortedList to attempt to keep the packets in order. i do this because i'm sending the data over an unreliable channel. i see a bug in the code now too where it could insert old packets to the front of the list - totally not accounting for that now. might cause problems if they come in really late, like after they should have been played.

 

Try to send over Unreliable Sequenced. Probably the way to go with audio. But thanks alot for this. I looked into Streaming. But I could not really wrap my head around it. I'll have a look and see how it goes and potentially update the original thread.

EDIT: Just a question, why are you sending all the MetaData with every packet? Won't that just eat bandwith for no reason?

 

well, i'm using the steamworks p2p stuff to send data. they recommend sending voice chats on the "k_EP2PSendUnreliableNoDelay" channel. so that's what i'm doing i don't think it even has an Unreliable Sequenced channel

 

Well, I will have to take some day when I'm off work to look at what you have provided with the Stream. You have structured it a bit differently. I only send the actual bytes of data.

But are you actually doing a client buffer? And if so how does it build up? Delayed playback after first packet?
And if not, mind elaborating on how you actually went ahead and proceeded with it. I appreciate the code example. But I much prefer to understand what I am writing haha.

 

sorry, i didn't really want to chop the code up & make it any more understandable than i had to it doesn't really help that this is like my 4th iteration at getting this to work, so the code has grown some pretty ugly hairs over time.

i had a client buffer in my old experiments, but i took it out for this test. the lack of a buffer might also account for some of the random pops i'm hearing, so thanks for reminding me! it would be pretty easy to not actually start writing in OnAudioRead until i've collected a handful of packets.

are you referring to the IsSilence stuff in my code? that was an old experiment that i should weed out. from back when i was copying/pasting stuff into the audio buffer. at the time, i had a hard time dealing with microphone silence, so i started sending a message even when there was no audio data. i added that filter that throws out silence just today - it seems sending silence messages is not necessary at all, though maybe it will be. idk

 

So you are creating a audioclip with a read and write stream method. But what do you actually do with those when you recieve the data from the server? Say I recieve an array of bytes from the server. I then decompress that. What would you do with that data then? Currently I'm just creating a new AudioClip every time I get new data and playing that.

 

i only create one AudioClip per player that you're talking to, and it's just reused continuously throughout the session

the code example here helped me figure out this streaming stuff - https://docs.unity3d.com/ScriptReference/AudioClip.Create.html ... it just plays a tone for a second.

if you change that example to make the audio source loop instead, it will play a tone continuously, and you will see the OnAudioRead function keeps getting called, allowing you to continually write new data into the stream.

my code is essentially identical to this example, except where i've added my own logic for deciding which sample to write in OnAudioRead - i just grab the next piece of data from the current packet that's playing.

so to more specifically address your questions -

1. you receive an array of bytes from the server
2. decompress it with Steamworks.SteamUser.DecompressVoice. this gives you a new byte array of uncompressed audio.
3. you convert that byte array into the float array that unity prefers (i do this in VoiceChatPacket.Decode)
4. hang onto that float array until your OnAudioRead function is ready for it (put it in a queue so that you can read them in the correct order!)
5. keep track of which float array you're currently playing, and the last position you wrote into the OnAudioRead array (my vars to keep track of this are called currentPacket and currentPacketSampleIndex). you want to keep track of this stuff so that you can pick up where you left off the next time OnAudioRead is called.
6. if you reach the end of the currentPacket's data, get the next packet from our queue and reset currentPacketSampleIndex to 0

does that make more sense? not sure if i'm doing a good job explaining it.

 

How I understand it, please correct me if I'm wrong.
You have a OrderedList (or some other datatype that is fit for the job and performant for the operations). This ordered list has float arrays in it.
When you recieve a float[] from the server (well, not really. You decompress and stuff before but you get what I mean). You put that float array into your list. And then everytime you call OnAudioRead. You feed it the first float array (oldest). And then remove that from the list. And if there is nothing in the list. I guess generate a new float array that contains silence.

If I understand it all correctly, (which i'm probably not). I have a few more questions, What frequencies have you found suitable?
What length should the streaming audioclip be? Or does it even matter when the stream option is true?

 

yeah, that sounds right actually. i'm about to do a test without writing any 'silent' arrays to see how that goes...i'm pretty sure it should work mostly fine without it - i just won't modify OnAudioRead's array if the packet queue is empty.

i don't have good answers for the rest of your questions yet.

i haven't messed around with frequencies much. i've been using 11025 because for some reason i saw that number in the steam docs, and have been using it as my baseline. i'll probably test out other frequencies once i get these popping issues handled.

as for the length of the streaming clip...i'm not really sure that size matters here. in my example, i tried making it quite a bit longer than it needs to be, in case the looping is a cause of audio pops. increasing the size doesn't seem to cause popping to be more infrequent. i'd guess there's a minimum size requirement but i haven't tested it.

if you pay attention to the size of the array that's passed into OnAudioRead, I'm seeing it usually ranges from like 300-4000ish (4096?) floats long. so i assume it probably needs to be at least that many samples long? clearly i'm figuring out a lot of this stuff as i go.

 

As for silent arrays. Dont see that being a issue too be honest. GetAvailableVoice Wont return OK if it just has silence data. Atleast from my testing.

 

ok, so a couple more points on this -

you have to write zero's into the OnAudioRead buffer if there's no data in the queue, otherwise you'll hear old voice chat data on a very short loop.

i added a simple buffer mechanism that disables writing in OnAudioRead until i have X number of snapshots accumulated. i also added a check to discard old data packets that would have already been played.

this sounds really good now. it probably could be higher fidelity, but it's passable in its current state. i'm using this for a VR game so I'd prefer to save cpu cycles wherever i can.

 

Do you mind sharing a gist or something? Would love to see your approaches to this.

 

Here's the complete file i'm using. i cleaned it up a bit and added some comments.

Code (CSharp):

1.  
2. namespace Robochase6000
3. {
4.     public class VoiceChatPacket
5.     {
6.         public ulong PacketId;
7.         public int Length;
8.         public byte[] Data;
9.         public float[] DecodedData = null;
10.         public bool IsSilence = false;
11.  
12.         // decodes from steam's uncompressed format to the float format that unity likes
13.         // note that this might need to change somewhat if i mess around with the frequency.
14.         public void Decode()
15.         {
16.             DecodedData = new float[Length / 2];// optimization todo :: pool this.
17.             for (int i = 0; i < DecodedData.Length; i++) {
18.                 float value = (float) System.BitConverter.ToInt16 (Data, i * 2);
19.                 DecodedData [i] = value / (float)short.MaxValue;
20.             }
21.         }
22.     }
23.  
24.     public class SteamVoiceChatPeer : MonoBehaviour
25.     {
26.         public CSteamID SteamID;
27.  
28.         public SortedList<ulong, VoiceChatPacket> PacketQueue = new SortedList<ulong, VoiceChatPacket>();
29.         private AudioSource m_audioSource;
30.         // how many packets we should collect before starting playback
31.         static public int PacketBuffer = 10;
32.         // whether or not we're currently waiting for more packets to be collected.
33.         public bool Buffering = true;
34.  
35.         // the current position of the playhead in the AudioClip
36.         private int m_streamPosition = 0;
37.         // the packet that is currently being played
38.         private VoiceChatPacket m_currentlyPlayingPacket;
39.         // our position in the packet that's being played.
40.         private int m_currentlyPlayingPacketSampleIndex = 0;
41.         // the last/current packet that was played.  if we get packets older than this, we can throw them out.
42.         private ulong m_lastPlayedPacketId = 0;
43.  
44.         void Start()
45.         {
46.             m_audioSource = GetComponent<AudioSource>();
47.             m_audioSource.loop = true;
48.             m_audioSource.clip = AudioClip.Create ("VoiceChat", 11025 * 10, 1, 11025, true, OnAudioRead, OnAudioSetPosition);
49.             m_audioSource.Play ();
50.         }
51.  
52.         void Update()
53.         {
54.             // if we're buffering, we're not anymore if we've gotten enough packets.
55.             if (Buffering) {
56.                 Buffering = PacketQueue.Count < PacketBuffer;
57.             }
58.         }
59.  
60.         void OnAudioRead(float[] data)
61.         {
62.             // wait til we have some packets saved up.
63.             if (Buffering) {
64.                 // write out silence and gtfo
65.                 int count = 0;
66.                 while (count < data.Length) {
67.                     data [count] = 0;
68.                     m_streamPosition++;
69.                     count++;
70.                 }
71.             }
72.             // we've got enough packets, start writing them to the buffer
73.             else {
74.  
75.                 // if we dont' have a packet to play, try grabbing the next one.
76.                 if (m_currentlyPlayingPacket == null) {
77.                     GrabNextPacket ();
78.                 }
79.  
80.                 int count = 0;
81.                 while (count < data.Length) {
82.                     // start at silence, and fill it in with the correct value.
83.                     float sample = 0;
84.  
85.                     if (m_currentlyPlayingPacket != null) {
86.                         sample = m_currentlyPlayingPacket.DecodedData [m_currentlyPlayingPacketSampleIndex];
87.  
88.                         // increment our current packet's playhead now that we've just read a sample
89.                         m_currentlyPlayingPacketSampleIndex++;
90.  
91.                         // mark down the last packet that was played so that we have an idea of which incoming packets are obsolete.
92.                         m_lastPlayedPacketId = m_currentlyPlayingPacket.PacketId;
93.  
94.                         // if we've reached the end of this packet, grab the next one.
95.                         if (m_currentlyPlayingPacketSampleIndex >= m_currentlyPlayingPacket.DecodedData.Length) {
96.                             GrabNextPacket ();
97.                         }
98.                     }
99.  
100.                     // write the sample to the AudioClip & update it's position
101.                     data [count] = sample;
102.                     m_streamPosition++;
103.                     count++;
104.                 }
105.             }
106.         }
107.  
108.         void OnAudioSetPosition(int newPosition)
109.         {
110.             m_streamPosition = newPosition;
111.         }
112.  
113.         private void GrabNextPacket()
114.         {
115.             if (PacketQueue.Count > 0) {
116.                 var pair = PacketQueue.First ();
117.                 VoiceChatPacket packet = pair.Value;
118.                 if (packet != null) {
119.                     m_currentlyPlayingPacket = packet;
120.                     PacketQueue.Remove (m_currentlyPlayingPacket.PacketId);
121.                 }
122.             } else {
123.                 m_currentlyPlayingPacket = null;
124.                 Buffering = true;
125.             }
126.  
127.             // reset the index.
128.             m_currentlyPlayingPacketSampleIndex = 0;
129.         }
130.  
131.         public void OnNewSample(VoiceChatPacket newPacket)
132.         {
133.             // throw out duplicates. this should never happen...
134.             if (PacketQueue.ContainsKey (newPacket.PacketId)) {
135.                 Debug.LogError ("already have packet " + newPacket.PacketId + ". aborting");
136.                 return;
137.             }
138.  
139.             // throw out old packets
140.             if (m_lastPlayedPacketId > newPacket.PacketId) {
141.                 Debug.Log ("throwing out old packet " + newPacket.PacketId);
142.                 return;
143.             }
144.  
145.             // ignore silence
146.             if (newPacket.IsSilence) {
147.                 return;
148.             }
149.  
150.             // convert immediately.
151.             newPacket.Decode();
152.  
153.             // shove it into our queue.
154.             PacketQueue.Add (newPacket.PacketId, newPacket);
155.         }
156.     }
157. }
158.  
159.  

 

Thats a nice find over here! Except none of this works. From my understanding, you guys are just kind of jump starting this?

Does anybody have a complete working script for the voice chat?
@robochase, I've tried your script and looked through it, and it seems like I cannot actually use it, since there is no function such KeyCode.V (example) to initiate a chat?

Cheers

 

None of it works? The code I posted in my post works great.

 

hey @saltymeow !

My script works quite well actually, but it's just one small piece of the puzzle. The script i posted is just to demonstrate how to decode the data and correctly play it back. It's missing a whole world outside of it that initializes recording, captures microphone data, and sends/receives that data to peers. I intentionally left all that stuff out because it's way outside the scope of this thread, and the primary concern was how to get the audio data to play back correctly.

Hopefully my code example is enough to get the gist of what you need to do for smooth voice playback.

 

It doesn't. I get few errors.

 

Gah. Well. I don't know much about voice chat and how to program it. Did my research and didn't find much.
You have a solution you could possiblly share?

 

What are your errors? And why don't you resolve those errors >.<

 

@TwoTen @robochase
I just wanted to let you guys know this thread saved me one or two days work. Thanks so much!

 

Np. This should also be compatible with Facepunch.Steamworks.

 

Hey guys. Thanks for doing this. I'm also working on Steam Voice right now. And this helped me solve the garbage audio issue I was having. But I'm now having a problem with some significant delay going on. I think its because we are looping the source all the time. I'll play around with it and post a fix here later if I ever figure it out.

 

Oh how silly. Don't need the packet buffer. Just set it to 1 or recode it so you are not using it. There will be some popping sounds. To reduce it, I'm not 100% if this is correct but what I did which worked for me is to add extra data to the last packet as silence so it will play that instead of an abrupt stop. I'm also setting the loop to false when buffering and just playing and setting it to loop again after it has buffered. Again I cannot guarantee my solution that's why I'm not posting my code here but it works good enough for me. I'll just revisit this one to make it better. Cheers!

 

cool thanks for taking a look @paolo_developer

I've noticed the voice is pretty delayed as well, but I haven't had time to verify if it's this code or maybe just steam or what.

 

@TwoTen Thanks for the script and we've been using MLAPI with Steam matchmaking and started looking for voice chat option. It will be great if this will be included in MLAPI transport/asset!

 

Just to confirm, the code works, here's my version for MLAPI where I start recording on Start straight away, Steam has the voice detection already and the voice buffer is only sent when there's data.

Code (CSharp):

1. using UnityEngine;
2. using MLAPI;
3. using MLAPI.Messaging;
4. using Steamworks;
5.  
6. namespace Multiplayer.Demo
7. {
8.     public class MultiplayerDemoPlayer : NetworkedBehaviour
9.     {
10.         public static MultiplayerDemoPlayer myPlayer;
11.  
12.         AudioSource audioSource;
13.  
14.         void Awake()
15.         {
16.             audioSource = GetComponent<AudioSource>();
17.         }
18.  
19.         void Start()
20.         {
21.             if (IsLocalPlayer) {
22.                 myPlayer = this;
23.                 SteamUser.StartVoiceRecording(); // Start recording automatically
24.             }
25.         }
26.  
27.         void Update()
28.         {
29.             if (IsLocalPlayer) {
30.                 EVoiceResult voiceResult = SteamUser.GetAvailableVoice(out uint compressed);
31.                 //Debug.LogFormat("MultiplayerDemoPlayer:Update - voiceResult={0}, compressed={1}", voiceResult, compressed);
32.                 if (voiceResult == EVoiceResult.k_EVoiceResultOK && compressed > 1024) {
33.                     byte[] byteBuffer = new byte[1024];
34.                     voiceResult = SteamUser.GetVoice(true, byteBuffer, 1024, out uint bufferSize);
35.                     if (voiceResult == EVoiceResult.k_EVoiceResultOK && bufferSize > 0) {
36.                         SendVoiceData(byteBuffer, bufferSize);
37.                     }
38.                 }
39.             }
40.         }
41.  
42.         [ServerRPC]
43.         void SendVoiceData(byte[] byteBuffer, uint byteCount)
44.         {
45.             //Debug.LogFormat("MultiplayerDemoPlayer:SendVoiceData - destBuffer.Length={0}, byteCount={1}", byteBuffer.Length, byteCount);
46.             var colliders = Physics.OverlapSphere(transform.position, 50, LayerMask.GetMask(new string[] { "Player" }));
47.             foreach (var collider in colliders) {
48.                 var networkedObject = collider.GetComponent<NetworkedObject>();
49.                 if (networkedObject.OwnerClientId == GetComponent<NetworkedObject>().OwnerClientId) { // Do not play voice on the player's own client
50.                     continue;
51.                 }
52.                 if (networkedObject != null) {
53.                     InvokeClientRpcOnClient(ClientPlaySound, networkedObject.OwnerClientId, byteBuffer, byteCount); // Send to sender too?
54.                 }
55.             }
56.         }
57.  
58.         [ClientRPC]
59.         void ClientPlaySound(byte[] byteBuffer, uint byteCount)
60.         {
61.             //Debug.LogFormat("MultiplayerDemoPlayer:ClientPlaySound - destBuffer.Length={0}, byteCount={1}", byteBuffer.Length, byteCount);
62.             byte[] destBuffer = new byte[22050 * 2];
63.             EVoiceResult voiceResult = SteamUser.DecompressVoice(byteBuffer, byteCount, destBuffer, (uint) destBuffer.Length, out uint bytesWritten, 22050);
64.             //Debug.LogFormat("MultiplayerDemoPlayer:ClientPlaySound - voiceResult={0}, bytesWritten={1}", voiceResult, bytesWritten);
65.             if (voiceResult == EVoiceResult.k_EVoiceResultOK && bytesWritten > 0) {
66.                 audioSource.clip = AudioClip.Create(UnityEngine.Random.Range(100, 1000000).ToString(), 22050, 1, 22050, false);
67.                 float[] test = new float[22050];
68.                 for (int i = 0; i < test.Length; ++i) {
69.                     test[i] = (short) (destBuffer[i * 2] | destBuffer[i * 2 + 1] << 8) / 32768.0f;
70.                 }
71.                 audioSource.clip.SetData(test, 0);
72.                 audioSource.Play();
73.             }
74.         }
75.  
76.     }
77. }
78.  

 

Hello!
I'm also trying to implement voice chat using Steamworks.
I have the following problem tho: In the first 4 seconds or so, SteamUser.GetAvailableVoice returns OK, but then after 4 seconds, it always only returns k_EVoiceResultNoData. I don't know what's the problem. I'm using the same code as @SweatyChair does.

Any help would be really appreciated!

 

Ok i found the issue... Windows has once again changed my default recording device to something else^^ Now it's working fine!

 

Hi! So glad I found this thread.
So, I'm trying to implement the Steamworks voice chat using Steamworks.NET + Mirror.
The voice chat works and it sounds decent for me, but I have one problem and it is when somebody is using the voice chat, they can listen to their own voice. I've been doing some workaround to modify the code so it can work in the lobby (not for the nearby player, like in the example) but still, everyone can hear their own voice.
Here I'm included my code so anyone could help me out, especially @TwoTen
Thanks!

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using Mirror;
5. using Steamworks;
6.  
7. public class PlayerVoiceChat : NetworkBehaviour
8. {
9.     public AudioSource audioSource;
10.  
11.     private void Update()
12.     {
13.         if (isLocalPlayer && Input.GetKeyDown(KeyCode.T))
14.         {
15.             SteamUser.StartVoiceRecording();
16.             Debug.Log("Record Start");
17.         }
18.         else if (isLocalPlayer && Input.GetKeyUp(KeyCode.T))
19.         {
20.             SteamUser.StopVoiceRecording();
21.             Debug.Log("Record Stop");
22.         }
23.  
24.         if (isLocalPlayer)
25.         {
26.             uint compressed;
27.             EVoiceResult ret = SteamUser.GetAvailableVoice(out compressed);
28.             if(ret == EVoiceResult.k_EVoiceResultOK && compressed > 1024)
29.             {
30.                 Debug.Log(compressed);
31.                 byte[] destBuffer = new byte[1024];
32.                 uint bytesWritten;
33.                 ret = SteamUser.GetVoice(true, destBuffer, 1024, out bytesWritten);
34.                 if(ret == EVoiceResult.k_EVoiceResultOK && bytesWritten > 0)
35.                 {
36.                     Cmd_SendData(destBuffer, bytesWritten);
37.                 }
38.             }
39.         }
40.     }
41.  
42.     [Command (channel = 2)]
43.     void Cmd_SendData(byte[] data, uint size)
44.     {
45.         Debug.Log("Command");
46.         PlayerVoiceChat[] players = FindObjectsOfType<PlayerVoiceChat>();
47.  
48.         for(int i = 0; i < players.Length; i++)
49.         {
50.             Target_PlaySound(players[i].GetComponent<NetworkIdentity>().connectionToClient, data, size);
51.         }
52.     }
53.  
54.  
55.  
56.     [TargetRpc (channel = 2)]
57.     void Target_PlaySound(NetworkConnection conn, byte[] destBuffer, uint bytesWritten)
58.     {
59.         Debug.Log("Target");
60.         byte[] destBuffer2 = new byte[22050 * 2];
61.         uint bytesWritten2;
62.         EVoiceResult ret = SteamUser.DecompressVoice(destBuffer, bytesWritten, destBuffer2, (uint)destBuffer2.Length, out bytesWritten2, 22050);
63.         if(ret == EVoiceResult.k_EVoiceResultOK && bytesWritten2 > 0)
64.         {
65.             audioSource.clip = AudioClip.Create(UnityEngine.Random.Range(100, 1000000).ToString(), 22050, 1, 22050, false);
66.  
67.             float[] test = new float[22050];
68.             for (int i = 0; i < test.Length; i++)
69.             {
70.                 test[i] = (short)(destBuffer2[i * 2] | destBuffer2[i * 2 + 1] << 8) / 32768.0f;
71.             }
72.             audioSource.clip.SetData(test, 0);
73.             audioSource.Play();
74.         }
75.     }
76. }
77.  

 

it looks like in Cmd_SendData, you're finding all the players and calling Target_PlaySound, which is why players can hear themselves.

ideally you skip calling Target_PlaySound for the sender.
alternatively, you could try adding something like this to the start of the Target_PlaySound method

if (isLocalPlayer) return;

you'd be sending extra data you don't need to send, but you could at least verify whether the early return fixes the problem

@MyHandStudio

 

Hello everyone, great code examples! I have a quick question though. I am currently using Fizzy Steamworks and I have the following channels configured:


When using [Command (channel = 2)], which channel does this use? Is this an index for the above settings? I guess not cause I have even tried writing 13 and there wasnt any error. I want to use the "k_EP2PSendUnreliableNoDelay" channel but I am not sure if I am using it or not.

 

I have done some digging in the code and apparently if you are using Steam Sockets (UseNextGenSteamNetworking), only the following two channels are used:

k_nSteamNetworkingSend_Unreliable (channel 0 in Steam)
k_nSteamNetworkingSend_Reliable (channel 8 in Steam)

It is determined by the following line. (NextCommon.cs - Line 20)

Code (CSharp):

1. int sendFlag = channelId == Channels.Unreliable ? Constants.k_nSteamNetworkingSend_Unreliable : Constants.k_nSteamNetworkingSend_Reliable;
2.  

So, to use the k_nSteamNetworkingSend_Unreliable channel, the property must be [Command (channel = 1)]. Anything other than that runs on k_nSteamNetworkingSend_Reliable. I have tested this and got better result with channel 1 due to the use of k_nSteamNetworkingSend_Unreliable channel. I will also modify the code to try the k_EP2PSendUnreliableNoDelay channel as well.

 

using System.Collections.Generic;
using Steamworks;
using UnityEngine;

 

@robochase I know it's been 5 years since you posted this, but would you be able to share the code you're using to record the audio samples and store them as VoiceChatPackets? I was able to get @SweatyChair 's example working, and wanted to switch to your method since you mentioned it resulted in much better quality. Thanks so much in advance!

 

@QPhysics it's tough to share all that, i have a lot of wrappers around steamworks methods .

Here's what i'm doing in a nutshell though

1. enable voice recording
Steamworks.SteamUser.StartVoiceRecording();

2. checking and getting voice recording

Code (CSharp):

1. // check if there's voice data available to send
2. var hasDataAvailable = Steamworks.SteamUser.GetAvailableVoice(out uint compressedBytes);
3.  
4. if (hasDataAvailable == EVoiceResult.k_EVoiceResultOK && compressedBytes > 0)
5. {
6.     // get the voice data to send
7.     bool wantCompressedAudio = true;
8.     byte[] compressedAudioBuffer = new byte[8000];// instantiate this somewhere else lol
9.  
10.     var getVoiceDataResult = Steamworks.SteamUser.GetVoice(wantCompressedAudio, compressedAudioBuffer, compressedBytes, out uint bytesWritten);
11.  
12.     if (getVoiceDataResult == EVoiceResult.k_EVoiceResultOK)
13.     {
14.         // at this piont, your compressedAudioBuffer array has bytesWritten number of entries filled with data to send
15.         // write the bytes to the socket however your networking layer prefers
16.     }
17.    
18. }

3. reading voice audio on the receiving end

Code (CSharp):

1. // read the data that was sent.
2. var packetIndex = reader.ReadInt();
3. var hasVoiceData = reader.ReadBool();
4. if (hasVoiceData)
5. {
6.     var audioBufferLength = reader.ReadInt();
7.     var compressedAudio = new byte[8000];// create this somewhere else.
8.     reader.ReadBytes(compressedAudio, audioBufferLength);
9.  
10.     var uncompressedAudio = new byte[22000];// create this somewhere else
11.    
12.     var desiredSampleRate = 11025;// this is what my code is using, i don't remember messing with this value or how i came to use it
13.  
14.     var decompressResult = Steamworks.SteamUser.DecompressVoice(compressedAudioBuffer, bytesReceived,
15.                     uncompressedAudio, uncompressedAudio.Length,
16.                     out uint bytesWritten, desiredSampleRate);
17.  
18.  
19.     if (decompressResult == EVoiceResult.k_EVoiceResultOK)
20.     {
21.         var voiceChatPacket = new VoiceChatPacket();
22.         voiceChatPacket.PacketId = reader.ReadInt();
23.         voiceChatPacket.IsSilence = !reader.ReadBool();
24.         voiceChatPacket.Data = uncompressedAudio;
25.         voiceChatPacket.Length = (int)bytesWritten;
26.         peer.OnNewSample(voiceChatPacket);
27.     }
28. }
29.  

 

@robochase Thank you so much! This was exactly what I needed. Just got it up and running and it sounds wonderful!

 

@robochase Perhaps I spoke too soon haha - I'm having issues with the audio being very delayed with frequent and long pauses where it cuts out entirely. I tried increasing/decreasing the buffer and bitrate, but nothing seems to improve my problem. I thought it could possibly have to do with my networking layer (Photon PUN2), but that doesn't make sense I think. Any suggestions?

EDIT: the solution was to change the compressed buffer byte array size from 8000 to just whatever

SteamUser.GetVoice

returned for

nBytesWritten

 

cool @QPhysics glad you got it working, and thanks for troubleshooting the array size thing!

 

Here's my solution for voice chat using Facepunch.Steamworks and Mirror networking.

I'll post the code below as well, but the gist linked above might be more up-to-date.

Code (CSharp):

1. using System;
2. using System.Collections.Generic;
3. using System.IO;
4. using Mirror;
5. using Steamworks;
6. using UnityEngine;
7.  
8. // TODO To use:
9. // Grab this and its dependencies: https://github.com/Chykary/FizzyFacepunch
10. // Attach this to a Mirror Player Prefab and assign an AudioSource
11. // Test with two different Steam accounts (on two machines or with a VM)
12. // Enjoy! :D  -Finnbarr from No Bloat Studios
13.  
14. public class MirrorSteamworksVoice : NetworkBehaviour
15. {
16.     [SerializeField] AudioSource _audioSource;
17.    
18.     readonly MemoryStream _compressedVoiceStream = new();
19.     readonly MemoryStream _decompressedVoiceStream = new();
20.     readonly Queue<float> _streamingReadQueue = new();
21.    
22.     void Start()
23.     {
24.         _audioSource.clip = AudioClip.Create("SteamVoice", Convert.ToInt32(SteamUser.SampleRate),
25.             1, Convert.ToInt32(SteamUser.SampleRate), true, PcmReaderCallback);
26.        
27.         _audioSource.Play();
28.     }
29.  
30.     void Update()
31.     {
32.         if (isLocalPlayer)
33.         {
34.             SteamUser.VoiceRecord = Input.GetKey(KeyCode.V);
35.            
36.             if (SteamUser.HasVoiceData)
37.             {
38.                 _compressedVoiceStream.Position = 0;
39.                
40.                 int numBytesWritten = SteamUser.ReadVoiceData(_compressedVoiceStream);
41.                
42.                 CmdSubmitVoice(new ArraySegment<byte>(_compressedVoiceStream.GetBuffer(), 0, numBytesWritten));
43.             }
44.         }
45.     }
46.  
47.     [Command(channel = Channels.Unreliable, requiresAuthority = true)]
48.     void CmdSubmitVoice(ArraySegment<byte> voiceData)
49.     {
50.         RpcBroadcastVoice(voiceData);
51.     }
52.  
53.     [ClientRpc(channel = Channels.Unreliable, includeOwner = false)]
54.     void RpcBroadcastVoice(ArraySegment<byte> voiceData)
55.     {
56.         _compressedVoiceStream.Position = 0;
57.         _compressedVoiceStream.Write(voiceData);
58.        
59.         _compressedVoiceStream.Position = 0;
60.         _decompressedVoiceStream.Position = 0;
61.        
62.         int numBytesWritten = SteamUser.DecompressVoice(_compressedVoiceStream, voiceData.Count, _decompressedVoiceStream);
63.  
64.         _decompressedVoiceStream.Position = 0;
65.  
66.         while (_decompressedVoiceStream.Position < numBytesWritten)
67.         {
68.             byte byte1 = (byte)_decompressedVoiceStream.ReadByte();
69.             byte byte2 = (byte)_decompressedVoiceStream.ReadByte();
70.            
71.             short pcmShort = (short) ((byte2 << 8) | (byte1 << 0));
72.             float pcmFloat = Convert.ToSingle(pcmShort) / short.MaxValue;
73.            
74.             _streamingReadQueue.Enqueue(pcmFloat);
75.         }
76.     }
77.    
78.     void PcmReaderCallback(float[] data)
79.     {
80.         for (int i = 0; i < data.Length; i++)
81.         {
82.             if (_streamingReadQueue.TryDequeue(out float sample))
83.             {
84.                 data[i] = sample;
85.             }
86.             else
87.             {
88.                 data[i] = 0.0f;  // Nothing in the queue means we should just play silence
89.             }
90.         }
91.     }
92. }

 

Hello @everyone, Im trying to make a voice chat in my game. Im using fizzysteamworks and Mirror, and Im using the code below. However, I am not able to make it work, so that it sends data and someone else can hear the data. Is there someone that could address the problems in my code? However, I hear a little pop when I stop pushing the voicechat button.