[RELEASED] USpeak Voice Chat - Voice Chat For All Your Multiplayer Games

Voice chat is an essential feature for any multiplayer game. Players appreciate being able to call out loot they find in the dungeon or Ogres hiding around corners, warn teammates about snipers on the rooftop, or just in general have fun talking to others - without the hassle of having to put down the controls to type out a message. Now you can have fully integrated voice chat in any multiplayer game for almost any platform - Windows, Web, Mac, or even Mobile!

See What USpeak Can Do For You Game

• USpeak can plug into just about any networking backend in minutes - and you don't need to host any servers.
• Super easy to use - Converting the Photon AngryBots demo took just 10 minutes!
• Incredibly lag-spike resilient - sounds good even with a ping jitter of up to 100ms and packet loss rate of up to 10%!
• Uses the Speex codec for low bandwidth usage and high quality audio! You can even code your own codec with a few simple functions, and includes optional Mu-Law and ADPCM codecs.
• Runtime microphone detection - you never need to restart the application, just plug in a mic and go.
• Multiple activation methods built-in - Always on, push to talk, or toggle talk. Or, code your own.
• Spatial audio support - Just tick a checkbox and enjoy directional audio for a truly immersive experience that players will thank you for.

What Are People Saying About USpeak?

See How Easy It Is To Add USpeak To Your Game
Check out a video tutorial for adding USpeak to a Photon game:

Players Will Thank You For Voice Chat
For those that prefer the trusty old keyboard messaging, USpeak doesn't force players into using voice chat - it even comes with a handy Mute function for those players who just want to turn it off altogether.
And, in a poll of nearly 400 gamers asking whether they favored text chat or voice chat, over 80% of gamers were in favor of voice chat.
Voice chat is an essential feature for serious and casual gamers alike.


Hear The Quality You Can Depend On From USpeak
Check out the upcoming multiplayer survival game Rust, by Facepunch Studios, which uses USpeak to power their ingame VoIP http://playrust.com/
And this popular FPSMMO using USpeak: Critical Strike Portable http://www.kongregate.com/games/IgorLevochkin/counter-strike-web-browser-based-port-v2
Check out a song from the game Quake 3, with it's audio piped over Virtual Audio Cable into USpeak.

What Are You Waiting For?
Buy It Now!
Asset Store Link:http://u3d.as/content/mo-pho-games/u-speak-voice-chat/3sR

 

I use it in my game already , works great!

 

I'm proud to announce that version 1.2 is already on the horizon, with some audio fixes that will improve the quality in high-latency situations, as well as the possible introduction of webcam video streaming!

 

Would it be possible for you to implement the Opus codec?

It is the successor of speex and completely open and royalty-free
http://opus-codec.org/

For our game design we are thinking of implementing voice chat, but since we are supporting hundreds of players, using too much bandwidth for voice is a big no-no, the Opus 8 kbits/sec codec seems very good for it's bandwidth usage.


If not, does your asset comes with full source code so I could add this myself.

 

At the moment, the Opus reference implementation is in C - to use Opus, I would have to port the entire codec (in a language I'm not too familiar with) to another language, and that includes dealing with how to port code that uses pointers, how to deal with the quirks of C, understanding optimizations made, etc. That's something I don't feel is currently necessary, as I'm already using Speex which should be perfectly acceptable in terms of data usage.
As it is, audio is compressed in the following ways:
- Each sample in a chunk of audio is crunched down into a single byte, already reducing the audio clip to 1/4 of it's original size without sacrificing much quality at all.
- The audio is then fed into Speex, which compresses it even further.
- Finally, the audio is compressed using the Deflate algorithm before being sent over the network.

However, yes this is a source distribution, so if you feel like adding Opus yourself you are perfectly welcome to.

 

Though a .net wrapper would probably do just fine in this case.

What kind of bandwith are we talking about with your implementation then?
Also, how many times per second are UDP packets send?

 

Well, the packets/sec depends on your networking solution. This by itself doesn't send any network data, it's up to you to implement the ISpeechDataHandler interface's USpeakOnSerializeAudio function. If you're using Unity networking / RPCs, I believe Unity RPCs are sent immediately and it sends about 32 per second. If you're using Photon Unity Networking / Photon RPCs, the amount of RPCs sent per second is actually a fixed rate (PhotonNetwork.sendRate I believe). It all depends how you decide to handle it, but the absolute maximum it will send is 32 per second.
And I actually forget the bandwidth, I'll get back to you on that ASAP (a bit busy ATM)

 

OK, Vincenzo, I just tested and currently uSpeak generates about 60 kbit/sec on the default quality sending (bandwidth mode set to 'Wide'). While this isn't nearly as good as the impressive 8 kbit/sec of Opus, this is still on par with many popular voice chat solutions such as Mumble (widely used for Team Fortress 2 servers).

 

Hey Phobic!

This might help you first of all, a .net wrapper around the opus library.
https://github.com/JohnACarruthers/Opus.NET
I am unsure about device compatibility with wrappers... possibly it would be limited to windows. not entirely sure at this time. though for my game that does not matter, we are focusing on windows.

Well in terms of packets per second, the less and still acceptable, the better, in my current game implementation I'm sending packets out with 100 ms interval (10 times per second) will the library support such delays or do i need to send it at 50ms interval for instance.

I see, 60 kbits is indeed quite high, what kind of settings are available to set the band to narrow and other tricks to force speex to a lower bitrate as speex by itself does support quite low bitrates.

Cheers,

 

Thank you Vicenzo, I will look into integrating Opus.
However, the current output of 60 kbits isn't terribly high in the scope of things, Mumble for example produces a little over 50 kbits and Ventrillo produces on average 70 kilobits, as far as I know.
Setting the bandwidth mode to Narrow of course reduces the data ouput size.

 

Hey, good news everybody. Version 1.1b is pending review on the asset store. 1.1b includes a re-worked audio engine that should dramatically improve the audio quality.

 

Pretty cool stuff here, will have to pick it up.

 

Indeed, kind sir!
Also, for those who purchase uSpeak, always be on the lookout for updates. I might find bugs or other things to improve and when that happens I try to submit new versions to the asset store as soon as possible.
And remember, I haven't tested this on a mobile device for the simple reason that I don't own one, but your purchase goes toward helping me buy one so I can test it out as thoroughly as possible.

 

New Video, shows the upcoming update which will include rudimentary webcam streaming (note that it will incur a noticeable performance hit due to the Texture2D.Apply function, as far as I know there's no way around this).

EDIT: I've decided after some consideration to drop video streaming, due to some limitations in Unity. Even sending frames at 10 FPS drops my framerate down from 60-70 to 4-5 due to Texture2D.Apply

[video removed]

 

EDIT: The following no longer applies to the latest version available in the Asset Store, v2.0a

So. Today I wondered what it would sound like if someone were to micspam one of my favorite game trailers, Meet the Engineer (property of Valve, I make no claim on the Team Fortress IP or any related characters/content). So, I tested it. On one computer, I set up Virtual Audio Cable and set up a video of the aforementioned trailer to output from VAC. On another computer, faithful Audacity capturing the audio being piped in from the other computer. Both have a test client using Unity Networking.
The result, while not perfect, is still pretty good (note that uSpeak tends to handle straight voice data better than many other kinds of audio)
Shown here is the captured audio used to replace the original trailer's soundtrack.

[video removed, no longer reflects the current status of uSpeak]

 

After some careful consideration, I realize the quality seen in that video just isn't going to do it for me, at least not for the final version.
I'm going back to the drawing board, so it may take a while but uSpeak will come out a far better product because of it. Rest assured, the API will remain exactly the same, so you won't have to change your code, but this time around I can fix some of the design issues I encountered the first time, and a few other gotchas I ran into (for example, as it turns out Speex always adds some number of zero or near-zero samples into your audio data, in the above video I'm dealing with it somewhat, but not nearly enough. What I think I'm actually going to do is set up a test - an audio clip with random data in it, encoded with Speex, decoded, then saved out to a WAV file so that I can analyze it in Audacity)

 

So, in the re-development of USpeak from the ground up, one thing I realized was that I needed a better container format for the packets.
Enter the USpeakFrameContainer class. As audio data is gathered from the microphone, it will be encoded via Speex (or not, depending on your settings) and stuffed into a USpeakFrameContainer, which is then pushed onto a send buffer to be processed later (in order to avoid flooding the network)
Just before the frame is passed to your custom Serialize function, it's converted to a byte array. Previously, your function was essentially passed a smorgasbord of data, including how many samples, the bandwidth mode, whether it's speex encoded, etc. Now, all your function is going to get is a single byte array, which already contains all of the necessary data in a much more compact format than before ( takes up a grand total of 5 bytes now, one byte to encode the bandwidth mode/speex encoding/deflation/whether it's 3D and four bytes to encode the number of samples in the given frame )
Previously, this data combined would have taken up:
Samples (4) + Bandwidth Mode (4) + Is3D (1) + IsSpeex(1) + IsDeflate(1) = 11 bytes
Now, consider if your data once encoded is 80 bytes in size. If you send 500 of these packets, currently that's 45,500 bytes total. With the new system, it'll instead be 42,500 bytes, a savings of 3 kilobytes. Now, that's not exactly a huge savings, but I understand that in realtime multiplayer every byte counts - and being wasteful is not an answer.

 

You're new header though better is still not perfectly optimized.

IsSpeex and IsDeflate etc are settings... Why do you send them at all, things like these are settings that don't need to be send time and time again since they generally do not change. You should instead make a form of event or settings callback that allows you to set these variables on both client + server which can be simply done in editor on the script settings I presume.

I notice you use 4 bytes to encode the number of samples? do you really believe the number will be higher then 255 samples per packet? if not, a single byte will do.

Speaking about samples actually, I assume you record samples of 20 ms a piece and encode them?
perhaps make a system where you allow the guys implementing uSpeak to define their update rate, and you generate packets with enough 20ms samples to fit exactly into the update rate, this way you have 1 header for a bunch of samples into one single packet send out over the network with the rate defined.

Since the UDP header by itself is around 58 bytes, excluding your own network layer headers, which usually are around 10-16 bytes every packet send out is a huge waste, unless you stuff it up completely with as much data as possible.

The way to do is to safe up some data, and stuff it in the packet, for our game implementation we save up all data and send it out 10 times per second (100ms). Thus we waste the least amount of header information.

 

Yes, I know I can still optimize headers further...
As for the four-byte 'samples' value, I think that was a mistake in my thinking (it was pretty late at night). I think Samples CAN go over 255, but not very far over, so I could probably go with a Short to save space and just to be safe.
Additionally, the reason I'm encoding that info in the container is so that the settings can actually change at runtime - however, what I think I'm going to do is have another user callback for sending a settings packet, which is sent on startup as well as if any of the settings change. Then the frame only needs to contain Samples and Data.
And as for the update rate, yes this time I have a custom update rate, as packets are recorded they are encoded in the frame and stuffed in a buffer. When it comes time to send out the packets, they're smooshed together into the same byte array (I added another four bytes which defines the length of each one) before being sent to your code - I understand in some networking systems it's better to send more data less often rather than less data more often.

 

OK, so I've followed your advice. In total, I've managed to shave 3 bytes off of the total message size (Samples has been reduced to 2 bytes, and the metadata, which was one byte, is getting moved to it's own message which will be sent once). Not huge, but it'll probably add up in the long run.

 

OK, I have an important announcement for everybody: The next version of USpeak is going to be sans-Speex.
Oh, the shock! Oh, the horror!
No matter what I try, I just cannot make the codec sound good. So, instead I'm going to be using the G711 standard, specifically the MuLaw variant. Recording frequency is going to be somewhat limited (if you're tight on network bandwidth, you'll have to use 8kHz or lower), but for speech this isn't too bad - human speech generally remains pretty clear as low as 2kHz. And it sounds crystal clear compared to the current audio codec, with bit rates of 60kbps when there's noise/speech (much lower when there's silence), plus the new codec is many orders of magnitude faster than Speex - most of the work is precomputed and stored in lookup tables. In fact, I'm fairly certain the algorithm has been ported to Gameboy for a portable music player project. I already have the codec implemented, I'll post a demo video soon.

EDIT: New video of the MuLaw codec in action (sounds WAY better now)

 

Hey everybody! The new version is on it's way to the Asset Store. What's new? Well, I've completely changed codecs, now it uses the MuLaw variant of the G711 standard instead of the Speex codec. It should sound dramatically better, and is also blazingly fast (if I wanted to, I could probably run it on a Gameboy. Yeah, that fast).
Additionally, I've rebuilt the entire API from the ground up. Although most if it is the same, there are a few key differences which will be outlined in the included Readme.pdf.
The new version is already pending approval and should be available within a few days.

EDIT: New version's up!

 

Hello again people, I'm proud to announce a new version update that adds support for custom codecs! Not only will that allow me to maintain new codecs much easier than before, it also allows you to write your own with a very simple two-function interface.
The update should be available within a few days. EDIT: Already out.

 

Hey everybody, another reason to go with USpeak for your games: USpeak has now been tested with simulated ping jitter of up to 100ms. The result? Still sounds perfect!
Note that a ping jitter of 100ms is extraordinarily high. My ISP, for example, gives an average of 5ms jitter.

 

BUMP

 

I still notice a lot of dropping of voice, what is the cause of that?

 

In the latest video? The dropping out happens when the system is starved of audio data. It buffers up audio it receives and then plays it on a 1/4 second timer (using AudioSource.Play(delay) with an audio source pooling system). If it plays through all of these, and 1/4 second later there's nothing to play, then it has to wait for at least another quarter second if not longer to receive more audio data and continue playing.
That's what you're hearing here, which isn't terribly surprising given the test (100ms ping variance). In most cases network conditions aren't going to be nearly that bad (players with 100ms constant ping jitter probably have a potato for an ISP)

EDIT: Also coupled with some limitations of the Unity audio API. Each quarter-second block will sound perfect by itself, but between blocks there can be a small and random delay due to the way Unity automatically sample-aligns audio clips. As far as I know there's not much to be done about this.

 

hello, PhobicGunner.
I have two questions.
1) After recoring sound, then send the sound to orther. Do it suported this function ?
2) In furture, have you think to make Video Chat package?

 

1.) If you want to, say, have the user hold a key to record, and THEN when they release the key everything is sent, you can certainly accomplish that. I'm not sure why you'd want to, but to accomplish that you might attach a DefaultTalkController and assign it any key (say V) and Push To Talk mode. THen, in your speech handler component, you might keep a list of byte[] (List<byte[]>). In USpeakOnSerializeAudio( byte[] data ), call list.Add( data ). Finally, detect when the user releases the V key, send all of this data over the network and call list.Clear().

2.) Possibly. I had a proof-of-concept working, but at the moment due to several Unity limitations you cannot get it working in real-time (mainly due to Texture2D.Apply, which when called regularly can slow your game/application to a crawl).

 

I understanded, thank you!

 

Small update - added a new bugfix that should improve audio quality even further (turns out I was padding audio chunks with empty samples, and then upon decompression I was attempting to trim those samples - sounds much better when I don't pad audio chunks in the first place), support for custom codecs, new ADPCM codec (for the price of lower quality, reduces bandwidth requirements by half), and updated copyright headers in the code files.

EDIT: Sample of the sound quality of the ADPCM codec uploaded to SoundCloud ( http://soundcloud.com/alanstagner/lets-play-mari0-ep-2-micspam )

 

Update: New web demo uploaded - Photon AngryBots demo with USpeak added. Took probably about 10 minutes total to integrate and uses the ADPCM codec which will be available in the next update. Just more evidence that USpeak is the easiest voice chat solution for the Unity game engine.

 

BUMP

 

Update: v2.2a is now live on the asset store. Includes a few bugfixes and the new ADPCM codec. Cool stuff indeed.
In other news, I managed to get another 2 sales overnight. Thanks to whoever those two guys/girls were, your contributions go toward improving USpeak in the future

 

Man threads fall behind fast...
BUMP

 

Update: The game Critical Strike: Portable, which has been using USpeak for a while now, finally upgraded to the latest version, which sounds loads better than before.
Check it out: http://www.kongregate.com/games/IgorLevochkin/counter-strike-web-browser-based-port-v2

 

Very small update: New version pending review, the static field USpeaker.Mute has been renamed USpeaker.MuteAll and now USpeaker instances have a public field Mute. As the names suggest, MuteAll mutes all incoming voice data, whereas Mute only mutes individual USpeakers. Handy for when you have a dungeon raid going and one person decides to be a troll (<-- See what I did there?).

 

BUMP
Seriously guys, I need your purchases and reviews!
Also, if you have absolutely any ideas/requests, don't be afraid to drop me a message via PM. Any feedback is welcome!

 

Yay, USpeak is now on UnityMagic.com
http://www.unitymagic.com/shop/en/featured-products/uspeak-2-2.html

 

A suggestion...

Make a video showing how easy it is to implement your system. There is another voice chat package in the Unity store and they show a nice video showing how you add it to a project.

A lot of us are hobbyists, and $30 to $40 is a lot to drop on an asset we might not be able to figure out how to use.

 

Coming soon
Working on it now.

 

Tutorial vid is out, enjoy

 

Quick bit of news: USpeak is now featured on the front page of the Asset Store!
Exciting stuff indeed.

 

Another small update.
Today I found a forum post in the Shadowrun Universe forums where players were talking about voice chat integration...
And somebody brought up my 'uSpeak' product as an option for the game
http://www.shadowrun.com/forums/topic/in-game-voice-communication-client

EDIT: Now have contacted the developers. Having Shadowrun use my VoIP would be very cool!

 

Small bugfix was uploaded to the asset store and should be available within a few days. The issue is that apparently Unity Networking doesn't support 'short' values (I could have sworn I've sent shorts over RPC before, but I guess not), so the InitializeSettings value can't be directly sent over the network as-is. I've converted the value to a full Int32 and submitted the update for review.
I apologize for any confusion that caused.

 

New version heading to the asset store, I've removed the 4kHz option as I've found it rarely ever results in intelligible speech (unsuitable for communication), and accordingly renamed the bandwidth modes - they are now simply Narrow ( 8kHz ) and Wide ( 16kHz ).
Additionally I've added a new inspector property 'Play Buffer Length' (corresponding script property: USpeaker.PlayBufferSize). This is how much audio is buffered up before being played in order to a.) overcome Unity audio sample precision limitations and b.) overcome ping jitter, it solves both of these by concatenating the specified amount of audio data before playing it. Smaller values may result in gappier / glitchier data but also reduces the latency between sending and playback, while larger values result in less glitchy/gappy audio but of course increase the latency.

 

I am considering getting a copy of uSpeak, but I was curious if with it's current feature set if it would be easily possible to restrict speech to certain teams rather than broadcasting to everyone. I saw that you had both "MuteAll" and an individual "Mute" property defined, so would you programmatically mute all members of the other team, or would you suggest just handling that within the networking so that the voice networking only sends to members of your team.

Or to phrase it another way, does Mute just ignore data received from certain places or is Mute more of a server parameter that doesn't send data between certain clients in the first place?

 

As a matter of fact, Mute and MuteAll are both purely client-side, so they simply drop data when it's received.
You do have control over when data is sent, and even when/how it's received, so it's certainly possible to implement team chat.
If you were using Unity Networking or some other peer-to-peer solution, you might assemble a list of NetworkPlayers on the same team and send voice chat to each player
If you were using Photon Cloud (which has a central server but no custom server-side code), the most optimal solution would be to drop received packets if the sender isn't on the same team.
If you were using self-hosted Photon, Smartfox, or Player.IO (allows you to plug in your own server code), the server itself would determine which players to send voice data to.
There are quite a few options available to you, depending on your networking solution.

EDIT: I think fairly soon I'll whip up a team chat demo and include it with the main USpeak package as an example of how to accomplish that sort of thing, since it seems to me that it might be a pretty common scenario in competitive games like FPS's.

 

Hey PhobicGunner,

I am highly considering using your VoIP for a project and will be using SmartFox instead of Photon. Does uSpeak work with SmartFox? If by any chance, can you possibly create a short video tutorial on how to set that up with SmartFox (if you cannot I completely understand)? Looking forward to your reply. Thanks!

 

In Smartfox, can you serialize ints, shorts, and byte arrays? If so, it's entirely possible to use Smartfox with USpeak. The only gotcha here is making sure you have a way to associate a USpeaker object with it's associated owner player (when a voice message is received you have to know which USpeaker should receive the message). Although, for a multiplayer game you'll probably have something like that anyway, for player objects and such

EDIT: Was reading up on Smartfox development, most likely you'd have a server extension that handles voice communication, and then your client code creates and sends a new ExtensionRequest containing the byte array passed to your custom USpeakOnSerializeAudio function. At the most basic, your serverside extension can just relay the byte array back to all connected clients, or if you want you can also have the extension check, for example, the teams of the player currently speaking and selectively send voice to only teammates.