uSpeak Voice Chat - work in progress

Today I'm proud to introduce an upcoming voice chat system for Unity - uSpeak.
uSpeak will enable your game with efficient low-bandwidth communication, instantly.

NOTE: THE FEATURE SET AND DEMO VIDEOS BELOW ARE OUTDATED.
SINCE THE BETA, USPEAK HAS UNDERGONE MASSIVE IMPROVEMENTS
PLEASE REFER TO THE OFFICIAL RELEASE THREAD:
http://forum.unity3d.com/threads/15...hat-Voice-Chat-For-All-Your-Multiplayer-Games

Features

• Uses a combination of the Speex codec and Deflate compression, for minimal bandwidth usage
• Plugs into any network backend - if you can serialize bytes and ints, you're good to go!
• Supports four kinds of voice trigger - push to talk, toggle talk, volume activated, and always on.
• Spatial audio support - simply tick a checkbox and your players' voices will pan in realtime 3D
• Supports all major publishing platforms - browser, standalone, and mobile, and all licenses - Indie, Pro, and Mobile!
• Runtime microphone detection - no need to restart the game

I plan on releasing this package in the asset store soon for the low price of $30

The beta period is now over. Beta testers can keep and use their downloads free of charge, and exactly one week from today I will release uSpeak on the asset store.

EDIT: The fluttering effect seen below and mentioned by wccrawford has now been fixed, barring internet latency uSpeak now has flawless playback.

Check out a video of uSpeak in action!

 

I can hear some flutter to their voices. I usually associate that with running out of bandwidth... Could be an artifact of Speex compression at the lower settings? The audio of the stream and their guns is crystal clear, but his voice flutters a lot.

Any idea what price you're planning to sell this for, and an ETA on release?

 

Well as for ETA the package itself is mostly complete. I'm going to try dealing with some of that flutter, add some extra features, polish off the documentation, etc. Shouldn't take too long (a couple days at most)
As for that flutter, I think it's actually a result of packet bunching. Occasionally, the voice sounds perfect, but every so often the flutter happens, and I think it's because currently my system immediately decompresses and plays back a packet when it's received (and if a bunch of packets come in at the same time, they are played at the same time). I'm going to add a buffer, which should mitigate those issues. Also, I do notice the audio seems to stutter more when I use Speex (if I turn off Speex, I don't hear much stutter), so I think it's partially an artifact of the algorithm.
As for price, I'm thinking something around $20 or so.

EDIT: Actually, as for ETA I think I'm going to wait a week before submitting this thing. It should be plenty of time for me to catch any bugs that might surface. I want this thing to be rock solid, if possible.

 

BUMP

 

Aha!
In the last couple of days I've been trying to deal with the fluttering issue. It's been driving me crazy!
I finally found out what was causing it. It turns out, this is actually an age-old problem in Unity, and it's the fact that Unity doesn't provide any good way to accurately time audio sequences. It wasn't anything to do with Speex at all.
Something that definitely seems to improve it is setting DSPBuffer Size to Best Latency.

 

New video demonstrating audio reverb zones (also demonstrates just how much better it sounds with the DSPBuffer setting)

 

Hey,

A few quick questions -

First being - How much does this (Plan to) Sell for on the asset store?

Second being - Can one team hear another team if they get too close?

Thanks

Ben

 

First - I plan to sell this for right around $20.
Second - No, there's no chance of that. The voice channel property actually filters out voice data, so if the local player receives voice data on a different channel than the local uSpeaker, it's completely discarded. Of course, if you wanted to you could leave the voice channel at -1 (which is a global channel) and code your own solution (so, if you actually wanted to have teams hear each other if they are near each other, it's entirely possible to do so)

 

@PhobicGunner

Thank you for the VERY quick reply

The price point seems fair for what is being offered. Let me know when you add this to the asset store.

Well at least it can be programmed in.

 

Will do

 

Argh, I have a competitor.
Normally I wouldn't care so much, except the competitor is offering it for $40, and on the Asset Store guidelines it says:

I think selling mine for half price counts as grossly undercutting a similar product.
I think I'm going to try to get away with $30 instead, especially since his does support one more codec that mine doesn't which could account for the $10 price difference. If that doesn't work, I'll try $35.

 

OK, one person has requested a beta package.
There's still 19 left! Completely free of charge!

 

I'll take a beta package. Happy to help!

 

I'd like a beta copy. I think i'm still in the first 20 lucky ones? ;-)

 

Does this require Pro? I'm assuming it does, unless you ported Speex, but asking to be sure

 

Actually, I'm using a .NET port of Speex, so it is entirely Indie compatible In fact, I developed it on an Indie license

 

Nice! I'd love to try it, then

Side-project #389 could really use voice communication

 

There are now 16 of these left.. they're going much faster than I anticipated

 

I would be interested in trying out a beta package as well. I am actually working on an online game but was unsure if I wanted to try and attempt handling VOIP when there were solutions out there already. It would be fun to try this out.

-Dane

 

Well, you have a nice package here. Besides, the only other person doing this never released a beta, so this is a chance for us to try voice chat in our games.

Also, you'll get feedback as well.

 

Yes, that is part of the reason I'm giving these away.
I'm only one person, so having 20 people be able to use it in real cases and report any bugs or oversights is incredibly useful

 

It's strange nobody had something like this officially available already. I was even considering it myself, but fortunately PhobicGunner got started first

 

I find it kind of strange too. Especially since Unity already uses RakNet, why they don't go ahead and use RakVoice (since it appears to be included in the whole RakNet package) is beyond me.
Oh well. Lack of built-in voice chat does provide us indie developers with market opportunities

 

For those of you who requested a beta download...
I wish to apologize for a mistake in my code. Remote speakers will not process voice data, to fix it do the following:
Move the entire IF block starting on line 235 (it starts with the comment 'receive voice packets') and the line below (receiveTimer -= Time.deltaTime) to before line 205 ( which reads 'if( !ready || SpeakerMode == USpeakerMode.Remote )' )
Sorry for the inconvenience.

 

A couple of users have reported some issues with uSpeak, which I'll outline here...
1.) It seems the codec I am using (NSpeex) is prohibitively slow. I'm going to be replacing the codec with Mu-law compression.
2.) On some microphones, uSpeak doesn't transmit anything in Volume Activated mode. I'm going to try replacing my current VAD algorithm with something much more robust, likely based on zero-crossing rate and short-term energy.

Once I'm finished implementing these changes, I will send everybody a new version. Thanks so much for helping me work out some issues

 

OK, on second thought scratch Mu-Law compression.
After some digging I found that even though encoding does take quite a bit of time (20ms on average), it's only ever called on one player in the scene, so you will very likely never notice a slowdown. Decoding, on the other hand, is blazing fast (about 1ms on average).
I did some experimentation, and now I think volume detection is a more stable.
I also did quite a few bugfixes I noticed while looking though my code.
All current beta users should be receiving a new version. If you don't get a new version within an hour, please contact me.
Thank you guys so much for testing this thing.

 

15 uSpeak beta packages left...
I really need help testing this thing! I want it to be as rock solid as possible before I release it.

 

Sign me up! I'll throw into our fps test lab and see how it performs under some load.

 

Sent!
Now 14 left

 

Would love to try this out Phobic. I won't have it implemented for a couple weeks as basic multiplayer code is on my todo list for the next couple of weeks, but I'm a big believer in the importance of voice chat to solidify a community. I'll be adding it to my stealth multiplayer game "Covet" which is currently staying public throughout development on the browser.

 

Sent via PM
There are now 13 left

 

Nice work. I'd love to beta test uSpeak.
We will consider buying it for our project when it is released.

 

No need to purchase, privilege of a beta tester

 

Great! thanks

 

This makes me very happy. Thanks phobic. Like I said I'll let you know when I get it implemented into my public version of Covet.

 

Sign me up. I doubt I can make it scale on an mmo, but I'll give it a try.

 

Well, it could get tricky on an mmo. Think of a couple hundred people in a region all trying to talk at once. Not only is that a lot of bandwidth, it's also going to be very noisy. More likely, you'd want to either have voice chat in a few specific areas (like PvP arenas, where the number of players in one area is likely to be much smaller), or use the voice channels feature to separate parties into different channels (so a party of players can voice chat amongst themselves)

 

If you have a proximity system that could work as well, do checks for distance every so often to turn on/off chat between particular people.

 

That could be done super easily on the server-side. If you have a quadtree system, you could very quickly locate players around a given player, so if he sends voice chat it queries the quadtree for nearby players and sends the data to them. You'd also make the voices 3D so the player doesn't notice a hard line where voice data stops.

 

LotRO and DDO does group-only chat. That's the best way to do it. A chatroom per party of up to 9. Special care can be taken to make a raid leader and each party's designated leader the only ones speaking in a raid, if you have that option. Then you can keep the bandwidth abuse low and people can hear orders barked over the tubes

 

Hm... a few more thoughts on bandwidth-saving.
One thing you could do is when a party is formed create a separate private room for them, the sole purpose of the room is to ferry voice communication. Depending on your multiplayer tech, you might even just be able to open up a P2P connection between party members (in my case, I'm using Player.IO for an upcoming game which will be pretty small in scale, I'll probably just create a separate voice chat room when a player hosts a room. I could just as easily use Unity Networking to open up a direct connection between the players)

 

Interested in a beta of this as well. Working on a game with a group of recent grads from my university to put our game design class we took into a video game. I have been looking into voice alternatives and was planning to port the Mumble client into Unity eventually....but this could save me the trouble

Would be using SmartFox as a transport layer for voice packets and since the project is open source you are more than welcome to steal the SmartFox code once I get it working

Sounds like you are handing out the source, if that is the case let me know if you don't want that public and I can package it and only put the package on the repo.

 

Well, I'm not worried about people distributing the source without permission. The chances of that happening are actually fairly low, and if it happens I haven't deleted any messages and have enough proof that I am indeed the original creator
Also, as far as I know Mumble uses the exact same codec I am using - Speex. So essentially you're getting at least comparable voice quality (I think in the C++ version of speex you get more quality control, but other than that nothing is different)

 

Throw me a Beta for this if you will. if thats ok with you ^_^

 

I believe that's the case as well.

 

Well I'll be going with the voice channels for party, guild, etc. That way I can mix the voice server side, make rules like if guild master / raid leader speaks others are faded down so orders can be barked, etc.

Unless you did p2p band width usage for free roaming voice would be insane. Imagine early Iron Forge scenario's in wow, where hundreds or even thousands of players would be hanging around in one plaza. And you definitely don't want to do p2p because then players can get each other's ip's. I'm certain that some people would resort to killing a router to disconnect an opponent in pvp.

Its worth a shot and I'll let you guys know what kind of bandwidth this consumes irl.

Cheers,
UnLogick

 

I've always wondered about bandwidth usage as well from solutions like this.

 

Correct. The biggest advantage of mumble is that it (and murmur) are already established. Not saying they are a superior product, but i would rather reuse something that has a proven track record then go through the pains of testing from alpha myself. Which is why I would gladly use your solution instead

 

It seems p2p is actually more likely to kill clients. Admittedly, the bandwidth requirement on your server(s) would need to be insane to handle free roaming voice when there is any sort of scale. However, image if you walked into a room and 100 separate peers all wanted to resolve a UDP port to use similtaneously? I think someone magically reverse engineering a peer's IP (which is hidden by NATs in most cases) would be the least of your worries...

Changing the voice mixing based on a set of rules is an interesting idea. I wonder how it would affect player experience if their party leader automatically was given 'command' when speaking? Would it support a more authoritative party structure in your game? Would it make players subconsciously less apt to argue/contradict a direct command from their raid leader? Essentially, you are allowing the leader to suppress freedom of speech...sorry I am straying. Interesting food for thought though

 

Yeah, I think for any moderately scaled MMO you'd probably just want to leave out free roaming chat and stick with party chat, and possibly direct chat (click on another player to talk to them or something). Free roaming is bound to generate an insane amount of data.

Also, for leader speaking, I think you could probably use the Gain for that, just check the isTalking function of the leader and if it is true set Gain of all other members to 0.5, otherwise 1.0. Maybe add a lerp to smooth it out.