[Released] Dissonance: Unity Voice Chat

Website | Documentation | Help Forum | Issue Tracker | Asset Store​

Dissonance is a voice chat asset for Unity which makes it easy to add high quality, low latency voice communication into your game no matter what network system you're using!

Features

Real time, high quality, low bandwidth voice communication provided by the Opus codec.

Written to be totally independent from the underlying network system. Use the built in networking support for UNet HLAPI, UNet LLAPI, Forge Remastered, Forge Classic, Photon Unity Networking, Photon BOLT or Dark Rift 2. You can also write your own integrations for any other networking system.

Positional voice playback makes other player voices sound like they're in the correct positions with no additional bandwidth or processing overhead. Even when you're using a VR spatializer plugin.

Collider based chat volumes automatically start and stop talking based on where players are in the scene. Attach the trigger to your player to get easy proximity voice chat.

Automatic voice detection algorithm starts and stops transmitting when players are speaking, no need for players to set up a push-to-talk key (PTT is also supported).

Priority Speakers automatically mute lower priority speakers when a high priority speaker is speaking.

All source code available when you buy the asset. All the C# source code for Dissonance as well as the C++ source code for the native dependencies (C++ is not included in the package, will be supplied upon request).

Requirements
Unity version 5.6+, Works with Windows, Linux, MacOS, iOS and Android.

 

I would be interested in trying this out for my Multiplayer FPS/TPS.

 

I'd like to be in on the beta. I am working on an educational VR app and I need voice chat. I have tried photon voice with very little luck.

 

Your website claims it is in the asset store but I could not find it there. Has it been released yet or when is it going to be released?

 

Hi Chuluney. I'm afraid we got a bit ahead of ourselves there with the website - it's currently being reviewed by Unity and will be available as soon as the review is done. I will post an update in this thread as well as on the website when it is finally available

 

I'm happy to announce that Dissonance is now available on the Unity asset store!

 

Do you know what would be required to integrate this with Photon Bolt?

 

Generally adding a new network backend should be relatively simple. Dissonance only requires 2 things from it's network back end: send an unreliable+unordered message (e.g. UDP) and send a reliable+ordered message (e.g. TCP). Everything else is done for you by Dissonance itself.

I haven't used Photon bolt, so unfortunately I can't say anything specific to it. It's definitely something we'd like to support in the future.

 

Dissonance now supports Android and macOS.

 

Why is this Unity 5.5+ only? We're on 5.4.4p3. You're shooting yourself in the foot if you're only supporting 5.5 and up.

 

Dissonance was developed using Unity 5.5 so that's all we supported for the initial release - we do intend to support older versions of Unity soon. In fact we already have a cloud build target for Unity 5.4 which works just fine

 

Hello, we are trying to use Dissonance but run into some issues :
- The communication is very intermittent, even if the VAD GUI says "Long speech" the other user sometimes doesn't receive anything.
- It also seems that at a certain point in the game, the communication ends definitely, with no apparent error in the console.
Can you help us ?

 

Hi RLasne, these sound like they may both be issues we're aware of.

We recently fixed an issue which caused voice to cut out completely in certain situations, that fix is currently in our test version and should be released to the store within a week (assuming our test group doesn't find anything which delays it). If you'd like to get your hands on that version right now send your invoice number to martin@placeholder-software.co.uk and I'll add you to the test list

Unfortunately the current VAD isn't great - it's ok once it's configured properly but it's extremely picky about your microphone and is very fiddly to tweak. We have a complete replacement coming soon (based on the fantastic webRTC VAD).

 

Dissonance 1.0.5 has just gone live on the asset store!

This version adds a new inspector for the playback component which displays realtime statistics on the playback system. This version also brings a number of important bugfixes for various issues which could cause desyncs, audio cutoff and crackly voice.

We're already hard at work on the next version which should be available in 4-5 weeks.

 

I'm trying to use Dissonance over UNET. I have 3 players connected:

• Host (Server with player): Doesn't hear anybody.
• Client 1 (Client with player): Hears Host and Client 2.
• Client 2 (Client with player): Hears Host and Client 1.

Any idea what the problem with the host might be? They are all using the same scene and the VoicePlayback prefab instances are duplicated and positioned (I attach them manually to our skeleton's head) correctly.

Sorry that I don't provide more information; I am unsure what exactly you need. Ask away!

 

That's not a problem I've seen anyone else have. Does it still happen if you don't fiddle with the playback instances? By UNET do you mean HLAPI or LLAPI?

What are you trying to achieve by moving the VoicePlayback prefabs around? Dissonance doesn't really let you do that - it recycles the playback instances as players leave and join so it completely manages the lifetime of the playback instances. If you're using the position tracking system (simply attach a component implementing IDissonancePlayer to whatever object you'd like sound to come from) you shouldn't need to change the prefabs at all to get positional audio in the right place.

The next version has some more debugging tools for seeing live information about packets in the voice playback pipeline. If you send me an email (martin@placeholder-software.co.uk) with your invoice number I can give you a test version with that stuff in it - that should help with debugging the problem

 

I mean HLAPI. And I don't really do much fiddling, I just attach this script (which should also work with the recycling, I hope) to the VoicePlayback prefab:

Code (CSharp):

1. public class DissonanceHeadAttacher : MonoBehaviour
2.     {
3.         VoicePlayback playback;
4.  
5.         void Awake()
6.         {
7.             playback = GetComponent<VoicePlayback>();
8.         }
9.  
10.         void OnDisable()
11.         {
12.             transform.parent = null;
13.         }
14.  
15.         void Update()
16.         {
17.             if (transform.parent == null)
18.                 return;
19.  
20.             foreach (var player in GameController.Instance.Players)
21.             {
22.                 if ((player.VoiceChatDissonanceClientId == null) || !player.VoiceChatDissonanceClientId.Equals(playback.PlayerName))
23.                     continue;
24.  
25.                 transform.parent = player.Skeleton.GetBoneTransform(ActorSkeletonBone.Tongue);
26.                 transform.localPosition = Vector3.zero;
27.                 transform.localRotation = Quaternion.identity;
28.                 return;
29.             }
30.         }
31.     }

The reason why we don't want to use positional tracking is that a) we already know the head position, so there is no need to send the data again and b) we want it to be strictly synchronized with the current head position, so the best way seems to be to just attach it.

I'll send you a mail later!

 

That script certainly doesn't seem like it would break anything - on the other hand we've already done almost exactly the same thing for you! The position tracking system built into Dissonance doesn't ever transmit positions and rotations over the network, instead it asks the *local* position tracking component where the player is and moves the playback instance to the same place. In your case you could implement IDissonancePlayer to return the bone transform for the tongue and Dissonance would do the rest of the work for you.

I'll ask you for some more details by email once you have the new version to help debugging

 

Okay! It'll be a few days until I can get back to this project, so don't worry if you don't hear from me immediately.

And I'm looking forward to new debugging ways. It certainly is weird: On the host, DissonanceComs shows the client player as "speaking" and "DEBUG [Dissonance : Playback] Player Playback: Began playback of speech session" and "DEBUG [Dissonance : Playback] Player Playback: Speech session complete" is logged, but the host just can't hear anything.

 

We are having numerous issues with dissonance right now. After a few minutes, the voice completely cuts out.

I've created a new scene without any of our other code/prefabs to test from scratch, but I'm having trouble even getting any audio through.

My current setup:
Default NetworkManager with HlapiPlayerTracker in spawnable prefabs
DissonanceSetup prefab with defaults, Added Voice Broadcast and Voice Receipt triggers with defaults

This is all the HLAPI quickstart instructed me to do, but when I connect over the network, I am hearing no audio... it seems like Unity would require an AudioSource to pipe the audio out to the speakers. Do I need to have a VoicePlayback object in the scene?

 

Hi will this work with Photon Bolt or is it just Photon PUN

 

> Photon Bolt

At the moment we don't support Photon Bolt - it's just PUN. However a few people have requested this so it's definitely something we'll be looking at soon.

> After a few minutes, the voice completely cuts out.

Could you send me (martin@placeholder-software.co.uk) your test scene and I'll have a look at what's wrong - it's probably just something that's not properly documented. Once we have that fixed up we can see if the cutout problem can be reproduced.

Just rereading through the quickstart docs I guess I know what the problem is. They're written assuming you already have a scene with functional networking - if Dissonance initialises in a scene *before* the networking things will probably break! That's probably something we can handle a bit better

> Unity would require an AudioSource ... Do I need to have a VoicePlayback object in the scene?

Nope. Dissonance instantiates a per player prefab which it uses to play the audio.

 

Hello, I recently bought Dissonance for us in my mobile project. On the asset store page it says that both Android and iOS is supported, but I am unable to run iOS in either the editor or on a device.

I've tested both the Demo scene as well as my own project using standalone as the platform and everything was working fine. When I switch Platform to iOS I get the following errors inside the editor:


And when I run the project on a device (iPhone 7 iOS 10.2) the following exception occurs almost immediately:


Are there any additional setup steps needed for iOS? It would be great if I got this asset to work so I wouldn't have to rely on Photon for Voice chat and Networking. This project is currently set up with Unet HLAPI.

 

Hi Shiratori,

I know there is an extra step to build for iOS - but I don't know exactly what it is! I've poked Tom (the other half of Placeholder Software) he developed all of the iOS support for Dissonance and he'll be able to answer your question.

 

Shiratori,

Your first issue, the crash when running in editor, is a bug with our dll imports for iOS. We have fixed this internally, but if you want to get your copy working right away then you need to replace all occurrences of "#if UNITY_IOS" with "#if UNITY_IOS && !UNITY_EDITOR" in SpeexDspNatives.cs and Opus.cs.

As to your second issue, we are having trouble reproducing the issue. Any more information would help us greatly:

• What version of Unity are you running?
• Do you have both "De-Noise" and "Automatic Gain Control" disabled in Voice Settings (in the Dissonance Comms component)?
• Have you been able to reproduce the issue in another project? If so, can you send us the reproduction?

 

Thanks for your answer. The fix for the first issue works great, thanks.

I'm still having issues with the build for the device though.
I have tried both with and without De-noise and Automatic Gain enabled. The first screen shot I sent was with both enabled. Now that I tried again when having both disabled I get another, very cryptic Bad Access Exception:


I'm running Unity 5.2.2f1, XCode 8.2.1.

I haven't had time trying it on another project, but I can do that this weekend and get back to you.

 

I just set up a brand new project with only Dissonance imported and it worked fine on the device. I'm currently debugging what differences exist between this and my real project.

I'll get back to you with my findings.

 

Dissonance 1.0.6 just went live on the asset store. Check out the complete changelog (also included in the package) here.

 

+ ​

Dissonance 1.0.7 just went live on the asset store. This is a very exciting release because we're releasing SalsaDissonanceLink - an integration we built in collaboration with Crazy Minnow Studio - to enable real time lip synchronisation for Dissonance speakers using SALSA lip sync!

The integration is freely available to anyone who owns both Dissonance and SALSA. Visit the SALSA website to download the integration and to get detailed documentation on how to use it.

 

Dissonance 1.0.8 just went live on the asset store. This release finally brings the brand new voice detector we've been working on for a long time - this one is based on the chromium webRTC voice detector which is widely considered to be the best there is. Check our website for a full changelog (also included in the asset itself).

 

The brand new voice detector is really great so far! In the old version we had lots of cases where we couldn't hear each other or the first syllable of a sentence was skipped, but now it's always working!

If anything it seems a bit too sensitive - it sometimes thinks somebody is speaking without them actually saying anything. I don't suppose there are any settings to adjust that?

 

I'm glad to hear the new VAD is working better

There is (in the underlying webRTC VAD) exactly one setting - you can set how "aggressive" the VAD is. Increasing this value basically increases the chance it classifies silence correctly, but also increases the chance it classifies some speech as silence. Given how much of a mess the configuration for the last VAD was we decided to be conservative and not expose that value yet, that might change though

 

Hey!

I found the issue I had to be the combination of both Dissonance and the latest Google Cardboard SDK. When both are in a fresh project the iOS bug I reported earlier appears. Now since 5.6 has native VR I've solved the issue for now.

I have two other points I've noticed though.

First of: HlapiPlayerStateTracker doesn't seem to be destroyed or recycled when players disconnect:



This is the case after two players disconnected and then reconnected. If I repeat the process of disconnecting and reconnecting the list gets longer.


The second issue I'm having is that the microphone is not working on some Android devices. I've tested it on a Lenovo phab 2 running android 6 and there the mic works. A Samsung S7 and an A3 plus 1 running android 7 both didn't work. The audio playback works fine, but nothing is sent from the devices. All devices tested with both headset and without. But there seems to be no mic picking up any sounds on the two devices that didn't work. On the Lenovo which does work the headset mic gets used when the headset is plugged in, otherwise the built in mic gets used.

Log output using adb reveals nothing strange.

This is the output from Samsung S7



It seems the mic gets started without any errors.

Any thoughts on how to approach this?

EDIT: I should mention that I'm using 1.0.8 of Dissonance, so maybe it has something to do with the new VAD? Unfortunately I didn't test with the failing devices before 1.0.8 was released.

 

Hi Shiratori,

Good job tracking that first bug down to the cardboard SDK - I've opened up an internal issue to investigate that that one.

For the HLAPI player state tracker bug that's very odd - I just tested this out and I can't reproduce it. There's something else odd about the screenshot you show - the state tracker prefabs should be spawned as children of the Dissonance game object but yours appear to be in the root. Check that you've followed section 1a of the HLAPI docs, not setting up that 'registered spawnable prefab' is a common mistake. If that doesn't fix it could you open up an issue on the issue tracker and post a few more things:
- Screenshot of the Network Manager inspector
- Screenshot of the HlapiPlayerTracker prefab inspector
- A complete log of a time this happened (on the DissonanceComms component click 'Diagnostic Settings' and change 'Core' and 'Network' to 'Trace' - this is very bad for performance so make sure to change them back to 'Info' afterwards).

For the microphone problem unfortunately I can't reproduce this because I don't own those devices! Once again could you open up an issue on the tracker and post a log. This time set 'Core' and 'Recording' to 'Trace'. The microphone capture is entirely using the Unity mic API so an incompatibility with certain phones would be odd!

The location of the logs for various unity platforms are documented here: https://docs.unity3d.com/Manual/LogFiles.html

 

Thanks for your answer. I probably did something wrong with the prefab and will investigate it.

As for the Android issue I've also found the issue after even more research. I'm happy to say that is has nothing to do with Dissonance.

I'm using unity 5.6 along with the native VR support for GoogleVR. It seems I've stumbled upon a bug regarding android permissions when using GoogleVR for Android. When having Virtual reality Support enabled Unity never checks for microphone permissions, so it can't use it. But apparently it also doesn't give any errors when trying to use the microphone without permission. It took me so long time to find it because I'm not used to building on Android

 

Ah that's an interesting problem with the VR support. I'm not certain how permissions are managed on Android but is it possible to prompt the phone to ask for permission before you enter "VR mode"? If there's no explicit way to request permission perhaps just by calling:

Microphone.Start(null);
Microphone.End(null);​

when you want to trigger the per missions check will work. It's particularly weird this only happens on certain phones :/

 

Yeah, I'm gonna work around it for now. I've also made a forum thread about the issue and I'll see what any Unity devs have to say about it.

Thanks for your time!

 

Regarding the other issue with weird state trackers. I'm not using the default Player spawning in Unet but my own.



As you can see I don't use the standard way of spawning the player, but rather override OnServerAddPlayer and spawn the Players myself.


This is my Player prefab I spawn in OnServerAddPlayer.

This is the spawning code.

Code (CSharp):

1. int spawnPointIndex = recycledSpawnpoints.Count > 0 ? recycledSpawnpoints.Dequeue() : lastUsedSpawnpoint++;
2.             Debug.Log(spawnPointIndex);
3.             usedSpawnpoints.Add(conn, spawnPointIndex);
4.             Spawnpoint spawnpoint = spawnpoints[spawnPointIndex];
5.             GameObject newPlayer = Instantiate(spawnPrefabs[1], spawnpoint.transform.position, spawnpoint.transform.rotation);
6.  
7.             NetworkServer.AddPlayerForConnection(conn, newPlayer, playerControllerId);

The HLAPITracker is the default one:



I'm most likely using the system the wrong way, but since I can't really use everything the default way I'm not sure how to do it properly.

 

Any news on that Bolt integration?

 

@auroxi No progress on bolt yet. It's definitely on our todo list!

@Shiratori Just to confirm - you're not manually spawning the HLAPI player tracker or interacting with it in any way from your own code? It should be spawned and destroyed completely automatically by the Dissonance HLAPI intergration (HlapiServer Line 44).

 

Correct. I haven't looked through the Dissonance code much. But you say it should be put as a child to the Dissonance GameObject? And in my case it's instead put under the scene root. Can you point my towards good debugging places? Which class is responsible for managing the HLAPI player tracker? I imagine you are subscribing to OnServerDisconnect from somewhere? Or is that only for Objects derived from NetworkManager? I'm a bit new to Unet since I just started prototyping my project moving from Photon which I was unhappy with previously. And since I discovered Dissonance as an alternative to Photon Voice I was sold instantly.

Btw I just had a product demo with the Product owner and he enjoyed the stability and performance of the voice quite a lot, so kudos to you!

 

The HlapiServer is responsible for creating the trackers. When a new player joins the Dissonance session it sends a message to the server, the server then creates a player tracker (HlapiServer Line 44 - AddClient). The code this is pretty simple:

var go = (GameObject)Object.Instantiate(_trackerPrefab, _network.transform);

Which is why I'm confused about your stuff going into the root, if that code runs there's no way the trackers will end up in the root!

Unfortunately we can't use OnServerDisconnect, that is only for things derived from NetworkManager. This is what we use the tracker for instead - when the player they're tracking quits the HLAPI automatically destroys the tracker for that player. Dissonance watches for the object to be destroyed and uses that to track disconnects.

 

Alright, when I have some time I'll dig into the issue, maybe my setup does something weird and I've hit a corner case or something. I'll post my findings.

 

So I am using dissonance with Photon self-hosted server. I had it working previously but I updated Dissonance and PUN to the latest versions (Dissonance for the new VAD and PUN because of a bug fix). I went through my project and replaced all of the old PUN and Dissonance stuff (scripts/game objects) with the new stuff. Now when I run my project I am getting this error and voice is no longer being transmitted [Dissonance:Network] CommsProcessor: Not received ID from Dissonance server; skipping voice packet transmission. I am currently testing with Photon server set to local so everything is staying on my local network. I know that dissonance must be started after a network connection has been established and I have 2 scenes one where a connection is established to the server, a lobby is joined, and a room is created, then the user is loaded into a scene where dissonance starts up, so the full network connection is set up and the users are not longer moving from server to server. Has anyone else had this error/ has any recommendations for ways to remedy the situation?

 

Quick update to my post above I neglected to mention that I am using the Network Simulation toggle that is accessed by clicking on the Diagnostic Settings on the Dissonance Comms object in my scene. I have it set to between 109 and 120ms of delay with a .5% packet loss. When I disable the Network Simulation I do not get the error but the audio still does not appear to transmit and the console logs do not show any of the usual [Dissonance Comms] Writing to buffer, Number of packets, etc.. logs.

 

Hi Novavr,

Looks like you've got two issues.

First the "Not Received ID" error. Dissonance has a few phases of network setup:

1. Wait until the underlying network integration is running (your multiple scene setup means this will happen almost instantly).
2. Client sends a packet to the Dissonance server instance. The server instance runs on whichever instance PhotonNetwork.isMasterClient == true. I don't know much about self hosting Photon - is it possible you don't have a master peer so there is no Dissonance server instance anywhere?
3. Server sends back an ID to the new client instance and informs all other clients of the new peer.

Only once all three steps are complete can a Dissonance peer send any voice. The "Not received ID" error you're getting indicates that step three hasn't happened, but of course it could be any previous step that the real problem. Check that in exactly one of your scenes the PhotonCommsNetwork inspector shows "Mode: Server & Client".

The second issue is something being different when the network simulation is enabled. That's probably a bug in the Dissonance network simulation dropping packets which should be reliable, I'll open an issue to investigate that.

 

Greetings,

I am having trouble implementing Dissonance with Photon Network, after following the Quick Start tutorial.

Apparently the connection is working properly and the audio is being sent and received correctly.

But on the client, no audio is being played and additionally on the following line of DissonanceComms.cs:

Code (CSharp):

1. playback.SetFormat(new WaveFormat(1, _transmissionPipeline.Encoder.SampleRate), (uint)_transmissionPipeline.Encoder.FrameSize);

Can I get some help? (Please see the image below, for more information).

 

Hi Andre,

This seems to be caused by the fact that your client does not have a microphone connected (as shown by the warnings in the image of the console). The Dissonance playback system queries the transmission system to detect what format it will be receiving audio in - since you have no mic, there is no transmission system, and thus the receiving system breaks.

This is a bug in Dissonance - I've created an issue on Github to track progress on fixing it.

 

That was it, everything is working perfectly fine now.

Cheers.

 

Hey there @Nyarlathothep !

I know you've been asked this a few times, but I was just wondering where you guys are on the Bolt Implementation (last mention was at the end of march )? If possible, do you guys have a time estimate on when it might be supported?

Thank you in advance!