Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Released: Visualizer Studio - A Music Visualization scripting package!

Discussion in 'Assets and Asset Store' started by AlteredReality, Aug 16, 2011.

  1. XY01

    XY01

    Joined:
    Jul 15, 2011
    Posts:
    24
    Hi, I just bought Visualizer studio and have built the example level using the microphone and it leaks memory, fast. Like 100k a second. Is there any fix for this?
     
  2. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    I just tested this on my Windows machine, both in the editor and standalone, and both with microphone on and off. I didn't see any memory leaks in this case by using the Unity Profiler. What platform are you running on and what are you using to determine you have a memory leak?
     
  3. XY01

    XY01

    Joined:
    Jul 15, 2011
    Posts:
    24
    Running a standalone build on Win7. It doesn't show in the editor, though when running the stand alone and watching the allocated memory in Windows Task Manager, it climbs steadily by ~100k a sec.
     
  4. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    I was able to reproduce this as well. I noticed it only happens when the microphone is being used. Unfortunately, I simply use Unity's built in support for the microphone as an audio source, so this issue seems like it is a Unity one. I did notice that the memory usage occasionally went back down, as if it was collecting garbage. So my guess is Unity's implementation of their microphone generates alot of garbage. Perhaps it would be worthwhile to contact Unity support and see if this is the case. Sorry I can't be of more help on this issue! :-(
     
  5. XY01

    XY01

    Joined:
    Jul 15, 2011
    Posts:
    24
    Bugger. Back to pumping FFT data from processing.
     
  6. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Hi Andy,

    Can you provide an example usage of the 'Target Property' filed in the Vis Material Property Modifier please? I've read the documentation but still have no idea ;)

    I am wondering if this can be used to change the alpha value of the Material.

    Thanks

    Paul
     
  7. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    It uses Unity material system to to set a float on the material. If it is setup to be used as a procedural material, it will call into this function...

    http://docs.unity3d.com/Documentation/ScriptReference/ProceduralMaterial.SetProceduralFloat.html

    And if not, it will call into this function...

    http://docs.unity3d.com/Documentation/ScriptReference/Material.SetFloat.html

    So, it is basically the name of a float variable on your shader. Also, I implemented some custom target material strings you can enter that directly change the Material.color, as follows...

    • colorred -- Changes the red component for the base material color
    • colorgreen -- Changes the green component for the base material color
    • colorblue -- Changes the blue component for the base material color
    • coloralpha -- Changes the alpha component for the base material color
    • color -- Changes all color components for the base material color

    Let me know if that helps!
     
  8. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Thanks for the Answer Andy! I will have a go this weekend.
     
  9. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    Cool, let me know if you have any questions at all!
     
  10. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Just to let you know that works great Andy! Exactly what I needed.

    I do have one other question. Sorry! ;)

    In the Vis Data Group components there are two options I am trying to get to work but with little success as I see no difference in response.

    I am trying to modify the Frequency Range Start and End Indexes, with the aim of making the frequency response ranges quite narrow.

    I am using the standard Visualizer Pro setup so there are: Bass, Mid, and High Vis Data Groups.

    An example would be: The bass to only react to frequencies between 100hz and 120hz. I am using a script to change them at run time.

    My script changes the values to: (Bass = 100hz to 120hz) - (Mid = 500hz to 600hz) - (High = 900hz to 1023hz)

    The values do change in the inspector panel, but when I send a test tone of say 550hz using a test tone generator, all objects seem to respond regardless.

    Am I even going about this the right way? Hope you can help Andy!

    Thanks

    Paul
     
  11. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    The problem you are seeing is likely related to leakage across the frequency bands with the FFT. This can be improved by adjusting window type on the vis manager. More info can be found here: http://docs.unity3d.com/Documentation/ScriptReference/FFTWindow.html

    The lower on the the list you go, the less leakage you will see, but it will be more computationally expensive.

    As far as the displayed frequency ranges, those are calculated based off of my knowledge of how FFT works. So they could be off, but I believe them to be accurate. I'm not an expert at sound theory, I'm just a good programmer that knows a little bit about sound visualization. :) Could you try a more accurate FFTWindow type and let me know if that gives you a more accurate result?
     
  12. fholm

    fholm

    Joined:
    Aug 20, 2011
    Posts:
    2,052
    Maybe this has been asked, but does it run on mobile (android and ios) ?
     
  13. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Ahh OK that starts to make more sense Andy, and thanks for taking the time to answer so quick. I am now in the process of setting up a clean, simple scene to test your suggestion outside of my project.

    One thing I have noticed is that the values you use seem a little odd to me. For example:

    Using the default VisualizerManager prefab, the vis data group for bass range is 0 to 10. However if that is in hz then that is incredibly low. Do those values actually represent hz?

    To further add to the example the vis data group for mid range is 15 to 125 which are still in the low frequency range and would be bass sounds.

    Also the max value seems to be 1023 which is at the start of high mid frequency range if in hz which seems a tad low.

    Feel free to tell me I am being stupid here lol! ;)

    Paul
     
    Last edited: Aug 31, 2012
  14. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Actually I am being stupid, but I'll share in case anyone else makes the same mistake. I have noticed in the debug panel, it shows the frequency ranges which are being used. So I guess my question is why do you use values which are different to hz and not just use true hz values in the frequency range indexes? I'm sure there is a logical reason as your skills are greater than mine! ;)

    Something else I noticed which may be why, is that when using audio sources with different sample rates (for example 41000hz or 96000hz) the index range values apply to different frequencies. Is there some kind of ratio at work? If so is there a formula to work them out quickly?

    Thanks

    Paul
     
    Last edited: Sep 1, 2012
  15. fholm

    fholm

    Joined:
    Aug 20, 2011
    Posts:
    2,052
    I realize that mobile support is possible, read the thread - but I have another question before I spend 100$ on the Pro version: What type of beat-detection do you do? I wonder because the song you have in your demo is *super* easy to detect beats on even using the most simple algorithms (not even any need for an FFT). How does this work with normal pop-songs with singing, rock, etc. where the beats are not as defined as the techno song you're using. I've been working on my own beat-detection, so that's why I wonder because the pop/rock/etc. songs are the hard ones, while doing techno/rap with very defined beats usually is super easy.
     
  16. fholm

    fholm

    Joined:
    Aug 20, 2011
    Posts:
    2,052
    I'd take a fair stab at the fact that the values you give are frequency bins in the FFT output and not Hz values.
     
  17. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Hi fholm / Hi Andy,

    Yeah, I think that is it. And I'm close to working out how it translates as it seems that each frequency range start value should be equal to the unit of resolution for a particular sample rate. So for example at 48000hz the resolution is 46.9hz.

    However testing shows something a bit different. The above is true for the frequency range start value. However the frequency range finish value is the resolution multiplied by the index value then add the resolution value.

    So for example:

    For a 48000hz file which will have a resolution of 46.9 hz:
    frequency range start Value 1 = 46.9hz ( 1 * 46.9hz)
    frequency range end Value 12 = 609.7hz ( 12 * 46.9hz + 46.9hz)

    This works true except for one thing. The greater the frequency range end Value is, then the ACTUAL value show seems to be less and less by small decimal amounts, when compared to the calcualtions shown on my pc calculator. I am wondering if there is some kind of rounding issue at work, or some other factor I have yet to discover.

    I realise that I mostly talking to myself here as Andy has not had a chance to reply yet, and he is probably glad about not having to

    explain this from scratch lol!

    If what I have worked out above is correct, the hopefully my ramblings will prove useful to other noobs like me.

    Andy - I have 2 questions at this moment:

    1 - Is there a rounding issue or is some other factor at play with regards to frequency range end Value?

    2 - Is there a way the get the Vis Microphone script to sample at 44100hz instead of 48000hz

    Thanks

    Paul
     
  18. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    Wow, lots of things to try and address! First off, yes Visualizer Studio works for mobile. However, I'd expect you'd want to adjust the FFT window type and size to be less expensive on the mobile devices.

    I don't do any heavy beat detection on the demo. It pretty much just is setup to visualize the peaks of the various frequency ranges that are configured. Basically I just tweak the threshold values until it feels good in most cases. Certain types of music is definitely easier to visualize than others.

    The index values are indeed the frequency bins of the FFT data. I then use some basic match to determine which frequencies each bin contains. Here is the code that is doing it for reference, incase you want more detail...

    Here is how the base frequency resolution is calculated...
    Code (csharp):
    1.  
    2. public void CalculateFrequencyResolution()
    3.   {
    4.     if (!m_bUseAudioListener  audioSource != null  audioSource.clip != null)
    5.     {
    6.       m_nFrequencyRange = audioSource.clip.frequency;
    7.       m_fFrequencyResolution = (float)m_nFrequencyRange / (float)windowSize;
    8.     }
    9.     else
    10.     {
    11.       m_fFrequencyResolution = 0;
    12.     }
    13.   }
    14.  
    And for the data groups, this is how the frequency ranges is calculated...
    Code (csharp):
    1.  
    2.       //calc frequencies
    3.       float startFreq = m_oVisManager != null ? (float)frequencyRangeStartIndex * m_oVisManager.FrequencyResolution : 0;
    4.       float endFreq = m_oVisManager != null ? (float)(frequencyRangeEndIndex + 1) * m_oVisManager.FrequencyResolution : 0;
    5.  
    So basically, to determine the frequency ranges, I just calcuate the resolution based off of the audio source. And then I apply the resolution to the frequency bins that the data groups use. Does that make sense? From what I understood of it, this was the correct way to estimate the frequency range.

    Also, for the microphone, it uses the Unity output sample rate for it's frequency. http://docs.unity3d.com/Documentation/ScriptReference/AudioSettings-outputSampleRate.html

    Does that answer all the questions at all?
     
  19. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Hey Andy! Great fielding of the questions ;)

    It all makes sense except for some reason my output is stuck at 48000hz. Everything hardware wise is set for 44100hz both input and output, or am I not understanding the unity output sample rate correctly. Is it that the default output for Unity is 48000hz? Sorry there are times when the Unity Documentation just confuses me more.

    Thanks

    Paul

    P.S. If I had bought this through the Unity Store I would be leaving a 5 star *must buy* review.!
     
    Last edited: Sep 2, 2012
  20. rollingdjs2

    rollingdjs2

    Joined:
    Jan 17, 2012
    Posts:
    4
    Is there a way of getting visualizer studio to accept fft info from iPod audio?
     
  21. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    Honestly, i'm not sure where exactly that value is set. I just followed an example of how to get the microphone working, and it showed them using that system value for the sample rate.

    And thank you for the kind remarks about VS! :)
     
  22. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
  23. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    No problem Andy. I am having so much fun with it!

    Another question. Is there a way to send a value to my own script? For example I use iTween for some stuff, and it would be brilliant to be able to send a new value to it.
     
  24. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    Yeah, you can definitely do that! You'd just have to create a custom modifier/trigger script. You could use one of the property modifier/trigger scripts as an example/starting point. Let me know if you have any questions about it.
     
  25. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Great news and I have started looking into it. I've duplicated and renamed the GameObjectPropertyModifierEditor and the GameObjectPropertyModifier. I have it showing up in the dropdown menu, and it applies OK, but obviously has the settings for the GameObject version. I now need to learn how to remove those settings and add my own.

    At the moment all I have is a script called TestTargetForValueChange with a single Var called makeMeChange, and will try to target that as a test. Any hints would be appreciated.

    My programming skills are at advanced novice level at the moment and they are all based in UnityScript which is why I struggle a bit.


    Thanks

    Paul
     
    Last edited: Sep 7, 2012
  26. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    Extending VS definitely requires some programming, as I had set it up to give a good amount of base scripts, while enabling custom scripts. Just keep in mind that for each custom script you make, you'll need to create and editor script to go along with it.
     
  27. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
  28. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Hi Andy,

    I've returned back to Visualizer Studio and need your help/advice.

    I am using the VisMessage trigger and changing the properties of it at run time via my own script. I have no problems changing most of the properties except for the Controller selection. I need to be able to change which controller is being used, for example from Bass to High.

    Below is an example of code. Where is am successfully changing the triggerReactivateDelay to 1.5, but I am unable to change the controller value to either of the Bass,High,Mid settings. I understand that you don't do JS but you will hopefully still get the idea from the following:

    Code (csharp):
    1.  
    2. var visStudio : Component;
    3.  
    4. function Start () {
    5.    
    6.     visStudio.triggerReactivateDelay = 1.5; //This works   
    7.     visStudio.Controller = 2; //Not working
    8.     visStudio.Controller = "Mid"; //Not Working
    9. print(visStudio.Controller); //This works
    10. }
    11.  
    12.  
    I've tried all sorts of combinations etc and looked at the source but can't seem to figure it out. Any ideas?

    Also to anyone thinking of getting this component ... do it! It's by far the component I have had most fun with in Unity and now building a game around it.

    Thanks

    Paul
     
    Last edited: Feb 2, 2013
  29. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    UFO Hunter, here is some code of pretty much what you'll need to do (in C#)

    Code (csharp):
    1.  
    2.         VisFrequencyController[] controllers = (VisFrequencyController[])GameObject.FindSceneObjectsOfType(typeof(VisFrequencyController));
    3.         for (int i = 0; i < controllers.Length; i++)
    4.         {
    5.             if (controllers[i].controllerName == "Bass")
    6.             {
    7.                 trigger.Controller = controllers[i];
    8.                 break;
    9.             }
    10.         }
    11.  
    Not exactly sure of the correct syntax of that in Javascript, but it should give you the general idea of what you need to do. Let me know if that works out for you.
     
  30. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Hi Andy!

    Got it working, you sir are a legend! For anyone else using this with UnityScript, here is a working version used in conjunction with my script example above:

    Code (csharp):
    1.  
    2.     var controllers : Component[] = GameObject.FindSceneObjectsOfType(Component);
    3.             for (i = 0; i < controllers.Length; i++)
    4.             {
    5.                 if (controllers[i].controllerName == "High")
    6.                 {
    7.                     visStudio.Controller = controllers[i];
    8.                     break;
    9.                 }
    10.             }
    11.  
    Thanks for the help Andy!

    Paul
     
    Last edited: Feb 5, 2013
  31. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    Hi Andy,

    I am using the VisMessageTrigger extensively and noticing something a little weird. Everything starts fine but overtime the values passed by the "Difference" message parameter gets less and less until eventually there is no output. This happens on multiple scripts. Any ideas?

    Thanks

    Paul
     
  32. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    I looked around in the code some, and I don't see anything that would cause this in the code. Does your input audio tend to have less differences over time? It is simply the difference of the previous and current frames, so if your audio levels out, then this would level out as well. Beyond that, I can't think of any reason it would do this. Maybe some more info on how you are using it would be helpful to try and figure this out. Feel free to reply here, or send an email to alteredrealityent@yahoo.com, if you'd like to provide me with more info.
     
  33. code-blep

    code-blep

    Joined:
    Oct 1, 2010
    Posts:
    308
    I'll keep looking into it Andy. It is more than likely something funky my end. To be honest VisStudio has been rock solid so it probably is me. I'll let you now.

    Thanks

    Paul
     
  34. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    Ok, let me know if you need help with anything else at all.
     
  35. Next Beat Games

    Next Beat Games

    Joined:
    Dec 18, 2012
    Posts:
    7
    This is fantasic. I am making a mobile music game and am trying to squeeze out every last drop of performance so I have a tiny custom 4 function way of getting what I want: a simple DJ Mixer's channel volume indicator. I will look into this as device hardware develops. Thanks for this!
     
  36. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    Awesome, glad to hear you like Visualizer Studio! Let me know if you have any questions at all.
     
  37. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    Last edited: May 8, 2013
  38. Tripwire

    Tripwire

    Joined:
    Oct 12, 2010
    Posts:
    442
    Hi,

    I have bought the Visualizer studio from the asset store. But i have a question, i'm making a minigame where the music is lighting up a GameObject, and when the GameObject lights up the player has to tap a drum. Is Visualizer Studio accurate enough to for instance only trigger on bass sounds? Because when i use the sample sound and set it to bass it's not only triggering to bass but to some other sounds too.

    Thx!
     
  39. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    There is a lot that can be done to tweak the settings so that an object will mostly only trigger on bass. The FFT used to get the base visualization data has some inherent bleed over between frequencies. However, you can adjust the window size to give more granularity. As well, you can set the window type to blackman harris, which has less bleed over. Increasing either of these settings does increase the cost of running Visualizer Studio.

    To be honest though, I wouldn't recommend relying on visualization data to drive rhythm based gameplay. For this, I would recommend using some sort of predetermined timing to que the player when to hit the drum. That way you can count on it always happening. I'd mostly use visualization data for auxiliary gameplay components, or for effects tying the music to your game visually.
     
  40. yuno117

    yuno117

    Joined:
    Apr 1, 2009
    Posts:
    6
    We created a game for Android called GelaTennis, (check it out here, It's free)

    We used your plugin, very cool by the way; however, the game is requesting record audio and microphone permissions (added by Unity on build). This is unwanted and is freaking out some of our users. We suspect it's coming from your plugin, can you help us out?
     
  41. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    Awesome, glad to hear you've released a game using our plug-in! :)

    As for the microphone warning, you should be able to get rid of that by getting rid of VisMicrophone.cs. At the very least, make sure it's not used in any level. That should do it for you. Let me know if that helps!
     
  42. Play_Edu

    Play_Edu

    Joined:
    Jun 10, 2012
    Posts:
    722
    nice work
     
  43. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    Thanks! :D
     
  44. Tehelee

    Tehelee

    Joined:
    Jan 26, 2013
    Posts:
    12
    Hey Andy!
    I've purchased your plugin through the asset store, but I have a few questions:

    1. How can I get a Data Group's spectrum range as an array? I wouldn't like to retrieve from the manager then splice it up manually each frame if you're doing that.

    2. Could you explain the "Value Types" of VisFrequencyController's Sub Value Type and Final Value Type?
    Also why is there a sub then a final? Wouldn't it make sense to have just a value type? The VisFrequencyController is very undocumented.
     
  45. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    I'm sorry for the delay in response. It's been a busy week at work getting ready for the break. Answers below...

    1. There is no way currently to access a Data Group's data directly. It was set up to be mostly a closed system so users didn't have to worry about the underlying data calculations. However, I am planning on releasing a plugin to expand some functionality of Visualizer Studio. As part of this release, I will be updating parts of the core VS to support the plugin. While working on this, I'll look into exposing access to the data groups directly (this would be released as an update to VS).

    2. Here is the documentation of it from the How-To doc that Visualizer Studio comes with:

    It's relatively brief because it focuses simply on what that implementation of a controller does. How a controller works is explained earlier in the document. Basically, those two value types define how it aggregates the data of the data group. For example, it could use the Average of the Maximum of each sub data group. So the SubDataType defines what value to pull from all the sub data groups. And the FinalDataType defines what value to pull from the results of the values grabbed from the sub data groups.

    Let me know if you have any other questions at all.
     
  46. silentslack

    silentslack

    Joined:
    Apr 5, 2013
    Posts:
    391
    Hi buddy,

    Quick question: Could this detect the BPM of a track loaded at runtime?

    Thanks
     
  47. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397

    There is no built in functionality to calculate BPM. I'm not planning on adding the functionality. However, with the tools with Visualizer Studio, you could likely estimate the BPM of a song using a VS trigger hooked up to the bass and counting the number of times it is triggered. I've not tried this, but I'd imagine the accuracy of this would vary based on the song.

    Let me know if this helps, or if you have any other questions.
     
  48. meansnothing

    meansnothing

    Joined:
    Dec 13, 2013
    Posts:
    3
    Thanks for the cool plugin AlteredReality! I bought the asset a week or two a go and have been playing with it. Have you noticed any lag using it for larger objects? I tried using it to move a whole mountain up and down and it basically makes the game un-playable. Maybe it's just not meant to move larger objects? Thanks again, it's still pretty cool!
     
  49. AlteredReality

    AlteredReality

    Joined:
    Aug 15, 2011
    Posts:
    397
    I haven't really noticed any lag in my tests. I think the issue you are seeing is because of what you are moving. The code that actually moves the objects around is very simple, and would not cause that kind of slow down. I'd think that moving the mountain in and off itself is the problem. You could confirm this by making a simple script to move the mountain around during the game.

    Let me know if you have any other questions.
     
  50. sonicviz

    sonicviz

    Joined:
    May 19, 2009
    Posts:
    1,051
    Hi,
    Is VS going to be upgraded to work with Shuriken particle system?

    ty!