Koreographer - Audio Driven Events, Animation, and Gameplay

A user (@daville) posted the following question to this thread after the "new" forums went live, prior to being reverted:

This was the reply:

Yes! Well, near enough. There is a way to retrieve events within a specified range of time. This is done on a per-KoreographyTrack basis, however. Further, the time is specified in samples. See:

Code (CSharp):

1. string eventID;    // Retrieved or set.
2. Koreography someKoreography;    // Retrieved or set.
3. KoreographyTrack targetTrack = someKoreography.GetTrackByID(eventID);
4.  
5. // Look at time 10.5 seconds.
6. int targetTimeInSamples = (int)(10.5f * someKoreography.SampleRate);
7. int sampleRangeWindow = 100;
8.  
9. // Look in the range 10.5 seconds ±100 samples (provides a window).
10. targetTrack.GetEventsInRange(targetTimeInSamples - sampleRangeWindow, targetTimeInSamples + sampleRangeWindow);

Keep in mind that the GetEventsInRange method simply searches the list of Koreography Events within the Koreography Track in a linear fashion. You can alternatively use the GetEventsInRangeTracked method to cut down on this. Especially recommended if you're doing things sequentially!

 

Hey everyone! We just submitted v1.4.0 to the Asset Store for review! Here's a sneak peak at two of the new features we're really proud of:

Rhythm Game Demo (both Koreographer and Koreographer Professional Edition)

​

FFT Analysis (Koreographer Professional Edition only)

​

This and more coming soon!

 

Hey everyone, version 1.4.0 is out! Here's a list of some of the features and enhancements to look for:

• The Rhythm Game Demo! This is designed to show users how to use Koreographer to enable classic rhythm gameplay. The demo is mobile-ready and includes support for both touch and mouse input.
  
• The Master Audio Integration! This enables Professional Edition users who also have Master Audio to connect to Koreographer's event system. The Music Time APIs are also supported!
  
• FFT Analysis! Koreographer Professional Edition users now have access to the FFT Analysis system. Results of FFT analysis are stored in the new Spectrum Payload type which stores multiple spectra across time.
  
• MIDI Lyrics Support! Koreographer Professional Edition users can import Lyrics from MIDI files.

And for those who love details, here's a thorough list of features, fixes, and enhancements that made it into v1.4.0:

• [NEW] Rhythm Game Demo showing how to create a game with simple rhythm gameplay! This also provides an example of using the Event Delay functionality to overcome issues related to audio latency.
• [NEW] [Pro] Master Audio integration! Supports both Koreography Event triggering and the Music Time APIs.
• [NEW] [Pro] New FFT Audio Analysis method! This includes a new Payload type (Spectrum Payload) that is generated by the FFT analysis and can be queried at runtime for frequency spectrum information! Check out the new functionality by opening the Analysis Panel and clicking the FFT tab!
• [NEW] [Pro] Lyric Meta Event support added to the MIDI Converter.
• [NEW] The Koreography Editor no longer snaps at the beginning and end of a song to the edges of the screen; the waveform no longer jumps when pausing playback.
• [NEW] Lots of new functions added to the Music Time API! Timings can now be accessed in Samples, Seconds, or Beats, and, yes, there's a BPM accessor!
• [NEW] Tempo Sections now support smooth tempo transitions! If the new "Start New Measure" flag is unchecked, the time-in-beats will pick up where the previous Tempo Section left off. [Pro] The MIDI Converter now unchecks this option by default, properly matching the way MIDI tempo changes are handled.
• [NEW] Timing calculations use 'double' instead of 'float' for far greater precision.
• [NEW] Report to the user if a previous Koreography processing pass did not finish properly. This can occur if an exception is thrown from within a callback.
• [NEW] The Koreography Editor playhead can now be moved by beat as well as by measure (hold down the shift key while pressing left/right).
• [NEW] Support waveform rendering on Retina and HiDPI displays in the Koreography Editor (Unity 5.4+).
• [NEW] The waveform can now be "scratched" in the Koreography Editor by dragging with the middle mouse button during playback.
• [NEW] Koreographer's Music Players now have an IsPlaying property to quickly check the status of playback.
• [NEW] Koreographer's Music Players now support setting a target Koreographer component with which to interface.
• [NEW] Loaded Koreography can now be queried from a Koreographer component.
• [NEW] Improved the portability and performance of the KoreographyTrack event retrieval functions. Old versions marked as Obsolete.
• [NEW] Added APIs to the Koreography class to retrieve Sample Time and Sample Time Delta values that are concurrent with the current (or most recent) Koreography Processing pass.
• Major optimizations to the waveform rendering system resulting in less memory consumption and faster computation.
• Slightly faster zooming by optimizing the waveform cache system.
• Fix for unloading Koreography during a callback causing undefined/difficult to track down errors/issues.
• Fix Koreography Editor issues related to changing the AudioClip (especially when setting it to "None").
• Fix showing the Scene tab when adding Koreography to a field in the Inspector.
• Fix unexpected selection box appearing when clicking on controls in the Koreography Editor.
• Fix for unresponsive Transport Displays (LCD readouts) in the Koreography Editor when clicking to change mode.
• Fix drawing waveforms upside down.
• Fix calculating Curve Payload deltas when the first key was not at time 0.
• Fix Audio Visor logic that used the global Koreographer component, ignoring a specified target Koreographer component.
• [Pro] Fix RMS Audio Analysis results not immediately appearing in the Koreography Editor when generated.
• [Pro] Fix an issue where Audio Analysis would fail to mark a Koreography Track as dirty when generating events.
• Fix not resetting internal Koreography time tracking values when loading a Koreography.
• Customized the script execution order of Koreographer's Audio Players to get them to update early.
• No longer allow creating Koreography Events beyond the end of the AudioClip.
• Removed the version of the GetAllEventsInRange API that had been previously marked as Obsolete.
• Remove reliance upon 'decimal' type in Koreographer functions.

If you purchased a previous version of Koreographer or Koreographer Professional Edition, v1.4.0 is a free upgrade!

 

Hi I was working with your suggestion to use GetEventsInRange(), and everything worked fine.

After the upgrade it says that function is deprecated, and I should use the new one.



But... If I switch them, suddently I won't get any data



it seems as if the koreoEvt is empty...

So I don't know if that's a bug, but the Deprecated version still works fine for me so I'm leaving it as it is.

 

Hi David! Thanks for reaching out!

Out of curiosity, are you getting any errors in the console? I ask because the old (deprecated) version has been refactored to use the new one internally.

Are you initializing the koreoEvt list before sending it in? Technically it should look like this:

Code (CSharp):

1. // Somewhere in Start() or possibly right in the field definition for your class:
2. List<KoreographyEvent> koreoEvt = new List<KoreographyEvent>();
3.  
4. //...
5.  
6. // In your event checking function:
7. koreoEvt.Clear();    // Clear out any previous events in the list so we can reuse it!
8. targetTrack.GetEventsInRange(targetTimeInSamples - sampleRangeWindow, targetTimeInSamples + sampleRangeWindow, koreoEvt);

By initializing your list this way and clearing it you sidestep the need to allocate the List every time you make the call (the deprecated version does this internally).

Does this help?

 

i got this error :
MethodAccessException: Method `UnityEditor.EditorGUIUtility:get_pixelsPerPoint ()' is inaccessible from method `SonicBloom.Koreo.EditorUI.WaveDisplayrawBeatLinesForSection (UnityEngine.Rect,SonicBloom.Koreo.EditorUI.WaveDisplayState,SonicBloom.Koreo.TempoSectionDef,int,int,double&,int&)'

SonicBloom.Koreo.EditorUI.WaveDisplay.DrawBeatLines (Rect contentRect, SonicBloom.Koreo.EditorUI.WaveDisplayState displayState, SonicBloom.Koreo.Koreography koreo)
SonicBloom.Koreo.EditorUI.WaveDisplay.Draw (Rect displayRect, SonicBloom.Koreo.EditorUI.WaveDisplayState displayState, SonicBloom.Koreo.Koreography koreo, Boolean bShowPlayhead, System.Collections.Generic.List`1 selectedEvents)
SonicBloom.Koreo.EditorUI.KoreographyEditor.OnGUI ()
System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)

and it can't show the audio WaveDisplay in editor

 

Hi @thanhle! We'll help get this resolved!

What version of Unity are you using when running into this problem? The call to EditorGUIUtility.pixelsPerPoint only exists in versions of Koreographer downloaded with Unity 5.4.0 and up. This could conceivably happen if you then attempt to open the project with an older version of Unity, where that property existed but was marked internal.

I was able to reproduce the issue with a project containing the version of Koreographer downloaded with Unity 5.4.0. I opened this project in 5.3.6 and attempted to view the waveform in the editor. The result was exactly the error you posted (four of them, to be exact).

It's not the most clear thing in the world, but publishers can upload multiple versions of their assets to the Asset Store to deal with situations like this. In Koreographer's case, the package has variations for several versions of Unity, as described in the Asset Store pages:



The versions listed are the first compatible version and are forward compatible until the next uploaded version. Therefore, anyone using Unity 5.1.x, 5.2.x, or 5.3.x will end up using the package uploaded with Unity 5.1.0. Users on 4.5.5 will get the version uploaded from Unity 4.5.0. New versions are used whenever Unity makes some breaking changes or adds certain features that help with functionality (in the Unity 5.4.0 case, the changes allow proper Waveform handling on Retina displays).

If this describes your problem, the following steps should fix the error:

1. Open the Asset Store window in the version of Unity you wish to get working (⌘+9 for Mac, CTRL+9 for Win).
2. Go to the Download Manager.
3. Press the Update or Download button.
4. Once the download completes, press the Import button. If you're using a recent version of Unity, the import window should automatically import only the files that are actually different into the project.

Does this help with the issue?

 

Nice support ! i will try with a new version unity
Thank you

 

Hello,
Is there a possibility to change tempo of music (real audio, not midi) without changing the pitch in Koreographer? For example like in Audacity.

 

Koreographer does not provide this feature, no. Any tempo changes performed with the AudioSource-based audio player components included in Koreographer are performed using the pitch property.

There might be something that does what you're looking for in the Native Audio Plugins package provided by Unity. That package comes with several pre-built demos and one may contain what you're looking for.

Koreographer can be easily made to work with such a system once you've got the underlying audio stream modulation functionality working!

 

Ok, thank you. I'll check that out

 

Koreographer Professional Edition has been nominated for the 2016 Unity Awards!

Koreographer has been included as a finalist in the Asset Store Award and Golden Cube categories. We seriously couldn't be more surprised by this. With the quality of work that's become available on the asset store in the 2015-2016 timeframe, this is a big honor and we're extremely humbled by it.

All that said, we now have two requests:

1. For current users of Koreographer, we are putting together a video that will be shown with Unity's award ceremony sizzle reel. If you have a game or demo that you put together using Koreographer and would like it to possibly appear in this reel, please reach out to us! We'd love to chat about how to get some footage together!
2. For everyone, please vote for your favorite nominees! We'd of course love a vote in our favor but we'll be your friend even if you vote another way. We want you to be happy!

And one last big thanks to everyone who's helped us out with bug requests, feature requests, kudos, or even a passing thumbs up! Your feedback and support is what made this all possible! <3

Finally, Unity gave us this nifty badge to use. I think it's nice!

​

 

Hi and congrats with the nomination. I read the thread once and searched again for answers regarding the asset and whether it'll do what I'd like, but am still not sure so I'd better ask:

Does Koreographer support step sequencing out of the box?

Does the MIDI feature in the pro version play SF2 files?

 

Thanks!

Koreographer does not support either step sequencing or the playback of SF2 files as it is not intended for playback (the included Music Player components are convenience wrappers for Unity's built-in audio system). Koreographer isn't an audio playback tool, but an event and timing system. It is a proactive audio tool that is designed to enable message passing from audio back to other game systems. The MIDI features in Koreographer Professional Edition are limited to MIDI file conversion (into Koreography data).

I hope this is helpful!

 

Ok thanks, that clears it up a bit. Which if any asset would you recommend for step sequencing? Also, would you consider making a web demo of your rhythm game demo.

 

The only sequencer asset for Unity that we're aware of is Stepp, but we've never tried it so we can't really speak to its capabilities. If you do try it (or have tried it), we'd love to hear how things went!

We are considering a web demo build for the included Rhythm Game Demo, yes. We don't currently have a timeline for it.

 

this "key node" going down you create by array with script ?
how i can create it with playmaker ?

 

The colored rectangles? Those are NoteObjects that are spawned from a pool managed by the RhythmGameController. The LaneController instance (of which there is one for each note lane) spawns them internally based on the position of the music relative to a tracked list of KoreographyEvents. For a more in-depth overview, check out the Rhythm Game Demo Overview in the documentation (also included in the Rhythm Game Demo package).

I'm not sure what you mean by "create by array". They are instantiated by the RhythmGameController in script based on an archetype object that you can specify (a disabled object in the hierarchy, but a prefab would also work).

When you say "it" do you mean a NoteObject instance or the entire Rhythm Game Demo? The NoteObjects themselves you can spawn by using the Create Object PlayMaker action and specify a NoteObject prefab (or scene object).

Reproducing the entire Rhythm Game Demo in PlayMaker without using the demo scripts is a different matter entirely and far, far more involved as it would involve effectively reimplementing the functionality in the different classes contained in the demo scripts in PlayMaker FSMs. Have you attempted this yourself? If so, what have you tried and where are you stuck?

But perhaps I misinterpreted your question. If so, any further detail about what you're trying to do would be super helpful!

 

Hey everyone, version 1.4.1 just hit the asset store! This is primarily a maintenance release, with lots of bug fixes and optimizations. That said, we were able to squeeze in a few new features:

• Audio Source Visor API Overhaul! We realized that the Audio Source Visor, a component intended to be simple as simple as possible, was far too simple! We expanded its utility by providing some extra APIs to help with those pesky edge cases!
• New PlayMaker Action! We've added the Get Koreography Event Timing Offsets action to the PlayMaker integration to help access sub-frame timing information when an event is begins and ends. This will help with spawn offsetting and more!
• [Pro] Wwise Integration! We've been hard at work getting Koreographer working with the Wwise Unity integration (2016.2.0+). Big thanks to the Audiokinetic team for expanding their APIs to enable the Koreographer integration! This is a beta-level feature announcement: the integration is hot off the presses and may still be a little rough around the edges. Rest assured that we're committed to smoothing everything out: we've got your back and are hungry for feedback!
  • Important Note: The Wwise Integration is not included in the Asset Store package. Please reach out to us if you'd like to make use of this feature!

And here's the nitty-gritty details of everything that went into v1.4.1:

• [NEW] Overloaded the AudioVisor.ResyncTimings() API to take a specific sample time value. This helps guard against race conditions between the Main and Audio threads.
• [NEW] The AudioSourceVisor now has a ResyncTimings() API to enable notification about intentional changes to its AudioSource.
• [NEW] The AudioSourceVisor now has an [optional] "Target Audio Source" field in the Inspector. This removes the implicit requirement of having a single AudioSourceVisor per GameObject. To support this, the AudioSourceVisor no longer auto-adds an AudioSource to the GameObject when added in the Editor. If no AudioSource is specified in the "Target AudioSource" field and no AudioSource is attached to the same GameObject, the AudioSource will log a warning to the console and disable itself.
• [NEW] The AudioSourceVisor now has a ScheduledPlayTime property that enables proper event handling when using the AudioSource.PlayScheduled() APIs.
• [NEW] Brand new PlayMaker Action: Get Koreography Event Timing Offsets! This provides sub-frame timing accuracy for reported Koreography Events. Reported Timing Offsets are intended for OneOff events or only frames wherein the Start or End of a Span are processed.
• Improved the accuracy and reliability of internal timing calculations. This helps improve the reliability of event tracking when looping, as well as better accuracy when the engine is under heavy load.
• Fixes for the Event Delay system to ensure proper event ordering, as well as consistent and correct reporting if time slices.
• Fixes for event triggering when looping with Event Delay.
• Fix events triggering early when the MultiMusicPlayer is configured with a large Sync Play Delay.
• Fix to stop CurvePayload.GetValueAtDelta() from allocating memory.
• Fix AudioSourceVisor initialization incorrectly occurring in Start() instead of Awake().
• Fix the Event ID Tooltip for string fields marked with the [EventID] Attribute appearing over other fields and controls in the Inspector.
• Fix spamming the Console with errors in Unity 5.0+ when scrolling the waveform near the end of the loaded AudioClip in the Koreography Editor.
• Fix for certain Sample Delta functions being off by one sample.
• Very slight math optimizations in internal functionality (reduction in floating point division).
• Fixes to script API documentation (information shown by IntelliSense or equivalent).
• [Pro] Fix for the MIDI Converter not showing MIDI Tracks Converter that had only Lyrics in them. The "Channels" dropdown is not shown for MIDI Tracks that only contain Lyrics.
• [Pro] Fix to stop hiding the actual type of unhandled MIDI Status messages, providing better debug information with certain MIDI files.
• [Pro] Fix for the MIDI Converter incorrectly handling MIDI files with certain SysEx messages. This would cause the next MIDI Event's timing (as well as subsequent events) to be off by a large margin.
• [Pro] Fix script warnings in the Master Audio integration with newer versions of MonoDevelop.
• [Pro] Fix to properly support Master Audio's Gapless Playback and Crossfading playlist features.
• [Pro] Reorganized the SECTR Audio integration script hierarchy to better match other integrations.
• [Pro] Fix Fabric integration scripts polluting the global namespace. The Fabric visors are now properly in the SonicBloom.Koreo.Players.Fabric namespace.

If you purchased a previous version of Koreographer or Koreographer Professional Edition, v1.4.1 is a free upgrade!

 

Has anyone tried the RMS and FFT analysis? If I want to make a DDR type of game with songs that do have voices, will it still do a good job of auto generating events positions I can use for the steps or will I end up having to adjust them anyways?

 

The RMS and FFT analysis output is either regularly distributed or continuous - it doesn't [currently] do any "event picking". The purpose of the RMS is to provide a sense of the overall volume of a piece of audio at a given moment. If you output a Curve payload you could drive a light with, say, the volume of a voice for a piece of dialog. The FFT output is a Spectrum payload - a Span that contains a list of frequencies-over-time [included frequencies are dependent upon the setup]. We are currently researching the best way to enhance what can be done with these and more algorithms.

The MIDI Converter would likely be a better tool to use to create a DDR type of game.

 

Hi,
I have purchased Koreographer PRO because of the FFT analysis, but I'm not sure how I could use these FFT payloads?! Is there any example scene which shows how to use the created spectrum? Is it possible to create some kind of visualizer with FFT spectrum?

Another question: Could you give me some hints how you created the "Disco Dodgeball" room? Is there a example similar to this available? I need to build a music room with different elements, but I'm not sure if the RMS overall volume is enough to get a good and matching variation?! Did you use the MIDI converter?
I really would like to use the MIDI converter, but the choosen music is only available as mp3 or wav! Probably this is not possible, but maybe you know some tools to convert wav to midi?

 

Hello!

The FFT payloads (the SpectrumPayload, specifically) are used to pre-process the audio spectrum, akin to what Unity's GetSpectrumData method does at runtime. The difference is that you get framerate-independent stability (you are guaranteed to get the same set of data each play-through) with a memory-CPU tradeoff (the FFT Analysis window shows you the estimated storage size when you run the analysis).

An example scene that showcases an example use of the SpectrumPayload does not currently exist. That said, to make use of it, you would do something like this:

Code (CSharp):

1. // Ensure you have 'using SonicBloom.Koreo;' at the top of your script.
2.  
3. [EventID]
4. public string eventID;
5.  
6. float[] spectrumData;
7.  
8. void OnEnable()
9. {
10.     Koreographer.Instance.RegisterForEventsWithTime(eventID, OnKoreoEvent);
11. }
12.  
13. void OnKoreoEvent(KoreographyEvent koreoEvent, int sampleTime, int sampleDelta, DeltaSlice deltaSlice)
14. {
15.     if (koreoEvent.HasSpectrumPayload())
16.     {
17.         koreoEvent.GetSpectrumAtTime(sampleTime, ref spectrumData);
18.  
19.         // DO SOMETHING WITH CONTENTS OF THE FLOAT ARRAY "spectrumData"!
20.     }
21. }

The contents of this array can be used in exactly the same way as the data retrieved from the built-in AudioSource.GetSpectrumData function. For an example of how to use this data to create a visualization, I highly recommend checking out the tutorial in this forum post.

The Disco Dodgeball scene was created by 82apps for their game. As I understand it, they created several KoreographyTracks and added KoreographyEvents with the keyboard (E or Enter key) in the Koreography Editor as the audio played back. These events were then tied with some simple scripts to wall animations, light movements, and colors in the scenes. They had already set up most of the scene when they integrated Koreographer so it was mostly a job of creating the Koreography data to get the tight audio-visual synchronization and connecting the event system's callbacks to the logic that was already driving the visualizations in their scenes. I'm sure that the developer would be happy to describe in better detail how they did it if you asked them directly (on Twitter or email [click the Contact link on their website]). I can say, however, that the MIDI Converter was not used in this case.

The MIDI Converter is a very powerful tool for generating tight, editable markup for music. Definitely use it if you can. Unfortunately, there are no good tools that convert WAV (or equivalent sample-based data) to the MIDI format. You may find some programs available to do this kind of translation but they almost exclusively target "stem" tracks (single instrument recordings) as it's a much simpler problem. While technology in this area is improving, it still has a long way to go and is an active area of research. If you do find software that does a workable job of building a MIDI file representation of a mixed song, please let us know! That would be a hugely valuable piece of software for the community!

(We've not tried them but Amazing MIDI and WaoN are two free options you might try out for MIDI-to-WAV transcription. Someone posted a results video of an Amazing MIDI test that seem pretty messy [see: the massive number of notes that appear])

I hope this is helpful!

 

Great! Thank you @SonicBloomEric for your fast, detailed and helpful answer. Now I'm aware of those subjects.
Amazing MIDI was already on my radar and I will test it, but I doubt that it is able to create a useful midi file.

Looking forward for potential Spectrum Payload examples

 

Hi,

I was wondering if it would be possible to set-up custom payload types ? It would be super-useful to be able to associate a class (may be a scriptableObject) that should be create for an event and be able to set-up its public properties. The point would be to be able to send custom datas that are notre directly related to the music but more about game events that should be fire at a very specific moment of the track with specific values.

Thanks

 

TL;DR - Custom Payload types are available to Koreographer Professional Edition users. This requires Source Code access, however (see below for more on why).

Koreographer's internal system isn't currently set up to deal properly with ScriptableObject subclass instances. The problem stems from the fact that each ScriptableObject instance is itself an Asset. With all current Payload types, Koreographer creates a default version when you create a Payload. For a ScriptableObject (or a Prefab), you would probably want to create a new Payload object that stored a reference to an asset. The default version in this case would be a "null" reference. The open question, then, is how do you tell what to do with a UnityObject at runtime? Perhaps this is an implementation detail best left to the user in the callback...

As a note, you wouldn't want to create a new ScriptableObject instance with every Payload instance because each ScriptableObject needs to be serialized into its own asset file. You could modify the Koreography Editor (again, with source code access) to push such objects into the KoreographyTrack they are defined within but then you'd lose a lot of the flexibility you get with the ScriptableObjects.

But Why Source Code Access? Why Not Support Custom Payload Types?
We want to support custom payload types and even built the system to be as flexible as possible to support this out of the box in the Professional edition. The core problem here is serialization. The KoreographyTrack uses Serialization Callbacks to unshuffle payloads from within its internal Koreography Event list into type-specific lists. This enables Unity's serialization system to work its magic and save everything properly to disk (and simplifies Undo/Redo logic). However, this also means that the KoreographyTrack is "locked" when built into a DLL, which means there's no room for custom payload types. This limitation, of course, disappears with the Source Code version.

We are investigating ways to add this out of the box. Some [very] recent tests were promising but led to completely broken Undo support and lack of good icon use in the Project View (not a necessity, but a bummer to lose). Some other ideas but it will take time to try them out and make sure that we don't break anything.

If Unity ever properly supports polymorphism of pure objects in their serialization system (beyond the ScriptableObject and MonoBehaviour classes), this will be a cinch to get working.

 

Whoops! Meant to point out that there are other ways to accomplish this! You could create a list of ScriptableObjects or Prefabs in some location that acts as a map of options to use with Koreography Events. You could then use the Int Payload as an index into that list. Or even the Text Payload to specify a name (for enum-like operation). The Text Payload version would probably be less error-prone.

This would also give you the ability to specify different mappings for different scenes, say, and keep the Koreography data as-is.

Just an option!

 

Hi,

Thanks for the answer. With the pro version, does come along an example on how to add a custom payload ?

Thx

 

Not currently, no. We are happy to help you get moving, though, and outline the steps via email (which will likely turn into a guide). It's a pretty simple class to build and you typically start by duplicating a pre-existing one (Payload classes are amongst the simplest in the system).

 

Maybe there is a little bug with the visual presentation of the Koreography Editor timeline, or I'm doing something wrong.
If I create a new Koregraphy and assign a audio clip to it, everything is working fine. The issue comes when I add a track, then the waveform (with auto scroll, without autoscroll its the cursor only) jumps around left and right very quickly while playing, it feels annoying. The strange thing is, the included example tracks work just fine ?!
I use Unity 5.5 and Koreographer Pro 1.4.1.

 

Thank you so much for reporting this! We have investigated the issue and it turns out that Unity 5.5.0 introduced a bug in the Editor: the static Time fields are not properly reset to default values (typically zero) when the Editor starts up or after leaving play mode. This has been confirmed as a bug by Unity and we are preparing a bug report for their tracker right now.

This also highlighted an area of code in the Koreography Editor that was not working as expected! We have a fix prepared and will be pushing a new build ASAP. As of right now there are two options available to you in the meantime:

1. Download a previous version of Unity (recommended Unity 5.4.4) and use it for Koreography development. The data itself is 100% portable between all versions of Unity. In-game playback is unaffected by this bug (it is limited to the Koreography Editor).
2. Reach out to us and request an early fix build. This will be the one we will submit to the Asset Store. We will ask to verify your purchase through the invoice number Unity supplied you upon purchase.

I will respond again once we've submitted to the Asset Store.

Again, thank you very much @MS80 for pointing this out to us!

 

@MS80: While preparing a bug report for Unity we found a workaround for the issue in Unity 5.5.0. While Unity's static Time fields do not get reset to zero as expected, they are updated whenever the Unity Editor Layout changes. It is likely that simply launching the Koreography Editor for the first time results in just such a change with relatively large values (EditorWindows are instantiated ScriptableObjects - it can take some time to allocate them). Large Time.deltaTime values are at the heart of the issue.

To reduce/sidestep this issue, simply move the Koreography Editor window while audio playback is paused. This will cause a very small Time.deltaTime value to be stored and subsequently used. In our testing, simply moving the Koreography Editor a few pixels got rid of the jerky playback!

We are submitting a bug report to Unity and submitting a fixed version to the Asset Store. Thanks again for pointing this out!

 

Thanks @SonicBloomEric, you give great support
Your little sidestep "move editor window while playback" did not work for me, I already tried that, but it seems the issue disappears randomly from time to time?! However, I will reach out to you or wait for the update.

 

Thanks! I'm glad that I'm able to help!

Just to be sure, the playback is paused when you try the sidestep, yes? (Just want to make sure that it was just the copy-paste that missed the "paused" part!) We did find that it could take a few tries to get it to work...

We are going aiming to have the fix submitted by end of day today. With luck it should be available from the Asset Store promptly!

 

@MS80: The latest version of Koreographer and Koreographer Professional Edition (v1.4.2) has been released on the Asset Store! Please update in your project and let us know if this resolves the issue you reported! Thanks again for informing us of the issue!

 

Thanks for the prompt update! The issue has been fixed

 

Hi there,

I'm interested and toying with making a vr rhythm game.

I see the demo for a rhythm game is there, so that will be a great start.

Can I do simple choreography with Koreographer? E.g. have a character that dances in time to the music. How much can it do? E.g would all my animations need to be a certain bpm, or start and end in a certain way?

My kids have been playing a rhythm game that has a dancing character each and they are having a dance off by tapping at the right spots on screen at the right time. My intention is to recreate this but with nicer visual's and in VR.

Thanks

 

Hi!

The short answer is "Yes! Koreographer supports all of that!" The longer answers follow:

1. The Rhythm Game Demo not only includes usable "out-of-the-box" functionality, but it shows one example of how to do "lookahead" style functionality (bypassing the main event system). Should be very useful!
2. Koreographer is very good at driving music-synchronized animations. You don't put animations directly into the Koreography (Koreographer's data is called "Koreography"), but use your normal animations with the Animator component. The one small change you need to make is that you disable the Animator component (set enabled to false) and then call the Animator Component's Update function yourself from a custom behaviour and pass it beat time (which Koreographer can provide). When you call Play on the animation, you also provide a slight offset based on the beat or measure time to which you want to "anchor", ensuring that not only is the timing tied to the music, but that the animation feels right. Koreography events can easily be configured to set up "Dance Choreography" - you could set specific events into the music timeline at points to change the dance animation that's currently playing. (The script to control how the animations change would be custom designed on your end to make of the Koreography events you set up with Koreographer.)
   • The one change to your animations with the above outline is that you put "impact" keyframes on the seconds timeline. This is because you replace the timing clock with beats, so those seconds become beats at runtime! Very simple change to the workflow.

The experience you describe sounds fantastic! If you end up using Koreographer for this project, please feel free to reach out with any more questions if you find yourself stuck during implementation. Happy to help here, with the main Koreographer forum (no login required to post in most boards, though ads disappear when you do), or via the support email with more sensitive questions!

 

Hey guys! Wow, this asset looks incredible! Great work. I have one big question!

Will Koreographer play nicely with the upcoming Unity 2017 Timeline feature?
Sorry if I'm asking in the wrong place, or if this has been covered already.

( Timeline beta link: https://forum.unity3d.com/forums/timeline-cinemachine.127/ )

 

Hi!

We have not yet tested Timeline compatibility, but we do plan to support it with Koreographer. In early conversations with Unity's audio team, we were told that all APIs necessary for Koreographer to work would be present. We have held off from diving in as the APIs are still fluctuating greatly (it's still preview/experimental, of course!) - especially on the Audio side. That said, we have reached out to see what, if anything, could be done in the near term.

That said, the current Timeline preview build is based on Unity 5.6b6. This version is only partially compatible with Koreographer due to a bug that Unity introduced in version 5.6b5: the Editor looks to the wrong assembly for type information, breaking any serialization in assets built with earlier versions of Unity. It should be stressed that all new content generated in the latest betas does work correctly, but we cannot promise that it will continue to do so when Unity fixes their bug. We recommend that Koreographer users use Unity 5.6b4 until we get confirmation of a fix in an upcoming beta or RC.

I hope this is helpful! If/when we hear more information about the Timeline Audio APIs we'll be sure to report more!

 

Hey Guys,

I am having a really tough time figuring whats wrong. Everything was working fine but all of a sudden things in the Wave from view in Koreography track settings became really hard to deal with. The problem that I am facing is the white line that runs along the wave form view, when in play mode, that shows you where in the track is the audio is coming from, is always flickering. When I disable auto-track the white line flickers but when I enable it the whole wave starts to flicker. This makes it really hard for me to be more accurate when placing events.

How do I deal with this?

I have attached an image to show you what I mean.

 

Hi @Megalithic! Thanks for reaching out and reporting the issue. I'm very sorry to hear that you're having problems with the Koreographer interface.

This sounds like it may be related to a problem reported a few months ago by another user. May I ask which version of Koreographer you are using? Further, which version of Unity are you using?

 

wow didn't expect such a quick response.

So after reading the post that you resolved for the previous user, I updated the koreography asset and it is working fine! haha. If only I had known there was a newer version available. I wouldn't have spent a whole week trying to figure it out!

Thank you for your support!

 

@Megalithic You're very welcome! I'm happy that the update was able to resolve the issue for you

Best of luck with your projects and please don't hesitate to reach out again if you come across some other issue!

 

Hello again,

Since you guys are so active on the forums, I thought I would just come here and ask you directly instead of spending ages to find the solution.

I am trying to figure out a way to make koreography music player play from a specific sample or time in the track.

Code (CSharp):

1. void SomeRandomFunction( KoreographyEvent evt, int sampleTime, int sampleDelta, DeltaSlice
2. deltaSlice)

The above function only gives me the specific sample time. I was wondering how to make the music player play from a specific sample time that I choose.

I am using the standard version btw not the pro one.

I hope you understand my question.

 

No problem!

The Music Player components that come packaged with Koreographer do support setting playback time. The API you are looking for is SeekToSample(int targetSample). In order to make use of it, you need to add the following using directive to your script:

Code (CSharp):

1. using SonicBloom.Koreo.Players;

You will then want to add grab a reference to the specific music player component you're using and call the SeekToSample method. You can easily achieve this through the following two Unity-standard methods:

1. Add a public SimpleMusicPlayer or MultiMusicPlayer field to your script such that it appears in the Inspector and connect it in the Editor.
   
2. Use GetComponent<SimpleMusicPlayer>() or GetComponent<MultiMusicPlayer>().

The SeekToSample method will cause the music to jump to the sample position specified.

For more information, check out page 34 of the Koreographer User's Guide PDF in the Assets/Koreographer/Documentation folder (also available here).

I hope this helps!

 

This is top notch! Thank you so much.

I have using the getcomponent method but didn't know I had to add

Code (CSharp):

1. using SonicBloom.Koreo.Players;

You have been great!

 

Hi there,

Been looking for an answer to this but have had no success.

I've just upgraded to KoreographerPro (running on Unity 5.603f) for the audio analysis tools but while RMS works fine when analysing audio assigned to an asset, FFT keeps throwing the following error:

Code (CSharp):

1. IndexOutOfRangeException: Array index is out of range.
2. SonicBloom.Koreo.EditorUI.FFT2.run (System.Double[] xRe, System.Double[] xIm, Boolean inverse)
3. SonicBloom.Koreo.EditorUI.AnalysisPanel.DoFFT (Int32 startSample, Int32 endSample, Boolean bReplace)
4. SonicBloom.Koreo.EditorUI.AnalysisPanel.DoFFTGUI ()
5. SonicBloom.Koreo.EditorUI.AnalysisPanel.OnGUI ()
6. System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)
7. Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
8. System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:232)
9. System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MethodBase.cs:115)
10. UnityEditor.HostView.Invoke (System.String methodName, System.Object obj) (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:262)
11. UnityEditor.HostView.Invoke (System.String methodName) (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:255)
12. UnityEditor.HostView.OnGUI () (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:84)
13.  

Any idea what might be wrong?

 

Hi @keviny! Thanks for reaching out! I'm sorry to hear that you've run into this issue but I will do my best to help!

Hmm... this seems to be a new one. We ran the FFT using the audio file in the demo Koreography without issue in Unity 5.6.0f3.

A couple of questions for you:

1. What settings are you using to run the FFT analysis?
2. How long is the audio file that you're analyzing?
3. Are you able to run the FFT analysis with the

We'll continue to take a look at what might be going wrong (the error report is very helpful!), but more information as to the setup will be very useful!

 

Hey Eric, thanks for getting back to me!

I've attached the settings I'm using. The audio file is a 3:30min AIF file but I just tried it on StarlitKoreography in demos (Starlit Black audio file) and I get the same IndexOutOfRangeException error.

I'm running on Windows10 and the project is an UWP project. I've imported the WindowsRT package in plugins too.