Microphone input plugin for Windows/MacOS

I see people ask about this on here from time to time, so I hacked up a plugin over a weekend and figured I'd publish it. There's a native library so you'll need Unity Pro, and the scenes might need Unity 3.4.1; I've included all of the source code if anyone wants to hack on it.

Here's a quick link to the Unity3D project; you can have a hunt around the google code site if you want to check out the source: https://code.google.com/p/unity3d-wavein/wiki/Unity3DProject

 

Nice one!

Afraid Im not having any luck getting this working on OS X though:

Cannot get fft example to work. It says its found 1 input, but I see nothing, regardless of whether I have line-in or mic selected as input device in the OS. I don't see any error messages in unity, and the problem is the same with Lion and Snow Leopard. If I load an audio file into the audio player then I see bars at the bottom of the screen, but the central fft display never does anything.

Record demo crashes unity editor when using OS X Lion. It doesn't crash if I use Snow Leopard, but Im not sure if its actually recorded anything.

I haven't tried rebuilding the plugin, just using the downloaded binary version.

Any ideas?

 

I don't have Lion, so I don't know if the plugin works there. From what you've said, I guess maybe not. The Mac version is kind of an afterthought I'm afraid. The Windows version should be fairly sturdy

Two things:
1) You'll need to call StartDevice etc. yourself in the FFT scene; really there's a missing component that wasn't mine to release with the rest of the project. How that worked was, the missing component found a WaveInFFT component on the same object using GetComponent, then called StartDevice in 'Start' to start receiving samples. I guess you could call StartDevice from a 'Start' added to WaveInFFTDemo (be mindful of the order of calls there I guess, since WaveInPlugin needs to do some stuff in 'Start')

2) Look at the console output in the record demo; it should include the name of the file(s) that raw samples have been written to.

 

Just checked; the record demo works on my Snow Leopard Mac. Output is written in ~/Library/Caches/colinbarrettdotnet

 

Many thanks for the info.

If I just add the line waveInFFT.StartDevice(0); to the end of the Awake function in WaveInFFTDemo.cs then it works!

Only on Snow Leopard though, the above crashes Lion in the same way the record demo did. I doubt I have the skills to fix this but I will investigate a bit anyway.

 

I installed Lion to have a look at this:

1) A standalone built with the 'record' scene doesn't crash (actually appears to work fine)

2) The editor does crash; worse, the Unity bug reporter comes up but the editor hangs up and has to be force-quit so there's no call stack for me to look at. Even worse, for whatever asinine reason, I can't attach the Xcode debugger to the Unity application (I've noticed this before, and I suspect it's a DRM thing; there's no such problem using Windows and MSVC).

So, you're not alone, but I don't know what to do to help you

 

Thanks for trying. I spent some hours looking at the source for the plugin, but I was unable to spot anything really obvious. Looking at editor log in console wasn't terribly illuminating to me, but this is what I got:

WaveInPlugin initialized; found 1 device(s)
UnityEngine.Debug:Internal_Log(Int32, String, Object)
UnityEngine.Debug:Log(Object)
UnityEngine.MonoBehaviourrint(Object)
WaveInPlugin:Start() (at Assets/Plugins/WaveInPlugin.cs:21)
WaveInRecord:Start() (at Assets/WaveInRecord.cs:19)

(Filename: Assets/Plugins/WaveInPlugin.cs Line: 21)

WaveInRecord started
UnityEngine.Debug:Internal_Log(Int32, String, Object)
UnityEngine.Debug:Log(Object)
UnityEngine.MonoBehaviourrint(Object)
WaveInRecord:StartRecording() (at Assets/WaveInRecord.cs:53)
WaveInRecord:Update() (at Assets/WaveInRecord.cs:36)

(Filename: Assets/WaveInRecord.cs Line: 53)

Receiving unhandled NULL exception
Launching bug reporter

*** Launching bug reporter due to crashbug.
Obtained 60 stack frames.
0 Unity 0x00fcb585 _Z17LaunchBugReporter13BugReportMode + 1077
1 libmono.0.dylib 0x020de6c2 mono_chain_signal + 76
2 libmono.0.dylib 0x02016a42 mono_sigsegv_signal_handler + 263
3 libsystem_c.dylib 0x9449d59b _sigtramp + 43
4 ??? 0xffffffff 0x0 + 4294967295
5 libmono.0.dylib 0x021ede03 GC_mark_some + 480
6 libmono.0.dylib 0x021e622a GC_stopped_mark + 467
7 libmono.0.dylib 0x021e6605 GC_try_to_collect_inner + 369
8 libmono.0.dylib 0x021e6793 GC_collect_or_expand + 134
9 libmono.0.dylib 0x021e6d41 GC_allocobj + 268
10 libmono.0.dylib 0x021eb127 GC_generic_malloc_inner + 176
11 libmono.0.dylib 0x021eb22a GC_generic_malloc + 92
12 libmono.0.dylib 0x021eb49f GC_malloc + 137
13 libmono.0.dylib 0x021668b8 mono_object_new_alloc_specific + 93
14 libmono.0.dylib 0x02166a13 mono_object_new_specific + 290
15 ??? 0x0892741c 0x0 + 143815708
16 ??? 0x0d69d91e 0x0 + 225040670
17 ??? 0x0d69d80d 0x0 + 225040397
18 ??? 0x0d69d6a2 0x0 + 225040034
19 ??? 0x0d69c7f0 0x0 + 225036272
20 ??? 0x0d69baad 0x0 + 225032877
21 ??? 0x0c0481ef 0x0 + 201621999
22 libmono.0.dylib 0x0201db6c mono_jit_compile_method + 1283
23 libmono.0.dylib 0x02165360 mono_runtime_invoke + 133
24 libmono.0.dylib 0x0216dad7 mono_runtime_invoke_array + 1711
25 libmono.0.dylib 0x0211b2ae mono_class_fill_runtime_generic_context + 20415
26 ??? 0x0cb9a568 0x0 + 213493096
27 ??? 0x0cb99c95 0x0 + 213490837
28 ??? 0x0d69b945 0x0 + 225032517
29 ??? 0x0cb9661d 0x0 + 213476893
30 ??? 0x0cb96587 0x0 + 213476743
31 ??? 0x0d693e54 0x0 + 225001044
32 ??? 0x0c0481ef 0x0 + 201621999
33 libmono.0.dylib 0x0201db6c mono_jit_compile_method + 1283
34 libmono.0.dylib 0x02165360 mono_runtime_invoke + 133
35 Unity 0x008dab3e _ZN13MonoBehaviour22DoCallInputEventMethodEPS_P10MonoMethodR10InputEventiPPvbbb + 334
36 Unity 0x008db9bc _ZN13MonoBehaviour5DoGUIER10InputEventbbbi + 124
37 Unity 0x00fde3cb _ZN7GUIView12OnInputEventER10InputEvent + 315
38 Unity 0x00fdf39d -[GUIOpenGLView renderRect:] + 237
39 Unity 0x00fc668a -[OpenGLView drawRect:] + 330
40 Unity 0x00fdcc3b _ZN7GUIView10RepaintAllEb + 427
41 Unity 0x00ee5254 _ZN11Application11UpdateSceneEb + 484
42 Unity 0x00ee5718 _ZN11Application19UpdateSceneIfNeededEv + 72
43 Unity 0x00eefe00 _ZN11Application9TickTimerEv + 672
44 Foundation 0x947673eb __NSFireTimer + 286
45 CoreFoundation 0x999eb256 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 22
46 CoreFoundation 0x999eabe7 __CFRunLoopDoTimer + 743
47 CoreFoundation 0x999c9ce0 __CFRunLoopRun + 1888
48 CoreFoundation 0x999c91ec CFRunLoopRunSpecific + 332
49 CoreFoundation 0x999c9098 CFRunLoopRunInMode + 120
50 HIToolbox 0x9af93487 RunCurrentEventLoopInMode + 318
51 HIToolbox 0x9af9adc3 ReceiveNextEventCommon + 381
52 HIToolbox 0x9af9ac32 BlockUntilNextEventMatchingListInMode + 88
53 AppKit 0x986418ec _DPSNextEvent + 678
54 AppKit 0x98641159 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 113
55 AppKit 0x9863d4cb -[NSApplication run] + 904
56 AppKit 0x988d0b54 NSApplicationMain + 1054
57 Unity 0x00fe19a5 main + 837
58 Unity 0x0003f36e _start + 216
59 Unity 0x0003f295 start + 41

 

That's consistent with what I see; the problem occurs somewhere in 'GC_mark_some'. It feels like this is a mono/Lion issue (the reverse p/invoke in the plugin is something that maybe not many other people are doing, and it's tricky with other threads calling into the mono runtime: it should work (and it appears to on other operating systems), but maybe there are problems on Lion. Anecdotally, I've heard that there are problems with older versions of mono on Lion. I can't find much in the way of detail about that though).

I logged a bug here: http://fogbugz.unity3d.com/default.asp?421543_9fqc7j96

 

Hey, thanks heaps for this, I am pretty stoked about getting it running since I am currently using a very round about method (Processing > UDP > Unity) to get the mic fft data.
I downloaded the project, on windows, had to upgrade it to 3.5 and I get an error saying:

DllNotFoundException: wavein
WaveInPlugin.Start () (at Assets/Plugins/WaveInPlugin.cs:31)
WaveInFFT.Start () (at Assets/Plugins/WaveInFFT.cs:106)

I can't see the .dll in the project either. Any help would be ace, thanks man

 

Actually I got it running, got the wrong version.
Though in the FFT demo I have to add a wave/mp3 to the Audio Source to get it to work?
Am I missing something for getting the mic to work?

 

I think, if I remember how that demo is supposed to work; you hook up an AudioSource to play sound through the speakers, and the already running WaveIn samples the playing sound. Then an FFT is calculated for both the playing sound and the recorded sound (my idea was that both of those FFTs should look similar, since they are generated by the same sound or close to it). But you shouldn't need to play a sound; you should be able to make a sound yourself and at least the FFT display of the microphone input should update (in that case there's no other FFT to display, since the computer isn't making the sound).

I thought- though I haven't checked- that microphone input was a built in feature of Unity 3.5? So maybe you wouldn't need my plugin at all anymore?