Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Kinect plugin

Discussion in 'Made With Unity' started by bliprob, Nov 21, 2010.

  1. Sunny

    Sunny

    Joined:
    Mar 21, 2011
    Posts:
    9
    Hi, I am trying to figure out how to work with the Unity wrapper by tinkerer.

    I already have Nite and OpenNi installed and working.....

    So far i've tried to create a sphere and then assign the handgen.cs script to the sphere but i keep getting an error that says the Depth node was not found...What am i doing wrong? Can someone guide me to creating a simple app on windows that moves a sphere based on hand movements captured by the kinect
     
  2. diabloroxx

    diabloroxx

    Joined:
    Jan 20, 2010
    Posts:
    71
    Haha, yeah I did the brute force method couple of times. But I do not want to risk it any further as for some reason the clicking sounds were uneven from time to time. An expensive hardware to be tinkering around with :)
     
  3. Tripwire

    Tripwire

    Joined:
    Oct 12, 2010
    Posts:
    442
    So you fixed the whole missing references then?
     
  4. Sunny

    Sunny

    Joined:
    Mar 21, 2011
    Posts:
    9
    Amir told me to try working it on the unstable releases and it worked so problem solved!
     
  5. Arveee

    Arveee

    Joined:
    Mar 23, 2011
    Posts:
    1
    Hey guys,

    I'm new here and new with working with Unity 3D. And I had no idea where to ask my noobish question, So therefore I ask it here.

    I started using Unity 3D for a experiment with the Kinect. Probably not the best way of starting with Unity, but it was necessary for a school project. After playing around with the sample project in the unity wrapper I tried to load in a 3D model to replace the Soldiers in the project.

    I renamed all the joints to the ones named in the NITE.CS, as I was told to do by someone who worked with it before, but everytime I try to load the "game" nothing works.

    Is there a trick to get a different 3D model to work with the script or can someone point me in the right direction? I hope I explained enough and sorry for a lack of information, I'm totally new to all of this. Be gentle :D.
     
  6. Phoenixx

    Phoenixx

    Joined:
    Mar 23, 2011
    Posts:
    3
    You just renamed them? That's not enough, you have to drag the joints onto the corresponding variable (Right Arm, Left Arm,..) in the script, which is attached to the cube as far as i remember.
     
  7. Sunny

    Sunny

    Joined:
    Mar 21, 2011
    Posts:
    9
    Hi, I recently downloaded the Unstable release and it works fine but each time i reboot my computer it stops working again. Giving me the device is not connected error. Anyone got any solutions to this ? to why is this happening
     
  8. Tripwire

    Tripwire

    Joined:
    Oct 12, 2010
    Posts:
    442
    Which scene do you have working? I got the WalkRunIdleBlend scene working now with all the scripts on the right place. If you have the other scene's working can you tell me how you did it?
     
  9. bryanleister

    bryanleister

    Joined:
    Apr 28, 2009
    Posts:
    130
    This is so much fun to work with, thanks Amir for providing this wrapper. I have two questions that I was wondering if anyone had any advice on.

    1. Why does the example require the OpenNI.xml file to be outside the build? This is the line that I'm talking about:

    Code (csharp):
    1. private readonly string SAMPLE_XML_FILE = @".//OpenNI.xml";
    It requires that I put the xml file outside the Assets folder in the editor, and on the final Build I need to put the file in the directory of the .exe. I tried using a TextAsset and inserting the text but it generates errors.

    Alternatively, is there a better way to get the file path within the build? I'm not that familiar with path shortcuts on Windows, and the GetApplication path didn't seem to work. I actually much prefer to put the text file into the script as a game object so it's always there, but I don't get why it doesn't work.

    2. Does anyone know why the pose calibration would be slower in the game build versus the Editor? In the Editor, it will find my pose quickly, within a second or two of striking the pose. I create and run the final build and it will take about 5-10 seconds of standing still before the pose is calibrated.

    Bryan
     
  10. Sunny

    Sunny

    Joined:
    Mar 21, 2011
    Posts:
    9
    First to be to ensure you have the latest unstable build installed,
    Then open up the poi scene with unity
    Then go to the project folder and there is a poi package file double click it
    Unity will prompt you if you want to extract that package to your project,
    Click Import
    run the poi scene and i it should work.

    Let me know how it goes
     
  11. RIkki

    RIkki

    Joined:
    Feb 8, 2011
    Posts:
    8
  12. diabloroxx

    diabloroxx

    Joined:
    Jan 20, 2010
    Posts:
    71
    Is there a way to change the calibration and detection procedure of Kinect? In case a person is disabled and cannot move one of his arms, how can I alter the game to make the calibration?
     
  13. Tripwire

    Tripwire

    Joined:
    Oct 12, 2010
    Posts:
    442
    HI i tryed doing like you told me. But all i see is a bunch of flames flying around... i'm guessing the camera's position isn't right
     
  14. Phoenixx

    Phoenixx

    Joined:
    Mar 23, 2011
    Posts:
    3
    I guess it's not possible. Maybe you can do it, but you would have to change the algorithms concerning the detection procedure that come with OpenNI.
     
  15. Sunny

    Sunny

    Joined:
    Mar 21, 2011
    Posts:
    9
    that means its actually working fine, try calibrating yourself and the concept of the game is that you'll be holding 2 "maces" and you can go a destruction spree (destroying the pink boxes)...The reason for the camera being like that is that it gives a first person view e.g. try moving your shoulders you'd see that the model moves also...Alternatively you could just delete the camera which is placed in the soldiers->Spine1->Spine2-Spine3-> neck/head object. After deleting you could create another camera with the desired view

    Video example of the game : http://www.youtube.com/watch?v=wuFpyyZDiVw&playnext=1&list=PL26C1E6E9C835C05F
     
    Last edited: Mar 28, 2011
  16. sukku

    sukku

    Joined:
    Mar 30, 2011
    Posts:
    1
    Hi,

    I tried your project. WIth setPixels, I could get close to 30FPS. I am using non-pro version. Could you explain why I am not able to use c# interop with the non-pro version?

    -Sukku
     
  17. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,035
    I'm suprised you are getting 30 fps with setPixels, are you sure you've changed the #defines in the code to use the correct codepath? From memory the default codepath is using either C# interop and referencing the opengl functions directly or via the supplied plugin.

    If you've tried changing the define and only setPixels works, then that means C# interop is somehow crippled in non-Pro Unity. I've not been able to test this yet, i'm waiting for my Pro demo to run out on the test machine (all other machines have Pro installed already).
     
  18. kissofdeath

    kissofdeath

    Joined:
    Apr 11, 2011
    Posts:
    1
    Hello Hackers!

    I've gone through most of the discussion in this thread and I'm now throughly confused.

    I'd like to know what is the best (easiest to get working) solution for hacking on Kinect on Mac 10.5.8 and using Unity wrappers to make a fun game.

    A small compilation of what I've heard so far is:
    i) OpenNI and NITE from openni.org
    ii) libfreenect (i dont think this came up in this thread)
    iii) FAAST - only for windows?

    I've also heard about openframeworks (theo watson has a number of cool demos using it) which is cross platform.

    Does the Unity wrapper have any dependencies on what I choose?
    If so, what are the options?

    Thanks for your time!
    Raja
     
  19. TheRaider

    TheRaider

    Joined:
    Dec 5, 2010
    Posts:
    2,249
    I have been trying to get this working.

    I have installed OpenNI and NITE and pointed the sensor (appears in my device manager) however when I installed the sensor it said it couldn't register with OpenNI which I think is my problem. Does anyone have a fix?

    Currently when I load any samples it just loads the cmd line window and closes without doing anything.

    Advice please!
     
  20. diabloroxx

    diabloroxx

    Joined:
    Jan 20, 2010
    Posts:
    71
    Can someone tell me where I can find the Poses currently supported by OpenNI and that supported by the UnityInterface.dll file.
     
  21. Mr. Me

    Mr. Me

    Joined:
    Apr 21, 2011
    Posts:
    11
    Hi,

    im a bit new to Unity and especially to shaders, im trying to turn the black background in the openNI_MaskImageDepth shader into transparency. anyone knows how to do this?


    Mr. Me
     
  22. fisyher

    fisyher

    Joined:
    Apr 27, 2011
    Posts:
    8
    Hi,

    I'm new to Unity and currently involved in a project that requires me to use it with Kinect (and possibly other depth sensing cameras). The UnityInterface wrapper with OpenNI and NITE mentioned a few pages looks really good and seems exactly like what we needed. However, we are abit worried about whether it can still be used in the free version of Unity since it is stated in the Licenses page that Unity Free does not support plugins. We would like to avoid having to pay for a Pro version as much as possible since we are currently low on budget. Can someone confirm whether this is true for this case?
     
  23. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    You need pro.
     
  24. fisyher

    fisyher

    Joined:
    Apr 27, 2011
    Posts:
    8
    But according to the UnityWrapper website ( https://github.com/OpenNI/UnityWrapper ), the readme section states that "Both the PRO and Free editions are fully supported!" and yet Unity only feature plugins in the pro version. This is rather confusing. Is there someone who can clear me up on this?
     
  25. Phoenixx

    Phoenixx

    Joined:
    Mar 23, 2011
    Posts:
    3
    Hippocoder is wrong. It also works with the free version.
     
  26. fisyher

    fisyher

    Joined:
    Apr 27, 2011
    Posts:
    8
    Thanks for the reply, Phoenixx. If that is the case, then how does Unity Free differ from Unity Pro with regards to plugins?
     
  27. the_marsbar

    the_marsbar

    Joined:
    Nov 17, 2009
    Posts:
    38
    What I don't understand is, why don't people use the free SDK from Softkinetic? It's much faster than the plugins I've tried so far...
     
  28. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
    How long until there's a kinect plugin for Unity that works without installing any drivers or dependent software?

    On the Mac, I see fernLightning which I just plugged in the kinect and ran the app and it works.
    http://fernlightning.com/doku.php?id=randd:kinect

    http://gamedev.stackexchange.com/questions/4971/can-you-access-kinect-motion-sensing-from-xna

    http://research.microsoft.com/en-us/news/features/kinectforwindowssdk-022111.aspx
     
    Last edited: May 16, 2011
  29. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
  30. pablofuerteventura

    pablofuerteventura

    Joined:
    Oct 10, 2010
    Posts:
    12
    Hello,

    Im looking to develop an small game that consist on an Avatar that reproduces the movements of the player in front the motion recognition game.

    like this example: http://archvirtual.com/?p=3113

    Please,anyone think can develop something like that and what could be the budget and timeline?

    I will provide the 3D character, and ofcourse it will be a payed proyect: 50% when I can see working demo, other 50% when I get final version + source code.

    thanks and regards
     
  31. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
    Windows 7 - 32-bit unstable version: I was able to get the Unity Kinect example to run by installing the unstable versions of everything.

    Windows 7 - 64-bit unstable version: Still trying to get the OpenNUI samples to run and detect the 64-bit drivers.
     
    Last edited: May 28, 2011
  32. Eva Maria

    Eva Maria

    Joined:
    Dec 21, 2010
    Posts:
    85
    Are you planning to release a stable version e.g. at the asset store any time soon?
     
  33. Costanel

    Costanel

    Joined:
    May 31, 2011
    Posts:
    3
    I can't seem to rebuilt this Unity Wrapper at all, using as well VS 9 as 2010. It gives an error with XnSkeletonJoint and POSIX names. I'm not really known with Visual Studio, please help. :)

    Would love to have an example scene of the Kinect with Unity3D for a project, would save me a bunch.
     
  34. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
    Make sure you install all the unstable versions.

    Make sure you install the drivers. OpenNI-OpenNI-8c24e16\Platform\Win32\Driver\Bin

    Make sure VS is closed when you install everything. And maybe reboot to get the environment dirs to activate.

    Also in your project settings you can manually enter the paths for includes and libs.
     
  35. ollak

    ollak

    Joined:
    Jun 10, 2011
    Posts:
    1
    I just want to thank everybody in this thread, i haven't posted here before but I have used this thread in order to develop this demo of a game based on the tv-show hole in the wall: http://www.youtube.com/watch?v=7IuBx6Idstk. Thank you guys!
     
  36. hackshaven

    hackshaven

    Joined:
    Aug 12, 2010
    Posts:
    1
  37. janpec

    janpec

    Joined:
    Jul 16, 2010
    Posts:
    3,520
    I am looking forward for game called Ryse (Cryengine 3 powered game) to see how good it will turn out. I am little doubtfull about kinect especially for games that arent sport oriented games or games becouse most players are just too lazy to stand to play game.
     
  38. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
    Don't forget there's an app that let's you kinect scan your pillows and put into games. :)

    The hardest part for me with Kinect is requiring a large empty space.

    Hopefully Kinect 2 will support a messy house and sitting down on various couch configurations.
     
  39. ricardo.mello

    ricardo.mello

    Joined:
    Sep 2, 2009
    Posts:
    14
    Dude... this will sound really stupid... since this happened to me too... Are you sure kinect is plugged into the energy? see... the green led blinks because the led energy came from the usb, but if you dont power it, it will not work. :)
    I discovered this by just trying the usertracker sample from openni and seeing that it was not working too. Then I looked at the cord and it was not going to the wall... :) heheh

    []s, gandhi

    Anyway... you should check the samples first
     
  40. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
    Nothing so simple. In the end it came down to installing all the unstable drivers. I'm thinking of making an easy installer for those wanting to avoid pain.
     
  41. Todilo

    Todilo

    Joined:
    Feb 1, 2011
    Posts:
    88
  42. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,035
    Hi all,

    Finally got round to setting up openNI and kinect on my desktop machine since my Macbook Air HD died and have taken the opportunity to get to grips with the new Unity Wrapper found in the NITE x86 install.

    I decided to roll in my optimisations for setPixel methods and add opengl texture access from my previous release in this thread, but ended up implementing new features and bug fixes as well. I've also decided to rename it from wrapper to framework as that seems more appropriate and it avoids name clashes with the original.

    This version should be much more usable than my test projects, as I've just implemented the single opengl method (via InteropServices), along with Unity's native SetPixels method.
    The new wrapper class architecture also helps to keep the code clean, though not necessarily optimised.


    Download (5mb): openNI_Unity_Framework_Release_0.8



    What's new in the OpenNI Unity Framework:

    * Fixes to Soldier skeleton set up.
    * Support for RGB image and user label maps.
    * Example dynamic heighmap mesh from depth.
    * Support for opengl to directly blit maps into textures avoiding Unity's slow native method of setPixels.
    * Dedicated shaders for opengl textures from kinect depth and user labels.
    * Added AlternativeViewpointCapability to allow registration of image/depth/label.
    * Improved organisation, some refactoring and optimisation of code.
    * New scenes to compare Unity setPixels vs opengl methods.
    * Changed soldier ground image from 15mb psd to 500k jpg to reduce download file size.

    Note: The opengl texture support is only available for Windows, see readme for more details.


    Assuming you have openNI/NITE/SensorKinect already installed on your machine you're good to go.
    Otherwise there is a quickstart guide in the zip.

    I'm currently using (32 bit versions on x64 machine)
    - OpenNI Unstable Build (v1.1.0.41)
    - PrimeSense NITE Unstable Build (v1.3.1.5)
    - avin2-SensorKinect-xxxxxxx.zip. (ver 5.0.1)


    I have left the previous openNI test projects on my site, but their locations have now moved.
    OpenNITextueTest_mk2
    OpenNITextueTest_mk5


    enjoy.
     
    Last edited: Jun 21, 2011
  43. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
    Has anyone been using OpenNI and the new Microsoft SDK side by side?

    Is there a way to switch between versions without reinstalling the drivers to switch?

    I got bug report that after installing the M$ SDK, the unity samples will crash...

    So the alternative is to reinstall the OpenNI Primesense drivers.

    Can they just live together? What if you want to run an OpenNI app and a M$ app at the same time?
     
  44. amir

    amir

    Joined:
    Oct 5, 2010
    Posts:
    75
    Hey tgraupmann, We are working on binding OpenNI with the KinectSDK as well as KinectSDK bindings to Unity. Join the google group:
    http://groups.google.com/group/unitykinect



     
  45. Mr. Me

    Mr. Me

    Joined:
    Apr 21, 2011
    Posts:
    11
    Thank you very much,


    Ive been using you previous test version trying to make virtual objects flying behind reallife objects. So this update is very helpfull.

    Here's an early video of a cube flying trough the office in case anyone is interested.
    View attachment $flyingcube_02.7z

    Anyway, i got the newer drivers working but everytime i try to start any of the scenes in this project it give an error:

    Code (csharp):
    1.  
    2. GeneralException: One or more of the following nodes could not be enumerated:
    3.  
    4. User: PrimeSense/XnVSkeletonGenerator/1.3.1.5: No license to use this module!
    5. User: PrimeSense/XnVSkeletonGenerator/1.3.0.17: No license to use this module!
    6.  
    7. OpenNI.WrapperUtils.CheckEnumeration (Int32 status, OpenNI.EnumerationErrors errors)
    8. OpenNI.Context.InitFromXml (System.String xmlFile)
    9. OpenNI.Context..ctor (System.String xmlFile)
    10. OpenNIContext.Init () (at Assets/NITE/OpenNIContext.cs:56)
    11. OpenNIContext..ctor () (at Assets/NITE/OpenNIContext.cs:19)
    12. OpenNIContext..cctor ()
    13. Rethrow as TypeInitializationException: An exception was thrown by the type initializer for OpenNIContext
    14. NITESessionManager.Start () (at Assets/NITE/NITESessionManager.cs:51)
    15.  
    I quess it has something to do with the .xml files as explained in the readme but i dont realy know, anyone else having this problem?!?


    - Mr. Me
     
  46. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,035
    You're welcome, glad its being of use and will check out the video.

    Edit: Just watched the video, love it, simple but very effective and loved your reaction to it working ;)

    You guess sounds right to me. Looks like you've not adding the Primesense license key to your openNI.xml files.

    Try updating your files
    C:\Program Files (x86)\PrimeSense\NITE\Data - this is the important one I think
    C:\Program Files (x86)\OpenNI\Data

    and of course any xml files in your project folder.

    Make sure you have the correct license, no typos and for safetydouble check that if a depth node is referenced in the xml its at 640x480x30 and not 320x240 , which I don't believe is supported.

    I can only assume NITE requires its root xml files on your C drive to have the license, as well as ones in the project. However It is just a guess based on the error report.
     
    Last edited: Jun 22, 2011
  47. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
    The Primesense installer prompts for the key.
     
  48. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,035
    True but it didn't always update the xml files, I explicitly wrote a note to myself after spending ages trying to work out why the NITE samples didn't work and finding the license information was missing from the xml files.

    If the license is present in those xml files, then I'm not sure what the problem is, since the error message gives me the impression it is missing/wrong.

    In which case my next question to the OP would be do the openNI NITE samples work? Does my old wrapper still work?

    Actually better would be, does the new Unity wrapper found in the Primsense/NITE folder work?
     
  49. Mr. Me

    Mr. Me

    Joined:
    Apr 21, 2011
    Posts:
    11
    - The OpenNI samples work
    - The NITE samples work
    - The old mk5 still works

    the new one does work in the Unity pro version of, lets say my friend Mr. Schme... but it doesnt work in the indie version. for some reason.
     
  50. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,035
    Interesting so its a Pro vs Indie issue then?

    Try the new unity wrapper in the NITE install in your programs folder on C: drive.

    if that doesn't work, then there is some change in the wrapper that has stopped it working in Unity Free.

    If it does work, then I suggest temporarily deleting my '_OpenGL_DataToTextures' folder within Nite in my framework and see if that helps.
    if it does then the method in openGLConext is probably the problem. In which case you'll have to change the opengl code here to use one of the other methods, which ever is working for you in mk5 project.

    I guess this must assume you are trying the opengl scenes as openGLContext shouldn't be initiated otherwise? In which case maybe before deleting the folder try running the native scenes and see if they work.

    Must admit though I'm a bit puzzled as to why you got the error above in the framework, since it clearly states PrimseSense not getting a license, but not say mk5 version.

    Please let me know if you manage to determine the issue, so I can make any fixes necessary.

    Oh and i'm also using Unity 3.3, but don't see that making a difference ?
     
    Last edited: Jun 23, 2011