More Fun with Unity and Kinect

Discussion in 'Showcase' started by amir, Jan 21, 2011.

  1. amir

    amir

    New Member

    Joined:
    Oct 5, 2010
    Messages:
    75


    (tinkerheavy.com) This is a demonstration of the Unity wrapper for OpenNI. My skeleton is tracked as I move in front of the Kinect. My feet are rigid-bodies that can kick the boxes. The boxes are also carryable if two hands interact with it simultaneously.

    If you already have OpenNI+NITE working with your kinect, the binary for this project (pc and mac) is here: http://tinkerheavy.com/unityskeleton.zip
    To get the others working you'll need to get OpenNI and NITE from OpenNI.org (to use the kinect get sensorkinect from https://github.com/avin2/SensorKinect instead of the openni.org sensor module)

    The code and unity project are available on my git:
    https://github.com/tinkerer/UnityWrapper
  2. pixelsteam

    pixelsteam

    Member

    Joined:
    May 1, 2009
    Messages:
    671
    Great Work Amir!!!!
    I am very excited about the various potential of Unity with Kinect....
  3. Kyuss77

    Kyuss77

    New Member

    Joined:
    Jan 20, 2010
    Messages:
    121
    Is the kinect running on an xbox or a pc. I can't tell from the video. Very cool though.
  4. ZJP

    ZJP

    Member

    Joined:
    Jan 22, 2010
    Messages:
    1,612
    Nice work :cool:

    JP
  5. zweihander111

    zweihander111

    Member

    Joined:
    Nov 7, 2010
    Messages:
    17
    Very cool! Just got my kinect home, I'll start right away!
  6. wmaass

    wmaass

    New Member

    Joined:
    May 24, 2010
    Messages:
    10
    If I may ask a silly question: How do you "connect" to the Kinect when running this? When I run this the scene with the boxes comes up but nothing else. My Kinect works with all of the OpenNI and NITE samples just fine. Any tips?

    EDIT: I got it, you have to do the pose. I was impatient. Works great, good work!
    Last edited: Jan 22, 2011
  7. amir

    amir

    New Member

    Joined:
    Oct 5, 2010
    Messages:
    75
    I just raise my hands to the air, look up and say "please god make it work" and then it does. Every testing moment is a miracle.

    Kyuss: this works on OS X or PC. I don't even own an Xbox!
  8. robotfunk

    robotfunk

    New Member

    Joined:
    Mar 8, 2010
    Messages:
    7
    I'm watching this with great interest, as we are looking to use the kinect for mocap into unity3d (at Film Academy). I've setup openNI/Sensor/NITE, the NITE examples work fine here including skeleton tracking.

    If I open this project in Unity3D, the floor and cubes are all bright magenta (missing texture/material?), the boxes have a missing script in the Mono Behaviour (I think the carryable goes here?). Also the first 2 spheres have a missing script (would the hand script go here?)
    Also the camera is missing a script (which one?).

    If I open the executable I see the floor and boxes textured fine, but striking the pose all I want, I don't see a character appearing... any tips would be highly appreciated.
  9. robotfunk

    robotfunk

    New Member

    Joined:
    Mar 8, 2010
    Messages:
    7
    OK I think the Nite3 script goes on the camera, when I do that i get DllNotFoundException: OpenNI
  10. Checco Esimple

    Checco Esimple

    Member

    Joined:
    Feb 14, 2008
    Messages:
    77
    Hi Amir... I installed all the libraries and dependencies as wrote on readme, and I got it working with the precompiled unityskeleton (and unitiskeleton2 found on your website)... but I can't make working the unity project because it has lost all the links to the monobehaviours (and also shaders and materials...)...

    I think you should use the "export package" feature to keep the logical links between the assets inside the project...

    btw, thank you for your work!.. we will start expreimenting it really soon!

    PS: I found a shorter way to install the libusb-1.0 library, downloading the roxlu openNI repo through github and manual copying the content of /Source/External/LibUSB/Mac/ inside the /opt/local/bin/ folder in my mac.
  11. Checco Esimple

    Checco Esimple

    Member

    Joined:
    Feb 14, 2008
    Messages:
    77
    Only a last question.. (I'm talking about OSX unityskeleton build): I noticed that the axis are inverted... I changed the mirror option to true inside OpenNI.xml and I get the right X movement, but I still have the wrong Z axis orientation...
  12. amir

    amir

    New Member

    Joined:
    Oct 5, 2010
    Messages:
    75
    I'm sorry I suck at sharing Unity Projects! I'll learn how to use export package now!
  13. amir

    amir

    New Member

    Joined:
    Oct 5, 2010
    Messages:
    75
    if you want it to be looking at you and mirror image i think you should just put a negative sign somewhere in there. also will do this to the project when i learn to export packages
  14. T.Lourenco

    T.Lourenco

    New Member

    Joined:
    Jan 23, 2011
    Messages:
    15
    looking at this,...my mind start to think about my new challenger!

    good job!
  15. zweihander111

    zweihander111

    Member

    Joined:
    Nov 7, 2010
    Messages:
    17
  16. Thomas Lund

    Thomas Lund

    New Member

    Joined:
    Jan 18, 2008
    Messages:
    465
    We are trying to get this to work here, but both on windows and Mac the error message that comes from running the binary is

    DllNotFoundException: OpenNI
    at (wrapper managed-to-native) xn.OpenNIImporter:xnEnumerationErrorsAllocate (intptr)
    at xn.EnumerationErrors.Create () [0x00000] in <filename unknown>:0
    at xn.EnumerationErrors..ctor () [0x00000] in <filename unknown>:0
    at xn.Context.InitFromXml (System.String xmlFile) [0x00000] in <filename unknown>:0
    at xn.Context..ctor (System.String xmlFile) [0x00000] in <filename unknown>:0
    at Nite3.Start () [0x00000] in <filename unknown>:0

    (Filename: Line: -1)

    Any clue on whats wrong?

    /Thomas
  17. kweiko

    kweiko

    New Member

    Joined:
    Mar 9, 2010
    Messages:
    100
    REALLY awesome job, this will be so much fun to look forward once I know enough (and have gotten a kinect =) )
    One thing though: with such bg music and such an avatar, you should have done the robot dance instead! :O

    /Kweiko
  18. GTaveira

    GTaveira

    Member

    Joined:
    May 20, 2009
    Messages:
    29
  19. diabloroxx

    diabloroxx

    New Member

    Joined:
    Jan 20, 2010
    Messages:
    71
    Nice work..Thanks for the info abt setting it up.
  20. t00cg

    t00cg

    New Member

    Joined:
    Dec 24, 2010
    Messages:
    2
    First Thanks for your work. Your ReadMe (for MacOS X) works really good. (Only thing that not work or i had to do also - i guess this is installed on your maschine was: *********1.1**** sudo port install git-core *********** after 1) Intall MacPorts from: http://distfiles.macports.org/)

    The Tests and your Skeleton-Programm (compiled) works perfectly on MacOSX.

    But my Problem is now the Unity-Project. If I download the whole thing over git. I get a folder with the Assets Folder etc. If i open for example skeletontest-project. there a lot of Missing-Scripts on the gameobjects. And i can't even compile it.

    What can i do to fix this? Do you have also a simpler version? Than i could start to bring all together again. Or am i the only one with this problem?

    Thanks for any help

    René Bauer
  21. fergu00

    fergu00

    New Member

    Joined:
    Feb 20, 2011
    Messages:
    2
    Thanks Mate! very good Job

    Sergi
  22. megisto

    megisto

    Member

    Joined:
    Dec 25, 2009
    Messages:
    82
    Why i see everything pink? Does this project works only on the full edition maybe?

    Attached Files:

  23. pandigital

    pandigital

    New Member

    Joined:
    Mar 20, 2009
    Messages:
    12
    Can someone post some screenies or list showing which scripts need to go on which Objects please.
  24. Ilikescifi

    Ilikescifi

    New Member

    Joined:
    Dec 15, 2010
    Messages:
    17
    @ checco77: open the openNi file. there you´ll find <Mirror on="true"/> false is mirror, true is right !
    best,
    Jot
  25. Ilikescifi

    Ilikescifi

    New Member

    Joined:
    Dec 15, 2010
    Messages:
    17
    @ "Why i see everything pink? Does this project works only on the full edition maybe?"

    I have the same problem :(
    Last edited: Feb 24, 2011
  26. HeywoodFloyd

    HeywoodFloyd

    Member

    Joined:
    May 27, 2009
    Messages:
    15
    Impressive work.

    I've got a nut bolts question about the OSX implementation: Obviously you were able to access the OpenNI library through a bundle. How'd you build the bundle? I've been trying to access a 3rd party library through a bundle, and I always get the System.DllNotFoundException, even when the bundle is obviously in place.
  27. nikemerlino

    nikemerlino

    New Member

    Joined:
    Jul 2, 2008
    Messages:
    66
    I can't install NITE because this free PrimeSense license key seems to be invalid 0KOIk2JeIBYClPWVnMoRKn5cdY4=

    How is it possible to do an application like "KinectViewer" in MacOS App Store without any installation of OpenNI,NITE and SensorKinect?

    Can someone have the time to explain us the right procedures to run the examples in this forum. It is not very clear. When I do sudo sh "Path to install.sh" I have many errors and I can't understand if the installation is done. What is your experience?
    Thanks to all
  28. pandigital

    pandigital

    New Member

    Joined:
    Mar 20, 2009
    Messages:
    12
  29. amir

    amir

    New Member

    Joined:
    Oct 5, 2010
    Messages:
    75
    it's cuz i suck and don't know how to distribute my projects. i'll fix it later. i may as well take down the scenes and just make everyone figure out how to use my source
  30. nikemerlino

    nikemerlino

    New Member

    Joined:
    Jul 2, 2008
    Messages:
    66
    It works fine!!!
    Many thanks to all
  31. pandigital

    pandigital

    New Member

    Joined:
    Mar 20, 2009
    Messages:
    12
    Hi Amir

    Could you post here which scripts go on which Game Objects?

    Many thanks
  32. Bushyo

    Bushyo

    New Member

    Joined:
    Apr 13, 2011
    Messages:
    8
    Once again: great job! I'm not the master at unity development but I got it running :)
    But only in debug mode... After publishing and testing your example as standalone app my kinect device doesn't fire at all ;[ what can I do to compile it to Mac app ( or windows-compatible ) and make it working? Is a C wrapper issue?
    Thanks for any help
  33. amir

    amir

    New Member

    Joined:
    Oct 5, 2010
    Messages:
    75
    @Bushyo: I believe you need to have the .NET.dll and .xml files in the same directory as your executable.
  34. Bushyo

    Bushyo

    New Member

    Joined:
    Apr 13, 2011
    Messages:
    8
    Thanks! Eth works fine. I'm creating virtual puppet for my little son. I gonna publish the source soon...
    I share I am :)
  35. Bushyo

    Bushyo

    New Member

    Joined:
    Apr 13, 2011
    Messages:
    8
    I can not figure it out how to detect collisions between model controlled by Kinect and any other object. RigiBody and Character Controller don't work at all. Do I need to write my own script or do I attach scripts to wrong element in library?
    Any help welcome ;]
  36. bryanleister

    bryanleister

    New Member

    Joined:
    Apr 28, 2009
    Messages:
    130
    You need a collider on both and a rigid body on at least one of them. Turn off gravity or make it Kinematic can help it behave as you want.

    Bryan
  37. minevr

    minevr

    Member

    Joined:
    Mar 4, 2008
    Messages:
    1,009
    I get it.....

    Thin waist~~~:p:p:p

    Attached Files:

  38. kaushalyjain

    kaushalyjain

    New Member

    Joined:
    May 30, 2011
    Messages:
    3
    Hi,
    can you please explain me how to modify the code and how exactly to incorporate OpenNI + NITE in Unity3D..
    I got the sample posted above working fine.. and i know OpenNI + NITE framework reasonably well..
    Thanks...
    Kaushal
  39. rahuxx

    rahuxx

    Member

    Joined:
    May 8, 2009
    Messages:
    363
    @ Minevr : help me too.
  40. Curious

    Curious

    Member

    Joined:
    Nov 19, 2009
    Messages:
    314
    Are you guys on windows or mac? I can't get it to work on win! Any instructions?

    @Minevr:
    How did you get the images at the corner of the screen?
    Last edited: May 30, 2011
  41. Curious

    Curious

    Member

    Joined:
    Nov 19, 2009
    Messages:
    314
    Heh, got it to work, impressive :)

    Thank you Amir ;)
  42. kaushalyjain

    kaushalyjain

    New Member

    Joined:
    May 30, 2011
    Messages:
    3
    i m on win 7 and i have got the sample running.. only thing is i want to know how to modify the code and understand it.. especially the integration of the gestures of NITE with unity3D... thanks in advance..!!
  43. Curious

    Curious

    Member

    Joined:
    Nov 19, 2009
    Messages:
    314
    So, are you going to upload the complete project again fixing the missing script and untextured objects any time soon?
  44. ben.aten

    ben.aten

    New Member

    Joined:
    Jan 10, 2013
    Messages:
    3
    Hi,

    Can we play this without having any external devices except our PC.

    Thank you,
    Ben