Unity Community


Page 1 of 15 12311 ... LastLast
Results 1 to 20 of 293

Thread: Kinect plugin


  1. Location
    San Francisco, CA
    Posts
    901

    Kinect plugin

    I've got a Unity plugin for Kinect working. Here's a screenshot. For simplicity's sake, plugin is stuffing the depth info into the alpha channel of the texture.



    Next I'll work on polygonizing the depth buffer... although a shader might be simpler. Does anyone have a parallax shader that uses one RGBA texture, where the alpha channel is depth?


  2. Location
    Greece
    Posts
    387
    Cool!
    Do you plan to release it?


  3. Location
    Venezuela
    Posts
    159
    Cooooooooooooooooooooooooooooooooooooooooooooooooo ooooooooooooooool.l!!!!!!!!!!!!!!!!!!!!!


  4. Location
    UK
    Posts
    292
    Thats pretty dang sweet
    My tl:dr; Development Blog - http://blog.greenslimegames.com
    Seeking world domination ...


  5. Location
    ohio
    Posts
    274
    are you sure you want to map it to a paralax shader? That doesn't seem like you would get the full representation of depth, or more important, to just use the depth value from the image to script any objects or particles into the scene, etc.. If you have the plugin reading the camera images into a texture - that's the basic functionality for any number of implementations after that - I would love to see what you've got working.

    I have been working on getting kinect into unity this weekend also. I got the motor and the led lights working, can read the serial, and turn the cameras on and off, but I am having difficulty figuring out how to read the image data from the pointer in the dll.

    I am using the dll from codelaboratories.com/nui/ - are you using that or are you using the openKinect code from
    https://github.com/OpenKinect/openkinect/ ?

    This is the format of the camera capture function in the dll - any suggestions?

    IMPORT(bool) GetNUICameraColorFrameRGB24(CLNUICamera cam, PBYTE pData, int waitTimeout = 2000);
    It's the PBYTE pData I can't figure out. I am trying this in the unity C# code:
    Code:  
    1.     [DllImport("CLNUIDevice.dll", EntryPoint = "GetNUICameraColorFrameRGB24")]
    2.     private static extern bool GetCameraColorFrameRGB24(System.IntPtr camera, System.IntPtr data, int timeout);

    but getting what I think should be a byte array out of the data is not working out for me so far.


  6. Location
    San Francisco, CA
    Posts
    901
    I wrote a plugin that uses the OpenKinect library. Seemed simplest at the time.

    The plugin will also give you the texture without alpha, and also give an array of depth values at full precision. I added the depth to the texture alpha solely so I could quickly determine if things were working -- I'm not sure how long it will take me to write code to polygonize the depths. (And it seemed like a no-brainer to add depth in a shader, rather than churning out fifty thousand polygons.) It's only three bits of precision less, and it avoided the pain of passing arrays around to native code, which I always find annoying.

    It's the PBYTE pData I can't figure out. I am trying this in the unity C# code:
    I'm not sure how that library works, but the depths aren't bytes -- they are 11-bit integers, and OpenKinect stores them as unsigned 16-bit ints. Mono maps them to the ushorts type. So maybe you need to make your array ushorts instead of bytes.


  7. Location
    ohio
    Posts
    274
    alright - thanks for the tip. I will keep at it. Do you have plans to share your plugin?
    I will if I get it working, but I suspect it will be pretty messy.
    thanks


  8. Location
    Canada
    Posts
    221
    That's awesome Rob!
    Unify.


  9. Location
    San Francisco, CA
    Posts
    901
    I'd like to get something more interesting working before I make any releases. Right now I feel confident it works well, but until I can really visualize the depth in 3D, I'm not 100% certain.


  10. Location
    ohio
    Posts
    274
    Your first question was about a parallax shader that uses one RGBA texture, where the alpha channel is depth - I started to look into that, but wondered why not use the parallax diffuse shader built in - and assign the one texture you are writing to both the Base and also the Heightmap, because the Heightmap input uses the A of the image and the Base uses the RGB. What are you looking for that would be different from that?

    What might also be interesting is to use something like the "heightmap generator" scene in the procedural mesh examples in the resources section of the unity site. This takes a texture and displaces the vertices of a mesh.


  11. Posts
    334
    great work man, would like to use some thing like this in my projects too.

    If you want some testing in others project please PM.


  12. Location
    The Netherlands
    Posts
    380
    I just saw this demo, seems like a cool thing to try: http://vimeo.com/17073934

    It appears a sphere is created at every pixel, and then placed at the measured depth.

    Edit: This one's better: http://vimeo.com/17075378
    Last edited by Tinus; 11-22-2010 at 10:04 AM.


  13. Location
    Amsterdam
    Posts
    96
    Quote Originally Posted by Tinus View Post
    I just saw this demo, seems like a cool thing to try: http://vimeo.com/17073934

    I appears a sphere is created at every pixel, and then placed at the measured depth.

    Edit: This one's better: http://vimeo.com/17075378
    Hahahahaha holy shit, I've seen many homebrew Kinect applications, but this is one of the coolest


  14. Location
    Brasil
    Posts
    207
    Really cool man. I made some tests with openFrameWorks and OpenKinetic and I was planning on doing something to integrate it with Unity too, but you made it first. Congrats.


  15. Location
    Boston,MA
    Posts
    146
    I do have question about Kinect and C# is anyway to convert a unity game on visual C# project and then put on XNA ? if you are working with KInect and C# how hard that can be.

    My reason for this question is i have and XNA license i can deploy games on my Xbox 360 i wonder if somebody got the idea to convert any of the games made with UNITY put them on Xbox live.


  16. Location
    San Francisco, CA
    Posts
    901
    Slightly more interesting example. A grid of spheres offset by their distance values (with texture drawn on the plane below). I pointed the camera at the wall to remove the background noise. The shader on the plane is drawing the depths as greyscale, so you can see how the depth values match the pixel values (and you can see why the XBox is always telling you to back up.)



    I do have question about Kinect and C# is anyway to convert a unity game on visual C# project and then put on XNA ?
    I heard that XNA licenses don't incude Kinect support, so you might want to ask Microsoft directly about that. If you did you wouldn't have to be hacking around with open source drivers like the rest of us.


  17. Location
    Amsterdam
    Posts
    96
    Would love to see some video bliprob! Any chance?


  18. Location
    Orlando, FL
    Posts
    724
    Great work!

    Here is a relief shader that uses an alpha for height:

    Relief.unitypackage


  19. Location
    Argentina, Buenos Aires
    Posts
    2
    Hi!, this would certainly be astonishing.
    IŽll keep an eye on this thread.

    Cheers and keep up the good work. =)


  20. Location
    San Francisco, CA
    Posts
    901
    reissgrant: awesome, thanks! will give a try. I've hacked at the marching cubes from the wiki some, too, and added simple background removal.

Page 1 of 15 12311 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •