Unity Community


Page 1 of 3 123 LastLast
Results 1 to 20 of 47

  1. Location
    Netherlands
    Posts
    50

    ARToolkit extension

    Hi There!

    Iím quite interested in ARToolkit ( http://www.hitl.washington.edu/artoolkit/ ), a software library for building Augmented Reality (AR) applications. These are applications that involve the overlay of virtual imagery on the real world.
    GameBlender and Virtools already have plugins to communicate with ARToolkit. Check http://www.ash.webstranka.info/?p=36 for an example movie.
    My question is, how difficult will it be to combine ARToolkit with Unity? And could it be possible to make a connection without owning Unity pro? I am not a programmer by any means. I actually just started trying Unity to learn programming in a more playful way.
    If anyone is interested in making a connection, ARToolkit plus ( http://studierstube.icg.tu-graz.ac.a...oolkitplus.php ) would be an even better base to build ontop.

    Links:

    GameBlender plugin: http://www.ash.webstranka.info/?p=45

    Virtools plugin: http://mirevicast.medien.fh-duesseld...ols-extensions

    As an example what kind of games you could create, check this amazing project by Julian Oliver: http://julianoliver.com/levelhead

    Greetings,

    Wim


  2. Location
    Fullrate, Copenhagen
    Posts
    499
    From an initial scan of the docs it seems like its written in C and i couldn't find any API in C# so think it will have to be done in a plugin but it should be possible. It might be hairy though to actually combine the rendering pipeline into unity, but can't say for sure as i havent done it

    Is a cool technology.

    Regards,
    Marc


  3. Location
    Australia
    Posts
    267
    Theres a c# version of NyArToolkit, im sure this could be used... i dont have time to test it out, but please someone do, id love this as well (indie)

    http://nyatla.jp/nyartoolkit/wiki/in...p?FrontPage.en


  4. Location
    France
    Posts
    17
    Hi,

    I'm also doing a plugin with ARToolKit.

    A few things I managed to do :

    - The pattern recognition works.

    - I've integrated the image grabbed by the webcam on a plane (Texture2D) which is far away from the Unity camera.

    - Thanks to openGL I have access to 2 matrices [16] of 16 elements that I convert in 2 matrices [3][4] so i can extract easily a position vector and a quaternion with an ARToolkit function "arUtilMat2QuatPos". The first one is the projection matrix for the Unity Camera (the real webcam is not in the position (0,0,0) and rotation (0,0,0) in Unity world, this is due to camera calibration). --> "glGetFloatv(GL_PROJECTION_MATRIX,MatrixProjection) ;"
    The other one is the model-view matrix for the Unity object which will appear on the pattern. -->"glGetFloatv(GL_MODELVIEW_MATRIX,MatrixModelView);"

    -The object (in Unity -> a cube) I want to be on the pattern follow the pattern when I move the pattern.




    Nevertheless, I have some projection (or position) problems : the object is not on the middle of the pattern but is shifted.

    I think that the problem is that one or two of my gameObject (the camera, the cube and the plane) is not is the right position. Because when the pattern is on the left extremety of the screen, the cube has a better position than if the pattern was on the right extremity of the screen.

    The distance plane-camera is a constant and is set to an arbitrary value ("4150" so that the camera look only at the plane in the Unity world)
    The plane move only one time with the camera in the "start" function (due to the projection matrix).
    The cube move alone (all the time) (due to the model-view matrix) in the "update" function.

    Does someone (who knows well Unity/OpenGL or ARToolkit) thinks that I've missed a step to determine the positions of my gameObjects.

    Or maybe the problem is somewhere else :s .
    Attached Images  


  5. Location
    Bloomington, IN, USA
    Posts
    367
    I've likewise implemented an ARTk plugin and got exactly to where solmyr is. Something gets mucked up in the translation from the plugin coordinates to Unity. I tried asking on the ARTk forums and emailed the maintainer about it and all I ever got was a rather brusque reply that I must be reversing the projection matrix (ie, calculating it in camera space rather than world space, which can be, and was, tested by adding a single function call to the plugin). There really isn't much support for ARTk anymore outside of the rather lackluster forums, unfortunately, and the maintainers didn't seem interested in looking into the problem at all.


  6. Location
    France
    Posts
    17
    yeah... it's really strange, it's working perfectly when the drawing are done in OpenGL (in ARToolKIt code samples).
    I've followed every function that is necessary in their code to get the 2 matrices. Then the adaptation to unity should have be easy


  7. Posts
    30
    looks interesting, is this useable for indie too ?
    if so, could you make an guide or example project ?
    i'd like to try it out


  8. Location
    Bloomington, IN, USA
    Posts
    367
    Quote Originally Posted by zeroZshadow
    looks interesting, is this useable for indie too ?
    if so, could you make an guide or example project ?
    i'd like to try it out
    It's Pro only. When last I checked the ARTk library is C and C++ only (with wrappers for some other languages), which means it has to be done via plugin.


  9. Location
    Halifax, Nova Scotia, Canada
    Posts
    154
    This is a long shot, but:

    as far as I can tell, Unity uses a left handed co-ordinate system. OpenGL uses a right handed co-ordinate system.

    If you are stacking your matrixes in Unity, do them in the reverse order. Let me know if this works.


  10. Location
    Bloomington, IN, USA
    Posts
    367
    Quote Originally Posted by Horsman
    If you are stacking your matrixes in Unity, do them in the reverse order. Let me know if this works.
    I can't speak for solmyr, but I wasn't mucking with the rendering pipeline at all (aside from using Graphics.DrawTexture() to overlay the incoming video image) since doing so would require getting the camera modelview matrix post-render, sending it to the plugin and drawing the AR objects there and then setting it again (which defeats the purpose of using a 3D engine). I was simply using the transformation values ARTk produced to update the position of Unity prefabs. There is a utility function in ARTk to get a right-handed transformation matrix, but just from looking at my notes (the project won't open in 2.5 without some conversion I can't really do right now), we were actually getting different transform values in the trace from the plugin versus what Unity was actually receiving.


  11. Location
    France
    Posts
    17
    Quote Originally Posted by burnumd
    I was simply using the transformation values ARTk produced to update the position of Unity prefabs.
    Yes I'm doing it in the same way (don't work either)

    Quote Originally Posted by burnumd
    There is a utility function in ARTk to get a right-handed transformation matrix
    I don't need it, i'm just switch Y and Z axes. Apparently it works because the cube is following the pattern in all directions (up/down/left/right/front/back)

    Quote Originally Posted by burnumd
    we were actually getting different transform values in the trace from the plugin versus what Unity was actually receiving.
    I'd rather think that the problem comes from an openGL transformation we don't understand (with the 2 matrices projection/model-view)

    Or perhaps it comes from another coordinates problem:
    -the projection matrix has to be inversed (because we need camera coordinates in the real world)
    -the modelview matrix musn't be inversed (because it's the pattern coordinates in the camera coordinates)

    3 possibilities , 3 failures ^^
    Attached Images  


  12. Location
    France
    Posts
    17
    Another strange thing (burnumd have the same problem I suppose)

    With ARToolKit code sample, the openGL modelview matrix is :
    -0.210773 | 0.411159 | -0.886861 | 0
    0.969434 | -0.0286372 | -0.243674 | 0
    -0.125586 | -0.911114 | -0.392556 | 0
    51.7304 | -38.3579 | 384.759 | 1


    With my code in Unity3d, the openGL modelview matrix is :
    0.02105 | 0.377307 | 0.925849 | 0
    0.99863 | 0.0364404 | -0.0375552 | 0
    -0.0479081 | 0.925371 | -0.376023 | 0
    51.9647 | 21.9308 | 391.948 | 1

    The pattern is exactly in the same position.

    The more important thing is the position of the pattern ( i'll deal with the rotation problem after). The position XYZ is the last line:
    51.7304 | -38.3579 | 384.759 in the first case
    51.9647 | 21.9308 | 391.948 in the other case.



  13. Location
    Bloomington, IN, USA
    Posts
    367
    That's pretty much exactly the problem we were having. We tried to do some corrections to the values, but it didn't seem to correspond linearly and I, at least, couldn't see a pattern to the deviation. I didn't use the modelview matrix directly (I used the utility function to convert the matrix to two double arrays. Looking at your data, I wonder now if it's a bug in the plugin system. Maybe an overflow or type conversion issue since ARTk gives the matricies as doubles rather than floats? From looking at my plugin, it doesn't seem like it would be the latter, since my plugin print statements display the converted float value and I saw the same issue. It didn't occur to me at the time to report it because I assumed it to be either an ARTk problem or something I was just doing wrong, but if you've got a working(-ish) project, you might consider submitting a ticket to Unity QA with your project (along with the reams of fiducial patterns so they can actually test it).


  14. Posts
    30
    just well.. did you try mirroring the camera texture >_> ?


  15. Location
    France
    Posts
    17
    First, thanks for your help .

    Quote Originally Posted by burnumd
    Looking at your data, I wonder now if it's a bug in the plugin system. Maybe an overflow or type conversion issue since ARTk gives the matricies as doubles rather than floats?
    Can't be that, I send a floats matrix to Unity.

    The main steps of my program are (in red C# script , in blue C++ plugin , green comments):

    -Start()
    -->initARToolKit(); // init the webcam and the projection matrix
    -->GetQuatAndPos(quaternionCam, positionCam, MODE_CAM,INVERSE_CAM); // get position and rotation for the cam using arUtilMat2QuatPos(newMatrix, q, p);
    // "quaternionCam" and" positionCam" are float matrices whereas "q" and "p" are double matrices

    --> UpdateCamera(...) // update positions and rotations of the Unity camera


    -Update()
    --> mainLoop(); // grab the image,detect the marker, calculate the "patt_trans" matrix
    -->UpdateTexture1(...) //Update the texture on the plane like in the texture plugin
    --> if(IsPatternDetected()
    ){
    ---->GetQuatAndPos(quaternionOrient, positionPatt,MODE_CUBE,NULL); // get the position and rotation of the cube in float matrices
    ---->UpdateCube(...) // update positions and rotations of the cube
    }

    Here the code for "GetQuatAndPos(quaternionOrient, positionPatt,MODE_CUBE,NULL)" . You can see that there is no problem of double or float conversion through the unity plugin
    Code:  
    1. {
    2. argConvGlpara(patt_trans, gl_para);
    3. glMatrixMode(GL_MODELVIEW);
    4. glLoadMatrixd( gl_para );
    5. glTranslatef( 0.0, 0.0, 25.0 );
    6. glGetFloatv(GL_MODELVIEW_MATRIX,matriceModelView);
    7. for(int ii = 0; ii < 3; ii++ ){
    8.     for( int jj = 0; jj < 4; jj++){
    9.             newMatrix[ii][jj] = matriceModelView[jj*4+ii];
    10.     }
    11. }   
    12.                     arUtilMat2QuatPos(newMatrix, q, p);
    13.  
    14. for( int ii = 0; ii < 4; ii++ ){quat[ii] = (float) q[ii];}
    15. for( int ii = 0; ii < 3; ii++ ){pos[ii] = (float) p[ii];}
    16. argSwapBuffers();
    17. }


    Quote Originally Posted by zeroZshadow
    just well.. did you try mirroring the camera texture >_> ?
    Already did because the camera grab a mirrored image (and when I move my pattern in the left side in reality, it does the same thing in the texture).
    Moreover the problem isn't here zeroZshadow.

    There is nothing that could explain the difference between the values in the modelview matrices in the 2 cases (ArTk sample of code and my unity plugin) with a pattern in a same position.
    -I follow every step they do
    -I don't modify the intermediate matrices
    -I compare the openGL modelview matrix that is used in the display function to my openGL modelview matrix that is used just before sending the coordinates to Unity.


  16. Location
    Altanta, GA
    Posts
    16
    Hi, I didn't read this very closely, but I am incorporating StbES into Unity and have change both the position and quaternions to Unity's left handed system. For the position of a marker (z up) I found I had to invert the Y value. For the quaternion, I had to invert the X and Z quat values. We are not putting the projection matrix delivered by StbES directly into the camera because it seems to interfere with picking. So we are just setting the transform and FOV from calibration data.


  17. Location
    Wiesbaden, Germany
    Posts
    24

    My ARToolkit implementation

    Hey Guys!

    This is my way to calculate the position and rotation of an object using ArToolkit. It works fine if the camera position is (0,0,0) and looks to +z axis, the image plane is at position (0,0,960) with a scale of (128,1,96):
    Code:  
    1. // position
    2. double[] trans = new double[3];
    3.  
    4. // rotation vector as quaternion
    5. double[] rot = new double[4];
    6.  
    7. // Get the translation and the rotation with
    8. // the AR function arUtilMat2QuatPos
    9. getDetectedTranslationRotation(trans, rot);
    10.  
    11. // Set position
    12. transform.position = new Vector3(-(float)trans[0] * 1.5f, -(float)trans[1] * 1.5f, (float)trans[2]);
    13.  
    14. // Set rotation
    15. transform.rotation = new Quaternion((float)rot[0], (float)rot[1], -(float)rot[2], (float)rot[3]);

    The getDetectedTranslationRotation is a DLL function where patt_trans is the dedected marker matrix:

    Code:  
    1. EXPORT void getDetectedTranslationRotation(double* translation, double* rotation){
    2.     arUtilMat2QuatPos(patt_trans, rotation, translation);
    3. }

    This implementation works fine in my case, even if the marker are in the corner of the webcam image.
    Attached Images  
    Christian Rathemacher
    Lead Programmer

    ------------------------------------------------------------------------------------------------

    weltenbauer. Software Entwicklung GmbH

    Humboldtstr. 7
    65189 Wiesbaden

    0611 / 97 14 314
    produktion.weltenbauer.com
    produkion@weltenbauer.com


  18. Posts
    69
    Can someone publish sources?


  19. Location
    Dublin, Ireland
    Posts
    86
    wow, the sources would be awesome:


  20. Location
    Melbourne, Australia
    Posts
    64

    rendering webcam

    Hi,

    I have experience in the AR side of things - I can code in C/C++ (amongst other things) and understand camera matrices.

    But I'm new to Unity3D. What do I need to do to get webcam video streaming into a Unity 3D authored demo?

    Will I need the pro version of Unity3D? Or will the indie version do it?

    Carl

Page 1 of 3 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •