Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

UnityAR - ARToolkit Interface

Discussion in 'Made With Unity' started by Christian, Sep 22, 2009.

  1. Christian

    Christian

    Joined:
    Dec 2, 2008
    Posts:
    24
    UnityAr provides an interface to the ARToolkit. Local connected webcams are used as input source for the pattern recognition. This interface allows to detect multiple ARToolkit marker from the videostream and provides the position and rotation of each marker in the 3D scene.

    A free beta version is available on our website: http://produktion.weltenbauer.com on the "entwicklungen." section.

    ARToolkit Details: http://www.hitl.washington.edu/artoolkit

    CodeVis VidCap Details (Video stream input): http://www.codevis.com/vidcapture
     

    Attached Files:

  2. Zappageck

    Zappageck

    Joined:
    Jul 31, 2009
    Posts:
    26
    Thx for sharing the beta. Looks quite promising :D
    I got a question though. On your webpage you have a curtain visualization that seems to behave like a cloth thing.
    Would you mind sharing how you achieved this effect ?
     
  3. polytropoi

    polytropoi

    Joined:
    Aug 16, 2006
    Posts:
    681
    wow, that looks cool. is a mac version possible?
     
  4. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    Here is a screenshot of my AR server streaming full frame video @ 25 fps, into Unity Indie.

    Will provide download links in due course. Windows 32 bit only.

    Carl
     

    Attached Files:

  5. liverolA

    liverolA

    Joined:
    Feb 10, 2009
    Posts:
    347
    nice work,keep up
     
  6. jtbentley

    jtbentley

    Joined:
    Jun 30, 2009
    Posts:
    1,397
    How much information is required, ie, focal lengths/height etc? Or do you derive all that information from statistical analysis from parralax etc?

    Essentially, is this a realtime equivilent of Bojou/PixelFarm/Matchmover/Syntheyez?
     
  7. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    Bojou/PixelFarm/Matchmover/Syntheyez/Icarus/Etc are markerless systems so they have a lot more work to do - but yes, the idea is the same.

    The ARToolkit (on top of which I've built my server version), tracks a known rectangular marker in the live video image.

    See here for more information:

    http://www.hitl.washington.edu/artoolkit/

    Intrinsic camera parameters such as focal length are pre-programmed (or auto-derived from prior analysis of a grid via the camera).

    Carl
     
  8. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    Hi Christian,

    you are asking about live video in Unity. The following code is my render loop and I've had it running up to about 35 fps. My video data runs raw over a socket connection as a byte stream, three bytes per pixel.

    The following code has been tested with PAL video, 720 x 576 pixels, rendering at 25 frames per second.

    I'm using a dual processor 2.8 GHz which probably helps. I don't know where the cutoff point is in terms of CPU power however the code is optimal as far as I can tell (short of writing a DLL).

    C# optimises inner loops and I've written the pixel data iteration as a single dimensional loop.

    It would go a lot faster if I didn't have to do the byte to float conversion (multiplication by m). But that would require sending floats over the socket ( 4 times more data!!!)

    Note the use of the function: setPixels (plural).

    Code (csharp):
    1.  
    2. // per frame render
    3.  
    4. int i=0;
    5. int k;
    6. float m = 1.0f/255.0f;
    7. float r,g,b;
    8. int miplevel = 0;
    9. int numPixels = frameWidth * frameHeight;
    10. int numPixelBytes = numPixels * 3;
    11.        
    12. // even though this is a per/pixel operation
    13. // it works quite fast in C#
    14. k = frameOffset;
    15.        
    16. for(i=0; i<numPixels; i++)
    17. {
    18.   r = dataBuffer[k++] * m;
    19.   g = dataBuffer[k++] * m;
    20.   b = dataBuffer[k++] * m;
    21.   cols[i] = new Color(r, g, b);
    22. }
    23.        
    24. int xpos;
    25. int ypos;
    26.        
    27. xpos = (1024 - frameWidth)/2;
    28. ypos = (1024- frameHeight)/2;
    29.        
    30. texture.SetPixels (xpos, ypos, frameWidth, frameHeight, cols, miplevel);
    31.    
    32. // copy texture to the GPU
    33. texture.Apply(false);
    34.  
     
  9. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    Hi Christian,

    just looking at your demo client code:

    Code (csharp):
    1.  
    2. Color[] val = new Color[width * height];
    3. float[] color = new float[3];
    4. for(int y=0; y<height; y++){
    5.  for(int x=0; x<width; x++){
    6.   getPixel(x, y, color);
    7.   val[y*width + x] = new Color(color[0], color[1], color[2]);
    8.  }
    9. }
    10.        
    11. tex.SetPixels(val);
    12. tex.Apply();
    13.  
    This is what I do in my Start function (ie. just once). Note class level variables accessible by update

    Code (csharp):
    1.  
    2.  
    3. // class level globals
    4.  
    5. Texture2D texture;
    6. Color [] cols;
    7. Color colour;
    8.  
    9.     void Start ()
    10.     {
    11.    
    12.         // create a new texture for the video frame data
    13.         texture = new Texture2D (1024, 1024, TextureFormat.RGB24, false);
    14.        
    15.         // do we need expensive texture filtering? Use the least expensive:
    16.         texture.filterMode = FilterMode.Point;
    17.        
    18.         // assign our new texture to the object's texture
    19.         renderer.material.mainTexture = texture;
    20.    
    21.         // apply the object's texture to the object (ie. copy texture to GPU)
    22.         texture.Apply(false);
    23.        
    24.         int i=0;
    25.         int miplevel = 0;
    26.         int numPixels = frameWidth * frameHeight;
    27.    
    28.         int xpos;
    29.         int ypos;
    30.        
    31.         xpos = (1024 - frameWidth)/2;
    32.         ypos = (1024 - frameHeight)/2;
    33.        
    34.         // init cols array by getting pixels from the object ??
    35.         //cols = texture.GetPixels (xpos, ypos, frameWidth, frameHeight, miplevel);
    36.        
    37.         // Can't we just do it explicitly - yes we can ...
    38.         cols = new Color[numPixels];
    39.        
    40.         // TEST - set all pixels white
    41.         for(i=0; i<numPixels; i++)
    42.         {
    43.             cols[i] = new Color(1.0f, 1.0f, 1.0f);
    44.         }
    45.        
    46.         // copy pixels back to texture
    47.         texture.SetPixels (xpos, ypos, frameWidth, frameHeight, cols, miplevel);
    48.    
    49.         // copy texture to the GPU
    50.         texture.Apply(false);
    51.  
    The update function's only responsibility is then to populate the cols array, setPixels to cols and apply to object (send to GPU).

    Carl
     
  10. attilam

    attilam

    Joined:
    Jun 2, 2007
    Posts:
    86
    Hi,

    I downloaded UnityAR and so far have tried it on a bootcamped MacBook Pro with iSight as well as a PC with a Philips webcam, but unfortunately non of them work.

    The demo can get as far as detecting modes, but it fails to set them (I have tried different ones, by changing modeNr in script).

    Any ideas why this is? Are there any other requirements to download perhaps?

    Thanks,
    attila
     
  11. Christian

    Christian

    Joined:
    Dec 2, 2008
    Posts:
    24
    Hm? That's is strange. Are there any error reports or exceptions that are coming up in the console?
     
  12. Christian

    Christian

    Joined:
    Dec 2, 2008
    Posts:
    24
    Thanks for your advice. I have nearly the same FPS on my test system with pal resolution. But do you realy think that using SetPixels() instead of SetPixel() for every pixel brings that speed-up. I think what SetPixel() is doing internaly, is setting a Color struct to a two dimensional Color array that is located in cpu memory. The function which slows down the whole process is tex.Apply() that copys this Color array to the gpu mem.

    The main question for me is, if there is a better way to apply all pixels from the webcam image in the gpu mem every frame. Maybe streaming the pixel data direct to gpu mem or something like that.

    But I think currently is SetPixel() and SetPixels() the only way to modify textures in gpu mem. Correct me if I'm wrong.
     
  13. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    Hi Christian,

    when I compare the rate difference between setPixel and setPixels there is a significant difference. One should expect this since a loop around setPixel involves the overhead (per pixel) of a function call that, apart from anything else, must calculate an address from the supplied x and y, ie. something equivalent to:

    i = y * width + x

    An extra mutiiplication and addition per pixel! Plus the loop iterators for generating x and y in the first place:

    x++
    y++

    And the bounds check on both:

    x<width
    y<height

    And when you go back to how your original data was obtained in the first place ... I'm just using ARToolkit's builtin camera capture (referenced in SimpleTest by "dataPtr" - a single dimensional array of 4 bytes per pixel ... well, you can start to see what I mean.

    With respect to getting data from the CPU to the GPU - I don't know any other way within the framework of Unity. However I didn't find that was the bottleneck.

    If I prerender a 1024 x 1024 array of Colors in Startup, I can send this to the GPU, on every update, at about 75 fps. But if I try to populate every element of the Colors array during every update call then the fps drops right down to around 20 fps.

    So the bottleneck is in how to populate the array of Colors.

    Now if your data was already an array of Color structs one could (maybe) pass that directly into setPixels ... but I haven't tried that. You'd have to reverse engineer the internal structure of the Color array - or make some eduacted assumptions and test such - and I'm not sure C# would be happy about it anyway. In C/C++ you can always ensure how every bit of every byte is structured ...

    I'm still learning how C# works.

    So I guess that's an experiment to try. Casting a C prepared byte array - to a Colors array! Is that even possible?

    Carl
     
  14. minevr

    minevr

    Joined:
    Mar 4, 2008
    Posts:
    1,018
    How to print "paper"?

    :eek:
     
  15. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    Ok, there is more optimisation one can do. C# provides a means by which to use what is called "unsafe" code. You qualify such code with the keyword "unsafe". That's an actual C# keyword. This tells the compiler not do any bounds checking on your code allowing you to access memory directly, ie. as you would in C/C++.

    The demo code I've got (from 'C# in A Nutshell' by Albahari and Albahari) is:

    Code (csharp):
    1.  
    2. unsafe void RedFilter(int[,] bitmap)
    3. {
    4.   int length = bitmap.length;
    5.   fixed (int* b = bitmap)
    6.   {  
    7.      int * p = b;
    8.      for(int i=0; i<length; i++)
    9.        *p++ = 0xFF;
    10.   }
    11. }
    12.  
    The above is C# code! I didn't realise until I read this that one could use pointers in C#. In other words one can do what is normally the province of C/C++. And according to Albahari and Albahari, it can be faster than calling an external C function because there's no overhead in leaving the managed environment.

    --

    Regarding nested loop iteration. Suppose we need to copy data from src to dest via a transport data structure. The simplest approach is:

    Code (csharp):
    1.  
    2. int src[100];
    3. int transport[100];
    4. int dest[100];
    5.  
    6. // source domain
    7. for(i=0; i<100; i++) transport[i] = src[i];
    8.  
    9. // destination domain
    10. for(i=0; i<100; i++) dest[i] = transport[i];
    11.  
    Compare the above to this:

    Code (csharp):
    1.  
    2. int src[100];
    3. int transport[10,10];
    4. int dest[100];
    5.  
    6. // source domain
    7.  
    8. i=0;
    9. for(y=0; y<10; y++)
    10. {
    11.   for(x=0; x<10; x++)
    12.   {
    13.     transport[y,x] = src[i++];
    14.  
    15.   }
    16. }
    17.  
    18. // destination domain
    19.  
    20. for(y=0; y<10; y++)
    21. {
    22.   for(x=0; x<10; x++)
    23.   {
    24.       // do what setPixel must do
    25.       i = y * 10 + x;
    26.       dest[i] = transport[y,x];
    27.   }
    28. }
    29.  
    As can be seen the first approach is going to be faster than the second approach, because there is simply much less for the compiled code to do.

    Carl
     
  16. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    The next question is how to get Unity's complier to accept "unsafe" code.

    Carl
     
  17. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    The next question is how to get Unity's complier to accept "unsafe" code.

    ...

    Ok, found it here:

    http://forum.unity3d.com/viewtopic.php?t=7533&highlight=unsafe+code

    On windows the default filepath is:

    C:\Program Files\Unity\Editor\Data\MonoCompiler.framework\compile_any.pl

    And the code to change:

    From:

    my @base_args=($mono_path, $mcs_path, '-debug', '-target:library', '-nowarn:0169', '-out:'.$output, );

    To:

    my @base_args=($mono_path, $mcs_path, '-debug', '-unsafe', '-target:library', '-nowarn:0169', '-out:'.$output, );


    ----

    BUT - having done the above, Unity is still asking me to set the 'unsafe' option. Although I closed down Unity and started it up again, it may be that the compiler is temporarily cached (sitting in memory for a little while). I'll try leaving Unity shut down for a while.

    ----

    No joy. Not sure what's happening there.

    ----

    Oh great. Found this advice:

    http://forum.unity3d.com/viewtopic.php?t=16355&highlight=unsafe+option

    Unity no longer supports the use of unsafe. The only way to use unsafe is compile your own dlls - which defeats what I'm doing since I'm working with Unity Indie.

    ----

    Some other possibilitys here:

    http://forum.unity3d.com/viewtopic.php?t=6153&highlight=unsafe+option




    Carl
     
  18. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    This works.

    Demonstrates writing unmanaged data into a Color.

    Code (csharp):
    1.  
    2.  
    3. using System;
    4. using System.Runtime.InteropServices;
    5.  
    6. public struct MyColor
    7. {
    8.    public float r;
    9.    public float g;
    10.    public float b;
    11.    public float a;
    12.        
    13. }
    14.  
    15. void Startup()
    16. {
    17.  
    18. MyColor myColor;
    19.        
    20. myColor.r = 0.42f;
    21. myColor.g= 0.0f;
    22. myColor.b= 0.0f;
    23. myColor.a= 0.0f;
    24.        
    25. Color unityColor = new Color(0.0f, 0.0f, 0.0f);
    26.        
    27. info = "unityColor = " + unityColor.ToString() + "\n";
    28.        
    29. // Initialize unmanged memory to hold myColor
    30. IntPtr pnt = Marshal.AllocHGlobal(Marshal.SizeOf(myColor));
    31.        
    32. info += "Size of myColor = " + Marshal.SizeOf(myColor).ToString() + " bytes\n";
    33. info += "Size of unityColor = " + Marshal.SizeOf(unityColor).ToString() + " bytes\n";
    34.  
    35. // Copy myColor to unmanaged memory.
    36. Marshal.StructureToPtr(myColor, pnt, false);
    37.        
    38. // display the address of unmanaged memory
    39. info += "pnt = " + pnt.ToString() + "\n";
    40.  
    41. // This is the import one
    42. // Copy the data at address pnt, from unmanaged memory, into Color
    43.  
    44. unityColor = (Color) Marshal.PtrToStructure( pnt, typeof(Color) );
    45.        
    46. info += "unityColor = " + unityColor.ToString() + "\n";
    47.    
    48. // Free the unmanaged memory.
    49. Marshal.FreeHGlobal(pnt);
    50.  
    51. }
    52.  
     
  19. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    Now I'm really quite perplexed, amazed and somewhat relieved.

    I've just discovered that I can access DLLs from within Unity Indie.

    I thought that wasn't possible. The comparison chart between Indie and Pro and says that Indie doesn't support the following Pro feature:

    C/C++/Objective-C Plugins Support
    Have a custom native library that you absolutely must use? Use a C, C++ or Objective-C plugin to directly call into it.


    So why does the following succeed in invoking the Windows OS message box in the Indie version of Unity3D?

    Code (csharp):
    1.  
    2.  
    3. using UnityEngine;
    4. using System.Collections;
    5.  
    6. using System;
    7. using System.Runtime.InteropServices;
    8. using System.Text;
    9. using System.IO;
    10.  
    11. class MsgBoxTest
    12. {
    13.     [DllImport("user32.dll")]
    14.     static extern int MessageBox (IntPtr hWnd, string text, string caption, int type);
    15.     public static void Main()
    16.     {
    17.         MessageBox (IntPtr.Zero, "Unity3D Indie version supports access to DLLs", "Attention", 0);
    18.     }
    19. }
    20.  
    21. public class DLLclientTest : MonoBehaviour
    22. {
    23.     void Start ()
    24.     {
    25.         MsgBoxTest.Main();
    26.     }
    27. }
    28.  
    29.  
    The entire time I've been working on getting ARToolkit data into Unity3D via a socket connection I could have just written a DLL instead.

    ?????

    Oh well - it was worth the effort anyway - I now know how to do sockets in C#.
     
  20. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    So here we go:

    C++:

    Code (csharp):
    1.  
    2.  
    3. #ifdef __cplusplus
    4.    extern "C" {  
    5. #endif
    6.    __declspec( dllexport ) int secretOfTheUniverse();
    7. #ifdef __cplusplus
    8. }
    9. #endif
    10.  
    11. __declspec( dllexport ) int secretOfTheUniverse()
    12. {
    13.     return 42;
    14. }
    15.  
    16.  
    C#:

    Code (csharp):
    1.  
    2.  
    3. using UnityEngine;
    4. using System.Collections;
    5.  
    6. using System;
    7. using System.Runtime.InteropServices;
    8.  
    9. class TestDLL
    10. {      
    11.    [DllImport("MyDLL.dll")]
    12.    static extern int secretOfTheUniverse();
    13.    
    14.    public static int Secret()
    15.    {
    16.       return secretOfTheUniverse();
    17.    }   
    18. }
    19.  
    20. public class DLLTest : MonoBehaviour
    21. {
    22.    static String info = "Testing DLL\n";
    23.    
    24.    void Start ()
    25.    {
    26.       int i;
    27.       i = TestDLL.Secret();
    28.       info += "The secret of the Universe is " + i.ToString() + "\n";
    29.        
    30.    }
    31.    
    32.    void OnGUI ()
    33.    {
    34.       GUI.Label (new Rect (0, 0, 200, 200), info);
    35.    }
    36. }
    37.  
    38.  
    The result in Unity:

    Testing DLL
    The secret of the Universe is 42


    So in relation to performance of an ARToolkit port I'm assuming the following will be pretty fast:

    On Startup:

    1. Allocate enough unmanaged memory (using Marshalling) to represent an array of Colors, getting back a pointer to such.

    2. Pass the pointer to a DLL, which holds it for future reference.

    On Update:

    1. Call a DLL function (which has the unmanaged memory pointer) to populate the unmanaged memory with four floats per pixel (which will be very fast).

    2. Copy the memory back into managed memory (using Marshalling) where it is now referenced by a Color reference.

    3. Pass the Color reference to texture.SetPixels.

    4. Call apply on the texture to copy the Color data to the GPU.

    On ApplicationQuit:

    1. Free the unmanaged memory.

    ---

    Now I found the only thing stopping my Indie version of Unity3D accessing a DLL was if I put the DLL where the docs (for the Pro version) say to put it, ie in:

    Assets > Plugins.

    Doesn't work.

    The Indie version (not surprisingly) doesn't look there. But if I put the DLL in the System search path (a normal location for DLLs), eg.

    C:\Windows\System32

    No problem.

    Or if I put it in the other place where DLLs typically become available to their hosts:

    The same folder as a published standalone executable.

    No problem.

    Or I put it in the same folder as the Unity editor executable:

    C:\Program Files\Unity\Editor

    No problem.

    So I guess the only thing the Pro version offers in this respect is just the convenience of it being in a single project specific location where both the editor can find it and from where it will be published. Fair enough. That's a value. But it's still bit of a stretch to sell such convenience as "plugins are a Pro only feature". Should I laugh or cry?

    Carl
     
  21. minevr

    minevr

    Joined:
    Mar 4, 2008
    Posts:
    1,018
    Why? :( :( :( :(

    In "snow leopard"'s "Parallels Desktop"'s "windows xp"


    Lost....
     

    Attached Files:

  22. attilam

    attilam

    Joined:
    Jun 2, 2007
    Posts:
    86
    I have the same error when trying on a machine where Unity is not installed.

    Otherwise I get "Error while set mode."

    When looking at output_log.txt, I get the following (cut out Filename stuff):

    Code (csharp):
    1.  
    2. Audio devices: Generic Hardware (default: Generic Hardware)
    3. Player: init engine
    4. Direct3D:
    5.     Version:  Direct3D 9.0c [nv4_disp.dll 6.14.11.8208]
    6.     Renderer: NVIDIA GeForce 9800 GTX/9800 GTX+
    7.     Vendor:   NVIDIA
    8.     VRAM:     512 MB
    9. desktop: 1680x1050 60Hz; virtual: 3360x1050 at 0,0
    10.      Found 1 devices.
    11.  
    12. 0: USB Video Device
    13. Connect to device.
    14. Found 2 supported Modes.
    15. 0: 320 x 240, 60 Hz
    16. 1: 640 x 480, 30 Hz
    17. Error while set mode.
    18.  
    19. NullReferenceException: Object reference not set to an instance of an object
    20.  
    That last line is repeated over and over again, since it's in ArDemo.Update()

    Any ideas?
     
  23. burnumd

    burnumd

    Joined:
    May 27, 2008
    Posts:
    367
    I don't think they're selling the "convenience" of drag-and-drop plugins in Pro, I think you deliberately used a fairly obscure workaround to subvert restrictions on the Indie license. But that's semantics. I think, technically speaking, you've got something very interesting, and while (from a cursory reading of the EULA) there doesn't appear to be anything specifically prohibiting this approach in the EULA, I wouldn't be surprised (especially since you posted it on the company's own forums), if this path were closed off in future versions of Unity.
     
  24. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    Well, no I didn't. It was neither deliberate, nor an obscure workaround. The location of DLLs is typically on the system path or the same folder as an executable. And this has been the case for as long I've been writing DLLs (10-15 years) - so hardly obscure.

    I was using some example C# code from a book on C# (which had nothing to do with Unity) and it accessed a system DLL all on it's own, ie. within Unity.

    And that's what surprised me.

    Now it's true I was deliberately subverting the Unity "understanding" regarding plugins (DLLs) by spending the last two months of weekends writing an alternative to DLLS: using a socket into Unity.

    But that was an alternative to DLLs! One that was allowed by the understanding

    But I didn't have to do all that in the first place. There was no need to write an alternative to DLLs in Unity Indie. Because Unity Indie - by turning off it's custom DLL path feature - defaults to accessing DLLs like any other windows application.

    I don't care if they do close off a path that every C/C++ programmer has used since DLLs were first invented.

    That's what I thought they had done in the first place. And in retrospect that would have taken the Unity developer's so much more work to do they'd want to charge more for the Indie version than the Pro version.

    I think that's exactly what they're doing.

    Carl

    NB. It should be noted that drag-and-drop plugins are a good feature (if and when they work), and definitely a selling point. My criticism was simply the way they are sold - not that the feature exists.

    If and when the marketing department decides that the developers should now ensure that all DLLs not work in the Unity Indie version at all, I pity the developer's task. I can see it right now:

    "So George, because you oversold the plugin feature you want us to now change how C# works, by hijacking it's DLL interface, so that it can't use DLLs in the Indie version.".

    "Yes, Fred. I explained this last year. Some upstart in Australia is now accessing DLLs he wrote himself and using them in the Indie version. And that's not how we agreed to sell the Indie version"

    "And I said, we can turn off the custom path feature, but if we're going to turn DLLs off altogether - that's another story"

    "And I said, well ok, lets just sell it as if we did. But now that we've done that your job is to ensure that is what we're selling"

    Press Release:

    Unity Indie Upgrade Available Today. A bug in previous versions meant you could use DLLs from C# without any problem. Bug is now fixed.


    :)
     
  25. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    Code (csharp):
    1. I have the same error when trying on a machine where Unity is not installed.
    In case there is some confusion I'm not the developer of UnityAR. I've developed my own version which uses sockets (which does work but I haven't yet publically released).

    Regarding Christian's DLL version I think you'll need to copy all the dlls from out of the various different folders in which they've been deployed - and put them into the demo folder's plugin folder. Or wherever the error messages indicate the app/editor is looking for them.

    I haven't tried this but just looking at the location of the dlls and the error messages that would seem to be the recommended remedy:

    So, find all dlls and put them in this folder:

    UnityAR_v01_beta\UnityArDemo\Assets\Plugins

    Note that some of the required dlls are in the Toolkit download - rather than the beta download.

    Carl

    Upon further reflection, the error messages appear to indicate that unity is actually looking in the right places - so my advice is misguided. Sorry.
     
  26. minevr

    minevr

    Joined:
    Mar 4, 2008
    Posts:
    1,018
    In"UnityAR_v01_beta\UnityArDemo\Assets\Plugins" have the file.

    :x ,But unity say no have :evil: ...


    ps:How to set the "Paper"?
    Customize?

    Thank you.
     

    Attached Files:

  27. attilam

    attilam

    Joined:
    Jun 2, 2007
    Posts:
    86
    OK guys, I've managed to get past the DLL not found errors (using ProcMon).

    Basically the error message is wrong: it's not UnityArLib.dll, that's missing, but the Visual Studio 2008 Runtime (get the redistributable here).

    Unfortunately the demo still cannot connect to the webcam here: it finds the USB camera, but then it comes up with "Error while connect to device".

    I will try it on more machines, but I guess something else is still missing.
     
  28. wimeck

    wimeck

    Joined:
    May 26, 2008
    Posts:
    50
    Hi Christian,

    Great work! I have a question though. I simply downloaded the beta and started the DemoScene.unity and it worked, but it's quite slow on my laptop (2GHz dual core), only getting 2 frames a second. Is this normal performance right now (I know it's a beta), or am I doing something wrong?

    Thanks again!

    Wim
     
  29. wimeck

    wimeck

    Joined:
    May 26, 2008
    Posts:
    50
    I just tried using my laptops camera, now I get 8fps. 2fps was with a Logitech Pro 9000.
     
  30. Animaleante

    Animaleante

    Joined:
    Apr 3, 2009
    Posts:
    28
    indy138:
    I have been getting the same error after I build the project.
    Problem is, from the same place you download this beta, there is the download for the unity project that only connects the webcam to unity and stream the image to the plane's material, and it works beautifully if you build the project. I have been analysing the dlls in both projects, and they seem pretty much the same. I believe that it is the .cs file that has not been written in the correct way. Hope this helps at all.

    Diogo.
     
  31. wimeck

    wimeck

    Joined:
    May 26, 2008
    Posts:
    50
    On second thought... Maybe the plugin just chooses the highest resolution the webcam offers, and since the Logitech offers very high resolutions it gets slower?
     
  32. Animaleante

    Animaleante

    Joined:
    Apr 3, 2009
    Posts:
    28
    wimeck:

    on the same site you downloaded this project, there is the UnityWebcamToolkit, download it, and when you run the project, you will see it offers the choice of modes your cam supports, that will help you see if the problem is in the cam's high resolution, or on your pc.

    Hope it helps.
     
  33. attilam

    attilam

    Joined:
    Jun 2, 2007
    Posts:
    86
    @Animaleante: You were right, the webcam handling part of both projects is the same.

    Looking at the code it turns out, that the error message 'Error while connect to device' is misleading again. The code doesn't fail, because it cannot activate the camera, but because it cannot find the AR marker description file.

    The path to it is hardwired in UnityAR.h:146:

    Code (csharp):
    1.  
    2. /**
    3.  * @brief Path to the multi-pattern definition file.
    4.  */
    5. char* model_name = "../Data/object_data";
    6.  
    (Update: there's another path to a config file in UnityAR.h:130, 'camera_para.dat' too)

    So basically it looks for the marker data in a folder that is right next to the executable's own folder. Because I put the build into a separate folder (outside 'UnityAR_v01_beta'), it couldn't find the descriptors and came up with the error message.

    After sorting this out it works now.

    Hope this helps,
    Attila
     
  34. Animaleante

    Animaleante

    Joined:
    Apr 3, 2009
    Posts:
    28
    I have placed the all the files that were in the Data folder from the demo, in the Data folder from the build, and it still throws the same error?
    have you changed anything else?

    thanks
     
  35. Animaleante

    Animaleante

    Joined:
    Apr 3, 2009
    Posts:
    28
    figured it out, the data folder must be placed outside the folder created with the build...working beautifully now, thanks a lot indy138 :D
    you really saved my ass here, the alternative would be paying 30 thousand dollars to a company to do it :(

    so, thank you!!
     
  36. attilam

    attilam

    Joined:
    Jun 2, 2007
    Posts:
    86
    @Animaleante: $30k!? Man I should rethink my pricing! J/K ;) ...seriously!?

    Otherwise thanks to Christian for releasing the source of his projects; it's a nice tutorial to learn how to program plugins for Unity, plus @carllooper for interesting findings, which I still have to try ;)
     
  37. wimeck

    wimeck

    Joined:
    May 26, 2008
    Posts:
    50
    Hi Animaleante,

    I tried the webcam-plugin, and indeed, the highest resolution gives the 2fps, while with lower ones I get around 12fps. It would be good to be able to choose webcam-resolutions in the AR-plugin as wel, maybe for the next version?
    Great stuff!
     
  38. wimeck

    wimeck

    Joined:
    May 26, 2008
    Posts:
    50
    Ah silly, the "mode NR" option in the AR Demo script lets you change the camera-resolution, for me mode 12 did the trick.
     
  39. minevr

    minevr

    Joined:
    Mar 4, 2008
    Posts:
    1,018
    Haha..Dll no found:setup vcredist. :D

    But,build to "exe" or "webplayer",why? How to fix it? :cry:
     
  40. etx

    etx

    Joined:
    Apr 7, 2009
    Posts:
    12
    From your signature I assume you're using a mac. So far the libraries are only built for windows so I'm not sure it will work for you.
     
  41. minevr

    minevr

    Joined:
    Mar 4, 2008
    Posts:
    1,018

    I am using Parallels Desktop.
    It supports webplayer? Eager to know the result.
     
  42. the_gnoblin

    the_gnoblin

    Joined:
    Jan 10, 2009
    Posts:
    722
    Hello!

    Is it possible to track motion with ARToolkit (or do I have to switch to openCV for that)?

    Also, I didn't get the idea how to "train" the example Project (Christian, thanks for UnityAr!) to recognize new markers (with some symbol written on it).

    Is it possible to track non-square shapes?

    Thanks a lot :D
     
  43. Animaleante

    Animaleante

    Joined:
    Apr 3, 2009
    Posts:
    28
  44. DaveA

    DaveA

    Joined:
    Apr 15, 2009
    Posts:
    310
    Regarding indie allowing DLL's: well, I just tried it with 2.5.1 and the console says something about a license violation or plugins only supported in pro. Anyway, was there a workaround for that (missing DLL's), or did Unity finally plug that hole?
     
  45. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    If I put a DLL in the system path (as where all system DLLs reside), or I put it in the same folder as a unity executable, and then reference that DLL (without any path-to-DLL) then the DLL is loaded by unity normally.

    This is standard practice and not a workaround.

    I assume, therefore, that a license violation occurs only when trying to reference a DLL at a particular location (eg. in the plugins folder).

    Carl
     
  46. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    A simple test of your system is the following. If you see a popup message then you have accessed a system DLL.

    Code (csharp):
    1.  
    2. using UnityEngine;
    3. using System.Collections;
    4. using System;
    5. using System.Runtime.InteropServices;
    6.  
    7. public class MessageBoxTest : MonoBehaviour {
    8.  
    9.     [DllImport("user32.dll")]
    10.   static extern int MessageBox (IntPtr hWnd, string text, string caption,
    11.                                 int type);
    12.    
    13.     // Use this for initialization
    14.     void Start ()
    15.     {
    16.         MessageBox (IntPtr.Zero, "You accessed a DLL", "Guess what", 0);
    17.     }
    18.    
    19.     // Update is called once per frame
    20.     void Update () {
    21.    
    22.     }
    23. }
    24.  
    And it's not just in the editor that it works, but in published executables.

    Original source code from here: http://www.albahari.com/nutshell/ch22.aspx

    Carl Looper
     

    Attached Files:

  47. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64
    Have just tested DLL access with version 2.6.0 of the Indie Editor and it behaves the same way as 2.5.1.

    That is, you can access DLLs that are in the system path, or in the same folder as your executable.


    Carl
     
  48. stephanws

    stephanws

    Joined:
    Nov 13, 2009
    Posts:
    1
    Hi, is it possible to take a picture with the webcam and save it out to a file (e.g. jpg)?
     
  49. jiangwave

    jiangwave

    Joined:
    Jun 5, 2009
    Posts:
    1
    hi!
    After i download the unityArtDemo.rar ,open it with unity3d,click file->build run ,then the program run ,all thing is perfect.
    this is output_log.txt 's content:
    ----------------------------------------------------
    Audio devices: Generic Hardware (default: Generic Hardware)
    Player: init engine
    Direct3D:
    Version: Direct3D 9.0c [nv4_disp.dll 6.14.11.134]
    Renderer: NVIDIA GeForce 8400M GS
    Vendor: NVIDIA
    VRAM: 256 MB
    desktop: 1280x800 60Hz; virtual: 1280x800 at 0,0
    Found 1 devices.
    UnityEngine.Debug:Log(Object)
    ArDemo:Start()

    (Filename: ..\..\Runtime\Export\Generated\BaseClass.cpp Line: 1658)

    0:
    UnityEngine.Debug:Log(Object)
    ArDemo:Start()

    (Filename: ..\..\Runtime\Export\Generated\BaseClass.cpp Line: 1658)

    Connect to device.
    UnityEngine.Debug:Log(Object)
    ArDemo:Start()

    (Filename: ..\..\Runtime\Export\Generated\BaseClass.cpp Line: 1658)

    Found 5 supported Modes.
    UnityEngine.Debug:Log(Object)
    ArDemo:Start()

    (Filename: ..\..\Runtime\Export\Generated\BaseClass.cpp Line: 1658)

    0: 160 x 120, 30 Hz
    UnityEngine.Debug:Log(Object)
    ArDemo:Start()

    (Filename: ..\..\Runtime\Export\Generated\BaseClass.cpp Line: 1658)

    1: 176 x 144, 30 Hz
    UnityEngine.Debug:Log(Object)
    ArDemo:Start()
    ...................
    -----------------------------------------------------
    But if i go into the execute directory and double click to run it , i can execute it ,but i can't see anything but a white plane.
    output_log.txt point to the erro is : "Error while connect to device."
    this is output_log.txt 's content:
    -----------------------------------------------------
    Audio devices: Generic Hardware (default: Generic Hardware)
    Player: init engine
    Direct3D:
    Version: Direct3D 9.0c [nv4_disp.dll 6.14.11.134]
    Renderer: NVIDIA GeForce 8400M GS
    Vendor: NVIDIA
    VRAM: 256 MB
    desktop: 1280x800 60Hz; virtual: 1280x800 at 0,0
    Found 1 devices.
    UnityEngine.Debug:Log(Object)
    ArDemo:Start()

    (Filename: ..\..\Runtime\Export\Generated\BaseClass.cpp Line: 1658)

    0:
    UnityEngine.Debug:Log(Object)
    ArDemo:Start()

    (Filename: ..\..\Runtime\Export\Generated\BaseClass.cpp Line: 1658)

    Error while connect to device.
    UnityEngine.Debug:Log(Object)
    ArDemo:Start()

    (Filename: ..\..\Runtime\Export\Generated\BaseClass.cpp Line: 1658)

    NullReferenceException: Object reference not set to an instance of an object
    ArDemo.Update ()

    (Filename: Line: -1)

    NullReferenceException: Object reference not set to an instance of an object
    ArDemo.Update ()
    ----------------------------------------------------
    why ? i can't understand it, anyone else get this error just like me?
     
  50. spacecat

    spacecat

    Joined:
    Jul 17, 2008
    Posts:
    16
    carllooper,
    2.6 comes with 30 day trial of Pro.
    Did you test this on 2.6 after the trial had expired?