Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

UnityAR - ARToolkit Interface

Discussion in 'Made With Unity' started by Christian, Sep 22, 2009.

  1. Q_B

    Q_B

    Joined:
    Nov 29, 2010
    Posts:
    14
    Just to illustrate what I'm trying to work out:

    This is just a WIP right now, and I don't really know if something will really come out of it, but;

    This is based on a new game that is about to be released for the PS3, Under Siege. I'm not part of the dev team of the game in no way, but I know them and worked in some projects together in the past. (btw, check the game, it'll be a blast if you're into strategy / RTS / RTT )

    I'm trying to create a simple AR card game to help advertise the main game, basically. Something like Avatar's action figures AR stuff ( http://www.youtube.com/watch?v=7JWk_JIE3Ow )

    It's all very unpolished right now, as you can see above. Just testing for line of sight, and enemy/friendly checking at the moment, no actions or game mechanics implemented till now. The 3D models and the artwork come from the main game, obviously, with the needed adaptations for Unity.

    Would be really nice if I can get something finished out of this, put it up on the web, with decent camera support and acceptable framerate :)

    Thanx!
     

    Attached Files:

  2. steffan

    steffan

    Joined:
    Jan 7, 2011
    Posts:
    12
    It's my first post, so hello to everyone.
    I have trouble with run the UnityAR from .exe, or .app- everytime when i do this, i see only a blank webcamplane, without any streamed video..
    But when i run it in the editor, everything works great..
    I've tried to
    but it didn't worked, so my question is; is there any other solution, to solve that problem ?

    thanks in advance for your help.
    Steffan.
     
  3. Q_B

    Q_B

    Joined:
    Nov 29, 2010
    Posts:
    14
    Well, the Data folder must be located one level below your executable. So, for instance, you have a "base" dir. On top of that you must have a "build" dir, and inside there goes the .exe and the [yourappname]_Data dir. On the same level as the build dir, that is, on top of the the "base" dir, you must have the "Data" dir. That works for me. Hope it helps :)
     
  4. sbrk

    sbrk

    Joined:
    Feb 1, 2011
    Posts:
    4
    Althought I read the whole thread: is this compatible with the Mac version of Unity?
     
  5. eduardo-dudis

    eduardo-dudis

    Joined:
    Feb 3, 2011
    Posts:
    4
    I am facing problems with build an application... the camera doesn't open after I build the project... Inside Unity and clicking on "play" button, all run fine...
    I am using MacBook Pro 10.6.6 and download UART from link https://research.cc.gatech.edu/uart/

    Can anyone help me ? Thanks !!
     
    Last edited: Feb 4, 2011
  6. AqusNovae

    AqusNovae

    Joined:
    Feb 13, 2011
    Posts:
    11
    No
     
  7. AqusNovae

    AqusNovae

    Joined:
    Feb 13, 2011
    Posts:
    11
    novacom check your email >:p


    dce evarhcet, e yesmaat de du iuo :D

    ttyat kinect ddvyulomiheeh udu :p
     
  8. Q_B

    Q_B

    Joined:
    Nov 29, 2010
    Posts:
    14
    Well, it's been some time now ... any advances? guys? anyone? :)

    I'd be really happy with some news here ... even if it's something along the lines "Sorry but I'm not sharing anymore" or something like that.
     
  9. AqusNovae

    AqusNovae

    Joined:
    Feb 13, 2011
    Posts:
    11
    I can't speak for novacom but I helped him get his project to work on the kinect as a additional feature, using the depth information of your hands as 'substitute markers'. He showed me a standalone program that connects to unity and can connect with firefox. It was pretty responsive and cool. Hes pretty paranoid with data though, I had to work on a spare computer in his apartment D:. He is still waiting his friend to design him a flashy interface and loading screen for a external program that 'feeds' information into unity. I can post a beta when its finished, as he promised me a copy. The software looks pretty close to completion at least when he showed me how it works. I've used unityar before but his software seems to be better at picking up markers, although that might be a illusion created by his dramatically better framerate.
     
  10. Q_B

    Q_B

    Joined:
    Nov 29, 2010
    Posts:
    14
    lol @ "spare computer in his apartment" :D

    Ok, thanks for the answer! As I said before, what I'm really interested in is the core part, the ability to use the camera on a web deployment, and the revised core dll (for faster camera support).

    I think I kinda started with the wrong foot with that "frankenstein-coder-type" thing. I was just trying to say I'm not a top-notch programmer. I now my way around C#, and I can take care of pretty much all the standard programming needs in Unity, GUI stuff, normal game mechanics stuff, etc etc.

    I think that in the example I posted above it pretty much shows I know how to use the .dll in my own code, and implement it according to my needs. So, I could really live without flashy interfaces and helpers and stuff. BUT, obviously, I'm just humbly asking for another person's work, so of course if novacom thinks it needs to be a whole thing with all the extra stuff, then ok, it's his stuff, and no one knows better than him :) Also, if he decides he's not giving it away for nothing, that's also completely ok by me, obviously.

    As for me, I just needed the core of it, really :) But it's all good!

    And again, thanks!
     
  11. priceap

    priceap

    Joined:
    Apr 18, 2009
    Posts:
    274
    hello -
    I have been following the thread for a while and have worked with the artoolkit plugin with plans to start developing a new project. I would like to know what updates there are in regard to the method for circumventing the texture apply() bottleneck.

    I have followed the links referenced in the thread, where details are discussed regarding about plugins transferring texture data, but none lead to a ready example that shares how this is accomplished, that I can tell.

    Can anyone pursuing this objective shed some light on this? thanks!
     
  12. AqusNovae

    AqusNovae

    Joined:
    Feb 13, 2011
    Posts:
    11
    ^ It depends strongly on your individual project/idea. To work within the bounds of unityar you have to be creative and many ideas just wont work. But if declare your idea the unity community may be able to help you be creative :D. The reason you need more fps would also be helpful. :p
     
  13. AqusNovae

    AqusNovae

    Joined:
    Feb 13, 2011
    Posts:
    11
    In terms of outright fixing the unityar bottleneck. I have a 'fix', but i cant release it publicly because of my university, hence why i am helping Novacom with his project. Try separating the videostream and AR portions of script into two different scripts is all i can say. Preferably get the video feed off a external program. Its not very hard, just ask a programmer to do it. It took me personally 2-2.5hrs with no initial direction. So any good computer science student, freelancer or teacher could probably do it in a day for you (you sound like your surrounded by such people). I personally achieved 340 fps (unity 2.61) 260fps+ (unity 3.0) on my i7 laptop with a high definition camera on mid-high settings by doing I described. Also remember that a web camera generally runs at 30fps or 15fps so anything above that number should generally just be 'food' for adding other stuff like fancy graphics (making it look better).
     
  14. priceap

    priceap

    Joined:
    Apr 18, 2009
    Posts:
    274
    Thanks for the reply. Just to clarify - by your statement "by doing as I described", are you referring to "Try separating the videostream and AR portions of script into two different scripts" ?

    Are you saying that you got the frame rate increase you have while still using setpixels and apply functions, but just managed in different scripts? Or are you using a method that bypasses those functions (a different plugin from UnityARs, etc.)

    thanks.

     
  15. AqusNovae

    AqusNovae

    Joined:
    Feb 13, 2011
    Posts:
    11
    yes

    Still using UnityAr for the AR (The AR works without the video apply call but you get a blank screen). Using another plugin and external application (both I wrote within the said time) for handling the video. Same AR result better frame rate and video quality.

    Cheers.
     
  16. priceap

    priceap

    Joined:
    Apr 18, 2009
    Posts:
    274
    Ok - that is what I thought. We switched to UART instead of UnityAR, which separates the two in the first place, among other things, and yes I get a much higher AR frame rate without the video displayed, and I will continue looking for a unity user that is willing to share advice on transferring video frames directly into a native unity texture without using setpixels. Otherwise, like you noted, I will figure it out for myself. Maybe starting with the threads referring to using gl_texture and then along with opencv or videowrapper - In any case it is a great head start that Christian R provided for all of us to build upon in the community!

    P.S. It would be interesting to see some images or description of your project sometime -
    thanks

     
  17. AqusNovae

    AqusNovae

    Joined:
    Feb 13, 2011
    Posts:
    11
    I have never used UART. I created my unityar 'version' a long time ago considerably before UART existed (at least I strongly believe so). My work was not recent but near the birth of unityar. I did not in any way copy the methodology of UART as I came up with the idea (I assume its a similar idea) before its creators did :p. I was only using unity3d because I was forced to by my university D: I don't really like the software as firstly it isn't designed around C++ D:< (grrr) and it isn't suited for creating professional applications even with the unity 3.0 release (novacom has the opposite view, but i think that's because he is lazy and has a problem finishing work) I'll post screenies of my work later.
     
  18. priceap

    priceap

    Joined:
    Apr 18, 2009
    Posts:
    274
    No worries there about when you created your version, AqusNovae, I was not comparing your method with UART from Ga Tech. ArToolkit is from HITLab under a GNU public license, and UART is under the same, so that the developer community can continue to contribute and benefit. It is entirely your choice not to explain methods or your school's choice not to release your implementation in unity, but if not, there really is no bearing whether your did yours before or after another group released theirs publicly, but thanks for letting us know about your success with it, which serves as an inspiration for others.

    I won't comment on whether I agree with your note that unity is for lazy people or that Novacom is lazy. I noticed he is from NZ, and we have not heard from him on the forum so I hope he and his family and friends are safe after the tragedy in Christchurch.

    As I continue to find places for improving performance I will share what I have found. So far using UART, I changed the camera to texture resolution to power of 2 and was able to double the frame rate. I wanted to have at least 640x480 res, so using a 1024x512 texture and then using setPixels like this:

    m_Texture.SetPixels (0,0,640,480, cam.m_vidframe, 0);

    And then calculate the texture scaling and offset in the renderer.material so that it is properly registered in the viewport. On my laptop, this takes the fps from 20 to about 45fps, greater rates on our quad core towers at work. This frame rate will be fine while developing the visual project and can continue in parallel on an alternate video transfer method.

    I will keep an eye open for your screen shots - thanks.
     
  19. novacom

    novacom

    Joined:
    Jun 30, 2010
    Posts:
    16
    Haven't been here in a while...

    Don't worry AqusNovae calls everyone lazy, because admittingly he is 'naturally talented' . He's the lazy one, you see he does this thing where he pretends work takes longer that it actually takes for him so everyone gets impressed how much 'time' he puts into the work he has done. In reality what takes normal people days takes him hours. This completely distorts his perception of how long projects take as he believes that people who have more experience than him should be faster than him. He's actually really nice in real life and helps a lot of people in his spare time which more than makes up for his habit. My personal progress is good but projects take time. I see there are other questions for me but I have to go. I'll log on later. p.s unity is for designers or lazy programmers that's a given :p

    Ok, I admit this is true D:
     
  20. AqusNovae

    AqusNovae

    Joined:
    Feb 13, 2011
    Posts:
    11
  21. sekhar77

    sekhar77

    Joined:
    Nov 22, 2010
    Posts:
    2
    Hi Friends,
    I'm new to this unity3d, i wanted to build an Marker based augmented reality application for iphone i've a full version of unity3d.
    could any one plz suggest me to implement an AR application on iphone using unity3d.

    Thanks in advance,
     
  22. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    does anyone know if this works on the iPhone or Android?
     
  23. Q_B

    Q_B

    Joined:
    Nov 29, 2010
    Posts:
    14
    Bump Bump!

    Hello again! :) Novacom, AqusNovae, and all the other people I fail to mention ;)
    Any news?
     
  24. Milad

    Milad

    Joined:
    Aug 17, 2010
    Posts:
    67
    Hi,

    When i build UnityAr demo,exe file can not open the camera,but i do not have any problem in unity editor.what must i do?

    Please help.thanks.
     
  25. wangjia

    wangjia

    Joined:
    Jan 9, 2010
    Posts:
    7
    Hi Christan,

    Thanks for the awesome plug-in. I have a question about the marker resolution. I am using multiple markers which are close to each other. And they got mis-identified for each other often. I noticed the default resolution of marker data file is 16*16, which seems to be defined in the "config.h" file in the "External" folder.

    Code (csharp):
    1. #define   AR_PATT_SIZE_X       16
    2. #define   AR_PATT_SIZE_Y       16
    However after changing them to 32 and recompile the UnityAR plug-in, the marker don't get recognized. By the way, I used the flash version of AR marker generator and set up the resolution to 32 when I generated the pattern files.

    Do you have an idea of what I did wrong? I appreciate your help!:)

    Best,
    Jia
     
  26. ermariete

    ermariete

    Joined:
    Apr 11, 2010
    Posts:
    5
    Hi all
    I downloaded UART but I can not make it works over iphone 4... it works perfectly over MAC system.
    I also tried using 3GS but no result

    Any ideas??
    many thanks guys
     
  27. eduardo-dudis

    eduardo-dudis

    Joined:
    Feb 3, 2011
    Posts:
    4
    Hi ermariete,

    I downloaded UART too and I use Macbook Pro... inside Unity all works fine... but I am not be able to build an application... Xcode fails... Could you explain to me how do you make it work ?

    Thanks !!
     
  28. flamy

    flamy

    Joined:
    Feb 7, 2011
    Posts:
    194
    any idea of how to make it work with unity web player
     
  29. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,601
    Not possible.
    AR on webplayer is only possible at very low framerates by streaming them in from a flash / java plugin.
     
  30. flamy

    flamy

    Joined:
    Feb 7, 2011
    Posts:
    194
    hey can i get the source code of the UnityAr
     
  31. flamy

    flamy

    Joined:
    Feb 7, 2011
    Posts:
    194
    can i get the source code of the UnityAr dll
     
  32. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,601
    that will not change it you can not use it then either. The webplayer does not allow any lowlevel access and thats requierd to talk to hardware elements directly like the cam.
     
  33. flamy

    flamy

    Joined:
    Feb 7, 2011
    Posts:
    194
    i understand web cam wont work in web player, i just need the pattern recognition algorithm to know how it works
     
  34. flanker

    flanker

    Joined:
    Jun 29, 2011
    Posts:
    9
    i've seen some examples on the web where a flash applet is used for providing webcam stream on the unity player app, so it's possible
    but wrap datas on the unityAR dll...
     
    Last edited: Jul 25, 2011
  35. eduardo-dudis

    eduardo-dudis

    Joined:
    Feb 3, 2011
    Posts:
    4
    Hi,

    Could you help me to make UART build the project on macbook pro ? I am facing Xcode problems while building the application...

    Thank you !!!
     
  36. flamy

    flamy

    Joined:
    Feb 7, 2011
    Posts:
    194
    how can i make my own marker file?
    with my width and height
     
  37. flanker

    flanker

    Joined:
    Jun 29, 2011
    Posts:
    9
    see in /data of the project, you can find some marker files as patt.hiro, so you have to create a file as them and reference it as others in the object_data file.

    -----

    I have some problems to resolve in my custom UnityAR project:
    - I'm adapting the system for AR, not on webcam stream, but on a loaded picture as jpg, so a fixed frame... so the detection is ok, 3d objects take a good perspective, but depending on pictures, I have some problems of precision in orientation and translation. I can cheat on vector3 and quaternion on c# script but for others pictures the result is bad. I don't have this problem on stream. No idea?

    - Then , in stream mode, when a marker is detected, it's ok, but when I don't target a marker, I would like objects display be off, I've tried to manage confidence indice and object.visible values in the dll to prevent perturbations but i still have 3d objects anarchically displayed.

    Thx for your help
     
  38. flamy

    flamy

    Joined:
    Feb 7, 2011
    Posts:
    194
    hey but with ar my web cam is not initialising(starting), even if i installed all the drivers, i.e. all web cam drivers
     
  39. BeiZhang

    BeiZhang

    Joined:
    Mar 9, 2010
    Posts:
    2
    After building a PC standalone exe file, did the following steps, it worked for me.

    The "Data" folder contains 8 files "camera_para.dat, object_data, patt.hiro, patt.kanji and patt.sample2" and 3 pdf files. Add the "Data" folder into your build.
    1. Create a folder -- WebCamBuild.
    2. Copy the "Data" folder to the WebCamBuild from the sample demo package after unizip it.
    3. Create a subfolder "build" under WebCamBuild
    4. Copy your exe file and its data folder to the subfolder "build".

    Run the exe file, and it should work.

    Just a clue here, you may find a better way.
     
  40. daus

    daus

    Joined:
    Oct 15, 2011
    Posts:
    1
    Hi guys
    I'm trying to make a solar eclipse simulation by using unity AR. When "moon" and "earth" had been detected on screen, the moon moved automatically to the place between earth and sun. But I couldn't make the moon "run" leave its marker,it always back to its original position

    I was wondering which parts of the script that I have to change in order to make the moon can move out from the marker

    thanks !!
     
  41. Starkudo

    Starkudo

    Joined:
    Jan 5, 2012
    Posts:
    3
    Hello friends, I'm having problems when testing an example, the following error appears when compiling:



    Opening the script appears that the error is here ...



    Does anyone have any suggestions of what might be wrong?
     
  42. falkonragno

    falkonragno

    Joined:
    May 31, 2010
    Posts:
    29
    Does anyone here have and working example project of this great technology that could be share, a lot of people sure will appreciate that..
     
  43. tonioseiler

    tonioseiler

    Joined:
    Feb 23, 2012
    Posts:
    5
    I have the Trackerwrapper Example up and running (Multitracker Example).
    For my project, I need to replace the videowrapper plugin with the standard webcamtexture available in unity3d pro (3.5)
    I get data from the webcam texture by calling getPixels32.

    When initializing the webcam, i create a pointer to the data:

    data = new Color32[webcamTexture.width * webcamTexture.height];
    dataHandle = GCHandle.Alloc(data, GCHandleType.Pinned);
    dataPointer = dataHandle.AddrOfPinnedObject();

    which I pass to the dll function

    TRACKER_getReports(handle,dataPointer, 640, 480,(int)format,repsPtr,ref num);

    Everything else is the same as in the example. The code compiles and seems to run, but it never finds any tracker.

    Thanks for help in advance.

    Tonio
     
  44. falkonragno

    falkonragno

    Joined:
    May 31, 2010
    Posts:
    29
    Wow Tonioseiler, do you have a working example?, could you share it here, maybe would be easier for the people here also help to migrate it to 3.5
     
  45. tonioseiler

    tonioseiler

    Joined:
    Feb 23, 2012
    Posts:
    5
  46. tonioseiler

    tonioseiler

    Joined:
    Feb 23, 2012
    Posts:
    5
    I finally found a solution to this.

    Place the followingfile in the Plugins directory and add the script to the MainCamera.



    In the TrackerManager, change the webcam node to WebcamWrapper and set the variable in the inspector to the MainCamera.

    In the TrackerManager.UpdateMarkers method, change the Pixelformat:
    Remove the VideoPlane and delete the VideoWrapper directory from the Plugins Directory.
    Remove the VideoWrapper entry in the Plugins file.

    Thats basically it.

    To render the Video on an object, just add a renderObject on the WebcamWrapper.

    Use this and you will experience a much better performance.
     
  47. zackivano

    zackivano

    Joined:
    Mar 20, 2012
    Posts:
    11
    Hi Tonioseiler,
    Thank you very much for sharing your code! :D

    I'm trying to make your changes in the UART examples to use WebcamTexture instead of video plugin but i cannot figure out how you did it.

    I deleted all the videowrapper plugin folder and did the changes you wrote, but in TrackerManager there are many references to VideoCamera class attribute that doesn't match the WebcamWrapper class. I tried to figure out changing the following things :

    - VideoCamera.m_vidpointer_byte with WebcamWrapper.frameBytePointer assuming that is the pointer to byte[] version of the frame to send to the Uart plugin.

    - VideoCamera.m_width and height with WebcamWrapper.cameraWidth and height.

    - in the UpdateMarker:
    Set the Pixel_Format format = Pixel_Format.PIXEL_FORMAT_RGBA;
    and commented out the if statement:
    if (CameraNode.m_depth == 32 CameraNode.m_pixelFormat == 32993)
    format = Pixel_Format.PIXEL_FORMAT_BGRA;


    - I changed the call to the plugin function in
    TRACKER_getReports(handle,CameraNode.frameBytePointer, CameraNode.cameraWidth, CameraNode.cameraHeight,(int)format,repsPtr,ref num);

    The WebcamTexture is working fine but no more tracking in the MultipleTracker example. I think I wrong something about frame format or pointers.

    Any suggestions?

    Ivano
     
    Last edited: Mar 20, 2012
  48. Thimon

    Thimon

    Joined:
    Mar 26, 2012
    Posts:
    1
    Hello all,

    For a school project I want to use AR. As I have some experience in Unity3D I would like to use the ARToolkit for unity. Problem is: I can not find it anywhere. All searches end up in this threat. If I understand correctly there is no link to the ARToolkit in the first post. But a addition to the ARToolkit.


    Does anyone know where to find it?
    Thanks!
     
  49. tonioseiler

    tonioseiler

    Joined:
    Feb 23, 2012
    Posts:
    5
    I use the following code for the TrackerWrapper:
    [i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i][/i]
     
  50. Lypheus

    Lypheus

    Joined:
    Apr 16, 2010
    Posts:
    664
    Is there a download link, going to your site I don't see a "entwicklungen" section :).