Arcolib - Augmented Reality Components

Hi!
To achieve this you definitely need a custom algorithm. I think the first step is to define the variables you are going to use for your classifiers (wings shape, wingspan, color and patterns in the wings). If the pattern in the wings are similar by species you can use some algorithms like fisherface or eigenface to classify your images due is for indoor use I assume the lighting conditions will be controlled. This is more a recognition feature than augmented reality but I can figure out how to create a detector for scenarios like this. I suggest you to try some OpenCV demos (available in C++, Python, or Java) about FisherFaces classifier, is widely used to detect ages or glasses on the faces, I think it should work for butterflies.

 

Hi!!. we are an indie team and already bought your asset. but I have a question..

We use to work with Vuforia, and we are working with some kind of experiences where the user can interact with objects and it is reality important to change gravity and other status in real time, when you lose or found the target..

In vuforia there are the methods "OnTrackingFound()" and "OnTrackingLost() ".. but here.. I can't see how we can handle it..

Anyways
Thanks for your awesome work!!

 

Hi!
Thanks for your feedback. We are working hard to improve the performance and tracking quality and then we will add new features (OCR, face tracking, geolocation). About your request it’s pretty easy to achieve using the trackingState property in the Symbol objects. When you call Tracker.queryFrame it return the latest frame computed by the detection algorithm. The first time, when a symbol is found it has a Detected state, later when it persist on next frame it has Tracked state. You can do your own hash table like Dictionary<id, TrackingState> symbolEvent and in each frame you must update that hash after queryFrame. If the symbol is not present on the hash is a Found, and when a symbol is not there anymore (in the new frame obtained by queryFrame) is a Lost. We can include this in coming version as a built-in feature, but anyway with this approach you can adapt it to your needs.

Regards

 

Finally got it to work and can for sure say it was a experience level issue not an asset issue, it's working great now and is easier/better than expected. The ease of use is the biggest draw for me, essentially a drag and drop AR once you get the hang of it. I'll be writing a good review for it soon.

I must say it seems, with no tweaking, that is a bit short on distance of recognition even with an "excellent" marker. It seems like if you lock on close you can pull your phone far away and it holds onto the marker, but if you try to start far away it can't find it. Although I imagine there is some tweaking or future updates that will make it better. I also had a pipe dream of allowing it to recognize user draw markers, like have them draw 3 dots for this Model, 4 dots and this one appears, etc. but I knew that was unlikely as it's not much of a quality marker.

Also a tad bit shakey, if I lock onto the target and then pan down with my camera it reorients/rotates itself a lot making for a jumpy view. Again though I'm sure this is a easily fixable issue and shouldnt discourage anyone reading this, I'm already very satisfied by this product.

 

All your comments are appreciated. We are now fixing the camera capture to make it blasting fast then we will add QR support and improve other features. There is also a sort of sketch detector on the way, I will put a road map on Trello so all the community can see what's next.

Thanks you for your review on the store, that help us a lot.

 

I am buying this on Monday! I am very excited to see how far you go with this.

I just used Vuforia for a client AR project (with the $499 usd fee) and was planning on using Easy AR for two smaller client projects and Vuforia for my next big one. This could change everything. :-D

 

@yezaldivar

When are you thinking of putting the road map on trello?

Cheers,
Marc

 

Hi again!!.. when I test the app in my cellphone.. you can see in the image attached.. the image camera is flipped..

In Unity I try to compile with default orientation "Autorotation" and forcing "Landscape" or "portrait" .. And happend the same..

My cellphone is a Xiaomi Redmi Note 3 Pro
(SnapDragon 650 1.8Ghz) 6 cores
3 Gb Ram

Also is extremely slow..

 

Hi!
This issue was already reported and fixed in the current version, please send me a private message with an email to send you a new package. (as it will take a few days until become public on the store). Thanks for the feedback that help us a lot!

PS The reason of this is because the first release does not support native camera capture on all platforms, and relies over Unity WebcamTexture that is not very fast on mobile devices. The current version has it own camera plugin (written in Java for Android and Objective-C for iOS) so the capture is faster.

 

Hi everyone!

Arcolib 1.0.1 beta has been submitted for approval to the Asset Store, the main features for this update are:

- Native camera capture is now supported on all platforms providing faster previews, low CPU load and better detection results.
- All detection algorithms are now parallelized using TBB.
- Improved input controller editor panel, now is possible to choose the default camera in mobile devices (front or back).
- Optimized tracking algorithms to be faster.
- The library now handles all device orientation automatically without affecting the detection during the flip.
- Fixed some minor issues with the Unity Editor reported by the community.
- New APIs documentation in HMTL format.

Since now when you export your projects to mobile don’t forget to choose native camera capture (see picture below). Of course the WebcamTexture input is still supported and updated to the new features like auto-orientation.

 

I just bought Arcolib!

Thank you for creating this asset. I hope you get many, many more sales.
I reckon some of the larger subscription based AR SDK's are going to be getting pretty worried soon.

 

Bought Arcolib earlier today! Looking forward to replace Vuforia with this!

 

It is amazing once you get going (which takes no time at all) and the support is some of the best I have ever experienced anywhere!

 

Thank you!
We are trying to do our best!

 

I, too, feel from the videos that the image tracking is a bit shaky compared to Vuforia. Otherwise your product looks fantastic with an affordable price tag.
Do you support only flat 2D image trackables? Is it possible now or will it be supported later to use e.g. box shaped (cuboid) trackables or other simple shapes, like a cylinder?
( I am much confused with your term for "(image) model" = "trackable image" as for me by deafult "model" means a 3D model...)

 

Yes, the tracking needs to be improved, we are already working on it. About models is a pretty similar concept to trackable dataset from Vuforia (or other AR solutions). Arcolib's trackers use models like a persistent source of data to be used during detection and tracking. Models describes the appearance of the objects to be detected, and handle some control and operations (save, load, create) over it life cycle. Only planar images can be detected on the current version. The most used objects in Arcolib are Models (data), Trackers (analysis), and Symbols (results). Once you get it you can't be missed at all.

 

I have been able to get some really solid (non-shaky) performance from Arcolib thanks to the fantastic support from @yezaldivar. It took a few small tweaks to my trackers and a few small alterations to my Unity scene. I am confident that this asset will get better and better.

I will post a video when I get a chance.

 

By the way. It is worth noting that if you are in the UK and want to use Vuforia in a commercial project you also need to be VAT registered.

I make under £80k GBP per year and choose not to be VAT registered because of all of the extra paperwork and stuff. It seems they don't really want us little guys as customers unless you are prepared to do all the VAT stuff too.

It's fine if you have a client who is VAT registered who is happy to pay but if you want to release a small AR project on your own, forget about it.

Its another great reason to buy Arcolib though.

 

Is there any extra cost OR royalty to publish the app on app store by using this Arcolib ?

 

No extra cost. Once you buy this asset, you can use it for as many projects / apps as you like with no royalties and no additional costs.

Obviously you have your yearly developer / publisher licences for iOS / Android, etc...

This is why it is such an amazing deal!
The price will go up once it is out of beta but @yezaldivar said he has no plans to move to a subscription model.

The support he provides is also worth noting. You would not get this level of assistance / interaction from any of the other AR SDK's out there.

 

It's exactly like Marc said! Arcolib will never be distributed under subscription model and it price will always be under $100 USD. That's for sure. (I am writing this here so if I die tomorrow everyone knows that was my will) The most exciting feature (besides all the modules on the way and the coming improvements) is that Arcolib will be part of a bigger ecosystem of digital interactive publicity. Yes, you will have the chance to earn money using Arcolib, together with multiple cloud services (or without the services). The business model for those services are not completely defined but it will always be a good deal for everyone. Arcolib will remain on the Unity Store and will always be under the pay once use forever model. You are free to use Arcolib as many time as you need it. We don't keep records of your apps neither your information.Thank you, just be patient there is a lot of good things on the way.

 

That sounds exciting! Feel free to use me as a guinea pig if you need any testers for the WIP modules. ;-)

It's in @yezaldivar's will now and recorded on the Unity forums for all eternity....

 

Hi there, just purchased this (another one who blaked at the price of vuforia), run into a wee problem though. Just followed your quick start tutorial and targeted my galaxy s6, when ever i try to open the app, my phone tried to launch into GearVR? any ideas? Thanks.
PS, very new to unity and not a very strong coder so please be gentle.

EDIT
AH, unticking Virtual Reality Support in Player Settings / Other Settings did the job. That's not been a problem before but a fix is a fix

 

Is there is extra cost to publish the app on app store by using arcolib in our application ?

 

There is no extra cost for using Arcolib in your application.

 

I hope the dev would improve on the stability of the tracking before branching out to other features.
Im still stuck with Vuforia, but looking for other alternative, as Vuforia team dont really give any sh*t to their customers, really bad support from them. The other alternative is Kudan which is very expensive.

 

I bought and had a quick play with the AR, great stuff.

Is there a way for only the camera to do the moving instead so that the target image helps in creating a groundplane instead? I want to have a whole game level load up and use AR to transform the camera instead.

 

Agree with you. We are working on that, the new release will have a new improved tracking algorithm. Thank you for buying the asset.

 

Hi!
It's possible (doing some tricky changes on symbol matrix and camera projection matrix) but if you have only one camera and your augmented world there is no difference in visual appreciation on moving the world or the camera (one relative to each other), but there is a lot of difference in performance when you have multi augmentation layers and have a camera for each layer. That’s why Arcolib uses the approach to project the objects to the same camera rig instead using one camera per object. Besides due it will support Unity VR mode (and Hololens) running in this mode Unity takes the control over the camera orientation, you can only place the other objects relative to the camera. A good way to place all the objects over your augmented layer (world) is creating a plane, then use that plane as ground of your world distributing all objects over the plane, even placing the light as child of the plane, later you can hide this plane or apply the MatteShadow shader to simulate shadows over real objects. See the picture below. Thank you for your feedback!

 

Excellent! I am looking forward to the next release.

The tracking is definitely acceptable for stationary objects at the moment.

One of my clients decided to start playing with an AR truck like a three year old today (I created some artwork for one of their trailers and they wanted to see it in situ) and it did lose the object a few times.

I would love to be able to create a mixed reality experience with trackable, augmented weapons using Arcolib.

 

Hey there,

I just bought your asset. I'm mostly frustrated about the inability to change the texture resolution for image recognition over at Vuforia. I just have two questions:

* If I understood correctly the resolution set in InputSourceController is only for the raw camera input, before applying any image recognition your scaling it down, right? How can I influence the size at which this happens. Do you have any other hints or ideas how to improve the distance/marker size ratio for initial detection?
* Are you already working on improving the overall shakyness (is that a word?)? Is there any possibility to beta test these? I'm currently developing an app for the next roadshow of a big client and would love get my hands on your most recent work

So far I enjoying this asset very much,
thank you!

 

Hey!
Thank you for your feedback!
When you set the resolution in the InputSourceController the capture will find the closest resolution supported on the current device. Increasing the size on the capture won´t affect the speed of the tracking and the quality may be slightly affected. The input image is scaled down using a fast pyramidal method, and then converted to integral images to perform detection over it. The current tracking algorithm will not improve the detection by choosing higher resolutions. We are working in a completely new tracking algorithm but is not ready yet. When we have something stable we can send you a preview. Currently we are working on:

- Improving the tracking in both BarcodeTracker and FeaturesTracker. (40%)
- Including QR detection on BarcodeTracker (90%)
- Early support for VR (Cardboard, and other Headsets) (25%)

 

Thank you for the quick response!

 

Hello, you have any android or ios example app we can test?

 

Very pleased with my investment in arcolib so far. Nice going.
Got a non arcolib but hopefully still relevant question. I want to create a virtual cavity on a real surface (think ceiling cat meme). Is there an easy way in unity to screen off or wipe out the unwanted polygons that are drawn beyond the 'real reality' surface? I've looked at culling but that doesn't appear to be the right approach. I'm trying to do something along the same lines as this:

thanks.

<EDIT> Doh, Depth masks. I do research these things before asking.. just appear to have a habit of finding the answer after asking.

 

With Arcolib you can easily achieve this using the OccluderMask shader included in Prefabs/Shaders folder. Just create a new material with the shader ( FX category) and set it to all objects you want to use as mask. Thank you for sharing your comments.

 

Yes I do! You can try this. This demos are very basic and uses the previous library version. I will update and remake the demos as soon as possible.
https://drive.google.com/file/d/0BxO_MNCv9eCXNWNTUnNMaEFab00

 

Out of interest, when do you think face tracking will come to Arcolib?

 

Works well, It need a little improvement in tracking and performance but is a good alternative to vuforia, really worth it

 

Once I get a fast and stable tracking for the FeaturesTracker I will start the face tracking/algorithm. It will take at least one month more. I prefer to refine the existing modules before add new features. The face tracking module has a lot of details I will release first the face position estimation, then the 3D orientation, later the landmarks detection and finally expression recognition (smile, anger). It’s a very complex module there is a lot of work in it.

 

Yes, I am working to improve that! Thank you!

 

Hi Yezaldivar,
I just bought Arcolib through the Asset Store today. Thank you very very very much.
Considering your great work I would argue that Unity3D should be working with you instead of Vuforia for AR support out of the box. Last week I've been in a conversation with a "pricing specialist" from Vuforia. They want to sell me a "Consumer Pro license" to enable me to use Vuforia on a Microsoft Surface Pro 4. Prices starting at $2500 per app! And this is for a very basic app that just puts a 3D augmentation on top of a reference image. Ridiculous. And completely unsustainable for a business of our size. We have been doing AR for 10 years now and we're in a bad place since Metaio was bought by Apple. AR development used to be reasonably priced. With a fixed price for the SDK ( and the permission to use it without limit ) we used to be able to earn back our investment in AR technology. Right now, parties like Vuforia, Wikitude, Catchoom, Kudan, Layar, etc... have ridiculous prices ( in most cases per app and sometimes even time limited ) that just don't allow normal sized companies to start playing with Mixed Reality. It's way to expensive and the quality and service is just not that great as it used to be in the Metaio days. Very nice to see initiatives like yours. Keep up the good work.

 

I completely agree with you @theolagendijk! I honestly think Unity should employ @yezaldivar to head up the AR technology division! Unity is already known as a great game engine and a fantastic platform for Augmented Reality with external SDK's. Imagine how amazing it would be if it just worked right out of the box! I am also really excited about the Otoy mixed reality capabilities that are coming to Unity 2017 but I am not sure if we will see anything until the end of April.

I am a happy owner of Arcolib anyway now and I am sure it is going to get better and better thanks to @yezaldivar and his vision for this asset. I would love Unity to give him a massive wad of cash and a full time job though. ;-)
He may be perfectly happy as a lone wolf though....

 

Would it be possible to use a US Dollar bill as a marker?

 

Yes, absolutely.

 

Yes, you can use any descriptive image (non repetitive patterns).

 

So building on the tutorial, I've been creating and attaching more complicated assets containing multiple objects to the features tracker instance script. So far so good until I try moving things around with script.
I've played around with google cardboard and vuforia, in both cases they appear to work by moving the camera around whilst your target scene stays still within the unity coordinate system.
Arcolib appears to keep the camera static and move the objects around. This is a bit of a pain as it means everything including all the built in physics now has to be calculated relative to the connected moving object.
Is there a simple way around this?

 

Hi!
Yes, Arcolib uses a different approach but there are good reasons for that. Arcolib is designed to build augmented reality by activating or deactivating specific components, each component can be used simultaneously with others and they must recycle the resources as much as possible. Camera is one of those reusable resources, imagine for instance the same camera being controlled by the device orientation to see real world placed objects and also detecting images patterns at the same time, with this scenario the symbols detected must be oriented respectively to the camera, because the camera is free to move anywhere (this happens with Hololens based apps), besides this approach is very accurate in terms of performance because you can have multiple augmentation layers at the same time with only one camera. Another good reason is actually the real physic simulations, when the camera move around the objects is hard to achieve physics behavior over multiple surfaces. With Arcolib this is pretty easy to do. In situations (like yours) when you have only one world and you want to simulate different camera view angles there is an easy way to do it. Just multiply the world (parent container) orientation quaternion to the local gravity direction and set the resulting vector to the physic engine.

https://drive.google.com/open?id=0BxO_MNCv9eCXZGVMdUl0Q2ZzdVk

This will simulate the gravity exactly as you want (keeping the objects hierarchy relative to the parent). To apply forces over local objects is exactly the same, just multiply the container quaternion with the local directions and you will get the proper vector (quaternions and vector multiplication is not commutative so you have to be aware of that). As simple as that, and you have a more versatile library adaptable to most situations. See the script used in the video.

 

Great!
Thank you for providing this example and for explaining some of the concepts behind Arcolib.

 

@yezaldivar I'm in the process of transferring some existing Vuforia projects to Arcolib and I noticed that your features tracker script toggles the visibility of the referenced GameObject by calling setActive(true) to show it when tracking the features and calling setActive(false) to hide it when not tracking the features.
This of course works perfect but it's slightly different from the approach that existing Vuforia developers like me are used too. The Vuforia DefaultTrackableEventHandler loops through all your GameObjects Renderer Components and Collider Components and enables/disables them like this;

Code (CSharp):

1.  
2. private void OnTrackingFound()
3. {
4.     Renderer[] rendererComponents = GetComponentsInChildren<Renderer>(true);
5.     Collider[] colliderComponents = GetComponentsInChildren<Collider>(true);
6.  
7.     // Enable rendering:
8.     foreach (Renderer component in rendererComponents)
9.     {
10.         component.enabled = true;
11.     }
12.  
13.     // Enable colliders:
14.     foreach (Collider component in colliderComponents)
15.     {
16.         component.enabled = true;
17.     }
18.  
19.     Debug.Log("Trackable " + mTrackableBehaviour.TrackableName + " found");
20. }
21.  
22. private void OnTrackingLost()
23. {
24.     Renderer[] rendererComponents = GetComponentsInChildren<Renderer>(true);
25.     Collider[] colliderComponents = GetComponentsInChildren<Collider>(true);
26.  
27.     // Disable rendering:
28.     foreach (Renderer component in rendererComponents)
29.     {
30.         component.enabled = false;
31.     }
32.  
33.     // Disable colliders:
34.     foreach (Collider component in colliderComponents)
35.     {
36.         component.enabled = false;
37.     }
38.  
39.     Debug.Log("Trackable " + mTrackableBehaviour.TrackableName + " lost");
40. }
41.  

This approach has some benefits.
As you can imagine my existing scripts attached to augmentations are based on the assumption that the tracking only affects the enabled state of Renderers and Colliders.
I'd like to do a feature request that will benefit developers like me that want to migrate their projects from Vuforia to Arcolib.

Feature request;
Can you add a Boolean checkbox to FeaturesTrackerInspector that would allow us to chose for the "enable/disable Renderers and Colliders approach" instead of your approach of toggling activeInHierarchy.

For now, I can write it myself, but I think it would help out all those developers like me that are testing to see if Acrolib is a viable alternative for their current tracking solution.
Cheers!