Arcolib - Augmented Reality Components

Arcolib v1.0.0 beta has born!

A new powerful alternative to achieve augmented reality in Unity. It's easy to use, flexible and has a beautiful APIs for those who love the coding. If you are not a code fan don´t worry, Arcolib has integration modules to build your basic AR prototypes without a line of code. Here some remarkable features.

1 - Native core (written in modern C++) to increase computational performance.
2 - Detection and tracking of traditional fiducial markers.
3 - Support for classical barcodes with colors. (Only generated with the editor tool)
4 - Detection and tracking of user defined images. (Natural images detection without borders or markers)
5 - Generic input system that allows to perform image detection from a device camera, file or remote resource.
6 - Allows multiple augmentation layers at the same time. (Multi-tracker and multi-target support)
7 - Natural images model generator in runtime. Anything can be a marker.
8 - Support for pre and post-processing thru input sources allowing to equalize the image or apply fun effects without affecting the detection.
9 - Integration with Unity editor. Built-in barcode and image features generator tools.
10 - Camera chooser with sensor presets to use on editor mode.

The asset will be available pretty soon. Don't miss it!

 

Not bad actually. Will there be subscriptions? Or just the asset you buy from asset store?
What do you mean by "Anything can be a marker"? Do you mean it has simultaneous localization and mapping (SLAM)?
When do you plan to release this? I am evaluating different AR solutions, and I will need to choose one probably within the next couple of weeks.

 

I agree. Looks very promising and would interested in buying the asset from the Unity store. Much less keen on a subscription model.

Do you need any testers? I have a project which uses AR, but had to be dropped due to Vuforia's unreasonable licensing terms/costs.

 

Hi
There won't be subscriptions. Users will pay onetime fee and use it forever on any platform or any app. No more wasting time generating keys, no more water marks. All you need will be available on your Unity Editor and will be yours forever. Due the library includes a detector for natural features on images you can use any image as pattern. If you use the built-in realtime model generation that allows you to learn from the camera input and store that info, adding the orientation and location of your device (this is not provided, you have to do it by yourself) you can create a simple but efficient SLAM. Anyway I will put that request on the road map to create something more robust using maybe 3D reconstruction based on stereo disparity or geometry generation from image features and particle filtering. Any ideas you have will be embraced. The version will be available in a week (or the time that takes to be accepted on the store). The initial price will be $55, I think is fair for more than three months of hard work . Be aware that this is will be a beta version if you are planning to use it on big projects (although it works great). Thank you for your comments.

 

No subscription, pay once and use it forever. But the initial price ($55) will change when the library grow up. For the people who buy the beta version the price will be always the same.

There are new features coming like support on other platforms (currently supported on Windows, MacOS, iOS and Android), face tracking, shapes recognition and much more. I think that we certainly will need testers that's for sure . This project started like a personal challenge precisely because the lack of alternatives with a low cost to build AR prototypes or just free apps. Now I am motivated with the early results and I will improve the library until it becomes a great tool.

 

Definitely be keeping an eye on this esp. since we have all been thru hell and back w the changes the last few years re. Metaio's acquisition, Vuforia's on again off again pricing scheme/s and the PTC buyout which, frankly, still concerns me.

 

If you need another tester I am definitely interested. Cheers!

 

My main feature request would be to provide a callback method for when the trigger image is recognised. I assume you do this already?

Can you explain the 'built in barcode generator' feature? Can you generate images similar to QRCodes (or Vuforia's VuMarks) which are recognised and trigger with an encoded data variable.

This is exactly what I would need. Images I can print (in books/posters) which when scanned would trigger a method in the app. e.g. Triggered(int encodedId)

 

This looks fantastic and I think a great opportunity for you. I have done a couple of projects with Vuforia. I really liked how there system was jitter free. Hint hint.
Keep up the great work, and I will gladly be a tester.

 

I've been testing it and so far so good! of course as a beta product it can always improved, but I'm amazed with the achievement so far, multiple image recognition!! , I believe @yezaldivar will upload a video soon, showing more features from the library so stay tuned

 

Yes, the library allows you to implement a trigger when registered images models are detected. You can query the symbols found per frame and then connect the result to your game logic.

The barcode generator and detector/tracker integrated in the library is an abstraction for all 2D barcode approaches used to encode data inside square patterns. Currently the library supports classic fiducial markers in grayscale and customized color markers only detected by Arcolib. In future versions this module (barcode) will support other visual fiducial system like QR codes, AprilTags and others, allowing you to choose the proper system for your project, and you will be able to detect multiple fiducial types over the same context without any effort. This is and example about how Arcolib color markers look like.

 

It's almost ready to go out!

 

Hello, is it possible to integrate NatCam or OpenCVForUnity with Arcolib?

 

Hi
Yes it is. All components inside Arcolib are designed to be easily extended. To integrate with NatCam you just has to inherit from InputSource class and override a few methods and everything will works like a charm. OpenCV for Unity will be a useful tool when you want to perform more sophisticated preprocessing task and InputSource class provide the proper interfaces to integrate with this. Arcolib comes with its own native camera plugin (currently supported on Windows, MacOS, iOS) but you are free to use any other of your preference.

 

Where can I buy this?

 

Hi
Thank you for being interested in buying the asset. The library will be available on the Unity Asset Store in a few days. I will post the link here when it becomes public.

 

Cool. I'm definitely interested in this.

Btw: Do you have other assets or projects I can look at what you've done before?

 

This asset is also mine.
https://www.assetstore.unity3d.com/en/#!/content/69812
These are some technologies and games I have developed with my team. The interaction of these apps are achieved with depth sensors (like kinect) or common video cameras. I love to integrate image processing on my apps. If you need a more robust portfolio I can get it for you. Regards.
https://drive.google.com/open?id=0B5XrAABWqTUZbDFPMC1wNDBkcEE

 

@yezaldivar very impressive!

Consider me your future customer (even though I'm not currently working on an AR project).

 

Sure we are waiting for this...... Can you please let us know the probable time of arrival of this asst in the store.
So that I can plan my project accordingly.

 

The package was submitted the last week. It depends on Assets Store verification process. I think this week it must be ready.

 

Is a mask shader available within the solution ? it's indeed a good feature in vuforia and ARToolkit.

 

I already added the OcclusionMask.shader and MatteShadow.shader to get similar results to what you get from 3Ds Studio. This will be available in the next update after I perform multiple tests in different situations.

 

How can we know the quality of a feature? Vuforia tells you in a ranking of one to five stars how well your marker works as a feature, one of the issues of working with AR is to have good markers.

 

I agree with you. To succeed in your AR projects you must choose the proper images which contains a lot of good features. You can see how good are the images by using the Features Model Generator integrated on your Unity Editor (it comes with the library). It performs several assessments to rate the model quality when generation is done. All FeaturesModels has the qualityLevel property which is a number between 0.0 (poor) to 1.0 (excellent), even when you save the model to file and load it back again the quality level is still available. See the picture bellow and see how the editor shows the quality level.

 

Any "SLAM" recognition are inside or in "study" stage? It would also be possible through the Web API to create a new marker? Support Cardboard? (for MR) and "Extended tracking"?

I'm working on several projects in AR (and MR), if you need other testers I'm available.

 

Hi there!
You can use some tools included in the library (like 2D barcode detection and tracking) to implement your own SLAM approach. But currently there is not a specialized module to achieve that because we initially are focused on augmented reality. By using the Unity Editor and the model generator (that comes with the lib) you will be able to create limitless markers. In contrast to other AR solutions with Arcolib you will be free to create as models (markers) as you need inside your workspace, without the need to upload the images to Websites or external places. But if you need to setup a service for your own application (I mean maybe allowing the people to upload pictures so the app automatically detect that images) it is also possible by using the native core libs and some customized app on the server side. It is also possible to create real time models on your apps, just take a picture and generate a model from it. The idea is to give you the freedom to create your own experiences. About the Cardboard support I can said YES, there will be support for VR headsets in coming versions so you can build mixed reality apps.

 

Nice man! When can I try / buy it?

I would like to try it as soon as possible!

 

Count me in! I just searched and it doesn't seem to be in the store yet. I assume you'll announce here when it is. I'm really looking forward to it!

Note that on one of my assets (PixelSurface), it took the Asset Store team weeks to get around to looking at it. I eventually had to write to support to get them to act on it. If it's been more than a week and you haven't heard anything, you might want to do the same.

 

Is the Arcolib plugin already available in the Asset Store? I could not find it.

 

I sent it for review three weeks ago but nope … is not available yet.

 

You should definitely write to asset store support and complain. It sounds like your submission has gotten lost in the shuffle.

And oh hey! I had a great idea last night for an Arcolib-based app, but have no time to develop it. So I throw it out to the group here, in hopes that somebody will run with it...

Crayola makes this cool (and cheap) product called Easy Animation Studio. It comes with a little poseable figure, with fiducial marks on all the limbs:


They have their own app that looks at this, and poses a character accordingly. You can string poses together into videos.

But for the Unity community, think what a great way this would be to pose any Mecanim humanoid character! You'd just set this figure in front of the camera, pose him as you like while your Mecanim character does the same on screen, and then capture it as an animation frame. I can think of no better/faster way to create humanoid animations, especially for amateur animators like me! And I think with Arcolib, this wouldn't be too hard to do; you would just get a clean image of the various markers, and define those as features to track.

So there you go. Awesome product idea. Somebody go for it!

 

Damn .. I have an augmented reality project that is about to be published, but would replace Vuforia gladly with your solution!

 

Finally Arcolib is available on the Asset Store!
https://www.assetstore.unity3d.com/en/#!/content/76937
Hope you enjoy it!

 

Awesome sauce! ...And, bought!

Keep up the great work!

 

Oh! bought!

 

Thank you guys for your support!

 

Hi, I just ran a sample from this asset on my device which comes with a 4Gb Ram with Snapdragon 810 CPU. here are my observations
1. the camera is not full screen and doesn't has an seamless auto rotation and the video is flipped.
2. The fps is very low.
3. The 3D model is not stable.

 

The video seems to be flipped when you export the examples in portrait mode because all demos are meant to be exported as landscape left. (Working in video tutorials on how to export to mobile). The full screen issue is because the default resolution is 640x480, just set the desired resolution on the object called ArcoCameraRig (in the scene) and then you can set any resolution. Choosing higher resolutions like 1920x1080 won’t affect the performance due internally the library compute integral images from input that are scale and rotation invariant. We are going to upload a demo supporting multi-orientation, this is not a library issue is just a demo missing feature. After some test on the hardware you mention (One Plus 2) we found the problem is maybe related with the implementation of TBB, only happens on specific hardware but we are going to fix it. We already test the library in other devices achieving a good fps.

The library (in our opinion) provides an acceptable 3D pose estimation that will be improved in coming version. The results also depends on the quality of the images used as marker and the camera sensor you have installed on the device. Thank you for your feedback we are working to improve/fix any issues you found.

 

Hi just bought Arcolib, epic potential by the way so thank you for your effort in bringing it to the asset store. I just bought it but its throwing an error and I just cant get beyond it. I hop it's just user error but any help you can offer is great appreciated.

get_dataPath is not allowed to be called from a ScriptableObject constructor (or instance field initializer), call it in Openable from ScriptableObject 'Features Mode Generator Window'

 

Fixed, apparently I had left the Features Building in an incomplete state but not building, seems to be ok now though.

 

Hey!
That issue comes out with the new versions of Unity Editor. (5.4 or higher ) It's a weird behavior that appears when you initialize a member variable on its definition. The fix is already applied to the next version (ready for the end of this week). This issue don’t affect the library at all but obviously the annoying message needs to be removed. On the script FeaturesModelGeneratorWindow just move the initialization of _latestImageUrl = Application.dataPath; to the void OnEnable() body...and that’s it!
Thanks for buying the library. New features will be available pretty soon!

 

You're welcome, looking forward to seeing what's coming next. Once again thank you for developing the asset.

 

I just bought this and it looks really cool! I'm having trouble figuring out how to create my own marker though. For instance, just using the SimpleFeaturesSetup example scene and replacing the "floral-women" image and training data with the "logo" image and training data (that the build created) and it isn't seeing it. Am I misunderstanding it?

 

Hey!
You are in the right path, just a couple of things...the logo texture must be in true colors (without DXT compression) if you are going to use it as input. Verify in the Unity Editor (Import Settings) the format must be RGB 24 bit or RGBA. Also increase the logo image size to 300x300 pixels or more, the logo image is too small to be considered a valid input. Do that and it should work!

 

Thanks! I'll try that when I get off work that makes sense! Knew it had to be something simple. Seems to be very well put together.

 

Hi community!
Doing some tests in multiple Android devices we discover that the Unity camera APIs can be extremely slow in some devices (mainly when it has 8 cores, that`s weird right?). That's why we are going to speed up the NativeCameraInput support on all platforms. Currently the default camera used is the Unity WebcamTexture but is becoming a bottle neck in performance.

 

Me again, I tried resizing to 512x512 and it is RGBA, I changed all the settings on the image to match the settings on the "floral-woman" but still no luck. The only difference I'm noticing is the image type is RGBA Compressed PVRTC not the same as your NPOT 32 bit, which I can't figure out why

 

The texture must have raw pixel data, no compression is allowed for input. See the image below your settings must look like that. It works for me.

 

Hello. I am developing an app that needs to recognize the species of a butterfly in the pics took by the user (in an indoor facility). The photos must be taken when the butterfly is stationary and with wings wide spread (not from every posible angle, plase check image below). I tried vuforia, but the image recognition is boolean and only recognizes the exact same image target. I was wondering if the image recognition scripts used for this assets would allow me to set a recognition threshold (even if i need to edit the code myself).If not, please consider this functionality since several other AR/image recognition libraries/frameworks dont support comparison threshold or are too expensive. Thanks for your time and support.

Example image:


P.D : By the way the app must work on both Android and iOS, thats why i am using Unity 3D (that and because cool graphic effects).