Resolved ARKit support for iOS via Unity-ARKit-Plugin

Thanks, migrating now !

 

Hey, not sure if you've already answered this, but ARkit is super grainy on my iPhone 6/7 right now, especially in low light conditions. Which is odd considering the camera app gives a really clear image. I've seen other ARkit videos which seem to have clearer images, so I'm not sure if its my phone/app or the plugin?

 

ARKit camera runs on lower resolution than usual. Apples choice. More about it earlier in this thread

 

Unity packaged UnityARKitScreen scene, Xcode packaged on the phone, why only show blue screen?I'm using the new plug-in

 

Posting a FAQ in the hopes that people do not repost the same questions over and over again. Please look through this, and if possible, through the forums before you ask your questions!

Unity ARKit Plugin FAQ

Q: What is the Unity ARkit Plugin?

A: Unity ARKit plugin will provide developers with friendly access to ARKit features like world-tracking, live video rendering, plane estimation and updates, hit-testing API, ambient light estimation, and raw point cloud data.


All the features exposed by the ARKit are available from the C# scripting API within Unity. There are also Unity Components for easy integration of these features with existing Unity game projects. Please see the README.md file in the bitbucket repository for more info.


Please join us at https://forum.unity3d.com/threads/arkit-support-for-ios-via-unity-arkit-plugin.474385 to get started.



Q: What are the prerequisites for making the Unity ARKit Plugin work?

A: This is an experimental preview plugin and it works with beta software, and it should not be used in its current form for production use. It requires the following:

• iOS 11 Beta 7
  
  
• XCode 9 Beta 6 with latest iOS SDK that contains ARKit Framework
  
  
• Unity version 5.6.2 or later, or version 2017.1.0 or later
  
  
• iOS device that supports ARKit (iPhone 6S or later, iPad (2017) or later)

Q: Where can I get the Unity ARKit Plugin from?

A: Unity ARKit Plugin (experimental) and its example projects are released as open-source in the bitbucket repository https://bitbucket.org/Unity-Technologies/unity-arkit-plugin . It is also updated on the asset store: https://www.assetstore.unity3d.com/#!/content/92515



Q: Is there any Step-by-Step Tutorial written to create ARKit Project for Unity (from scratch) yet so we can quickly grasp basic concepts?


A: You could start with TUTORIAL.txt that is in the bitbucket.org repo and follow the links there.


Q: Is there a description of the various examples included with the Unity ARKit Plugin?

A: See SCENES.txt included with the plugin distribution.

Q: I’m trying to get the included project to load, but I get compile errors like these: “Assets/Plugins/iOS/UnityARKit/UnityARVideo.cs(59,31): error CS0117: `UnityEngine.TextureFormat' does not contain a definition for `R8’
Assets/Plugins/iOS/UnityARKit/UnityARVideo.cs(66,31): error CS0117: `UnityEngine.TextureFormat' does not contain a definition for `RG16'”
What have I done wrong?


A: You need to upgrade your Unity version. The minimum Unity versions are:

Unty 5 - version 5.6.2 or newer.

Unity 2017 - version 2017.1.0f3 or newer


Q: When running, I get this error message in XCode console:


“2017-06-05 17:22:43.726156-0700 ProductName[757:217524] [Session] Unable to run the session, configuration is not supported on this device: <ARWorldTrackingSessionConfiguration: 0x1c429b7b0 planeDetection=Horizontal worldAlignment=Gravity lightEstimation=Enabled>

2017-06-05 17:22:43.788207-0700 ProductName[757:217524] AR FAIL”


What is wrong?


A: Your device must have a minimum of an A9 processor (so iPhone 6s and later, iPad (2017) or later). If in doubt, check your processor in this table


Q: But the documentation says that there is a lower level of AR (3DOF) that should work for devices older than iPhone 6S. But it doesn’t work when I try it?


A: There was a little note on iOS 11 beta release notes: “ARSessionConfiguration is only supported on iOS devices with an A9 or later processor and will be enabled for all iOS 11 devices in a future beta.”


Q: Cannot see the content of assets folder in project inspector. It seems that unity cannot read assets folder and library folder. How to solve it?


A: You may have upgraded to macOS 10.13 and chosen to use the APFS filesystem. There is currently an incompatibility between Unity and APFS filesystem. Please check this link for updates.


Q: Will there also be support for the new Vision framework in iOS 11? How about CoreML?

A: This plugin exposes only the ARKit functionality that is currently available.


Q: If I want to use front facing Camera, for tracking is it possible to do that using the ARKit?

A: This is a question for Apple, but the answer currently is that ARKit only works with rear facing camera.


Q: How do I record a video of my awesome ARKit app?

A: iOS 11 has functionality to record video of your screen. See this link.


Q: Can I use Unity ARKit plugin with Unity Vuforia support?

A: Currently, not simultaneously. Both of them require exclusive use of the camera that is on the device, so when one is active, the other cannot be.


Q: I get a linker error like "Geo::GetiOSAppDocumentsDir()", referenced from:” . How can I fix it?

A: If you see this, make sure the Build Setting in Xcode for Dead Code Stripping is set to Yes


Q: I have an app that I want to support ARKit with, but I also want to support devices that do not have ARKit support. How do I do that?

A: Create an ARKitWorldTackingSessionConfiguration just as you would normally, but before using it to RunWithConfig, query IsSupported on that struct. Then you could decide on a path through your game based on the result of that query.


Q: I have created a gameobject in the scene, but it seems to move with the device camera movement. I cannot seem to get any closer to it.

A: This could be because you created it really far away and with a large scale, and it is really moving closer to you, but you don’t notice it due to perspective. It could also be that you have started ARKit with 3DOF, which will allow you to rotate, but not move in the world.


Q: I got the plugin, and when I export it to my iOS device, it gives me a blank screen with “Waiting for connection..” on it.

A: Go to your Build Settings and specify the “UnityARKitScene” as the scene to export.


Q: The resolution of the video image I get on my app seems lower than that produced by the camera app on the same device?

A: That is correct, ARKit uses a lower resolution camera feed that we upscale to fit your screen. It’s possible they do this to speed up all the processing needed for AR.

Q: I'm trying to get the Unity ARKit Remote to work, but it doesn't succeed in connecting Editor to Remote with messages like "Failed to connect"?

A: Read and follow the instructions carefully. The most common issue is forgetting to make the Remote app use "Development Build" in step 2 of the instructions.


Q: I got the plugin, and I have the correct scene, but when I build and run on my device, it gives me a blue screen.

A: This could happen if you have a device that is not supported (e.g not A9 or better cpu), or your provisioning profile or signing credentials are not setup correctly in XCode. See the XCode console log for clues on what it might be.

Q: Can I use the Stimulator when I build in XCode or I have to connect with a real iphone with iOS11?

A: Simulator does not support ARKit at this time - please use a real device.

Q: I have read everything above, but this plugin is not working as expected. What should I do?

A: Most issues have been solved already and there are hundreds of developers using this plugin to develop apps, so more than likely, your problem is one of the ones in this FAQ, or someone has posted on the forums about it already. If you think you have a legitimate new issue and want to post a question about it, please include the following information:

Xcode logs attached in a txt file (not screenshots of Xcode)

Name of Scene built with unity

Device info (iPhone 7, iPad Pro gen 2, etc.)

iOS beta version

Xcode beta version

Plugin version and whether from bitbucket or asset store

 

I plug in directly, and the configuration files are automatically configured by the plug-in. I haven't changed any place

 

Fellow awesome forum-crew!
I am currently working on a project where my asset is already placed on startup, and calibrated as soon as the ARKit anchors down. I'm working with an asset that is supposed to float in the air, but i want to mimic a shadow with the shadowcatcher underneath the asset, which means that i need to know where ARKit places the ground on the Y-Axis.

What are your easiests ways of accessing the "ground level" in Y?

Thanks in beforehand

 

I also encountered this problem, and later found unity is not supported in ARKIT unity

 

Hi, after update to latest ARkit plugin and Xcode 9beta 6, the app is crash on launch, the debugger shown as in image. Tried with clean ARkit project, same result.

I run the demo successfully on previous version, but crash on latest update, what did I do wrong? Any other info you need?

 

Are you using the latest ARKit version? Because I'm working with both the latest Xcode beta and IOS 11 Beta, and mine works.

 

Checklist (I think it's 99% the case that one of these is not done):

• update iOS11 beta
• update Unity to latest 2017 patch release
• update XCode beta to latest
• Download latest UnityARKIit plugin from bitbucket from the link in OP and replace it in your project
• Maybe: replace XCode build instead of append (can someone clearify if this is important?)
• Use a A9 or better iOS device.

 

I have updated to the version you have mentioned. But the same issue happens to me. Could you please see the attached and let me know if the setting is incorrect? Thanks.

 

Can I use the Stimulator when I build in XCode or I have to connect with a real iphone with iOS11?

 

Your question has been added to FAQ - answer is Simulator does not work, you have to use a real device.

 

Yes I have used the real device, and I got this error message.

clang: error: linker command failed with exit code 1 (use -v to see invocation)

Could you let me know how to fix it? Thanks!

 

Something of a minor issue, but I'm using the latest Xcode beta (9M214v), and yesterday when I archived our project and tried to upload to the asset store (for use in TestFlight), I got an error about an invalid toolchain. I didn't copy it down at the time, but it was to the effect that the version of Xcode I was using was not authorised to push to the App Store.

I tried again today with the same configuration, and this time uploading worked. Since this was a very new version of the Xcode beta I was wondering if -- in purely hand-waving talk -- TestFlight takes some time to "catch up" with the latest Xcode versions in the wild. I feel like this has happened before with the same solution.

Obviously this is more of an Xcode problem than Unity/ARKit specifically, but I thought maybe others had experienced something similar given how fast they're updating the software.

 

Oh I have fixed and everything worked! Instead of open ".xcodeproj" I used ".xcworkspace".

One more question, is there any way to preview the work in Unity? Because when I click "Play" in Unity, the screen is green like this. (I used the default ARkit scene)

 

Thanks!

One more question, is there any way to preview the work in Unity? Because when I click "Play" in Unity, the screen is green like this. (I used the default ARkit scene)

 

I got this bluescreen with
- ARKit Plugin 1.0.8 from Assetstore / ARKitTestScene
- XCode 9 beta 6
- Unity 2017.1.0f3
- iPod 7.1 6thGen / iOS 11 beta 7

Same project build with same XCode + Unity version works on:
- iPhone 7 Plus / iOS 11 beta 5

 

Thanks for all the work you guys have put into this and answering all of our questions on here!!

Before we can use AR in our apps we have to give the app permission to use the camera through the native iOS popup. I'd like to show some info to our users before the popup appears, and then after confirmation show the native popup. Once they've made their selection, I'd like to either continue to the game if the camera is enabled, or go back if they've decided not to give permission.

I'm assuming it's best to just show any info before enabling the ARKit plugin? Once it's enabled, the popup will appear automatically? After the user has made their choice, is there any way we can tell? I noticed in the XCode log I can see an error if I don't give permission, as follows:

[Session] Session did fail with error: Error Domain=com.apple.arkit.error Code=103 "Camera access not authorized." UserInfo={NSLocalizedRecoverySuggestion=Make sure that the application has the required privacy settings., NSLocalizedDescription=Camera access not authorized., NSLocalizedFailureReason=The app does not have permission to use the camera.}

But what if they've not given permission then restarted the app, or disabled the camera from the app's settings? I can't see the the above error in my log, is there another way to check if the camera is enabled or not?

 

Alright! I'm just piping through all of the methods! Thanks!

 

yes. search the thread for "remote"

 

Hi there... in order to get mouseclicks in the editor working when using the remote, I ended up just using
Camera.main.ScreenPointToRay (Input.mousePosition). Is there any particular reason you use UnityARSessionNativeInterface.GetARSessionNativeInterface().HitTest in the MakeBall script, rather than testing against the plane gameobjects already created?

p.s. love your work!

 

Is there a method to find the plane coordinate not using HitTest?
I want to get the 4 points of planes detected.

 

I wrote the removal of objects that are associated with AR, but Planes remain on when the scene is restarted.

Code (CSharp):

1. foreach (GameObject go in destrObj)
2.             Destroy (go);
3.  
4.         SceneManager.LoadScene ("Map");

 

Could you guys remove from the arkit plugin, so that it does not change the project settings and does not nullify layers. It’s a pain in the ass to set those again.

Nevermind. I got it. Just tick off the project settings from the package import. Lol.

 

Hello Unity team

Pls See my fork https://bitbucket.org/LiveAnimationsGit/unity-arkit-plugin-vr

1. Added supported device Flash Light. Need when working with poor lighting.
Example:
UnityARSessionNativeInterface.UnityARSessionFlashLight(true); // Light ON
UnityARSessionNativeInterface.UnityARSessionFlashLight(false); // Light OFF

2. Update debugPlane added World tiling, added tiling shader for debugPlane

3. Added MatrixARWorldSize for ability to change the coordinate multiplier and dimension in ARkit. It is necessary to change the size of the entire grid.
UnityARMatrixOps.MatrixARWorldSize = 1.0f // Standart World grid size(Defaults)
UnityARMatrixOps.MatrixARWorldSize = 100.0f // More than 100

4. Off green screan on Windows and mac. Added YUVRGBShader.shader supports YUV and RGB, Update UnityARVideo.cs added support WebCamTexture For Windows and Mac WebCamera

Use this fork AS IS. Perhaps this will help you in developing ARkit...

 

Just tried the editor support and its running really, really slow for me. I would prefer to be able to just pick a colour we could render the background to instead?

 

In the ball generation case, your method works just as well. Thanks for the suggestion - most other folks can use the same method for remote (since remote does not support hittest yet) and game. The main reason you would use HitTest API is if you wanted to find the nearest feature point where there was no plane, or if you wanted to find a hit against a plane that will be found on the current frame.

 

I'd like to animate a blur effect on the ARKit video feed. I see that UnityARVideo.cs is using a CommandBuffer to blit the video frames, but I'm not very familiar with CommandBuffers.

I'm guessing I need a blur shader on a material (anyone have one handy?) and then would I want to add that to the existing CommandBuffer or make a second CommandBuffer for it that goes after the existing one?

-A

 

You could make your own version of YUVShader that could do this for you. Since the command buffer uses that shader to combine the Y and UV textures, you can put your blur changes into your version of the shader, and set it to be the shader for the ClearMaterial.

Alternatively, you could use a shader with GrabPass, or follow some of the advice here: https://blogs.unity3d.com/2015/02/06/extending-unity-5-rendering-pipeline-command-buffers/

 

Hi!

I'm having some issues in loading new scenes. My game has 5 levels, so at the first one I recognize planes and build my level based on it. When player finishes the level, I load the next level using SceneManager.LoadScene. App crashes immediately. I would like to load the new scene but keep the planes information to continue tracking the ambient. Is it possible? My crash is relatable to it? I tried to DontDestroyOnLoad all AR references (UnityARCameraManager and ARPlaneDetector basically) but crashes too.

 

Make sure you have all the ARKit related gameObjects set as DontDestroyOnLoad, and your new scene does not contain any ARKit related gameObjects. You can also disable "plane finding" once you have found the world position where you're going to put your levels.

 

Heads up - I've reorganized the project on BitBucket (basically moved files around so that they are in more obvious directories). I've tested that all the parts work still, but let me know if you find something messed up. I'll now be looking at some of the PRs and feature requests to see what I can integrate. The Asset Store version will not be updated until a later date.

 

Hello all. Thank you very much for this asset and thanks for replying so fast.

I have a problem. Plane detection starts always after a few secs (avg. 5 secs). Once it starts to find some planes, it detects very well. However until the first plane detection, sometimes it takes too much time to find a plane. I have very good table with a good texture. Once it starts to detect something, it detects the whole table very quickly and very well. But as I said, until the first detection, sometimes, app needs 10 secs. And sometimes, it never detects at all..


Has anyone experience such situation? Is this a normal behaviour? If not, do you have any idea why it is like this?

 

This is normal behaviour - you can check the tracking state reason, it will most probably be "Initializing". Mainly its due to having to detect enough feature points to create a plane. You can help it along by moving the device around a bit to help with feature detection.

 

Unity really should have an In-Build feature for simple AR marker traking to mix with the current ARKIT.
Its the very last step to make it great!...
Doesn’t ARKIT includes this already anyway?...

 

Hi all! I have a question regarding arkit and unity in general, So I integrated Arkit and Google Cardboard for a new experience in Unity and found that the field of view on a cardboard is very limiting and so to combat this i have brought wide angle lenses, such as a fisheye lens (about 180°) and a super wide angle lens (140°). Both have failed completely with arkit tracking. Hence this leads me onto my question how would i be able to increase the FOV on a Google Cardboard (basically the camera) and still retain good tracking with Arkit? Similar to this https://www.zappar.com/zapbox/
Thanks a lot!

 

Hey guys whenever i try to build the demo scene I'm getting the following error I'm Using Xcode 9 beta and iOS 11 Beta 6

 

Checkout our Product: "ARZombi" (Fall 2017 Release) built with Unity and ARKit. We've been making AR products for years now - it is such a relief to have an SDK supported in Unity that does so much that we need to make a fun product!

 

This has been answered many times - get latest XCode 9 Beta!

 

How big of a team are you guys? Because this is awesome! And to have a finished project by fall is just a dream, I am just starting out as a developer and would love to know.
Thanks and I will definitely be the first to download!