Resolved ARKit support for iOS via Unity-ARKit-Plugin

Found my problem. Integrated ARKit into an existing project that was using Natcam for camera support. Initializing Natcam was screwing up the ARKit camera matrix. Heads up.

 

These are great! Would love to get insight around your scene and physics set up for the "minigame" with the catapults. Have been trying to get the physics system to work with smaller objects and haven't quite nailed the right approach there.

 

Great to have ARKit working within Unity3D!
I am trying to optimize out rendering performance, but the standard FPS prefab always indicates 60FPS when I run the example App on iPad (2017, no pro). Even when I notice the difference (lag time) when focusing on a large (>4M triangles) mesh.

 

I am trying to get the samples running on my iPad. I'm using the latest Xcode build 9 beta 5, iOS 11 and Unity 2017.1.0f3. I built the Unity project for iOS successfully and was able to open it in Xcode. When I try to run the app on the device, however, both the app and Xcode crash. If I re-try, sometimes Xcode does not crash itself and shows a SIGABRT in line 419 of ARSessionNative.mm, in the method sessionDidUpdateFrame(). There's also an warning saying 'displayTransformwithViewportSizerientation' is deprecated on the same line. Any ideas how to get the sample running? I tried the same with Xcode 9 beta 3 a while ago, and there it worked. Unfortunately, Apple does only offer beta 5 now.

 

Hi,
We need to update the plugin for Xcode/iOS Beta 5. This will be done later today. Sorry for the inconvenience.
Cheers,
Chris

 

Hi everyone,

Does someone know a way to reduce the quality of the image returned by the camera with ARKit?

Thanks in advance

 

Why exactly would you want to do that?

 

I don't know if this is happening for everyone, but on my two dev phones the Unity ARKit plugin crashes on load on the new iOS 11 Beta 5. Happens even from the standard scene from the plugin in the asset store. About to try the bitbucket repo to see if it has a fix.

Code (Boo):

1. UnloadTime: 2.501125 ms
2. 2017-08-07 18:01:18.861614-0500 arkitscene[315:13501] [MC] Reading from public effective user settings.
3. 2017-08-07 18:01:19.371961-0500 arkitscene[315:13345] -[ARFrame displayTransformWithViewportSize:orientation:]: unrecognized selector sent to instance 0x1c02e9200
4. 2017-08-07 18:01:19.377308-0500 arkitscene[315:13345] Uncaught exception: NSInvalidArgumentException: -[ARFrame displayTransformWithViewportSize:orientation:]: unrecognized selector sent to instance 0x1c02e9200
5. (
6.     0   CoreFoundation                      0x0000000185584d74 <redacted> + 148
7.     1   libobjc.A.dylib                     0x0000000184a98528 objc_exception_throw + 56
8.     2   CoreFoundation                      0x0000000185592218 <redacted> + 0
9.     3   CoreFoundation                      0x000000018558a708 <redacted> + 1380
10.     4   CoreFoundation                      0x000000018547035c _CF_forwarding_prep_0 + 92
11.     5   arkitscene                          0x0000000103e9f980 -[UnityARSession session:didUpdateFrame:] + 756
12.     6   ARKit                               0x00000001a00c7b54 <redacted> + 132
13.     7   libdispatch.dylib                   0x0000000184f0946c <redacted> + 24
14.     8   libdispatch.dylib                   0x0000000184f0942c <redacted> + 16
15.     9   libdispatch.dylib                   0x0000000184f15e70 <redacted> + 1016
16.     10  CoreFoundation                      0x000000018552d138 <redacted> + 12
17.     11  CoreFoundation                      0x000000018552ad14 <redacted> + 2012
18.     12  CoreFoundation                      0x000000018544b558 CFRunLoopRunSpecific + 436
19.     13  GraphicsServices                    0x00000001872cef84 GSEventRunModal + 100
20.     14  UIKit                               0x000000018f116984 UIApplicationMain + 208
21.     15  arkitscene                          0x0000000103b051cc main + 172
22.     16  libdyld.dylib                       0x0000000184f6ddb4 <redacted> + 4
23. )
24. 2017-08-07 18:01:19.381619-0500 arkitscene[315:13345] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[ARFrame displayTransformWithViewportSize:orientation:]: unrecognized selector sent to instance 0x1c02e9200'
25. *** First throw call stack:
26. (0x185584d5c 0x184a98528 0x185592218 0x18558a708 0x18547035c 0x103e9f980 0x1a00c7b54 0x184f0946c 0x184f0942c 0x184f15e70 0x18552d138 0x18552ad14 0x18544b558 0x1872cef84 0x18f116984 0x103b051cc 0x184f6ddb4)
27. libc++abi.dylib: terminating with uncaught exception of type NSException

 

Same issue using the version from the BitBucket repo. Time to roll back to beta 4...

 

Thought i would post a little AR Kit showcase thingie since i got reminded about the use of realtime pseudo reflections in AR using the camera feed, so here is a very metallic statue in dark conditions and brighter conditions. As you can see the metallic surface adjusts nicely to the change in brightness, though i find materials like these extreme and the results would likely be better with a lesser reflective surface.




Edit - Bonus image using relatively shiny wood:
http://abload.de/img/img_012185s9u.png

 

If I had read above I would have seen this is already a known issue. Sorry about that.

 

I see some of demo when app cannot detect plane show custom message like "The room is to dark", "Find more space" it's a random message or UnityARKit can detect that?

 

How do I access the timestamp in ARFrame in C#? I'm having issues figuring this out.

 

Originally each brick was 2x1x1 meter, but I later scale the entire thing to 2%. The final mass is 0.2 kg. the ball is 8 kg. and the gravity for the project is set to -0.2 to give it this slow motion feeling

 

This is a damn good plugin ! We are working on a ghost hunting device, see it in action here:

I'm a bit stuck now with beta 5 - do you know any easy way to rollback to beta 4 on the device ? I can't find this release anymore in the iOS dev center.

 

got a fix for this : in the file "ARSessionNative.mm" located in the Plugins/iOS/UnityARKit/NativeInterface, replace:

Code (CSharp):

1. s_CurAffineTransform = [frame displayTransformWithViewportSize:nativeSize orientation:[[UIApplication sharedApplication] statusBarOrientation]];

by

Code (CSharp):

1. s_CurAffineTransform = [frame displayTransformForOrientation:[[UIApplication sharedApplication] statusBarOrientation] viewportSize:nativeSize];

Works here at last, and will help while the official fix gets in !

 

For the first, they are probably using https://developer.apple.com/documentation/arkit/arcamera.trackingstate.reason , which is returned via Unity ARKit Plugin, while for the second, they are probably checking the extents of the plane that is found and making sure its large enough for their content (again can be obtained by checking plane extents, which are received by plugin).

 

For anyone having trouble with remote: for me it did not work with the asset store version, but it works with the bitbucket repo.

Update on the reflections: this is a plane and a block hoovering above it.


The main Camera is used to render the reflections, but still shows the solid background color instead of unity AR Video in reflections.

I should probably change the 4th line to something else?

Code (CSharp):

1.            Camera reflectCamera = go.GetComponent<Camera>();
2.  
3.             reflectCamera.backgroundColor = clearColor;
4.             reflectCamera.clearFlags = reflectSkybox ? CameraClearFlags.Skybox : CameraClearFlags.SolidColor;

 

Hi, first good job on implementing this!

2 questions:

1. I want to place large GameObjects physically quite far from each other, think 20 meter large objects, 50 meters apart. Should I place an anchor at each of the GameObjects ? Or is 1 anchor enough ?

2. Is there a way to lock those GameObjects in place, so if the user picks up the app a few days later, the GameObjects will be in the same spot ?

Many thanks in advance!

 

If you copy the link for beta 5 but switch the 5s to 4s you should be able to download Xcode again

 

I'm using the image returned by the camera for sending it through a high performance websocket and I need to find the best transfer/performance relation.

Am I able to change this setting?

Thanks

 

This works for us as well. My partner found the same fix last night/early this morning. Thanks for posting!

 

Did you ever figure out the problem with this? I have experienced something similar with additive loading of another scene - where the ARKit Scene's "Random cube" is the only thing properly synched with Pointcloud tracking, while the other objects were only bound by IMU and loosely stayed where they were originally placed. Also, I noticed that larger scenes needed more time for ARKit plugin to localize...

 

Thank you! This worked for me as well.

 

Answering my self: this could not be done by now as the following post says https://forums.developer.apple.com/thread/81301

 

hi, maybe we've updated our iPhones to quit, but it's not working anymore... ios 11 beta 5
any news

 

Running into a small problem with the ARKitRemoteConnection on 2015 iPad Pro.

Everything is as would be expected through launching the app and the connection, however, while the video feed shows up in Unity (albeit very laggy), the display on the device is still black with 'waiting for editor to connect'. Any ideas as to where I've gone wrong?

 

@christophergoy:

Thanks a lot for your very fast reply! Looking forward to the new version.

 

@biscito It might be this...?

 

Hello!

I have some troubles...when I place my object, it is not on plane but in air. I was following this http://www.instructables.com/id/Apple-ArKit-Augmented-Reality-App/ tutorial but I am not able to achieve object that is place on plane :/
Is this something wrong with this tut?
Thanks in advance!

 

What is the proper way to prepare the ARKit plugin before switching scenes? I need to be able to switch to and from an scene using the ARKit plugin. I can load the ARKit scene fine using SceneManager.LoadScene, but when I switch away from it the app crashes. Also, this only when on an iOS device, scene switching works fine in the editor. I can replicate this in my custom scenes as well as in the example scenes from the ARKit plugin.

 

Hey Chris,
Any word on this? Kinda stuck atm and don't want to downgrade back to iOS 11 beta 4.
Thanks

 

Likely an error where the UnityARVideo script can no longer reference the camera, you probably have to fix that yourself.

 

Yes, that worked for me as well. Thank you!

 

I just got word from Apple that they believe they have fixed it in beta 5. I'm in the process of upgrading all the things . . . so I haven't confirmed the fix yet.

 

You should make all ARKit related gameobjects (most of the gameobjects in the original example scene) be "DontDestroyOnLoad", since these need to persist even when you change scenes.

 

There is a temporary fix above: link.

 

If you look at how we send the video across from Unity ARKit Remote, you can see how we capture the video at full resolution into byte arrays. You can compress and/or scale it down those byte arrays (we currently don't do it, but it's probably something we could so to make the transfer faster).

 

Hey guys!

Just out of curiosity. I'm making edits to the UnityARHitTestExample class and I'm trying to find a way that will rotate my object along the Y axis so it's always facing the camera, but i find no appropriate way to "hijack" the methods to place my rotation scripts. I've been playing around with Debug.Log but i can't figure out how to rotate the object at the same time as it's being translated on finger press.

How did you guys approach it, if you did?

 

thx

 

In that script, we translate and rotate based on hittest result. You could create an gameobject at the hittest location that is scripted to always look at the camera using one of the solutions here. i.e. instead of the cube that exists in the scene, create a prefab that you can instantiate which has your rotation scripts on it.

 

It´s not possible to make the rotation to be based on the camera´s position?

The effect im kinda aiming for is to rotate the object only when i press the screen so a the object transforms to the hit test. Currently, it seems to only rotate the object to the origin position of when the original track was initiated

 

I replaced that but I having other error...






Using last version of Unity, Xcode Beta 3 and the last version of iOs11 (beta 5)

 

You need XCode beta 5, which includes the latest ARKit SDK.

 

Fixed. That was the problem. Thank you so much

 

Other than obviously a potential higher rendering cost is there any reason to not set the camera far clipping plane to say 500?

 

Shadow quality drops the wider the clipping range is.

 

Hi all, so I read a post on anchoring GameObjects and to be able to see the game objects in the same position when you reopen the app. However your device has to be at the exact location you closed the app, this leads me onto my question I realised that if you completely close the app so its not running anymore, is it possible to save this data and the next time you open the app in the same position you will be able to see those GameObjects anchored? And if so this is possible how would you implement to have this saved data in the next session?

 

Hello Guys,

We love Unity's ARKit implementation! That being said, we anticipate releasing a game this September/October and wanted to ask/bring up the device heating up issue...

We are concerned that our characters (2500 polis/tris) which will eventually become fairly numerous (up to 30) will limit game play time - i.e. older phones will heat upend must be shut down. We've talked to other teams that (using different AR methods the ARKit) melted processors doing SLAM (3 years ago on a pre iPad Air 2). We obviously don't want to damage players devices (Yikes)! I am willing to add a method that times out the game with a friendly UI (unlike the shut down I've experienced with other IOS products) based on sustained performance or, better yet, high temperature readings.

(BTW: we are testing on iPhone 6s')

Question #1: Does anybody know of such a method that we can access in Unity to queue player to "take a break" and let their device cool down?

Question #2: We should be going with Unity's mobile shaders all the way right (this is our first metal project) or are their more optimized "Metal Unity Shaders"?

I realize this is a little early in ARKit's development... Thanks!

 

Looking for quick sanity check; does this look like it's including all compatible iOS devices which support ARKit?

Code (CSharp):

1.     public void CheckDeviceType(){
2.  
3.         if(Device.generation == DeviceGeneration.iPhoneSE1Gen){
4.             DeviceIsCompatible = true;
5.         }
6.  
7.         else if(Device.generation == DeviceGeneration.iPhone6S){
8.             DeviceIsCompatible = true;
9.         }
10.  
11.         else if(Device.generation == DeviceGeneration.iPhone6SPlus){
12.             DeviceIsCompatible = true;
13.         }
14.  
15.         else if(Device.generation == DeviceGeneration.iPhone7){
16.             DeviceIsCompatible = true;
17.         }
18.  
19.         else if(Device.generation == DeviceGeneration.iPhone7Plus){
20.             DeviceIsCompatible = true;
21.         }
22.  
23.         else if(Device.generation == DeviceGeneration.iPadPro1Gen){
24.             DeviceIsCompatible = true;
25.         }
26.  
27.         else if(Device.generation == DeviceGeneration.iPadPro10Inch1Gen){
28.             DeviceIsCompatible = true;
29.         }
30.  
31.         else if(Device.generation == DeviceGeneration.iPad5Gen){
32.             DeviceIsCompatible = true;
33.         }
34.  
35.         else {
36.             DeviceIsCompatible = false;
37.         }
38.  
39.         Debug.Log ("iOS Device = " + Device.generation.ToString());
40.         iOSSystem = Device.generation.ToString();
41.     }
42.