Resolved ARKit support for iOS via Unity-ARKit-Plugin

@jimmya when I break inside of `SetCameraNearFar` inside of ARSessionNative.mm I see something interesting. The first time I hit the breakpoint I see the correct far value of 1000, the second time it hits the breakpoint the value has gone back to 30.

 

OMG! My hero(es)! Thanks for sharing! <3

 

Do you maybe have 2 cameras with that script on it?

 

This bug is now fixed in the bitbucket repo. Asset store update will take a day or so.

 

This fixed me up. Thank you!

 

i get a few errors with your test scenes
UnityARBallz
Assertion failed: Screen position out of view frustum (screen pos 1397.124023, 1048.491577) (Camera rect 0 0 1502 1127)
UnityEngine.SendMouseEventsoSendMouseEvents(Int32)
which I can get around by deleting the camera and creating another

then i get 2 more errors

NullReferenceException: Object reference not set to an instance of an object
BallMaker.Update () (at Assets/Examples/BallMaker.cs:41)

NullReferenceException: Object reference not set to an instance of an object
BallMover.Update () (at Assets/Examples/BallMover.cs:27)
not sure how to work around these i cant get any of the test scenes to work

 

Get latest from bitbucket repo to fix these. If you need Asset Store update, I'm afraid this will take a day or so. Also, look at recent posts if they have had the same issue - information about this and the fix was just posted.

 

Thanks!
"Fix error in editor play mode if you don’t have ARKitRemoteConnection in scene"

 

It looked like a prefab was missing that maybe wasn't included? Thanks for fixing the issue! @LRavenwolf had the same issue as I did. The latest commit on bitbucked fixed the issue! Thanks @jimmya !

 

that was indeed the problem, thank you.

 

I'm hoping to store point cloud data recorded using ARKit and Unity such that it can be retrieved and displayed as it was originally recorded. Let's say that I am simply displaying point cloud data exactly how the UnityPointCloudExample.cs script does. However, when a point is detected and displayed, I also store some relevant information about that point. Then I close the application completely. When I open the application again, I want to be able to reload the data in the same positions relative to the camera as it was when originally detected, is this possible using ARKit and the Unity plugin as-is?

 

ARKit cannot automatically relocalise within an area, this is something a lot of people (including myself) have asked on this thread. Its not something Unity related, more ARKit related. Hopefully Apple will look into adding this in for the next major release of the kit.

 

Hey, Unity newbie & full time iOS developer here

I tried to get started using Unity for ARKit by using the HitTestExample in connection with a small controllable car. Let me explain what I did to get to the current state of my little project:

- create a new project with the assets "ARKit plugin" (btw, does Unity download the latest version from the asset store when new projects are created or does it use the version that is saved on disk?) and a free cartoon car model (http://u3d.as/gRG)
- replace children of HitCubeParent with the car prefab, assign UnityARHitTestExample script and Hit Cube Parent (Transform) to the car, rename its children to "wheel_front_left" etc.
- HitCubeParent transform position is 0;1;-2.98 (I want the car to be placed slightly above the hit test result and fall down on the anchor plane), scale 0.1
- Car transform position + rotation is set to 0, scale is 0.6 (I guess this just depends on the model that is being used)
- add canvas with 2 sliders (throttle and steer); I could have gone with buttons but decided otherwise
- using https://docs.unity3d.com/Manual/WheelColliderTutorial.html:

- add Rigidbody to the car prefab

- reposition and scale the box collider (bottom side of the box matches the bottom side of the car, not the wheels)
- mass 1500
- use gravity​

- add controller script (code below)

- 2 axes, front axe is used for steering and motor
- torque 60, max steering angle 30​

- add wheel collider to the wheels

- changed radius to 0.2 (= equal size to each wheel)
- set Force App Point Distance to center of wheel
- reposition collider by changing its center position​

CarController.cs:

Code (csharp):

1.  
2. [System.Serializable]
3. public class AxleInfo {
4.     public WheelCollider leftWheel;
5.     public WheelCollider rightWheel;
6.     public bool motor;
7.     public bool steering;
8. }
9.  
10. public class CarController : MonoBehaviour {
11.  
12.     public List<AxleInfo> axleInfos;
13.     public float maxMotorTorque;
14.     public float maxSteeringAngle;
15.  
16.     // Use this for initialization
17.     void Start () {
18.     }
19.  
20.     // Update is called once per frame
21.     void Update () {
22.     }
23.  
24.     public void FixedUpdate() {
25.         float throttleValue = GameObject.Find("SliderThrottle").GetComponent<Slider>().value;
26.         float steeringValue = GameObject.Find ("SliderSteer").GetComponent<Slider> ().value;
27.         float motor = maxMotorTorque * throttleValue;
28.         float steering = maxSteeringAngle * steeringValue;
29.  
30.         foreach (AxleInfo axleInfo in axleInfos) {
31.             if (axleInfo.steering) {
32.                 axleInfo.leftWheel.steerAngle = steering;
33.                 axleInfo.rightWheel.steerAngle = steering;
34.             }
35.             if (axleInfo.motor) {
36.                 axleInfo.leftWheel.motorTorque = motor;
37.                 axleInfo.rightWheel.motorTorque = motor;
38.             }
39.         }
40.     }
41. }
42.  

My current project setup looks like this:

http://imgur.com/a/gbe79

Now, when I tap on an anchor plane, I see the car model for a few frames, then it disappears. For testing purposes, I set Time.timeScale to 0.05 and took a video of what happens.

Yes, I printed those sheets to make plane detection on our office desks possible ;-) The video might show a different Rigidbody mass than shown above, I changed a lot for testing purposes.

From my limited Unity experience, I think:
- the car physics do not get reset each time I tap the screen, i.e. the acceleration of the car is applied after it has been repositioned. That's why the car flies off the screen much faster on the 2nd or 3rd tap.
- the car is somehow positioned "in" the plane and gets pushed away. I tried positioning it (resp. the HitCubeParent) at Y=1 unit (btw, does Unity use meters?), but that doesn't seem to help.

Can somebody tell me what is going on and how to fix it? I hope this is the right place to ask, I can't tell if it's more related to the ARKit plugin or my lack of understanding vehicles & wheel collider physics.

 

Hello everybody,

I am using arkit plugin in Unity and i work fine, but in my project I need to transform the CameraTexture to a byte array and then, transform this byte array to base64string.

Someone found the best practice to do this? Because I didn't find a solution to resolve this problem, i couldn't catch the texture of the camera.

Thanks for your help!

Juan.

 

Please read the forum before asking the same question again: in this case, this is answered here.

 

It might be possible to do something by using LocationService and gravityAndHeading, though I haven't tried it out myself.

 

Asset store update has landed.

 

I feel like I'm doing something elementally wrong here. I can get Unity to place objects around in the world but these objects positions appear to "shift" after they've first been placed. Does anyone know what is causing this? Is this ARKit repositioning the camera in the world based on new data? Should I be turning off updating after all pertinent objects have been placed?

 

Are you placing them in the world or as children of some other gameobject? e.g camera or plane? They should be placed in the world.

 

Nope. I place them as follows

Code (CSharp):

1. List<ARHitTestResult> results = UnityARSessionNativeInterface.GetARSessionNativeInterface().HitTest(point, ARHitTestResultType.ARHitTestResultTypeFeaturePoint);
2. foreach (ARHitTestResult hit in results) {
3.     transform.position = UnityARMatrixOps.GetPosition(hit.worldTransform);
4.     transform.LookAt(Camera.main.transform);
5. }

Right out onto the root of the stage. They seem to place and then face correctly, however as I move around their position appears to "shift". My assumption has been this is due to the virtual camera position updating and not the objects themselves moving around?

 

More than likely its because the object is not placed where you think it is. You are using the closest feature point to hit test against, and that may be providing a position that is really inaccurate. You should read up on ARHitTestResultType - they are listed from least accurate to most accurate.

One way to test this is to place objects in the world at specific locations and see if they work as expected - for example, the checkered cube in the UnityARKitScene - does that appear to stay in the right place? If so, there is something that your HitTest logic might be doing to place things where you don't expect it to be.

 

That's very possible. Just to play devils advocate if something was incorrect about that hit logic why would the objet place and then "drift" shouldn't it just place unexpectedly? I was actually using the feature point because I'm looking for a surface that is functionally a vertical plane.

 

Hi!

I was testing the plugin and found something that might be a bug. I have 2 equals rooms, one in front of the other. Same geometry but one of them had a -1 scale in z (lazy me). When running on my device, the scene seemed to be Ok but after turning around, the camera started rendering like when you set the camera clear flag to "Don't Clear" (it was set to depth only, as it should). Here's a video of the problem:

After a lot of fiddling with the scene, I found out that the negative scale was the culprit. Setting it back to 1 fixed the issue.

Should this be considered a bug? How should I report this? There's no bug reporter on Bitbucket

Thanks!

 

Hi,

I am still getting the following exception:

2017-07-24 15:59:06.652919-0700 ARKitZombie[613:114567] *** Terminating app due to uncaught exception 'UnrecognizedARTrackingStateReason', reason: 'Unrecognized ARTrackingStateReason: 1'

*** First throw call stack:

(0x185257e58 0x184768528 0x185257da0 0x10153c9a4 0x10153bc6c 0x19faac780 0x184bd9e04 0x184bd9dc4 0x184be6f6c 0x185201cd8 0x1851ff7f0 0x185121bf0 0x186f7bfac 0x18ecf9ec4 0x100e0f964 0x184c461e0)

libc++abi.dylib: terminating with uncaught exception of type NSException

(lldb)

My Dev Environment is:
Xcode Version 9.0 beta 3 (9M174d)
Unity version 5.6.1p1
latest ARKit plugin from bitbucket
Iphone 6s with ios11

I have a free developer profile with Personal Team and I have set it up in settings.

Please help! Thanks!

SG

 

@jimmya just wanted to provide an update on the issue I've had with the ARPlaneAnchors behaving erratically when the world alignment is set to gravity and heading. I've been able to replicate it using native ARKit here: https://github.com/AfterNow/arkit-by-example/commit/548731bef07d72864f0f7c921c0b3bcf952a93ef, with a new familiar video here:

I'll submit a ticket with Apple and see what happens.

 

great work with the plugin guys, have been playing and having alot of fun with this!

couple of questions...

1) so it takes a few seconds for ARKit to get enough geometry together to be able to do anything... we can listen for when planes/anchors are added but there's usually enough point cloud data created within a few seconds that allows 3d objects in the scene to start tracking properly... is there someway of me listening for an event to know when this might be? can sometimes take an age before anchors are made and floating stuff is already tracking by that point... (I basically want to display some UI whilst ARKit warms up)

2) wondering what the best approach people are using for identifying the floor is... currently checking the size of the plane is greater than a limit and just added a hittest to confirm it but it still feels like it could be improved and automated a bit more...?

3) I have a 3d environment of my house where I start in the centre of a room. The environment is all navmeshed and static so can't be moved. I've moved the camera parent to the location where I should be starting and it seems to be working fine but reading through the pages of this thread there's lots of mentions of the camera being reset to 0,0,0...? You're referring to the actual camera right, not the parent? I can put the parent wherever I want? I can't move my environment due to the navmesh being baked and though I've seen the navmesh runtime stuff I suspect it will kill performance on the phone whilst running ARKit

4) related to the above... if I was to move the cameraparent after say 10 seconds of running ARKit shouldn't care right?

Thanks!
M

 

This is probably not a bug, since when you put a negative scale, all the polygons now face in the opposite direction, so you are seeing the backface culling in action. If you think its still a bug, take it to https://forum.unity3d.com/forums/general-graphics.76/

 

You either do not have the latest plugin from bitbucket, or you have an older XCode 9 than Beta 3.

 

Thanks for your followup! Appreciate it. Let me also reach out to Apple to see if there's any information they can provide.

 

1) You shouldn't need a delay - world tracking starts immediately (see the checkered cube in the example). Feature points are detected when you move around, and when enough feature points are detected to estimate a plane, the plane will be created. So I'm not sure what state you're looking for.
2) Your method is best for now
3) Unfortunately, you cannot put anything but identity on the parent as well, since ARKit will return world coordinates to the camera, and if you put in an offset on the camera parent, it will get confused as to where it is compared to what ARKit thinks.
4) see answer to 3)

 

Improvement to Unity ARKit Remote checked into bitbucket :

You can specify in ARKitRemoteConnection on the editor side how you want ARKit to be configured and run when you connect to the Unity ARKit Remote.

 

Hey @jimmya, thanks for all your work on this. Does the ARKitRemoteConnection require insanely high specs? After starting a remote session, my refresh rate on both my device and unity game tab are about 1 frame for every 3 seconds. Sadly, it's unusable at this speed. I suspect there is a problem with my setup. I'm using an iPhone 6S and a late 2013 MBP Intel Core i7 2.8 GHz with 8 GB memory.

 

HI, I am trying to sync two device in the same world until I see this post ,so ,is there any example that setting device postion with the 6DOF in my custom postion in the real world that become 0,0,0 in unity world then start the ARkit Tracking.I am quite confuse with when ARkit get the camera postion.

THX

 

Is there any way to disable Unity's rotation tracking when using Native Unity VR? I would like to use ARKit's rotations when integrating cardboard with ARKit (otherwise there seems to be a significant amount of drift from ARKit's Rotations and throws my world off).

 

It should not be that slow. My mac is similar in spec to yours. I suspect your bottleneck may be connection/network traffic (may be throttling)? Are you using 5.6.2 or 2017.1? You should try and use the latter, since the connection is way more reliable on 2017.1. Also, once you have established connection, you can close the profiler window. Also check whether you have too many debug logs that may be slowing you down etc.

Oh, and I hope you're using the usb connection.

 

For the first couple of seconds I only get rotational movement on the checkered cube and then positional takes hold and you can move toward it without it floating with you... feels like ARKit takes a couple of seconds to build the world data (which is what I was asking for a notification of) but you're saying that's not the case? Using iPhone 6S...

Hmmm. This is going to be troublesome for I suspect quite a few people since normally you never move the world, you always move the camera. If you've got physics joints, navigation, lighting etc then the scene will literally explode if you move it... what's the best approach to tackle this?

5) On a different note I also get really unusable performance through the remote
Using Mid 2014 MBP 16GB, Latest Unity 2017 beta, iPhone 6S via USB , very few debugs... :/

 

Not sure if others would find this useful but currently updating and integrating this plugin into other projects isn't the cleanest solution as there are alot of folders and scene files etc and generally it's nice to keep plugins within their own folder to keep a project cleanly structured...

Is it possible to move the whole plugin into a root folder on the repo please, i.e. "ARKit", this would make it so much easier and quicker to merge new changes into other projects...

Thanks!

 

Ive been doing this myself already. I use an asset from the store called Package2Folder which imports a package into a folder of your choice. Named it UnityARKit and all works fine. It would be nice if the package could be updated to a self contained folder.

 

Is there any way that if I have already detected the surface then I should stop adding other anchors into the game and continue with only that specific anchor ?

 

Hi everyone! I'm trying to make something with arkit like this, but i cant think of a way to make those "tags" with the names. Can anyone give an idea.


Thanks.

 

Try to use UnityARSessionNativeInterface.ARAnchorAddedEvent to detect when an anchor has been added and then "make it stop".

 

I've inherited code from some very early work with ARKit, and it has references to UnityARAnchorManager.OriginPosition/Rotation. Looking at the project on GitHub, I wasn't able to find any previous references to these properties.

When were these deprecated, and are there new use cases to achieve the same functionality? The sense I get is that these were used to set the real-world anchor's position in the virtual scene (or vice-versa).

 

Ah you could hook into the newly exposed ARSessionTrackingChangedEvent, from which you can get UnityARCamera.trackingState

Yeah I haven't found a good solution for this - check earlier posts for some suggestions (e.g. use a root transform etc). If you do find a good solution, please let everyone know.

Hmm - I've got usable performance on my end (not 60fps), and I know some others who have as well. I'll try with a 6S today. Also, check that you do not have script debugging or other options checked on the build settings (only Development Build)

 

After you get your plane, you can send Start to ARKit again, this time have the config.detectPlanes set to None

 

I don't recall ever having those references in there. You can check the history in our repo. Maybe your predecessor wrote their own functionality into the plugin?

 

Just updated bitbucket repo with update to allow light estimation ambient intensity to be send to editor via Unity ARKit Remote.

 

What is the expected timeline to see -HitTest API added.?

Thanks

 

Hi thank for your job , it s really great ARkit integration work really well.
but for remote i follow the instruction and i got error :

Attempting to connect to player ip: 127.0.0.1 with a 1 ms timeout
UnityEditorInternal.ProfilerDriverirectURLConnect(String)
UnityEditor.AttachProfilerUI:SelectProfilerClick(Object, String[], Int32)

Have you an idea of where to look (i m on macbook Sierra 10.12.6 and Unity 2017.1.0f3)

 

Try https://docs.unity3d.com/ScriptReference/VR.VRDevice.DisableAutoVRCameraTracking.html

 

Yes, Thank you!! I rechecked my ARKit plugin needed to be updated. Thank you for the plugin. It works great! Amazing job