Using the controller to interact with Unity UI

Hey,

I created a project earlier for Cardboard and have been trying to port it to Daydream and use the controller for input instead of the "gaze" functionality of the cardboard.

Previously, I had the GazeInputModule on the EventSystem and that allowed the reticle to be used as a sort of cursor to select objects. This worked on both 3D objects and the UI elements.

I have tried to replicate this functionality to where the controller is aiming, and have it working with 3D objects, but its not working with the UI elements.

Here is the code I have tested:

RaycastHit hitInfo;
Vector3 rayDirection = GvrController.Orientation * Vector3.forward;
Debug.DrawRay(transform.position, rayDirection * 100000, Color.green);

PointerEventData cursor = new PointerEventData(EventSystem.current);
cursor.position = rayDirection;
List<RaycastResult> objectsHit = new List<RaycastResult>();
EventSystem.current.RaycastAll(cursor, objectsHit);

This doesn't have any effect on the UI elements. Is there a method similar to the cardboard implementation that I had? That allowed me to have a single code base and set events for the elements in the Unity editor.

If not, whats the best way to proceed?

 

The problem is most likely that you are still raycasting from the HMD UI camera. You will need a separate UI camera on the controller.

So in Awake() in your custom input module you'll need to create a new UI camera and make sure that all of the canvases in the scene use this new camera as the UI camera.

Have a look at this for an example:
https://github.com/wacki/Unity-VRIn...nputModule/Scripts/LaserPointerInputModule.cs

Let me know if you have any trouble.

 

Thanks for the help. I have implemented the following code, and the hover events enter the objects but don't exit. Am I missing something?

Code (CSharp):

1.     public static DaydreamControllerInputModule instance { get { return _instance; } }
2.     private static DaydreamControllerInputModule _instance = null;
3.  
4.     private Camera UICamera;
5.  
6.     protected override void Awake() {
7.         base.Awake();
8.         if (_instance != null) {
9.             Debug.LogWarning("Trying to instantiate multiple.");
10.             DestroyImmediate(this.gameObject);
11.         }
12.         _instance = this;
13.     }
14.  
15.     protected override void Start() {
16.         base.Start();
17.  
18.         // Create a new camera that will be used for raycasts
19.         UICamera = new GameObject("UI Camera").AddComponent<Camera>();
20.         UICamera.clearFlags = CameraClearFlags.Nothing;
21.         UICamera.cullingMask = 0;
22.         UICamera.fieldOfView = 5;
23.         UICamera.nearClipPlane = 0.01f;
24.  
25.         // Find canvases in the scene and assign our custom UICamera to them
26.         Canvas[] canvases = Resources.FindObjectsOfTypeAll<Canvas>();
27.         foreach (Canvas canvas in canvases) {
28.             canvas.worldCamera = UICamera;
29.         }
30.     }
31.  
32.     private PointerEventData eventData;
33.     private GameObject currentPoint;
34.  
35.     public override void Process() {
36.         UpdateCameraPosition();
37.  
38.         eventData = new PointerEventData(eventSystem);
39.  
40.         eventData.Reset();
41.         eventData.delta = Vector2.zero;
42.         eventData.position = new Vector2(UICamera.pixelWidth * 0.5f, UICamera.pixelHeight * 0.5f);
43.      
44.         eventSystem.RaycastAll(eventData, m_RaycastResultCache);
45.         eventData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
46.         m_RaycastResultCache.Clear();
47.      
48.         // Send control enter and exit events
49.         var hitControl = eventData.pointerCurrentRaycast.gameObject;
50.         currentPoint = hitControl;
51.  
52.         // Handle enter and exit events on the GUI controlls that are hit
53.         base.HandlePointerExitAndEnter(eventData, currentPoint);
54.     }
55.  
56.     protected void UpdateCameraPosition() {
57.         UICamera.transform.rotation = GvrController.Orientation;
58.     }

 

That problem would probably have something to do with other parts of your implementation. Can't see how anything here would cause that.

 

Every other method of input works fine (including the gaze functionality if I enable it). The only thing I can think of is that I'm missing something crucial with the implementation of this input module. Is there some sort of "guideline" or something to help make sure I haven't missed something obvious?

 

You can send me your input module and I can have a look.

 

I've attached my best attempt. Thanks

 

OK found your problem. You weren't saving the event data between frames. In Process() look under the commented section "NEW CODE" to see the change I made to get it to work.

Let me know how that goes for you.

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.EventSystems;
3.  
4. public class DaydreamControllerInputModule : BaseInputModule
5. {
6.     public static DaydreamControllerInputModule instance { get { return _instance; } }
7.     private static DaydreamControllerInputModule _instance = null;
8.  
9.     private Camera UICamera;
10.     private PointerEventData eventData;
11.  
12.     protected override void Awake() {
13.         base.Awake();
14.  
15.         if (_instance != null) {
16.             Debug.LogWarning("Trying to instantiate multiple.");
17.             DestroyImmediate(this.gameObject);
18.         }
19.  
20.         _instance = this;
21.     }
22.  
23.     protected override void Start() {
24.         base.Start();
25.  
26.         // Create a new camera that will be used for raycasts
27.         UICamera = new GameObject("UI Camera").AddComponent<Camera>();
28.         UICamera.clearFlags = CameraClearFlags.Nothing;
29.         UICamera.cullingMask = 0;
30.         UICamera.fieldOfView = 5;
31.         UICamera.nearClipPlane = 0.01f;
32.  
33.         // Find canvases in the scene and assign our custom UICamera to them
34.         Canvas[] canvases = Resources.FindObjectsOfTypeAll<Canvas>();
35.         foreach (Canvas canvas in canvases) {
36.             canvas.worldCamera = UICamera;
37.         }
38.     }
39.  
40.     public override void Process()
41.     {
42.         UpdateCameraPosition();
43.  
44.         // NEW CODE: Only create a new instance of event data if there isn't an existing instance
45.         if (eventData == null)
46.             eventData = new PointerEventData(eventSystem);
47.         else
48.             eventData.Reset();
49.  
50.         eventData.delta = Vector2.zero;
51.         eventData.position = new Vector2(UICamera.pixelWidth * 0.5f, UICamera.pixelHeight * 0.5f);
52.  
53.         eventSystem.RaycastAll(eventData, m_RaycastResultCache);
54.         eventData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
55.         m_RaycastResultCache.Clear();
56.  
57.         // Send control enter and exit events to our controller
58.         var hitControl = eventData.pointerCurrentRaycast.gameObject;
59.         var currentPoint = hitControl;
60.  
61.         // Handle enter and exit events on the GUI controlls that are hit
62.         base.HandlePointerExitAndEnter(eventData, currentPoint);
63.  
64.         if (base.eventSystem.currentSelectedGameObject != null)
65.         {
66.             ExecuteEvents.Execute(eventSystem.currentSelectedGameObject, GetBaseEventData(), ExecuteEvents.updateSelectedHandler);
67.         }
68.     }
69.  
70.     protected void UpdateCameraPosition()
71.     {
72. //        UICamera.transform.position = controller.transform.position;
73.         UICamera.transform.rotation = GvrController.Orientation;
74.     }
75.    
76.     private Vector3 GetControllerPosition()
77.     {
78.         return GvrController.Orientation * Vector3.forward;
79.     }  
80. }

 

Oh thanks so much! It seems so obvious now. It works great, I'll implement the other functionality now

 

@SubhanUrRehman Are you doing something different than provided demos?
Here is how i adapted controller input for World space UI:
1. Add GVRPointerInputModule on EventSystem object.
2. Add GVRPointerGraphicsRaycster to the canvas object(s).

 

As far as I know these were included in GVR13, but not in the earlier versions (which I was using when I make the scripts). So the desired functionality was the same but it was not bundled at the time. Now I've implemented a mixture of my script and the GVRPointer

 

@UmairEm Okay, for buttons it works, but for e.g. sliders not. It seems that the controller does not know the exact position of the slider. No matter where I click the slider, the handle is always in the middle.

 

I have created a thread on this issue already. Please keep an eye on that.

 

Hi r3dthrawn,
I'm using the same code that you linked (LaserPointerInputModule) in a HTC Vive app.
But I'm trying to do something slightly different: I would like to get the position of the hit on the world canvas, since I'd like to be able to (for example) drag it around.
This seems to be difficult, since the pointerCurrentRaycast.screenPosition always returns the (UICamera.pixelWidth * 0.5f, UICamera.pixelHeight * 0.5f) values, and worldPosition always returns Vector3.zero.

Am I missing something obvious here? Maybe I need a different approach of raycasting?
(for the sake of completion, I'm using Unity 5.4 and SteamVR 1.2)

Thanks in advance!