Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Camera WorldToScreenPoint Bug in OnSceneGUI?

Discussion in 'Immediate Mode GUI (IMGUI)' started by SweetJV, Aug 24, 2016.

  1. SweetJV

    SweetJV

    Joined:
    Aug 22, 2016
    Posts:
    11
    I'm trying to implement selection for my component in the scene view. It appears to work mostly fine when I'm zoomed out. But if I zoom in closely I am getting incorrect results. After a bit of debugging, it appears that Camera.WorldToScreenPoint() is returning erratic values.

    Basically, I have a list of line segments that I draw and am attempting to detect which segments, if any, should be selected when you drag a rectangular area in the scene view. To do this, I take the start/end point of the line segment and transform it into "GUI" space and then run my own Rect vs LineSegment intersection test. My intersection test appears to be fine. The problem seems to be that some segments are returning wildly different results for their start/end point when using Camera.WorldToScreenPoint().

    My conversion from Local to GUI space works as follows:

    Code (csharp):
    1.  
    2. private Vector2 TransformPoint_LocalToGUI( Vector3 local_pos )
    3. {
    4.     Vector3 world_pos  = m_handle_transform.TransformPoint( local_pos );
    5.     Vector3 screen_pos = Camera.current.WorldToScreenPoint( world_pos );
    6.     Vector2 gui_pos  = new Vector2( screen_pos.x, Camera.current.pixelHeight - screen_pos.y );
    7.     return gui_pos;
    8. }
    9.  
    Where:
    m_handle_transform = target.transform in my custom Editor

    Some segments produce reasonable results. For example with an object located at (20,0,0), using the code above transforms these Local values into these GUI values:

    local_start = (1.4, 0.0, -1.1)
    local_end = (1.2, 0.0, -0.9)

    gui_start = (730.8, -423.3)
    gui_end = (698.6, -426.9)

    However, others produce erratic results. For example, the first false positive when used with my intersection code is as follows:

    local_start = (-2.7, 0.0, -8.0)
    local_end = (-2.5, 0.0, -8.0)

    gui_start = (-134469.7, 12776.3)
    gui_end = (268561.9, -28288.5)

    I tried to look into how Camera.WorldToScreenPoint() works, using dnSpy, but it appears to be an internal call into C++. So, I'm not sure what's going on. But I suspect either some kind of overflow error is occurring or that it does some kind of frustum test, bails early, and is giving me junk values. Or, maybe it's something else?

    Anyway, clearly the results are wrong. Is this a known issue? And is there a known work-around?
     
  2. shawn

    shawn

    Unity Technologies

    Joined:
    Aug 4, 2007
    Posts:
    552
    Hey, I'm not aware of any issues with WorldToScreenPoint. Could you make a bug report so we can dig into it a bit more? We use these transformations heavily when doing our own gizmo/handle drawing, but it's possible that you're hitting a corner case that we haven't yet.
     
  3. SweetJV

    SweetJV

    Joined:
    Aug 22, 2016
    Posts:
    11
    Hey Shawn, thanks for the reply!

    As soon as I get a bit of spare time, I'll try to see if I can figure out some way to whip up a simple repro case and submit a bug with it.

    In the mean time, I'm curious if you take a quick look at the function if you notice any early-outs or branches where it might not fully compute things. Or is it pretty straight-forward?
     
  4. shawn

    shawn

    Unity Technologies

    Joined:
    Aug 4, 2007
    Posts:
    552
    Yeah, I looked when I first saw your post. Didn't see anything out of ordinary.

    If you're curious:
    Code (csharp):
    1. Vector3f Camera::WorldToScreenPoint (const Vector3f& v, bool* canProject) const
    2. {
    3.     RectInt viewport = GetScreenViewportRectInt ();
    4.  
    5.     Vector3f out;
    6.     bool ok = CameraProject( v, GetCameraToWorldMatrix(), GetWorldToClipMatrix(), viewport, out, GetTargetTexture() != NULL );
    7.     if( canProject != NULL )
    8.         *canProject = ok;
    9.     return out;
    10. }
    11.  
    12. bool CameraProject( const Vector3f& p, const Matrix4x4f& cameraToWorld, const Matrix4x4f& worldToClip, const RectInt& viewport, Vector3f& outP, bool offscreen )
    13. {
    14.     Vector3f clipPoint;
    15.     if( worldToClip.PerspectiveMultiplyPoint3( p, clipPoint ) )
    16.     {
    17.         Vector3f cameraPos = cameraToWorld.GetPosition();
    18.         Vector3f dir = p - cameraPos;
    19.         // The camera/projection matrices follow OpenGL convention: positive Z is towards the viewer.
    20.         // So negate it to get into Unity convention.
    21.         Vector3f forward = -cameraToWorld.GetAxisZ();
    22.         float dist = Dot( dir, forward );
    23.  
    24.         outP.x = viewport.x + (1.0f + clipPoint.x) * viewport.width * 0.5f;
    25.         outP.y = viewport.y + (1.0f + clipPoint.y) * viewport.height * 0.5f;
    26.         outP.z = dist;
    27.  
    28.         return true;
    29.     }
    30.  
    31.     outP.Set( 0.0f, 0.0f, 0.0f );
    32.     return false;
    33. }
    34.  
    35. inline bool Matrix4x4f::PerspectiveMultiplyPoint3( const Vector3f& v, Vector3f& output ) const
    36. {
    37.     Vector3f res;
    38.     float w;
    39.     res.x = Get (0, 0) * v.x + Get (0, 1) * v.y + Get (0, 2) * v.z + Get (0, 3);
    40.     res.y = Get (1, 0) * v.x + Get (1, 1) * v.y + Get (1, 2) * v.z + Get (1, 3);
    41.     res.z = Get (2, 0) * v.x + Get (2, 1) * v.y + Get (2, 2) * v.z + Get (2, 3);
    42.     w     = Get (3, 0) * v.x + Get (3, 1) * v.y + Get (3, 2) * v.z + Get (3, 3);
    43.     if( Abs(w) > 1.0e-7f )
    44.     {
    45.         float invW = 1.0f / w;
    46.         output.x = res.x * invW;
    47.         output.y = res.y * invW;
    48.         output.z = res.z * invW;
    49.         return true;
    50.     }
    51.     else
    52.     {
    53.         output.x = 0.0f;
    54.         output.y = 0.0f;
    55.         output.z = 0.0f;
    56.         return false;
    57.     }
    58. }
    59.