Search Unity

EditorWindow Rendering Corrupts RenderTextures/Camera on Scroll Events

Discussion in 'Immediate Mode GUI (IMGUI)' started by kode80, Sep 29, 2015.

  1. kode80

    kode80

    Joined:
    Aug 1, 2013
    Posts:
    151
    This past month I have been working on a volumetric clouds renderer for Unity. The effect itself is close to completion and so this week I have begun working on a custom editor in preparation for release on the Asset Store.

    Some background: the clouds are rendered to a render texture via ray casting at a lower resolution, the workload is spread over multiple frames by reprojecting previous frames, resulting in a full resolution image converging after several frames. This approach works great in-game, essentially giving you full resolution raycasting for a fraction of the cost... however since it's an image effect and takes several frames for the image to converge, editing the various cloud properties until this point required the game is running. This is why I started creating a custom EditorWindow.

    My current custom EditorWindow works like this: I handle basic FPS controls, when the 'scene' needs updating I update the current cloud frame, render the scene + full screen quad for clouds to the window via a hidden camera, allow several frames afterwards to also be rendered so the clouds converge.

    The above approach sort of works but I have run into many bugs, all centered around custom rendering in the editor itself and documentation in this area is sparse to say the least. Below is what I've faced so far and where I'm currently at (for which this thread title references):

    • Camera rendered to an EditorWindow are restricted to the game viewports dimensions, anything higher clips. This isn't documented anywhere I could find - but I fixed it by setting the camera's hideFlags to HideAndDontSave.
    • Setting the camera's pixelRect from EditorWindow.Update work's randomly, meaning that the size of my camera's output keeps changing. By stepping through in the debugger I literally see; line1 - set's pixelRect, line 2 - pixelRect has not been set. The incorrect size when this happens is again, the size of the game viewport.
    • At this stage I'd tried all variations of drawing a camera directly in an EditorWindow; Handles.DrawCamera, Camera.render etc. all exhibit the same issue.
    • I also noticed that scrolling anywhere on the screen (via mac trackpad 2 finger drag) seemed to increase the chance of the issue occurring.
    • Since the main issue seemed to be with setting the camera's pixelRect, I switched to rendering the camera to a render texture and then displaying that texture in OnGUI. I figured maybe setting Camera.targetTexture would force unity to respect the pixelRect...
    • ...this half worked! My cloud effect was now being rendered to the correct pixelRect of the camera however, again, when I scroll anywhere on the screen my render texture becomes corrupted with the panel that was under the mouse when I scroll. e.g. if the mouse is over the inspector panel I see the panel inspector scrolling *in* my clouds render texture AND the inspector panel itself stops updating. This happens with any of the editor panels - if I scroll the console I see the console scrolling *in* my clouds render texture and the console panel itself stops updating.
    • Whenever the above occurs I have to switch that panel to fullscreen (pinch gesture) and then back again to unfreeze it.
    • On a side note, I'm guessing the above freeze/unfreeze 'fix' is due to windows being instantiated a 2nd time when switching to maximized - something I had to figure out myself with EditorWindows and *really* wish was documented somewhere...
    If I had to make a guess as to what's causing all this, I'd say it's due the other editor panel's rendering hijacking my custom rendering. But with zero documentation as to the correct place/time/way of doing stuff like this, who knows. Right now it's causing a huge roadblock in me finishing this project as I'd rather not release an Asset that's buggy/difficult to edit.

    Attached is a screenshot of the inspector panel corrupting my clouds render texture in my custom EditorWindow. If you'd like to see previous videos of my clouds progress check https://www.youtube.com/user/kode80apps
     

    Attached Files:

  2. kode80

    kode80

    Joined:
    Aug 1, 2013
    Posts:
    151
    For anyone that may stumble across this thread, the solution (kindly posed by @Dave Carlile) is that Graphics.Blit() calls *must* restore RenderTexture.active to it's previous state. This only seems to be the case when rendering in the editor itself, in-game it's not an issue.

    Hopefully Unity will update their documentation as I've yet to find any mention of this.