Search Unity

Multiple cameras, image effects and the deferred rendering path

Discussion in 'General Graphics' started by Chman, May 2, 2015.

  1. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    721
    I'm working with the Deferred rendering path and I'm having issues with image effects and my multi-camera setup.

    Here's a simple scene setup :
    So, to sum up, Camera_0 renders the Default layer and Camera_1 renders the Test layer on top of the first camera. So far so good.

    But if I add an image effect to Camera_0 (let say the default "Sepia" effect), it simply does nothing. At all. If I put it on Camera_1 it works but that's not what I want, I only want to apply it to Camera_0.

    It works as soon as I switch to the Forward rendering path. Unity 4.x had the same issue with the legacy Deferred path but there was a workaround. Of course, this doesn't work in Unity 5 anymore.

    Has anyone found a solution to this issue ? Is it a bug or "by design" ? I can post the test scene if anyone's interested but it's easy to reproduce.
     
    JoRouss likes this.
  2. AlteredPlanets

    AlteredPlanets

    Joined:
    Aug 12, 2013
    Posts:
    455
    Yes I have had this issue as well even in 4.x cycle
     
  3. Torigas

    Torigas

    Joined:
    Jan 1, 2014
    Posts:
    63
    Same issue here. Seems like a bug to me.
     
  4. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    247
    Hi @Chman, thx for your awesome Colorful Asset, did you have find a solution this no brainer must fix bug?

    I had some luck with script on this page using the standard unity effects, maybe with some kind work it could work with Colorful?

    http://forum.unity3d.com/threads/de...-posts-ssao-bugged-ufps-onrenderimage.198632/

    It's essential for me to try and keep a full stack of deferred cameras with separate FX otherwise my game goes back to the 90's vibe. :)
     
  5. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    247
    Think I found a fix. Please read the last post on that thread.
     
  6. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    721
    The only "complete fix" is to render each camera manually to a global render texture and blit it to screen. It works fine, it's not really hard to setup but adds another layer of messiness to the scene...
     
  7. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    247
    Like this?

    Code (CSharp):
    1. using System.Collections;
    2. using UnityEngine;
    3.  
    4. // fixes the deferred lighting missing final copy&resolve, so the next camera gets the correctly final processed image in the temp screen RT as input
    5. // NOTE: The script must be the last in the image effect chain, so order it in the inspector!
    6. [ExecuteInEditMode]
    7. public class CopyToScreenRT : MonoBehaviour
    8. {
    9.     private RenderTexture activeRT; // hold the org. screen RT
    10.  
    11.     private void OnPreRender()
    12.     {
    13.         if (GetComponent<Camera>().actualRenderingPath == RenderingPath.DeferredLighting) {
    14.             activeRT = RenderTexture.active;
    15.         }
    16.         else {
    17.             activeRT = null;
    18.         }
    19.     }
    20.  
    21.     private void OnRenderImage(RenderTexture src, RenderTexture dest)
    22.     {
    23.         if (GetComponent<Camera>().actualRenderingPath == RenderingPath.DeferredLighting && activeRT) {
    24.             if (src.format == activeRT.format) {
    25.                 Graphics.Blit(src, activeRT);
    26.             }
    27.             else {
    28.                 Debug.LogWarning("Cant resolve texture, because of different formats!");
    29.             }
    30.         }
    31.  
    32.         // script must be last anyway, so we don't need a final copy?
    33.         Graphics.Blit(src, dest); // just in case we are not last!
    34.     }
    35. }
    This only works with a Standard Effect like fisheye included in the chain. When disabled it has no effect, do you know why?
     
  8. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    721
    This is the old fix for the old deferred path in Unity 4.x. I'm not sure why it would work with a standard effect and not one of Colorful FX. To be honest I haven't really investigated it.

    What I meant previously was to manually render the cameras one by one in a render texture. Something along these lines (typed on the fly so it's untested but just so you get the idea) :

    Code (CSharp):
    1. using UnityEngine;
    2.  
    3. [RequireComponent(typeof(Camera))]
    4. [ExecuteInEditMode]
    5. public class CameraGlobalRT : MonoBehaviour
    6. {
    7.     public Camera Camera0;
    8.     public Camera Camera1;
    9.     // Add more cameras or use a list instead
    10.  
    11.     protected Camera m_Camera;
    12.     protected RenderTexture m_RenderTexture;
    13.  
    14.     void OnEnable()
    15.     {
    16.         m_Camera = GetComponent<Camera>();
    17.     }
    18.  
    19.     void Start()
    20.     {
    21.         if (Camera0 == null || Camera1 == null)
    22.             return;
    23.  
    24.         Camera0.enabled = false;
    25.         Camera1.enabled = false;
    26.     }
    27.  
    28.     void OnPreRender()
    29.     {
    30.         if (Camera0 == null || Camera1 == null)
    31.             return;
    32.  
    33.         m_RenderTexture = RenderTexture.GetTemporary(m_Camera.pixelWidth, m_Camera.pixelHeight);
    34.  
    35.         Camera0.targetTexture = m_RenderTexture;
    36.         Camera0.Render();
    37.         Camera0.targetTexture = null;
    38.  
    39.         Camera1.targetTexture = m_RenderTexture;
    40.         Camera1.Render();
    41.         Camera1.targetTexture = null;
    42.     }
    43.  
    44.     void OnRenderImage(RenderTexture source, RenderTexture destination)
    45.     {
    46.         if (Camera0 == null || Camera1 == null)
    47.             Graphics.Blit(source, destination);
    48.         else
    49.             Graphics.Blit(m_RenderTexture, destination);
    50.  
    51.         RenderTexture.ReleaseTemporary(m_RenderTexture);
    52.     }
    53. }
    Once again, this is very rough code just to give you an idea of what I meant. Add HDR & depth support if needed when requesting m_RenderTexture. That should give you a good starting point. Put this on a new Camera object somewhere on your scene.
     
    wxxhrt and SunnySunshine like this.
  9. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    247
    Ok I've finally got round putting some time aside to learn more about RenderTexture usage and try this out. Thanks for it. If not too much trouble I have a question:

    So is this a 3 camera config, 1 cam on which the script is placed and another another 2 cameras which are passed in as camera object params? Does the script cam combine the other 2 passed in into one renderTexture and render it on to the script cam. I think that's it? I best try it out I suppose... Just trying to get the logic right, the disabling of the 2 cams is confusing me. Thanks
     
  10. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    721
    Yep exactly, one "main" camera that doesn't render anything (only used to blit the global render texture to screen) and two other cameras that get rendered into the global render texture one by one (I used two as an example, you can add more of course).
     
  11. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    247
    Wow, just tried it. Yep works a treat. Are there any gotchas with this method? UFPS has gone a little crazy but thats because I've dropped cam 2 in with it. Thank you so much, it's like magic. Got a big smile :)
     
  12. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    721
    The only gotcha I can think about is an increased VRam usage for the global render texture. If you need HDR, use ARGBHalf instead of ARGBFloat to save some memory. Otherwise ARGB32 is fine but the bigger the screen resolution is, the more VRam it will require (not that it matters much on desktop platforms nowadays anyway, most GPU come with 2GB/4GB).
     
  13. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    247
    Thanks, very helpful.
     
  14. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    247
    I've had a interesting day messing with this and noticed a couple things and wondered if I can pick your brains again please?

    16 bit depth buffer gives a little performance boast over 24 for obvious reasons, is 16 ok to use, whats the default?

    My second cam with Depth Only (Deferred) is casting no shadows for items included on the mask? Thanks again, making great headway. I'll be buying Lutify soon ;)
     
  15. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    721
    24 comes with stencil buffer, 16 should be fine in most cases. The default value when you call GetTemporary() is 0.
    By 'mask' you mean the camera on top ? In this case I wouldn't expect it to cast any shadow on objects rendered by other cameras (remember, the camera is clearing the depth).
     
  16. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    247
    Cheers, oh yeah durr. It's trying to cast onto the other camera which has already rendered the shadows with that cameras depth buffers. What I'm aiming to fix is my scattering fog effects bleeding through walls so tried putting my buildings onto another camera with cull masks. I've solved it by putting the buildings onto both layers. The top layer blocks the fog bleed now, simple but not used to having objects on 2 cams in this manner.

    You really helped out Chman, I owe you one. Once I solve my Z-Fighting flicker on terrain my game will really look the part. ;)

    Is Scion your work? It mentions Lutify works with it, can I use Lutify perfectly well with my own stack. Thank you.
     
  17. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    721
    Nah, Scion is from the talented @Aieth ;) Lutify is compatible with Scion if you already own it but it can work on its own if you only need a color grading pass.
     
  18. Beloudest

    Beloudest

    Joined:
    Mar 13, 2015
    Posts:
    247
    Cool I'm going to get Lutify first, from checking my performance the image effects don't have much overhead anyway. I've already paid out for single effects. Should I just go for it in your opinion?
     
  19. Chman

    Chman

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    721
    Honestly, it depend on your needs. Lutify is a LUT library first, so if you need a lot of pre-made LUTs then yeah, go for it :) If you just want it for the color grading effect itself and don't care about the pre-made LUTs, then I would recommend getting something else (part of a bigger package) or using the default LUT effect that comes with Unity.

    I think we're getting out of topic ;) PM me if you want to talk more about Lutify.
     
  20. Manny Calavera

    Manny Calavera

    Joined:
    Oct 19, 2011
    Posts:
    205
    Hi @Chman,

    Now that you are an insider, have you gained any more insights about this issue? Is this setup really not supported?

    I've found multiple posts about this problem dating as far as 2010.
     
  21. danger726

    danger726

    Joined:
    Aug 19, 2012
    Posts:
    184
    Hey all, I just posted up a workaround solution here: http://forum.unity3d.com/threads/de...-ssao-bugged-ufps-onrenderimage.198632/page-2

    Here it is again in case you haven't seen that other thread:-

    Code (CSharp):
    1. using UnityEngine;
    2. using UnityEngine.Rendering;
    3.  
    4. // UNITY BUG: When using the deferred rendering path it seems that the result after image effects does not get copied back into the
    5. // camera's render target.  So, when daisy chaining multiple camera passes with image effects, the final processed image from one
    6. // camera does not get used as input to the next camera.  In other words, image effects from all but the last camera are lost.
    7.  
    8. // FIX: Add this to each camera in the chain, it copies the final processed image back into the camera's render target.
    9. // Note that this script must be ordered after all other image effects!
    10.  
    11. [RequireComponent(typeof(Camera))]
    12. public class MultiCameraImageEffectFix : MonoBehaviour
    13. {
    14.     RenderTexture resultAfterImageEffects = null;
    15.  
    16.     void Awake()
    17.     {
    18.         // Add a command buffer to blit the result after image effects back into the camera's render target.
    19.         CommandBuffer commandBuffer = new CommandBuffer();
    20.         commandBuffer.name = "MultiCameraImageEffectFix";
    21.  
    22.         commandBuffer.Blit( resultAfterImageEffects as Texture, BuiltinRenderTextureType.CameraTarget );
    23.  
    24.         GetComponent<Camera>().AddCommandBuffer( CameraEvent.AfterImageEffects, commandBuffer );
    25.     }
    26.  
    27.     void OnRenderImage( RenderTexture src, RenderTexture dest )
    28.     {
    29.         resultAfterImageEffects = src;
    30.     }
    31. }