Search Unity

[Bug?] Surface Shaders with pragma alpha or pragma alphatest ALWAYS get ColorMask RGB

Discussion in 'Shaders' started by DodgeRollBrent, Aug 29, 2014.

  1. DodgeRollBrent

    DodgeRollBrent

    Joined:
    Dec 15, 2012
    Posts:
    7
    I am attempting to composite several rendertextures together, and am running into a strange problem.

    Any surface shaders that use the pragma alpha or pragma alphatest automatically get ColorMask RGB in the compiled version of the shader. This means the alpha channel is not written properly for compositing.

    If I manually go into the compiled surface shader and set ColorMask RGBA, everything works properly, but this forces me to replace my surface shader with the fully generated surface shader, and is a lot of work.

    If I remove pragma alpha/alphatest, this also fixes the compositing issue, but obviously causes a number of other errors related to the shader's blending.

    A few people have noted this problem, but no one has found a workable way around it to my knowledge.

    Aras, any ideas? Or anyone else?
     
    DodgeRollBrent, Aug 29, 2014
    #1

  2. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    It's not a bug insofar as this is something surface shaders simply weren't designed to do. Surface shaders are mostly about control over input to lighting functions, and there are a lot of blending and compositing techniques that they don't support. For instance, glass can pretty much only be done using decal:add, which means it has to be perfectly clean.
     
    Daniel_Brauer, Aug 29, 2014
    #2

  3. DodgeRollBrent

    DodgeRollBrent

    Joined:
    Dec 15, 2012
    Posts:
    7
    That may be so, but it still seems odd to me that the surface shader specification mandates ColorMask RGB when dealing with alpha blended or tested shaders, especially when the shaders seem to function just fine with other ColorMask options.

    I was hoping there was an undocumented way to override the ColorMask behavior of the pragmas; it doesn't seem like something INTRINSIC to the idea of surface shaders that I couldn't do this.
     
    DodgeRollBrent, Aug 29, 2014
    #3

  4. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    Some assumptions and restrictions that probably play into this:
    • By default, everything that writes to the alpha channel in Unity is doing so in order to record overbrightness for the fullscreen bloom effect.
    • The built-in Blinn-Phong lighting function, for instance, adds the specular result to whatever alpha you specify in your surface function.
    • Surface shaders don't give you access to the Blend command, because they generate multiple passes which use different Blend commands.
    • Instead of direct access to the Blend command, we get the decal and alphaTest parameters. These use the alpha you provide in order to do alpha testing, lerp alpha blending, or additive blending.
    • Without the ability to specify the blending coefficients (specifically those of the alpha channel), the alpha output from a shader using alpha to blend or test is generally not something you want written to the screen.
    Of course there are many reasons you might want to modify how surface shaders work, but the above points should help explain why they only work the way they do.
     
    Daniel_Brauer, Aug 29, 2014
    #4

  5. DodgeRollBrent

    DodgeRollBrent

    Joined:
    Dec 15, 2012
    Posts:
    7
    That all makes sense to me, and it certainly does a good job of explaining why the default behavior is to not write the alpha channel to the framebuffer with alpha blended or tested surface shaders.

    As you say yourself, though, it is "generally not something you want written to the screen." I happen to find myself in an exception to that rule, and it seems as if the only way out is to give up all of the other nice benefits that using the surface shader framework gives me, just to go from "ColorMask RGB" to "ColorMask RGBA."

    If that's what I have to do, that's what I have to do. I was hoping for a less unwieldy solution.
     
    DodgeRollBrent, Aug 29, 2014
    #5

  6. Daniel_Brauer

    Daniel_Brauer

    Unity Technologies

    Joined:
    Aug 11, 2006
    Posts:
    3,355
    That's probably your best bet. That said, I'm really curious why you want your alpha alpha-blended with the frame buffer alpha. What is that useful for?
     
    Daniel_Brauer, Aug 30, 2014
    #6

  7. DodgeRollBrent

    DodgeRollBrent

    Joined:
    Dec 15, 2012
    Posts:
    7
    Without getting too deep in the technicalities of it, I'm rendering a 2d game from an unusual perspective, and I'm compositing the primary view together from several different cameras.

    In order to get the separate layers to blend properly, I need each camera's alpha channel; colormasked objects on anything but the base composited layer end up getting "punched out" due to their lack of proper alpha.

    I hope that was a clear enough explanation.
     
    DodgeRollBrent, Sep 2, 2014
    #7