Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

how to disable the Stencil block via shader properties?

Discussion in 'Shaders' started by benblo, Oct 5, 2015.

  1. benblo

    benblo

    Joined:
    Aug 14, 2007
    Posts:
    476
    I'm trying to write a shader for making holes (masking). Here's the bare-bone minimal shader I came up with:
    Code (csharp):
    1. Shader "Test/stencil read"
    2. {
    3.     Properties
    4.     {
    5.         _StencilMask ("Stencil Mask", Int) = 1
    6.     }
    7.  
    8.     SubShader
    9.     {
    10.         Tags { "RenderType"="Opaque" }
    11.    
    12.         Pass
    13.         {
    14.             Stencil
    15.             {
    16.                 Ref [_StencilMask]
    17.                 ReadMask [_StencilMask]
    18.                 //WriteMask [_StencilMask]
    19.                 Comp NotEqual
    20.                 Pass Keep
    21.             }
    22.         }
    23.     }
    24. }
    25.  
    So by default this will only draw pixels if there's nothing in the mask for the given layer.

    Now the thing is, I'd like it to be an uber-shader so that for any material we can just set a property, and it becomes masked by objects of a given layer. So I can make the comparison function a property as well:
    Code (csharp):
    1.     Properties
    2.     {
    3.         [Enum(CompareFunction)] _StencilComp ("Stencil", Int) = 0
    4. ...
    5.             Stencil
    6.             {
    7.                 Comp [_StencilComp]
    8. ...
    so that we either check against the mask with NotEqual, or we bypass it with Disabled (see http://docs.unity3d.com/ScriptReference/Rendering.CompareFunction.Disabled.html).
    Problem is, contrarily to what the docs say, either way the stencil is used, I can see plainly in GPA:
    Code (csharp):
    1. IDIRECT3DDEVICE9_SETRENDERSTATE( D3DRS_STENCILENABLE, True )
    2. ...
    3. IDIRECT3DDEVICE9_SETRENDERSTATE( D3DRS_STENCILFUNC, D3DCMP_ALWAYS )

    So my question is, is there a way to disable the Stencil block entirely via properties/keywords? Just want to make sure there are no hidden cost when I don't want masking.
    As far as I know, there's no way to #define state blocks using keywords...?
    I also tried stuff like
    Code (csharp):
    1.             Stencil
    2.             {
    3.                 Mode Off
    4. ...
    5.             }
    like for the Fog... no luck, parse error :( !
     

    Attached Files:

  2. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    1,520
    Just set everything to the default values. So mainly:
    Code (csharp):
    1.  
    2. Comp Always
    3. Pass Keep
    4.  
    This is the most you can do and should not have any hidden costs. If Comp is set to Always, there is no stencil read. If Pass is set to Keep, there is no stencil write.
     
    WayneJP likes this.
  3. benblo

    benblo

    Joined:
    Aug 14, 2007
    Posts:
    476
    Okay, so what you're saying is Unity is setting states anyway, but if the values are right the hardware will just do nothing?
     
  4. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    1,520
    There is an actual on and off, but not in Unity it seems. You can almost assume that the driver will disable it for you anyway, but still...
     
  5. benblo

    benblo

    Joined:
    Aug 14, 2007
    Posts:
    476
    Yeah that's a bit frustrating. Thanks for the answer!
     
  6. tanoshimi

    tanoshimi

    Joined:
    May 21, 2013
    Posts:
    297
  7. benblo

    benblo

    Joined:
    Aug 14, 2007
    Posts:
    476
    See my initial post, either way the stencil state is visible in GPA:
    ... though I don't know if it has any cost; fingers crossed.
     
  8. kmalkic

    kmalkic

    Joined:
    Sep 28, 2016
    Posts:
    2
    founded that you can actually do it.


    Code (CSharp):
    1.     Properties {
    2.         _Stencil ("Stencil ID", int) = 4
    3.         _StencilComp ("StencilComp", Int) = 3 //3 = equal
    4.     }
    5.     SubShader {
    6.         Stencil {
    7.             Ref [_Stencil]
    8.             Comp [_StencilComp]
    9.             Pass keep
    10.             Fail keep
    11.         }
    and then at runtime use 'always' (8 is 'always' in the enum)
    Code (CSharp):
    1.  material.SetInt ("_StencilComp", 8);
    and 'disable' is 0
     
  9. tcmicka

    tcmicka

    Joined:
    Aug 2, 2017
    Posts:
    9
    Was a consensus found on the proper way to disable stencil operations via script?
     
  10. Grinning-Pickle

    Grinning-Pickle

    Joined:
    Feb 4, 2013
    Posts:
    7
    I'd like to also know this. We use this on a lot of objects and I would love to know I can do this without performance impact.

    Edit: I went with jvo3dc's suggestion. At least functionally it appears to work correctly. For performance, I just can't be a 100% sure.
     
    Last edited: May 20, 2019
  11. yjxyuna

    yjxyuna

    Joined:
    Aug 29, 2017
    Posts:
    1
    Properties:
    Code (CSharp):
    1. //stencil ref
    2. _StencilRef ("StencilRef", int) = 1
    3.      
    4. //0:Disable 1:Never 2:Less 3:Equal 4:LessEqual 5:Greater 6:NotEqual 7:GreaterEqual 8:Always
    5. [Enum(UnityEngine.Rendering.CompareFunction)] _StencilComp ("StencilComp", Int) = 8
    6.      
    7. //0:Keep 1:Zero 2:Replace 3:IncrSat 4:DecrSat 5:Invert 6:IncrWrap 7:DecrWrap
    8. [Enum(UnityEngine.Rendering.StencilOp)] _StencilOp ("StencilOp", int) = 2
    SubShader:
    Code (CSharp):
    1. Stencil{
    2.      Ref [_StencilRef]
    3.      Comp [_StencilComp]
    4.      Pass [_StencilOp]
    5.      ...
    6. }
     
    Last edited: Jan 28, 2021
  12. noethis

    noethis

    Joined:
    Nov 11, 2013
    Posts:
    129
    Reviving this thread, wondering if anybody knows for a fact whether there's a performance impact from having StencilComp=Disabled vs not having the code at all in the shader?

    I just did a small profiling test myself and there seems to be up to a .5ms difference (CPU renderering time) in the scene I was testing but I'm not 100% sure if that's it or not.
     
  13. Zapan15

    Zapan15

    Joined:
    Apr 11, 2011
    Posts:
    186
    Once you define a "Stencil" info in the file, even if it contains only the default values:
    Code (CSharp):
    1.         Stencil
    2.         {
    3.             Ref 0
    4.             CompBack Always
    5.             PassBack Keep
    6.  
    7.             CompFront Always
    8.             PassFront Keep
    9.         }
    10.  
    it is slower :(