Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Stencil buffer

Discussion in 'Shaders' started by Richard_B, Sep 9, 2006.

  1. Richard_B

    Richard_B

    Joined:
    Jul 22, 2005
    Posts:
    436
    Any way of accessing the stencil buffer? I am trying to create a circular (or elliptical) viewports on screen from several camera's and stencil buffers seems the simplest way to achieve this.

    thanks,
    Richard.
     
  2. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    Not at the moment.

    What you could do now is render something like "inverted ellipse" (i.e. an object that is an outside of the ellipse) with ColorMask A in the shader very close to the camera, so that depth buffer values become very small. Then render your regular stuff, it should be invisible outside the ellipse because it will always be "further away" than this invisible thing.
     
  3. Richard_B

    Richard_B

    Joined:
    Jul 22, 2005
    Posts:
    436
    Ok - I'll give that a try - but stencil buffers would be nice :)

    thanks,
    Richard.
     
  4. Richard_B

    Richard_B

    Joined:
    Jul 22, 2005
    Posts:
    436

    Attached Files:

  5. Samantha

    Samantha

    Joined:
    Aug 31, 2005
    Posts:
    609
    I'm interested in this. Would you mind outlining the steps you took to get a circular viewport working?
     
  6. Richard_B

    Richard_B

    Joined:
    Jul 22, 2005
    Posts:
    436
    Ulognep,

    I created it using an orthogrphic camera parented to the player and pointing down. The map camera only renders objects on the "map" layer, which currently only contains a plane with the map image as a material (which is actaully created on startup by another camera using render-to-texture, but you can equally well use a screen snap). The trick to give you the circular map is to use a second "masking" camera also parented to the player. The making camera only renders a single mesh (also parented to the player) wihich is in the "mask" lalyer. This mesh has a hole (which gives the circular shape - change the hole to any shape to change the map shape) -- see image below. The mask mesh also uses Neil Carters DepthMask material:

    http://www.unifycommunity.com/wiki/index.php?title=DepthMask

    Study Neil's project to see how to use the material.

    Richard.
     

    Attached Files:

  7. antenna-tree

    antenna-tree

    Joined:
    Oct 30, 2005
    Posts:
    5,324
    Any update to this? Is there any way to mask layers with a texture now... anything? I just ordered the Cg Tutorial book to learn more about shaders and to try and accomplish this specific effect. My knowledge of this stuff is really shaky at the moment, but I'm assuming the stencil buffer is what is needed to mask camera layers with a texture. Can we access this yet in ShaderLab?

    The ColorMask effect is great, but I need to be able to mask more complex shapes that aren't really possible performance-wise with polygons.

    I know I'm botching the terminology here, but any insights to what is or is not possible at this time with Unity and stencil buffers would be appreciated.

    Ethan
     
  8. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,770
    Couldn't you use the same shader with alpha testing on?
     
  9. antenna-tree

    antenna-tree

    Joined:
    Oct 30, 2005
    Posts:
    5,324
    Brilliant! That did the trick. Thanks a bunch Aras.
     
  10. thylaxene

    thylaxene

    Joined:
    Oct 10, 2005
    Posts:
    716
    Can you post the revised depth mask shader that takes an alpha image?

    Cheers.
     
  11. Alex

    Alex

    Joined:
    Nov 19, 2008
    Posts:
    122
    *performs arcane resurrection ritual*

    Sorry for the big bump, but I had a question related to this topic. I have a working round-edged minimap in the upper right corner of my screen. It's being put there by a normalized viewport and was created via the method outlined here. However, I want to add a GUI overlay on top of it (a simple ring graphic) to give it a border and mask out the straight edges of the hole in the mask plane.

    I'm noticing that the viewport is scaling in the game window when I toggle between fullscreen/2-split mode. This is problematic for a couple of reasons-- it's making getting the size of the GUI border and lining it up properly a pain. The machine I'm on is also not the same resolution as the target platform, so if I get it lined up in the editor if it keeps resizing there's no guarantee it'll look right on the kiosk.

    Any suggestions on how to tackle this?
     
  12. Imp of the Perverse

    Imp of the Perverse

    Joined:
    Apr 2, 2011
    Posts:
    10
    I'd like to adapt this for use as a scope - the main camera continues to render the scene at the default FOV, but the scope cam renders it at the zoomed FOV, displaying the image in a round viewport that lines up with the scope lens (guess this could also be done via render to texture, but that's a Unity Pro feature.) The trouble is, with this method the scope cam won't render the skybox. One way around this would be to not use the default skybox - actually model the box as a sphere or planes surrounding your scene. Any other way around this?

    *Edit - I'm using a big sphere with a double sided texture as a temporary skybox, and things are almost working - the trouble is the depth mask doesn't seem to effect terrain. Is there a way around this?
     
    Last edited: Apr 3, 2011
  13. JasonMo

    JasonMo

    Joined:
    Dec 14, 2011
    Posts:
    3
    I was looking for something similar, and updated the DepthMask shader to use a texture cut out. This will be extremely useful for anyone making 3d GUIs. Just use a 1x1 plane with texture and use the original method for renderqueues (see above). I also decided to make a Masked shader rather than the SetRenderQueue script, just works for me, totally optional. Enjoy!

    Code (csharp):
    1.  
    2. Shader "Transparent/Mask" {
    3.  
    4.     Properties
    5.     {
    6.         _MainTex ("Base (RGB) Alpha (A)", 2D) = "white" {}
    7.         _Cutoff ("Base Alpha cutoff", Range (0,.9)) = .5
    8.     }
    9.    
    10.     SubShader {
    11.         // Render the mask after regular geometry, but before masked geometry and
    12.         // transparent things.
    13.        
    14.         Tags {"Queue" = "Geometry+10" }
    15.        
    16.         // Don't draw in the RGBA channels; just the depth buffer
    17.        
    18.         ColorMask 0
    19.         ZWrite On
    20.        
    21.         Pass
    22.         {
    23.             AlphaTest LEqual [_Cutoff]
    24.            
    25.             SetTexture [_MainTex] {
    26.                 combine texture * primary, texture
    27.             }
    28.         }
    29.     }
    30. }
     
  14. Zeanog

    Zeanog

    Joined:
    Jan 10, 2012
    Posts:
    5
    Is it still not possible to write to the stencil buffer? I am attempting to have an opaque outline on a translucent model and Im currently having to dirty up the depth buffer causing be to be unable to blend with whats behind my object. Just curious. Thanks
     
  15. Bezzy

    Bezzy

    Joined:
    Apr 1, 2009
    Posts:
    75
    Zeanog - yeah it's doable. I've been messing with that exact trick for a while.

    The key is to make quite a lot of separate shaders, and treat each thing like a layer.

    You'll want to make a specific shader, and play with the Queue order a lot for that.


    it's sort of a hacky way to clear depth without clearing previously rendered stuff. But most of graphics is smoke and mirrors, so try to learn where it does and doesn't work, and design around that.

    So you:

    1) draw your normal geometry at Queue = Geometry
    2) draw your translucent "middle" (give the mask shader a draw order after geometry that you DON'T want it to occlude, before geometry)
    Color looks like nothing, but depth is reserved for your sillhouette innards.
    3) draw geometry that you DO want to have occluded by your central piece: i.e. draw your outline next( give the silhouette shader a queue order after 1's.). make it respect ZTest LEqual
    Back faces draw behind the earmarked depth pixels, but not in front: you can now see an outline.

    There's a few problems you might have when two of the same invisible areas change draw order, based on depth.. popping and whatnot. And the silhouettes will intersect one another, unless you do even more work and make one shader per thing you want silhouettes on. Actually, I've seen shaders which flatted the depth of the silhouette itself so that they're all at the same layer, and only draw order, rather than depth (since it's equal) affects which silhouette gets drawn first if the two overlap. But yeah, that's just a case of repeating the process over and over.

    The key is to treat the Queue almost like photoshop layers - one "layer" drawing on top of the other, but with this extra aspect of depth.

    Remember that you could also use a mesh with a ZTest Always shader (in the appropriate queue position/draw order) to force depth to whatever you're rendering. Like, you could attach a (massive) quad to a secondary camera you want to *only* clear the depth on (set its flags to "don't clear", so long has you have your normal world geometry drawn by a first camera which DOES clear with solid colour or skybox). Put the quad on the camera you want to depth clear with a local z of about camera.far -1, and then you can re-draw objects which actually, technically, are behind others, but just happen to be later in the draw order.