Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

So I got dual-paraboloid 97% working.. would you gurus co-brainstorm with me for the last 3%? :)

Discussion in 'Shaders' started by metaleap, Jun 23, 2014.

  1. metaleap

    metaleap

    Joined:
    Oct 3, 2012
    Posts:
    589
    ( Warning, wall of text but with some screens and some codez also a little further down :D )

    So I set my mind to implementing this tutorial: http://graphicsrunner.blogspot.ie/2008/07/dual-paraboloid-reflections.html which in turn is based on http://www.gamedev.net/page/resourc...paraboloid-mapping-in-the-vertex-shader-r2308 --- so far I'm just generating the 2 reflection maps (front and back), not even sampling into them (yet).

    Kyle Hayward (aka graphicsrunner linked above) also has a sample code package which I downloaded and worked off of. But it's a custom little mini-engine / basic DX renderer app in the zip, not Unity, so a few things are different: for generating both front and back maps, all geometry is rendered with no culling, and his technique does some manual "discard"-clipping based on Z direction in the frag shader based on the input from the vert shaders.

    Anyway it's a very cool reflection approach and still has its use and place: even today with current/next-gen mobile devices, dynamically generated cubemaps have the issue of super-ugly/dumb mip-maps and the still-not-insignificant cost of "6 full scene passes" to fill a usable reflection map.

    So "I got dual-paraboloids to work" for the most part but the devil is in the freaking details :/ I can't reflect the skybox (which is actually absolutely crucial no matter how you slice it) and I'm seeing stray artifacts "on the ground only" where I can't tell whether it's the artifacts the article warns about for "low tesselation geometry" or a side-effect of doing this (perhaps also slightly wrongly) in Unity as opposed to Kyle's own original custom mini-engine / DX renderer app.

    Let me give you a few screens to show the context in which I'm doing this and what I got so far, then further down some code.

    First off, I'm using a colorful debug skybox so don't be confused by all the funky colors in the following screens, here's how it's set up:

    https://dl.dropboxusercontent.com/u/136375/img/screens/unity-dprefl-06.png

    Its shader is set to "azFx_Skybox" but this is a verbatim line-by-line copy of the built-in RenderFX/Skybox shader, I made this copy to be later on modified for dual-paraboloid compatibility.

    So here's my playground scene:

    https://dl.dropboxusercontent.com/u/136375/img/screens/unity-dprefl-01.png

    Some more shots also show-casing the ugly-but-useful debug-skybox

    https://dl.dropboxusercontent.com/u/136375/img/screens/unity-dprefl-03.png

    https://dl.dropboxusercontent.com/u/136375/img/screens/unity-dprefl-04.png

    So generating the dual-paraboloid reflection means rendering into two reflection render-textures, front hemisphere and back hemisphere. Each includes a 180° view into the scene. In order for Unity not to frustum-cull anything that should go into such a reflection render-texture, field-of-view is set for the reflection camera to the maximum-possible 179:

    https://dl.dropboxusercontent.com/u/136375/img/screens/unity-dprefl-07.png

    Since there is apparently no known linear projection matrix to transform vertices into the paraboloid parameterization, what we do is set the camera to Perspective but in all vertex shaders ignore the projection matrix. Instead of UNITY_MATRIX_MVP, all vertex shaders just use UNITY_MATRIX_MV and then do the extra vertex-repositioning steps outlined in Kyle's tutorial.

    So in my set-up the reflection-camera position is always set to the "player's head" position (so world-Y is 1.5 units when the player is grounded at a world-Y of 0). Then it sets its global rotation to 0,0,0 for rendering the "front" reflection-map and then to 0,180,0 for rendering the "back" reflection map. Flipping back and forth every other frame. (The cam is also set to VertexLit which triggers specific "LightMode=VertexLit" shader Passes that are in my setup only used during reflection renderings, outside of reflections I'm doing some custom Deferred lighting but during reflections: no bumpmapping, no shadows, cheap-a$$ per-vertex Lambert-only NdotL-lighting).

    Let's see the reflections! OK when the player is positioned like in the very first screen shot above, with skybox disabled it renders the following reflection texture:

    https://dl.dropboxusercontent.com/u/136375/img/screens/unity-dprefl-02.png

    And from the same position, but for the opposite reflection direction and now with skybox enabled (but no paraboloid logic in it yet), the camera renders the following reflection texture:

    https://dl.dropboxusercontent.com/u/136375/img/screens/unity-dprefl-05.png

    You can tell how the skybox isn't set up for paraboloid so it just uses the FoV=179° perspective-projection.

    How does it work? Here's the typical vertex shader, relevant excerpt, obviously among my #pragmas there's a multi_compile AZ_DUALPARABOLOID_OFF AZ_DUALPARABOLOID_ON

    Code (csharp):
    1.  
    2. struct v2f_surf {
    3.     float4 pos : SV_POSITION;
    4.     float2 pack0 : TEXCOORD0;
    5.     fixed3 lightCol : COLOR;
    6.     #if AZ_DUALPARABOLOID_ON
    7.     half clipZ : TEXCOORD1;
    8.     #endif
    9. };
    10.  
    11. inline v2f_surf vert_surf(const in appdata_base v) {
    12.     v2f_surf o;
    13.     o.pack0.xy = TRANSFORM_TEX(v.texcoord, _MainTex);
    14. #if AZ_DUALPARABOLOID_ON
    15.     o.pos = mul(UNITY_MATRIX_MV, v.vertex);
    16.     o.pos.z = -o.pos.z; // we dont need the 'zDir' uniform from Kyle's demo as our cam rotates, but as I'm targeting GL, only with '-z' do *both* reflection-maps render what they should
    17.     const float L = length(o.pos.xyz);
    18.     o.pos.xyz /= L;
    19.     o.clipZ = o.pos.z;
    20.     o.pos.xy /= o.pos.z + 1.0;
    21.     half near = _ProjectionParams.y;
    22.     half far = _ProjectionParams.z;
    23.     o.pos.z = (L - near) / (far - near);
    24.     o.pos.w = 1.0;
    25. #else
    26.     o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
    27. #endif
    28.     const float3x3 mv = (float3x3)UNITY_MATRIX_MV;
    29.     const float3 vNormal = SCALED_NORMAL;
    30.     o.lightCol = calcLightCol(normalize(mul(mv, vNormal))); // from another .cginc, just a plain supercheap Lambertian NdotL
    31.     return o;
    32. }
    33.  
    So if this vertex shader code is present in the VertexLit shader Passes that only get triggered for geometry rendered by the reflection-camera, then I can render my scene into front and back paraboloid reflection maps such as linked in the above screenshots.

    The fragment shaders do their usual simple frag stuff (_MainTex.rgb * IN.lightCol) but the first thing they have is this:

    Code (csharp):
    1.  
    2. #if AZ_DUALPARABOLOID_ON
    3.     clip(IN.clipZ);
    4. #endif
    5.  
    And it's necessary, otherwise stray polygons from the wrong hemisphere that didn't get frustum-culled (due to their bounding-boxes or whatever) often pop in "from behind" and occlude most of the correct content.

    Well, good stuff!.... well, almost :/

    Now, the trouble is that there are 2 issues remaining where I'm at my wits end, no doubt in no small part due to an incomplete grasp on solid fundamentals :D

    First: the ground plane. It's the "kraut_plane" mesh from the "Water" (or Water Pro or Water4, can't remember) Standard Assets package, has quite some tesselation at 5100 triangles), I have a box-collider on it and the game-object scale is at 3,1,3. First, I had a simple box for this temporary ground but this only has very few triangles and so got almost completely clipped from the reflection maps, ie. basically didn't show up at all.

    But even the ground plane with its much finer tesselation reflects really bad as the above reflection screens show. Remember the reflection-cam is located 1.5 units above ground, I could understand it if it was at ground level but, oh well :D

    Now one could think if I remove the clip() at least the ground might reflect completely and properly, but it's still incomplete and artifacty and deranged/deformed -- somewhat differently (even weirder) than with clip(), but still.

    Now I'm wondering what's causing this weird ground rendering?

    Theory 1: does the paraboloid positioning simply de-generate towards the near-plane? I don't think so, I can move the player into the car and I get a nice 360° panorama from inside the car which similarly clips into the cam but does not de-generate like this, it looks crisp and correct.

    Theory 2: is the ground-plane not finely tesselated enough? Perhaps, but then it still has quite some tesselation and most other geometry is very low-poly in comparison yet even just as close to the reflection cam again does not de-generate like this.

    Any other theories?

    Second: making it work for the skybox. I have no freaking clue. As I said before, I have a verbatim copy of the out-of-box RenderFX/Skybox shader. Seems like it uses 6 passes, aka 6 draw-calls so I have no clue what geometry-primitives it is used with. Still, by default it is doing the good old mul(UNITY_MATRIX_MVP, v.vertex) so reusing my above vert-shader modifications for PARABOLOID_ON should totally work the same, but with that the skybox just doesn't show up at all in the reflection textures, just black background (ie. just my refl-cam's clear-color).

    And that makes me wonder what's really going on with the skybox. What geometry is used? Reason would suggest a cube, but then why have 6 passes. Ah yes for the 6 faces, they don't use a cubemap here. So do they render 6 quads, or a single cube 6 times? Also the usual camera-world-perspective matrix probably isn't used as the sky is infinitely far away so world coords won't make any sense..

    Does anyone know the intimate rendering internals of the Skybox? Perhaps even an idea how I'd have to tweak the paraboloid-vertex-repositioning logic for it? I'd rather not write my own skybox-replacement solution and also don't wanna use the "mobile skybox shader only with 1 texture", using (my verbatim clone of) the built-in 6-faces skybox shader would be optimal. And figuring out what special handling it needs will be enlightening, once I do, if ever... :D

    Without skybox, I can throw away the dual-paraboloids right there. And that'd be a shame because you can see it generates some very nice reflection textures! And if I don't use cubemaps, I can even afford slightly higher resolution, as could others wishing to adopt this technique. But how can we get our skybox in there?
     
  2. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,051
    I suggest uploading a simple demo project for people to test with. Though you've provided plenty of details the only people who will be able to help are those who have directly had the same experience, whilst others might be able to speculate. Providing a demo project will allow others to take a look, check settings etc and would probably get better replies.

    As for skybox, I would guess that Unity is rendering it with a custom a projection using a unit cube. it doesn't need to render into the depth buffer since nothing is behind it. Not sure when its rendered though. As the first object it will obviously increase fillrate/overdraw, as the last object it needs depth compares. Used to be rendering it last was a good bet as it saved on fillrate, but I think that may have changed over time (last decade or more ;) )

    Have you tried using skybox_Cubed and a cubemap for the skybox instead?
     
  3. kebrus

    kebrus

    Joined:
    Oct 10, 2011
    Posts:
    415
    +1 for the demo, i never heard of this technique before so i wont be of much help but it totally picks my interest
     
  4. metaleap

    metaleap

    Joined:
    Oct 3, 2012
    Posts:
    589
    OK gonna prepare a minimal demo scene and package it up later today ;)
     
  5. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,051
    cool I'll try to find some time to check out out, but been hit today with half a dozen things to sort out so it might be some time.

    I did skim the links you provided and I guess if you aren't using a custom skybox shader then that would explain why its not working. More than that though since the projection is wholly based on manipulating the vertices and that this manipulation is more extreme than normal perspectives it would make sense that your skybox is going to need to be heavily tessellated too.

    Now I guess that's going to be a bit of an issue since I don't remember being able to specify the skybox cube anywhere. That leaves you with two options. First ignore Unity built in skybox rendering and add your own box instead or create a geometry based shader that tessellates the default unity skybox.

    Though Geometry shader is more fun, I think the best method is probably just to write your own skybox rendering routine, perhaps use a camera OnWillRender() type function to issue a draw call that renders a custom tessellated cube with your own skybox shader based on the Unity skybox_cubed shader, but with your custom vertex code.
     
  6. metaleap

    metaleap

    Joined:
    Oct 3, 2012
    Posts:
    589
    I think you're spot on with the low-tesselation cube. Ultimately "in production" I'm actually using a fairly highly-tesselated sky-dome/hemisphere mesh and guess what, I've just converted its vert-shaders and it does reflect beautifully!

    I use the 6-color debug skybox for verifying accuracy and consistency/alignment of various reflection methods so would have been very neat to get it to work quickly. But no biggie, not gonna dive into geom-shaders just for that :D
     
  7. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,788
    Why not going with skydome??
     
  8. waldob

    waldob

    Joined:
    Mar 20, 2013
    Posts:
    24
  9. Prohell

    Prohell

    Joined:
    Oct 14, 2017
    Posts:
    2
    So late....
     
  10. Prohell

    Prohell

    Joined:
    Oct 14, 2017
    Posts:
    2
  11. MadeFromPolygons

    MadeFromPolygons

    Joined:
    Oct 5, 2013
    Posts:
    3,967
    was a 2 year old thread mate, noone is responding here.