Search Unity

Screen Space Local Reflection

Discussion in 'Works In Progress - Archive' started by castor76, Oct 17, 2013.

  1. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Hi all.

    I have banged my head against all the different "spaces" in Unity3D projections maths stuff and gave a go with what people call "Screen Space Local Reflection"

    The idea is certainly not new, and some people here in Unity community has already tried them and I know for sure that the hard surface shader has it working really nicely, but there is nothing like doing it yourself.

    So here it is...

    $sslr01.jpg

    As you can see from the picture, that I have basic stuff going... but there are still lots of improvements needs to be done.

    The one particular improvement that I am looking for is :

    When you have wrong ( or no info ) to reflect because the marched ray hits under the model or back of the model, how do you mask them and fade them away from the reflection contribution?

    Any comments or ideas especially from the ones who has already done this before would be more than welcome.

    My method is basically based on this post.

    http://www.gamedev.net/blog/1323/entry-2254101-real-time-local-reflections/

    The above method also suffers from the same problems...

    Another thing is to find some better method of ray marching...

    Thanks!
     
  2. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    I have tried to make this shader as a post process shader...

    But I am having trouble converting viewPos into screenPos.

    Basically, I need to convert view space position into screen space position using projecting matrix, but I can't make it work for the post processing shader.

    It works for non-post process shader.

    Please note that this is an experiment shader code.

    ---------

    Code (csharp):
    1. Shader "Custom/SSRPostProcess"
    2. {
    3.  
    4.     Properties
    5.     {
    6.         _MainTex ("Base (RGB)", any) = "" {}
    7.     }
    8.            
    9.     SubShader
    10.     {
    11.         Pass
    12.         {
    13.             CGPROGRAM
    14.             #pragma exclude_renderers d3d11 xbox360
    15.             #pragma vertex vert
    16.             #pragma fragment frag
    17.             #include "UnityCG.cginc"
    18.             #pragma target 3.0
    19.             #pragma glsl
    20.            
    21.             sampler2D _MainTex;
    22.             float4 _MainTex_ST;
    23.             float4 _MainTex_TexelSize;
    24.            
    25.            
    26.             uniform sampler2D _backgroundTexture;
    27.             uniform float _fadeToView;
    28.             uniform float4x4 _inverseProjection;
    29.             uniform float4x4 _ProjMatrix;
    30.             uniform float4x4 _ProjectionInv;
    31.             uniform float4 _ProjInfo;
    32.            
    33.             sampler2D _CameraNormalsTexture;
    34.             sampler2D _CameraDepthNormalsTexture;
    35.             float4 _CameraDepthNormalsTexture_ST;
    36.            
    37.             sampler2D _CameraDepthTexture;
    38.             float4 _CameraDepthTexture_ST;
    39.  
    40.             struct v2f {
    41.                 float4 pos : SV_POSITION;
    42.                 float2 uv : TEXCOORD0;
    43.             };
    44.  
    45.             v2f vert (appdata_img v)
    46.             {
    47.                 v2f o;
    48.                 o.pos = mul (UNITY_MATRIX_MVP, v.vertex);
    49.                 o.uv =  v.texcoord.xy;
    50.                 return o;
    51.             }
    52.            
    53.             float3 ReconstructCSPosition(float2 S, float z)
    54.             {
    55.                 float linEyeZ = LinearEyeDepth(z);
    56.                 return float3(( ( S.xy * _MainTex_TexelSize.zw) * _ProjInfo.xy + _ProjInfo.zw) * linEyeZ, linEyeZ);
    57.             }
    58.            
    59.             float4 ReconstructViewPosition(float2 S, float z)
    60.             {
    61.                 // for reference
    62.                 float4 clipPos = float4(S*2.0-1.0, (z*2-1), 1);
    63.                 float4 viewPos;
    64.                 viewPos.x = dot((float4)_ProjectionInv[0], clipPos);
    65.                 viewPos.y = dot((float4)_ProjectionInv[1], clipPos);
    66.                 viewPos.w = dot((float4)_ProjectionInv[3], clipPos);
    67.                 viewPos.z = -z;
    68.                 viewPos = viewPos/viewPos.w;
    69.                 return viewPos;
    70.             }
    71.  
    72.              
    73.             half4 frag (v2f i) : COLOR
    74.             {
    75.                 float4 geom = tex2D(_CameraDepthNormalsTexture,i.uv);
    76.                 float3 viewNormal;
    77.                 float  depth;
    78.                
    79.                 // ths is to get a view normal
    80.                 DecodeDepthNormal(geom,depth,viewNormal);
    81.                
    82.                 // just get Z again from depthmap because I don't want linear Z from 0-1 initially..(I think)
    83.                 float d = ( tex2D(_CameraDepthTexture, i.uv.xy) );
    84.                 float3 scrPos = float3(i.uv * 2 -1,d);
    85.                
    86.                 float4 viewPos = ReconstructViewPosition(i.uv,d);
    87.                 float3 vspReflect = normalize(reflect(normalize(viewPos.xyz), normalize(viewNormal)));
    88.                
    89.                
    90.                 // I think the problem is here
    91.                 //float4 vspPosReflectT = mul (UNITY_MATRIX_P,viewPos+float4(vspReflect,1));
    92.                 float4 vspPosReflectT = mul (_ProjMatrix, viewPos +float4(vspReflect,1));
    93.                 float3 vspPosReflect = vspPosReflectT.xyz / vspPosReflectT.w;
    94.                
    95.                                
    96.                 // subtract to get a vector.. (sub from end to start)
    97.                 float3 sspReflect = normalize( vspPosReflect - scrPos );
    98.                
    99.                 sspReflect = float3(sspReflect.xy*0.5f,sspReflect.z);
    100.                
    101.                 //initial stepping scalse value.. not sure about this neither
    102.                 float stepScale = 2/_ScreenParams.x/length(sspReflect.xy);
    103.                 stepScale = 0.002;
    104.                
    105.                 float4 c;
    106.                 float sampleDepth;
    107.                 float currentDepth;
    108.                 float depthTest = 0;
    109.                 float len = 0;
    110.                 int maxCount =164;
    111.                 float4 r = 0;  
    112.                 float bias = 0.001;
    113.                
    114.                 float rcpfadefact = 1/(1.0 - _fadeToView);
    115.                 float faceviewerfactor = (vspReflect.z - _fadeToView) * rcpfadefact;
    116.                 faceviewerfactor = 1-faceviewerfactor;
    117.                
    118.                 float3 startPosSS = float3(i.uv,scrPos.z);
    119.                 float3 sspReflectFinal = sspReflect;
    120.                 sspReflectFinal  *= stepScale * 1;
    121.                
    122.                 c = tex2D (_MainTex,i.uv);  
    123.                
    124.                 float deltaD = 0;
    125.                 float3 samplePos = startPosSS + sspReflectFinal;
    126.                 float3 oldPos = startPosSS;
    127.                 int cnt=1;
    128.                
    129.                 // ray march
    130.                
    131.                 while (cnt < maxCount)
    132.                 {
    133.                      
    134.                     geom = tex2Dlod (_CameraDepthNormalsTexture, float4(samplePos.xy,0,0));
    135.                     sampleDepth  = DecodeFloatRG(geom.zw);
    136.                     currentDepth = Linear01Depth(samplePos.z);
    137.                    
    138.                     if ( sampleDepth < currentDepth)  
    139.                     {  
    140.                    
    141.                         deltaD = (currentDepth - sampleDepth);
    142.                        
    143.                         if (deltaD < bias)
    144.                         {
    145.                             depthTest = 1;
    146.                             cnt = maxCount+1;
    147.                             break;
    148.                         }
    149.                         else
    150.                         {
    151.                             sspReflectFinal *= 0.5;
    152.                             samplePos = oldPos + sspReflectFinal;
    153.                         }                      
    154.                     }
    155.                     else
    156.                     {
    157.                         oldPos = samplePos;
    158.                         sspReflectFinal *= 1.1;
    159.                         samplePos += sspReflectFinal;
    160.                     }
    161.                     len += 1;
    162.                     cnt += 1;
    163.                 }
    164.                
    165.                float3 endP = float3(samplePos.x*2-1 , samplePos.y*2-1,samplePos.z);
    166.                
    167.                float dist = abs((distance(scrPos.xyz,endP)));
    168.                
    169.                r = tex2D(_MainTex,samplePos.xy);
    170.                r *= faceviewerfactor* faceviewerfactor;
    171.                r *= (1- pow(dist,1));
    172.                r *= depthTest;
    173.                
    174.                return  c + saturate(r)*0.5f ;                
    175.                
    176.             }
    177.             ENDCG
    178.         }
    179.     }
    180. }

    ----

    I think this line here is the problem.. (not 100% sure but)

    float4 vspPosReflectT = mul (UNITY_MATRIX_P,viewPos+float4(vspReflect,1));

    This works for non post process shader, but for some reason doing this and then doing the next line

    float3 vspPosReflect = vspPosReflectT.xyz / vspPosReflectT.w;

    doesn't give me the correct result. For some reason vspPosReflectT.w part is wrong...

    Any ideas?
     
    Last edited: Oct 18, 2013
  3. Lulucifer

    Lulucifer

    Joined:
    Jul 8, 2012
    Posts:
    358
  4. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Hi Lucifer

    I think I saw that one some time ago. very nice.

    Tell me did you try doing that as a post processing effect?
     
  5. Lulucifer

    Lulucifer

    Joined:
    Jul 8, 2012
    Posts:
    358
    there is something was missed in my implementation,but i didnt figure out how to fix or improve that,before that i wont try post effect
    after all it is something few months ago,never have time to try more :)
     
  6. CTPEJIOK22

    CTPEJIOK22

    Joined:
    Sep 4, 2013
    Posts:
    85
    looks nice)
     
  7. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Ok, after yet another head banging against world / view / projection space stuff I got this to work using post processing shader.

    My main struggle was to convert the screen space positions into view space positions and then back to the screen space. Also had hard time converting world space normal into view space normal.

    I could not use the view normals from Unity provided view+depth texture. Because view normals were too inaccurate and caused jump when I turn camera around. I had to land up using _CameraNormalsTexture and convert it into view space normal instead. But this ment I had to use deferred shadering. :( oh well...

    Performance wise, I think it must be slow or at least could go much faster. I couldn't really tell because fps tells me I am getting crazy 4000 fps.. but I know that can't be true.. looking at the profiler tells me that my post process is taking around 3ms so maybe it is around 200fps or something..?

    It is still the same basic SSLR without any advanced optimizations or visual improvements either than fading off on the edge of screen and obvious reflection vector pointing towards view...

    $ssr03.JPG
    $ssr04.JPG

    I think I need to come up with some better ray marching techniques and need to improve on depth compare algorithm..
    Also thinking about sampling and ray marching on smaller reduced texture to reduce the number of ray marching steps... I have seen from the other posts that their steps are like 16-30 etc.. but my steps are like 40 so I am sure there must be some better ways.
    Simple blur may help too..
    There are just so many things to learn..

    Any comments or feed back welcome. I would like to very much share and learn from others who have attempted this technique.
     
    Last edited: Oct 23, 2013
  8. superjayman

    superjayman

    Joined:
    May 31, 2013
    Posts:
    185
    good work. How are you computing _ProjMatrix in c#, why are you not using UNITY_MATRIX_P, can you please post your code or PM me please..cheers
     
  9. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Here is pic of my progress so far.

    I have added some blur pass after the reflection pass to reduce the artifacts from the false reflection catch and z depth in accuracy from ray marching. This one is combination of cube map reflection used for the sky.

    $ssr05.JPG

    To calculate projection matrix from C# :

    Code (csharp):
    1.         P  = this.camera.projectionMatrix;
    2.  
    3.         bool d3d = SystemInfo.graphicsDeviceVersion.IndexOf("Direct3D") > -1;
    4.         if (d3d) {
    5.             // Scale and bias from OpenGL -> D3D depth range
    6.             for (int i = 0; i < 4; i++) {
    7.                 P[2,i] = P[2,i]*0.5f + P[3,i]*0.5f;
    8.             }
    9.         }
    and pass it using Shader.SetGlobalMatrix

    The reason why I am calculating it instead of using UNITY_MATRIX_P is that in image effect (post processing) UNITY_MATRIX_P becomes an identity matrix.. I spent so long trying to figure that one out. When you know it , it kinda makes sense but it was not useful for me and confusing as hell. I don't think this is documented anywhere in the official Unity doc.
     
  10. 121

    121

    Joined:
    Nov 26, 2009
    Posts:
    199
    What a sexy shader.
     
  11. save

    save

    Joined:
    Nov 21, 2008
    Posts:
    744
    Nice work Castor! Looks promising. :)
     
  12. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Thanks Save...

    I am trying to figure out better method for doing ray marching but more I dig into it more I think I am approaching this realm of super maths.
    (Should have paid more attention at school! :D )

    There is this "distance field" method which I understand the principles but I don't think it can just be used in this case. Depth map per frame is not some mathematical formula. And calculating 3d distance field map sound like too expensive for realtime.
     
  13. superjayman

    superjayman

    Joined:
    May 31, 2013
    Posts:
    185
    Yes that UNITY_MATRIX_P issue is weird, any other tricky things to look out for? are you still using CameraDepthNormalsTexture to read depth? im also trying to implement this to learn from..cheers again
     
  14. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    No, I am using :

    depth = UNITY_SAMPLE_DEPTH(tex2Dlod (_CameraDepthTexture, float4(samplePos.xy,0,0)));

    which is non linear depth. It is the same as .z of the screen position z (from vertex shader if not using post process)
    Thing is that because it has very good accuracy around near plane of camera, I convert it into Linear01Depth when I do the compare.
    But I don't know if this is the best way because it just means more instructions per ray march loop.. :(

    Another thing to note is that view normal from CameraDepthNormalsTexture is almost free because Unity renders anyway, but its accuracy is not really good when it comes to the long flat plane. I think it is due to the compressing it into 2 RG channel and then decoding it back... The issue is that because it is not accurate enough, your refilection vector "jumps" in steps for large flat planes. ( I guess the error just propagates and multiply when you calculate reflection) so I could not really use them.

    What I landed up is going to deferred rendering path and used _CameraNormalsTexture instead. That gives you world normal so I had to convert it into view normal. (Again using projection from C#)

    Code (csharp):
    1. Matrix4x4 V = this.camera.worldToCameraMatrix;
    2. Shader.SetGlobalMatrix ("_ViewMatrix",V.inverse.transpose);
    So my approach is kinda, full of conversions which I don't think they are very optimal.
    Performance wise, I can't tell because it is rendering at 2-3ms so it is not too bad on my PC but I got i7 930 running with GTX 660.
     
    Last edited: Oct 23, 2013
  15. superjayman

    superjayman

    Joined:
    May 31, 2013
    Posts:
    185
    what did you do with the following line of code

    //float4 vspPosReflectT = mul (UNITY_MATRIX_P,viewPos+float4(vspReflect,1));

    float4 vspPosReflectT = mul (_ProjMatrix, viewPos +float4(vspReflect,1));

    float3 vspPosReflect = vspPosReflectT.xyz / vspPosReflectT.w;

    are you dividing with .w or .z also is Linear01Depth a unity function? This has been buggin me:)
     
  16. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Here is web player of what I am working on.. Use Maya like navigation. (use alt + mouse click drag)

    http://www.castorlee.com/script/shader/SSR/WebPlayer.html

    Superjay, I am dividing with .w Nothing has changed there..Also Linear01Depth is function from Unity included in UnityCG.cginc file.

    As you can see from the player, that I have issue with false reflection catch... If I make my depth compare bit more tighter, then I get very noisy result around that false reflection. The web player version is the one with relaxed depth compare. I need to come up with better ray marching technique to get rid of false reflection without too much noise. No blur filter will get rid of that...
     
    Last edited: Oct 23, 2013
  17. 121

    121

    Joined:
    Nov 26, 2009
    Posts:
    199
    It looks nice. What's causing the cylindrical shaped reflection error with the spheres?
    I can't believe this is running on my 8800GTS with WinXP :cool:

    You have a PM.
     
  18. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    It is caused by the what I call "false reflection". Basically, when you want to get a reflection pixel on certain area, the reflection is supposed to point to the "back" of the model you can see. Which you can't get because of the obvious limitation of the screen space reflection. In SSR you can only reflect what you can see in the screen, nothing behind the back of model or off the screen etc..

    I tell my ray march to stop when the difference between reflection ray's depth and depth from depth map is less than certain value. When I let the difference to be tight, I get very noisy artifacts and if I let it be bit more forgiving, then it shows that cylindrical error. (it thinks it found right reflection point but in fact it is not <- but shader doesn't know it is wrong..)

    If I can be more super accurate with my depth compare, or come up with some other way of doing ray marching ( or no ray march at all ) then this issue may be fixed...

    OR... I am thinking rendering the scene again from the opposite of the camera angle and then use that as pixel information at the back. But this is rendering the scene twice so it could be a crazy idea. hehe.
     
  19. WGermany

    WGermany

    Joined:
    Jun 27, 2013
    Posts:
    78
    Once again nice job! Is the glow around the meshes in the WebPlayer caused by SSR? SSAO? It seems really noticeable but the reflections are still nice! Maybe its just a sampling issue.
     
  20. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    The glow is probably caused by doing blur pass on the reflection.

    Edit:

    I tried to magnify the z value before I do the compare so the difference gets proportionally larger as it goes...
    It has helped to get rid of "some" false reflection but it still makes the pixels at around those border of max z difference to become very noisy.

    When I want to do very tight z compare, some "good" reflection gets filtered out.. I think this contributes to the noisy reflection.

    Still thinking about the ways to improve ray march and z compare..

    Also , I was thinking, maybe non pixel perfect fetch of depth map is causing this?
    I am sure at the moment, depth is sampled at texture filtered method so maybe I am getting some average depth..
    I think it could be the same issue Brn had..

    hummmmm....
     
    Last edited: Oct 24, 2013
  21. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Ok I have some news...

    I tried point sampling for my reflection pass ... and it did indeed got rid of lots of false reflections!

    $ssr06.JPG

    I still got some nasty zagged ztest issue around border of the z compare value, but I dont know if that can be fixed other than applying some heavy blur...

    black areas around the reflection / under the spheres are actually correct. (not the shadow area)
    That area is NOT black.. it is just darker than the area with the reflection. Because I am currently doing additive for applying reflections. so when there is nice black area, it means it is not catching any false reflections. This is known limitation of the technique.

    http://www.castorlee.com/script/shader/SSR/WebPlayer.html
     
    Last edited: Oct 24, 2013
  22. Kirbyrawr

    Kirbyrawr

    Joined:
    Jul 23, 2012
    Posts:
    945
    The other day i was in the park looking at the lake and how the people was reflecting on the water and i was wondering how to do it in a shader or in any form, this is like i saw, so it's very awesome.

    KirbyRawr
     
  23. Lulucifer

    Lulucifer

    Joined:
    Jul 8, 2012
    Posts:
    358
    In fact ,i want to see your work without blur or fadeout at all,just the raw stuff you achived :D
     
  24. superjayman

    superjayman

    Joined:
    May 31, 2013
    Posts:
    185
    What do you mean exactly by point sampling of the reflection pass? cheers
     
  25. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,788
    That's pretty cool, looks better that the previous version. Why not add some anisotropic blur to reduce the ztest issue? and also you can get stretched reflection :)
     
  26. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Rea. Can you explain more? How does adding anisotropic blur will help with ztest issue?
    I think the distance each ray can travel before it runs out the step number is limitation to how far the reflection can catch other surfaces, and that can cause reflections either to look short or long...

    Here is example with 256 steps per ray... :D on my work HD6850 pc...
    And another without blur or fade away. (Raw Stuff)

    $SSR07.jpg $SSR08.jpg
     

    Attached Files:

    Last edited: Oct 25, 2013
  27. CTPEJIOK22

    CTPEJIOK22

    Joined:
    Sep 4, 2013
    Posts:
    85
    looks great, i want this shader)
     
  28. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,788
    Whoops sorry i mean in the final reflection image so the jaggies reflection can be reduced, just a visual addition.

    And i guess you already fixed it here :p.
    Latest version looks great man.
     
  29. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    I am still doing too many ray steps to be really useful so it can be used in the games etc, Down sampling the rendered screen into /2 or /4 helps a lot with the speed, but reflection details gets too pixelated. Maybe I have to compromise it... but my other brain tells me that there must be a better way!

    Running like 16-30 steps per pixel and then having the reflection in the original resolution means the reflection is very short so it is not a good approach for long streak reflections... Maybe I need to set something up so that I can control the step sizes depending on the material? But this means I need to store this value somewhere so it could mean another render texture... or even pass.. humm....
     
  30. superjayman

    superjayman

    Joined:
    May 31, 2013
    Posts:
    185
    How are you computing _ProjectionInv in c# , is it like below

    Also did you do any changes to ReconstructViewPosition code, Im getting a result but the reflection is not upside down, any help will be appreciated..
     
  31. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    I land up not using ReconstructViewPosition code. I could not get that to work.
    Instead, I just used inverse projection matrix. And yes just like P.inverse
     
  32. superjayman

    superjayman

    Joined:
    May 31, 2013
    Posts:
    185
    so, how are you computing the reflection vector or how are you using inverse projection matrix? can you post some relevant code to this please
     
  33. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    I am using inverse projection matrix to get from screen space to view space. (for position) The code is the same as going from view to screen space using projection matrix , but just use the inverse matrix instead.

    All the relevant code is already posted up in the beginning of the thread. Nothing much as changed. The only thing different from the one from the code is that the one from the code is not using post processing so the view position came straight from the vertex shader. The post proces one, you have to calculate view position using the inverse matrix.
     
  34. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Ok, I think I made some improvements.

    First : ray marching algorithm.

    Instead of doing sort of binary search with one go , I separated it into 2 parts. First part is to find any geometry hit ASAP. using lower resolution scan.
    Second pass is just like in the code... use binary search like steps..

    With this, I was able to get slightly more accurate ray march with less steps overall.

    It still creates stepping artifacts if I want to do a long reach reflections. However, I remember from Crysis approach paper that they used some jitter to get rid of the stepping artifacts. I am wondering how this is done...

    Second : I tried making reflection pass with lower resolution render texture, but this caused the reflection to become too pixelated. So I tried another method. That is to scan at lower resolution for the depth scan only. And doing the down scale resolution for blur pass only.

    Web player is updated with the latest version.
     
    Last edited: Oct 26, 2013
  35. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Ok, I made some minor improvement to the shader which makes things go bit faster in certain circumstances.
    For example, amount of reflection value is stored at alpha channel of render buffer so i used to cull the shader where there is no reflection is needed.

    In practice I think there will be mixed of reflective and non reflective surface in the game so I think this is good to have. Cull out the whole shader process when there is no need to do it.

    This applies to the areas of the screen like Sky and places where it is too far away to consider the effect to be culled out.

    All of them can be done by testing the current depth and alpha channel of the render texture.

    Updated the web player with the latest version. (I have removed SSAO and Bloom)

    At the moment, it is only slightly slower than the SSAO from Unity, which I think it is not too bad. But not good enough yet to be called "optimized'

    While I was doing this, I realized that doing reflection is not as simple as "add" some color value on top of the render buffer screen. I think this is not physically correct, and thus beginning to understand why Brn is going for the physical model of lighting... But in order to do this, every shader that one uses must be based on the same physically based lighting shader which is very big task of its own.

    Not sure if I want to go there yet... Sounds like a mega project to me. :D

    $SSR09.jpg

    http://www.castorlee.com/script/shader/SSR/WebPlayer.html
     
    Last edited: Oct 28, 2013
  36. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    When it comes to the reflection, I noticed that things are becoming more and more towards physically correct model for lighting.I guess this can't be avoided as the reflection itself is physical phenomenon and we expect things to look like it naturally does in real life.

    I was simply just doing additive operations to my color buffer to add the reflections, but this obviously is not physically correct. For an instance, if you have very dark black object and in real life, this object could be reflective on its own, (or not) , but the thing is it does indeed shows in the reflective surface as dark object.

    When you just add colors together 1+0 = 1 which is not correct. (because 0 is black object and it won't show anything) This leads me to think in different terms for values shown in the screen. Instead of they are represented as pure RGB value, the final result must be worked out using some kind of energy model. My observation tells me that no matter how bright the source of the reflection is , the reflection itself can't be brighter than the source itself. Also reflection amount is not a amount of RGB value coming in and go, but it should represent some kind of light energy reflecting off the surface.

    I have tried to bring this basic idea into the composition phase of the shader, and it kinda... kinda.. does what I think it should do. But I mean obviously, it is still way not correct to be called "physically correct" reflection... but kinda approximating with what I can work within Unity.

    I think in order to be able to do more physically correct lighting+reflection, you need to replace whole set of default Unity shader and make them work with Unity's deferred shader pipeline too. I have seen from 4.3 video that some people in Unity is working on the physically based lighting model.. I wish that comes soon!

    I feel that my conclusion toward this technique is that :

    1. This is approximation and should only be used to add some dynamic feel to the scene.
    2. Can't rely on this technique for replacing good old cube map reflection. Rather, combination can work better.
    3. In order for the reflection to look more correct, whole lighting base and shaders needs to change too.
    4. Performance wise, It is slightly slower than SSAO in test scene and slightly faster in practice. Because in practice not everything is reflective.

    $SSR10.jpg
     
  37. save

    save

    Joined:
    Nov 21, 2008
    Posts:
    744
    Wonderful insight Castor! I think it looks completely amazing and I really think that this is making Unity come closer to AAA. Thanks for sharing man!
     
  38. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    cheers Save...

    I am wondering what I should do next at the moment. Like I said from my previous postings, that I think I can take this bit further with the following direction.

    1. Learn more about ray marching to make things go faster and increase accuracy , increase quality of the reflection ray marching.
    2. Learn about physically based light model and go from there.

    Both of them are not an easy task for me. But then what is now days... :D
     
  39. Marrrk

    Marrrk

    Joined:
    Mar 21, 2011
    Posts:
    1,032
    Well it looks good for my eye, what should I look for to see the big mistakes/errors?

    I would say, create some nice editor tools to setup everything inside Unity for the normal users.
     
  40. superjayman

    superjayman

    Joined:
    May 31, 2013
    Posts:
    185
    looking good caster. I have a trivial question, still new to all unity shaders, you're using a while loop in your code, cg cannot unroll them, so are you still using a while loop? Is this shader programmed in CG? cheers
     
  41. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    I am not 100% cg expert neither. But I think when you compile into pixel shader 3.0 mode and let it compile to glsl mode then you can do dynamic branching and while loop.

    I ended up using the for loop but it was the same really. use return or break or force the condition to go over to break out of the loop. And yes shader is what we call CG, where vertex and frag shader is used. It's the best when you got to do a post process effect or you don't have to deal with the lights inside Unity.
     
  42. AndrewRH

    AndrewRH

    Joined:
    Jan 24, 2012
    Posts:
    2,806
    MMMMmmmm great work Castor!!
     
  43. Ava42

    Ava42

    Joined:
    Aug 4, 2012
    Posts:
    33
    nice work! i was doing this several years ago, on my pure c++ / directx engine.
    Castor, can u post shader files for download, or just put it into asset store that we could buy it?
     
  44. gridside

    gridside

    Joined:
    Jul 26, 2013
    Posts:
    97
    It looks great castor76!
     
  45. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    Ta, everyone.

    At the moment, I am trying to see how I can do some physically based model lighting with this.
    On my previous project, what I tried to do was use HDRshop and extract diffuse and spec convolution and use cube map for doing all lighting.
    I am thinking about going back to that but this time use the values and model the surface material based on roughness etc..

    Having said that, I think .... Skyshop? is the closest asset store item that comes close to what I am thinking, and it does look great so I think I will nag my boss to get me one on Monday and see if I can combine the techniques togather.

    My worry is that doing post processing using Unity's deferred shader path gives me limited options on how to store material values, so I am not sure if this can work well. Unity's deferred path uses alpha as specularity but what can i do about the roughness? :(

    I can probably do with the forward path but then it would be cheaper to do it non post processing way. Will see...

    Arh, and about releasing this shader, I am willing to do so when things are more usable in productions. And I think it will be free or dirt cheap. I am not going to sell this unless it is really worth it to have it but at the moment, it is not. It has too many limitations such as deferred only, speed, setup etc.

    But if you want to learn things like myself, then the initial code is about 90% the same as far as the reflection pass goes. And other changes I have mentioned throughout the thread.
     
  46. EvilFox

    EvilFox

    Joined:
    Jul 23, 2013
    Posts:
    83
    It looks interesting...
     
  47. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,788
    What is unroll and for loop use for actually?
     
  48. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    @rea.

    When you use loop inside shader , Unity may unroll the loop ( inlines repeated shader instructions ) or actually do dynamic looping.

    We now have bought Skyshop and I have tried to integrate SSR into Skyshop. The result is okish so far, except as I have expected, there is little way I can pass gloss information to my post process shader to control blurness of the reflection.

    So I am not sure if this is possible doing it post processing way... or I may need to do another pass to generate what I need...

    I think I will hop into Skyshop forum and see if the author can help.

    Here is images for comparison. It is only slightly noticeable but I think it is the way it should be.

    $SSR11.jpg $SSR12.jpg
     
  49. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    some more shots you can see SSR around seat..
    $SSR13.jpg $SSR14.jpg
     
    Last edited: Nov 5, 2013
  50. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,467
    That is a beauty... is that a way to get reflections to be done in real time?