Screen Space Local Reflection

It is. But since SSR is screen space based method, it can't do 100% reflection about everything. Obviously , you can't reflect things you can't see in the screen, and there are other limitations. What I was hoping for is that SSR can adds to what Skyshop does already and give that little bit of "dynamic" feeling when things move around animate etc. I am hoping to blend best of two together to see if I can produce decent effect that looks good enough while giving illusion of the dynamic reflection.

I think Killzone and other games uses this kind of method too. They just do them really fast and good. Doing it in Unity is limited because it is really not made to do this kind of things well.

 

I did see this for Shadow Fall and it looks kick ass... and I like the previous renders you showed earlier too, especially the ground reflections.... very impressive work.

 

Great work Castor76!!

 

would it work on mobile devices?

 

@ dev

I would not think so. Even if I can make it work on Mobile, this kind of effect is just not feasible to run in the mobile.. Not just yet. I know realtime GI and stuffs like that are coming along, so I wouldn't be surprised if working example of this appears on mobile. Having said that I think it "could" work for OpenGLES 3.0 just very slowly.

 

*standing ovation*

 

Very interested in this. No reflections on brand new iMac with nVidia GPU.

 

so i tried to implement the shader on the first page as post process aka ImageEffect, but it only painted white borders around my objects. i couldnt fix it so i tried writing my own Shader. im only using the DepthNormal and Main textures, so im trying to calculate the z component of the surface normal, but its messed up after reflecting ( almost always negative)

any help appreciated

Code (csharp):

1. Shader "SSLR" {
2. Properties {
3.  _MainTex ("", 2D) = "white" {}
4. }
5.  
6. SubShader {
7.  
8. ZTest Always Cull Off ZWrite Off Fog { Mode Off } //Rendering settings
9.  
10.  Pass{
11.   CGPROGRAM
12.   #pragma vertex vert
13.   #pragma fragment frag
14.   #include "UnityCG.cginc"
15.   #pragma target 3.0
16.   #pragma glsl
17.  
18.   sampler2D _MainTex;
19.   sampler2D _CameraDepthNormalsTexture;
20.      
21.   struct v2f {
22.    float4 pos : POSITION;
23.    half2 uv : TEXCOORD0;
24.   };
25.  
26.   v2f vert (appdata_img v){
27.    v2f o;
28.    o.pos = mul (UNITY_MATRIX_MVP, v.vertex);
29.    o.uv = MultiplyUV (UNITY_MATRIX_TEXTURE0, v.texcoord.xy);
30.    return o;
31.   }
32.    
33.   half4 frag (v2f i) : COLOR{
34.    //Depthnormal texture, normals in xy, depth in zw
35.    float4 depthN = tex2Dlod(_CameraDepthNormalsTexture,float4(i.uv,0,0));
36.    
37.    //camera Vector in SS, z axis is positive in cam direction
38.    float3 ViewVec = float3(0.0,0.0,1.0);
39.    
40.    //Surface Normal Vector in SS, convert range from 0to1 to -1to1
41.    float3 SSSurfN = float3(depthN.xy*2-1, 0);
42.    //calculate missing z componenet with length = 1.0 = sqrt(x²+y²+z²). its negative because pointing towards cam
43.    SSSurfN.z = -sqrt(1 - SSSurfN.x*SSSurfN.x - SSSurfN.y*SSSurfN.y);
44.    
45.    //surface point in SS
46.    float3 SSPos = float3(i.uv, DecodeFloatRG(depthN.zw));
47.    
48.    //Reflect camera Vector on Surface
49.    float3 Reflected = reflect(ViewVec, SSSurfN);
50.    
51.    //raytrace
52.    int t = 1;
53.    int tMax = 100;
54.    float travelDist = 0.0;
55.    
56.    while(t < tMax){
57.     //add ray vector to current position
58.     float3 newSSPos = SSPos + Reflected/1000;
59.     //get new depth to compare
60.    float4 newDepthN = tex2Dlod (_CameraDepthNormalsTexture, float4(newSSPos.xy,0,0));
61.     float newDepth = DecodeFloatRG(newDepthN.zw);
62.     //compare depths    
63.     if(newSSPos.z < newDepth){//newSSPos.z <= newDepth){
64.      SSPos = newSSPos;
65.     }
66.     t += 1;
67.    }
68.    
69.    //initially rendered Color
70.    fixed4 renCol = tex2D(_MainTex, i.uv);
71.    
72.    //disp Surface Normal
73.    //renCol = float4(SSSurfN.xy/2+.5,SSSurfN.z/2+.5,1);
74.    
75.    //disp Reflected Normal
76.    //renCol = float4(0,0,Reflected.z/2+.5,1);
77.    
78.    //disp Position vec
79.    //renCol = float4(float3(Reflected.z/2+.5), 1);
80.    
81.    //grab reflected Color
82.    float4 reflCol = tex2D(_MainTex, SSPos.xy);
83.    
84.    //assemble
85.    return renCol + reflCol*.3;
86.   }
87.   ENDCG
88.  }
89. }
90.  FallBack "Diffuse"
91. }
92.  

 

Umm...ok

1. Normal from DepthNormal is view space normal. You are working everything in Screen space so there is obvious mismatch.
2. ViewVector is not always 1 facing towards Z axis... viewvector actually mean vector from camera position to the pixel position in your screen . So it should be like a .. 2d vector from center of the screen to the pixels on the screen.

If you want to work just in screen space, then make sure everything is in screen space. convert normal from depthnormal into screen space first.

My approach was use view normal given either by DepthNormal or normal texture from deferred render path and then with view space "view vector" , get the reflection vector from those two and then convert them into screen space.

I guess you could try reflecting off the screen space vectors... I am not sure if it will work or not.. but my guess is that ould work.

I could be wrong on all this... hehe it's been some time.

 

LOL.. OK Mac. Darn. Only if I have it to test it...

Only platform I have tested this with was Windows and WiiU.... (YES WiiU lol)

 

1.yeah thats true, my fault. how did you end up transforming view to screen space?
can i simply multiply it with the proj matrix? that sounds too easy
2.yeah with ViewVector i meant the ray from the pixel to the object it hits which then would be exactly the z-axis in ss. or am i wrong?

oh the web player is working today, looks really nice, especially the self-reflections on the space ship thing
do you do a normal check to hide "backfaces" in reflections or is this because of your bias check
and you should consider continuing the tracing if your bias doesnt fit, i think you could get some more reflections like that

do you consider uploading a newer script version? and do you have an idea why your shader doesnt work for me?ive set the depthtexturemode to DepthNormals and i can acess all the textures in the shader

 

my view to screen space transform is just like exactly shown in the code :

float4 vspPosReflectT = mul (_ProjMatrix, viewPos +float4(vspReflect,1));
float3 vspPosReflect = vspPosReflectT.xyz / vspPosReflectT.w;

_ProjMatrix is needed (passed from c# script.) because in post process, UNITY_MATRIX_P is identity matrix.. If you are not doing it in post process you can just use UNITY_MATRIX_P

No... I think screen space view vector is not always (0,0,1) the origin of the view vector will always be centered at (0,0,0) so it for very top left corner pixel's view vector would be like (-1,1,z) it would also be better if you normalize it..it depends..

You are also not decoding the view normal value from _CameraDepthNormalsTexture ... I think it is encoded value so you can't just get it and use it as *2-1 ... decode it first and then do that.

 

i cant get the passing of the ProjMatrix done, i tried both Shader.SetGlobalMatrix("_ProjMatrix", P); and m_SSLRMaterial.SetMatrix("_ProjMatrix", P); (before applying the shader after creating the material), but in both cases the normals get messed up pretty bad.

How can this be done not as post process, where do the normal and depth Information then Comes from?

Anyway, can you post the c# code? i seem to be too stupid for this

 

Have you taken a look at Lux to see if you could integrate it into its PBS pipeline? http://forum.unity3d.com/threads/221985-Lux-%96-an-open-source-physically-based-shading-framework

 

Interesting...

I did test this with Skyshop framework and it did work pretty well. But at the moment, I need to set up the camera a bit and it doesn't work for forward rendering if post processing is used. If I don't use post processing then you have to use some specific shader in order for it to work, so it is not very flexible yet to be integrated into some official frame work. At least not just yet.

I am planning to release this sometime, but I want to do it through Asset Store. (for free) but in order for it to get there, I need to work on this a bit more and need to get asset store access first.

But good news is that today is my official last day at my work. After been in the games industry for almost 15 years , I am heading off to the new world as an independent developer working for myself so I should be able to get this thing moved soon. (so I hope)

 

if you do not use post process then you have to calculate view normal and depth yourself. Or you can still get it through depthNormal texture.

if you enable depthNormal texture in camera, Unity will generate it by redrawing the whole scene again. So it doesn't matter if you use it or make your own.

But when you do not use post process, you have to rerender the scene using replacement shader. The replacement shader is your reflection pass and you can blend on top of the screen to composite it back. This ofcourse means more drawcalls.

For the shader code see reason I posted before, and for c# code to get matrix I have already posted it.

Just don't give up... it also took me like 3-4 weeks to figure out all this and I can't say I am through too...

 

Is there going to be a code release for this?

 

when i'm going fullscreen (accidently) the space ship becoming white...

 

so my shader starts to make the right things, thx to your help, but i still cant get the matrix thing working.
im using

in the cs script and then

in the shader but it aint working....

Edit: im pretty sure the matrix is the problem, without using it, the shader is working in the center of the screen, an the reflections are bend

by the way, i did the raytracing now a bit different. instead of stopping when hitting an object, i continue tracing for a surface that is hit more precisely.

like

you should get far better results with this when you look through fences or have pillars standing around like in your web demo.
Edit: Maybe i should only continue tracing when diff is bigger than a target bias...

 

castor this looks fantastic. Mad props