Modifying vertex position shader in a Surface shader - Shadow problems

Hello, I have a simple vertex shader which is attached to a built in surface shader. It basically curves the models with the camera distance, however the shadows aren't working.

As you see, I convert the vertex to view space for the calculations, then convert back to local space, as the surface inputs wants it like that. But the result is not correct. It's like if the real vertices aren't the displayed ones...

Anyonw got any ideas?

Code (csharp):

1. ...
2.  
3. #pragma surface surf BlinnPhong vertex:vert addshadow
4.  
5. ....
6.  
7. // Important stuff
8. void vert (inout appdata_full v)
9. {
10.     float4 vPos = mul (UNITY_MATRIX_MV, v.vertex);
11.     float zOff = vPos.z/_Dist;
12.     vPos += _QOffset*zOff*zOff;
13.     v.vertex = mul (vPos, UNITY_MATRIX_IT_MV);
14.     v.texcoord = mul( UNITY_MATRIX_TEXTURE0, v.texcoord );  
15. }
16. // end important stuff
17.  
18.  
19. void surf (Input IN, inout SurfaceOutput o) {
20.     fixed4 tex = tex2D(_MainTex, IN.uv_MainTex);
21.     o.Albedo = tex.rgb * _Color.rgb;
22.     o.Gloss = tex.a;
23.     o.Alpha = tex.a * _Color.a;
24.     o.Specular = _Shininess;
25.     o.Normal = UnpackNormal(tex2D(_BumpMap, IN.uv_BumpMap));
26. }

 

My custom shaders didn't have shadows if I didn't define a fallback to a built in shader, so from that I assumed that the shadow pass needs a specific definition.

In the built in shader collection it seems to be in the VertexLit shader:

Code (csharp):

1. // Pass to render object as a shadow caster
2.     Pass {
3.         Name "ShadowCaster"
4.         Tags { "LightMode" = "ShadowCaster" }
5.        
6.         Fog {Mode Off}
7.         ZWrite On ZTest LEqual Cull Off
8.         Offset 1, 1
9.  
10. CGPROGRAM
11. #pragma vertex vert
12. #pragma fragment frag
13. #pragma multi_compile_shadowcaster
14. #pragma fragmentoption ARB_precision_hint_fastest
15. #include "UnityCG.cginc"
16.  
17. struct v2f {
18.     V2F_SHADOW_CASTER;
19. };
20.  
21. v2f vert( appdata_base v )
22. {
23.     v2f o;
24.     TRANSFER_SHADOW_CASTER(o)
25.     return o;
26. }
27.  
28. float4 frag( v2f i ) : COLOR
29. {
30.     SHADOW_CASTER_FRAGMENT(i)
31. }
32. ENDCG
33.  
34.     }
35.    
36.     // Pass to render object as a shadow collector
37.     Pass {
38.         Name "ShadowCollector"
39.         Tags { "LightMode" = "ShadowCollector" }
40.        
41.         Fog {Mode Off}
42.         ZWrite On ZTest LEqual
43.  
44. CGPROGRAM
45. #pragma vertex vert
46. #pragma fragment frag
47. #pragma fragmentoption ARB_precision_hint_fastest
48. #pragma multi_compile_shadowcollector
49.  
50. #define SHADOW_COLLECTOR_PASS
51. #include "UnityCG.cginc"
52.  
53. struct appdata {
54.     float4 vertex : POSITION;
55. };
56.  
57. struct v2f {
58.     V2F_SHADOW_COLLECTOR;
59. };
60.  
61. v2f vert (appdata v)
62. {
63.     v2f o;
64.     TRANSFER_SHADOW_COLLECTOR(o)
65.     return o;
66. }
67.  
68. fixed4 frag (v2f i) : COLOR
69. {
70.     SHADOW_COLLECTOR_FRAGMENT(i)
71. }
72. ENDCG
73.  
74.     }
75. }

So I guess you'd have to add these shadow caster parts to your shader and then make sure the vertex shaders of the shadow passes are also properly modified.

 

I'm just starting with shaders so I didn't quite understand your answer.

I'm using as base shader the bumped specular (it's a surface shader), which as far as I know it supports shadows, then I'm adding the vertex part showed in the first post. What exactly do I have to add?

Do I have to implement my own fragment shaders? Is it posible to re-use the already done surface shaders?

 

Essentially, the surface shader only accounts for the way the geometry is rendered on the screen.

There are two other passes that are required to cast/receive shadows (a shadow caster pass and a shadow receiver pass).

So, via the surface shader, you're currently only editing the positions of the vertices as they're rendered to screen - the shadow caster/receiver passes don't know you're altering the positions of the vertices and so, as far as shadows are concerned, your mesh hasn't changed.

Meaning you'll need to add in those passes yourself (the code that jvo3dc posted) at the end of your shader and ensure that you do the same transforms to the vertices inside their vertex shaders as you do inside your surface shader.

 

I was just looking at the #pragma list for surface shaders for other reasons and I noticed this:

"addshadow - Add shadow caster collector passes. Commonly used with custom vertex modification, so that shadow casting also gets any procedural vertex animation. "

So a #pragma addshadow might make things easier.

 

Though note that while rendering shadow maps, the view matrix does not convert to camera space, but to light space. I guess you'd need to pass the correct camera view matrix by yourself.

 

Adding addshadow doesn't work, I think it's needed to add the shadow passes, having some trouble tho. I don't understant how can I add my vertex shader in this type of pass:

Code (csharp):

1. Pass {
2.          Name "ShadowCollector"
3.          Tags { "LightMode" = "ShadowCollector" }
4.          Fog {Mode Off}
5.          ZWrite On ZTest LEqual

For example, this is what I have:

Code (csharp):

1. Shader "Curved/Mobile/Diffuse" {
2. Properties {
3.     _MainTex ("Base (RGB)", 2D) = "white" {}
4.     _QOffset ("Offset", Vector) = (0,0,0,0)
5.     _Dist ("Distance", Float) = 100.0
6. }
7. SubShader {
8.     Tags { "RenderType"="Opaque" }
9.     LOD 150
10.  
11. CGPROGRAM
12. #pragma surface surf Lambert noforwardadd vertex:vert
13. #pragma addshadow
14.  
15. sampler2D _MainTex;
16. float4 _QOffset;
17. float _Dist;
18.  
19. struct Input {
20.     float2 uv_MainTex;
21. };
22.  
23. void vert (inout appdata_full v)
24. {
25.     float4 vPos = mul (UNITY_MATRIX_MV, v.vertex);
26.     float zOff = vPos.z/_Dist;
27.     vPos += _QOffset*zOff*zOff;
28.     v.vertex = mul (vPos, UNITY_MATRIX_IT_MV);
29.     v.texcoord = mul( UNITY_MATRIX_TEXTURE0, v.texcoord );  
30. }
31.  
32. void surf (Input IN, inout SurfaceOutput o) {
33.     fixed4 c = tex2D(_MainTex, IN.uv_MainTex);
34.     o.Albedo = c.rgb;
35.     o.Alpha = c.a;
36. }
37. ENDCG
38. }
39.  
40. Fallback "Mobile/VertexLit"
41. }
42.  

What needs to be done to add shadows.

 

Just did a simple test here. The addshadow pragma didn't work here either, but I was able to modify the vertex positions also for the shadow.

I didn't test the code below, but this should be the thing to add:

Code (csharp):

1.  
2. // Pass to render object as a shadow caster
3. Pass {
4.     Name "ShadowCaster"
5.     Tags { "LightMode" = "ShadowCaster" }
6.            
7.     Fog {Mode Off}
8.     ZWrite On ZTest LEqual Cull Off
9.     Offset 1, 1
10.      
11.     CGPROGRAM
12.     #pragma vertex vert
13.     #pragma fragment frag
14.     #pragma multi_compile_shadowcaster
15.     #pragma fragmentoption ARB_precision_hint_fastest
16.     #include "UnityCG.cginc"
17.  
18.     float4 _QOffset;
19.     float _Dist;
20.              
21.     struct v2f {
22.         V2F_SHADOW_CASTER;
23.     };
24.              
25.     v2f vert( appdata_base v ) {
26.         float4 vPos = mul (UNITY_MATRIX_MV, v.vertex);
27.         float zOff = vPos.z/_Dist;
28.         vPos += _QOffset*zOff*zOff;
29.         v.vertex = mul (vPos, UNITY_MATRIX_IT_MV);
30.         v2f o;
31.         TRANSFER_SHADOW_CASTER(o)
32.         return o;
33.     }
34.              
35.     float4 frag( v2f i ) : COLOR {
36.         SHADOW_CASTER_FRAGMENT(i)
37.     }
38.     ENDCG
39. }
40.        
41. // Pass to render object as a shadow collector
42. Pass {
43.     Name "ShadowCollector"
44.     Tags { "LightMode" = "ShadowCollector" }
45.            
46.     Fog {Mode Off}
47.     ZWrite On ZTest LEqual
48.      
49.     CGPROGRAM
50.     #pragma vertex vert
51.     #pragma fragment frag
52.     #pragma fragmentoption ARB_precision_hint_fastest
53.     #pragma multi_compile_shadowcollector
54.              
55.     #define SHADOW_COLLECTOR_PASS
56.     #include "UnityCG.cginc"
57.  
58.     float4 _QOffset;
59.     float _Dist;
60.  
61.     struct appdata {
62.         float4 vertex : POSITION;
63.     };
64.              
65.     struct v2f {
66.         V2F_SHADOW_COLLECTOR;
67.     };
68.              
69.     v2f vert (appdata v) {
70.         float4 vPos = mul (UNITY_MATRIX_MV, v.vertex);
71.         float zOff = vPos.z/_Dist;
72.         vPos += _QOffset*zOff*zOff;
73.         v.vertex = mul (vPos, UNITY_MATRIX_IT_MV);
74.         v2f o;
75.         TRANSFER_SHADOW_COLLECTOR(o)
76.         return o;
77.     }
78.              
79.     fixed4 frag (v2f i) : COLOR {
80.         SHADOW_COLLECTOR_FRAGMENT(i)
81.     }
82.     ENDCG
83. }
84.  

 

Where is the surface funcion? If there was a surface function, from the built-in shaders and I wanted to add my vertex modification how should it look like?

 

The surface function is in your part. You just have to add these passes to add shadow based on your vertex position modification.

 

I really don't understand, what's the order of the passes? Does the surface shader go between the two passes? Can you post an example of the simplest surface shader with vertex modification? It would really help.

 

hi there,

have a look at the built in "tree creator leaves optimized" shader for example to find out how you add the additional shadow collector and shadow caster passes.

unity 4.2 introduced batching for the shadow passes which i find pretty difficult to handle when doing your own vertex animation.
so the best is to do all vertex animation in worldspace.

but things are even more complex: next to the shadow passes you also have to modify the shaders that calculate the depth texture. otherwise deferred lighting and image effects like ssao will not work (so perhaps you do not have to do so if you use unity free).

lars

 

Your original code would be at the top and you just add the shadow passes under it.

 

Hi jvo3ds, I find your sujjestion very usefull. I'm working on a shader witch is having the same problem. I follow your instruction, using the code you posted online, but I don't know why is not working.
May I ask you some help?

this is the shader I am using:

Code (CSharp):

1. Shader "Curved Diffuse" {
2.  
3. Properties {
4.  
5.     _MainTex ("Base (RGB)", 2D) = "white" {}
6.      _Color ("Main Color", Color) = (1,0.5,0.5,1)
7. }
8.  
9. SubShader {
10.  
11.     Tags { "RenderType"="Opaque" }
12.  
13.     LOD 150
14.  
15.     CGPROGRAM
16.  
17.     #pragma exclude_renderers flash
18.     #pragma surface surf Lambert noforwardadd
19.     #pragma vertex vert addshadow
20.  
21.     sampler2D _MainTex;
22.     float4 _QOffset;
23.     float _Dist;
24.     float4 _Color;
25.  
26.     struct Input {
27.     float2 uv_MainTex;
28.     };
29.  
30.     void vert (inout appdata_full v) {
31.  
32.         float4 vertex_view = mul(UNITY_MATRIX_MV, v.vertex);
33.         float zOff = vertex_view.z / _Dist;
34.         v.vertex.xyz += mul(_QOffset*zOff*zOff, UNITY_MATRIX_IT_MV).xyz;
35.  
36.     }
37.  
38.     void surf (Input IN, inout SurfaceOutput o) {
39.  
40.         fixed4 c = tex2D(_MainTex, IN.uv_MainTex);
41.         o.Albedo =c.rgb;
42.         o.Albedo *= _Color.rgb;
43.         o.Alpha = c.a;
44.  
45.     }
46.  
47.     ENDCG
48. }
49.  
50. }

Floowing your instruction and everithing that I found about the argoment I produced this modified version of the script. I don't know why but is not working. I ask also on Unity Answer but no one by now helped me:

Code (CSharp):

1. Shader "Curved Diffuse Mod shadows" {
2.  
3. Properties {
4.  
5.     _MainTex ("Base (RGB)", 2D) = "white" {}
6.      _Color ("Main Color", Color) = (1,0.5,0.5,1)
7. }
8.  
9. SubShader {
10.  
11.        Tags { "RenderType"="Opaque"  }
12.  
13.        LOD 150
14.  
15.        CGPROGRAM
16.  
17.        #pragma exclude_renderers flash
18.        #pragma surface surf Lambert
19.        #pragma vertex vert  addshadow
20.  
21.        sampler2D _MainTex;
22.        float4 _QOffset;
23.        float _Dist;
24.        float4 _Color;
25.      
26.        struct Input {
27.            float2 uv_MainTex;
28.        };
29.  
30.        void vert (inout appdata_full v) {
31.  
32.            float4 vertex_view = mul(UNITY_MATRIX_MV, v.vertex);
33.            float zOff = vertex_view.z / _Dist;
34.            v.vertex.xyz += mul(_QOffset*zOff*zOff, UNITY_MATRIX_IT_MV).xyz;
35.  
36.        }
37.  
38.        void surf (Input IN, inout SurfaceOutput o) {
39.  
40.         fixed4 c = tex2D(_MainTex, IN.uv_MainTex);
41.         o.Albedo =c.rgb;
42.         o.Albedo *= _Color.rgb;
43.         o.Alpha = c.a;
44.  
45.     }
46.  
47.        ENDCG
48.  
49. // Pass to render object as a shadow collector
50. Pass {
51.     Name "ShadowCollector"
52.     Tags { "LightMode" = "ShadowCollector" }
53.        
54.     Fog {Mode Off}
55.     ZWrite On ZTest LEqual
56.  
57.     CGPROGRAM
58.     #pragma vertex vert
59.     #pragma fragment frag
60.     #pragma fragmentoption ARB_precision_hint_fastest
61.     #pragma multi_compile_shadowcollector
62.          
63.     #define SHADOW_COLLECTOR_PASS
64.     #include "UnityCG.cginc"
65.  
66.     float4 _QOffset;
67.     float _Dist;
68.  
69.     struct appdata {
70.         float4 vertex : POSITION;
71.     };
72.          
73.     struct v2f {
74.         V2F_SHADOW_COLLECTOR;
75.     };
76.          
77.     v2f vert (appdata v) {
78.         float4 vPos = mul (UNITY_MATRIX_MV, v.vertex);
79.         float zOff = vPos.z/_Dist;
80.         v.vertex.xyz += mul(_QOffset*zOff*zOff, UNITY_MATRIX_IT_MV).xyz;
81.    
82.         v2f o;
83.         TRANSFER_SHADOW_COLLECTOR(o);
84.         return o;
85.     }
86.          
87.     fixed4 frag (v2f i) : COLOR {
88.         SHADOW_COLLECTOR_FRAGMENT(i);
89.     }
90.     ENDCG
91. }          
92.  
93. // Pass to render object as a shadow caster
94. Pass {
95.     Name "ShadowCaster"
96.     Tags { "LightMode" = "ShadowCaster" }
97.        
98.     Fog {Mode Off}
99.     ZWrite On ZTest LEqual Cull Off
100.     //Offset 1, 1
101.  
102.     CGPROGRAM
103.     #pragma vertex vert
104.     #pragma fragment frag
105.     #pragma multi_compile_shadowcaster
106.     #pragma fragmentoption ARB_precision_hint_fastest
107.     #include "UnityCG.cginc"
108.  
109.     float4 _QOffset;
110.     float _Dist;
111.          
112.     struct v2f {
113.         V2F_SHADOW_CASTER;
114.    
115.         //V2F_SHADOW_CASTER is this structure:
116.         //float4 pos : SV_POSITION; float3 vec : TEXCOORD0
117.     };
118.          
119.     v2f vert( appdata_full v ) {
120.         float4 vPos = mul (UNITY_MATRIX_MV, v.vertex);
121.         float zOff = vPos.z/_Dist;
122.         v.vertex.xyz += mul(_QOffset*zOff*zOff, UNITY_MATRIX_IT_MV).xyz;
123.    
124.         v2f o;
125.         TRANSFER_SHADOW_CASTER(o);
126.         return o;
127.     }
128.          
129.     float4 frag( v2f i ) : COLOR {
130.         SHADOW_CASTER_FRAGMENT(i);
131.     }
132.     ENDCG
133. }  
134. }
135. }

This is the post on Unity Answer:
http://answers.unity3d.com/questions/936982/custom-shader-curved-diffuse-lighting-not-working.html

Thanks a lot to everyone will help.

 

Hello everyone. I know this is an old thread. I figured I'd ressurrect it because I've had exactly the same issue and I figured out how to fix it, incase anyone else has this issue in the future and looks up this thread like I had done. This was really confusing and I saw a lot of people online struggling with it, and couldn't find any satisfactory answers.

The problem is the UNITY_MATRIX_MV property changes during the shadow rendering pass. It doesn't retrieve the MV matrix for the camera, it retrieves the MV matrix for whereever the shader is currently rendering, and when shadows are created, it does a render based on the position of the lightsource. The surface shader doesn't use the new vertex locations calculated in your vertex shader, it instead applies the vertex shader function to the shadow rendering during the shadow pass, therefore the shadow's vertexes applies UNITY_MATRIX_MV transform based on the perspective of the lightsource, not the camera, so shadows are drawn at incorrect locations.

The solution to this problem is to never use UNITY_MATRIX properties in a vertex shader to modify vertex locations if you plan on rendering shadows. Instead, create a new global matrix in the shader using Shader.SetGlobalMatrix() that contains the world to camera transform every time the camera transform is changed:

Code (csharp):

1.  Shader.SetGlobalMatrix("_World2Camera", _camera.worldToCameraMatrix);

And then inside your shader:

Code (csharp):

1.  
2. float4x4 _World2Camera;
3. void vert(inout appdata_full v)
4. {
5.     float4 vertex_view = mul(_World2Camera, mul(_Object2World, v.vertex));
6.     ...
7. }

That's all there is to it. No need to write your own shadow passes or anything like that. The surface shader does that itself when you add the "addshadow" #pragma. I hope this helps anyone.

EDIT: It seems like someone else had the same idea years ago in this thread and nobody noticed (including me).