Lighting using tangent vector and also using surface shaders

Hi,

I am attempting to write a custom lighting shader that uses the tangent vector with only limited success. I want to write a shader that can access the tangent vector for per-pixel lighting purposes, and I want it to be able to work with any lights in the scene, including the situation where multiple lights affect the same object.

The options that I am aware of for writing shaders are:
1) custom vertex and fragment shaders
2) use a surface shader. This can optionally include a custom function in the vertex shader to pass interpolants to the pixel shader.

However, each of these methods, as far as I can tell, has a drawback that prevents me from accomplishing my goals. I have experimented with both, and these are the roadblocks that I am hitting.

Method 1 (custom vertex and fragment shaders)
I can get the tangent vector in the space of my choice, along with a single light in the space of my choice. However, there does not appear to be a way to access multiple lights. So as an object moves in the scene, the lighting can suddenly pop as the dominant light that the engine chooses changes.

Method 2 (surface shaders)
This method supports multiple lights. And I can pass along a tangent vector in the space of my choice. However, the shader appears to change the space that lighting (and view) vectors are in, rendering it impossible to hard code a tangent vector that will be in the same space. As a sanity check, I wrote a lighting function that simply outputs the lightDir as the color. If I move an object with this shader applied to it around the editor scene, then I see flickering. I presume that this is because the choice of space in which to represent the vectors continually changes.

What can I do to accomplish my goals? And if there is no current way, is there anything in the roadmap?

Thanks,
Dan

 

If you're using method 1, you have to write two passes, one for forward base, one for forward add.

Otherwise it will only accept one dominant directional light.

 

That's not really true... you can shade up to 4 lights in the first pass only. Observe this code:

Code (csharp):

1. float3 ShadeVertexLights (float4 vertex, float3 normal)
2. {
3.     float3 viewpos = mul (UNITY_MATRIX_MV, vertex).xyz;
4.     float3 viewN = mul ((float3x3)UNITY_MATRIX_IT_MV, normal);
5.     float3 lightColor = UNITY_LIGHTMODEL_AMBIENT.xyz;
6.     for (int i = 0; i < 4; i++) {
7.         float3 toLight = unity_LightPosition[i].xyz - viewpos.xyz * unity_LightPosition[i].w;
8.         float lengthSq = dot(toLight, toLight);
9.         float atten = 1.0 / (1.0 + lengthSq * unity_LightAtten[i].z);
10.         float diff = max (0, dot (viewN, normalize(toLight)));
11.         lightColor += unity_LightColor[i].rgb * (diff * atten);
12.     }
13.     return lightColor;
14. }

Nothing prevents you from using this data for per-pixel lighting. The only problem is that the cost will be fixed no matter how many lights you have around you.

 

Thanks for the replies, guys. Is there any documentation on how to write a forward-add pass? I would definitely prefer this method for performance reasons, as well as for automatically handling shadows (and possibly other light sources such as shadow maps and spherical harmonics).

Dolkar, it appears from your code snippet that unity_LightPosition is in view space. Is that correct?

 

By default, you won't get shadows on forward add passes anyway (you can force it in surface shaders with the fullforwardshadows argument, but I've not worked out how to force it in standard vert/frag shaders).

This is a hastily stripped down version of another shader, so it might not work straight up, but I've added comments at the important bits for forward rendering and lighting info.

Code (csharp):

1. Shader "ForwardRendering" {
2.     Properties {
3.         _Color ("Main Color", Color) = (1,1,1,1)
4.         _SpecColor ("Specular Color", Color) = (0.5, 0.5, 0.5, 1)
5.         _Shininess ("Shininess", Range (0.03, 1)) = 0.078125
6.         _MainTex ("Base (RGB) Alpha (A)", 2D) = "white" {}
7.         _BumpMap ("Normalmap", 2D) = "bump" {}
8.     }
9.     SubShader {
10.         Tags {"Queue" = "Geometry" "RenderType" = "Opaque"}
11.  
12.         Pass {
13.             Tags {"LightMode" = "ForwardBase"}                      // This Pass tag is important or Unity may not give it the correct light information.
14.             CGPROGRAM
15.                 #pragma vertex vert
16.                 #pragma fragment frag
17.                 #pragma multi_compile_fwdbase                       // This line tells Unity to compile this pass for forward base.
18.                 #pragma fragmentoption ARB_fog_exp2
19.                 #pragma fragmentoption ARB_precision_hint_fastest
20.                 #pragma target 3.0
21.                
22.                 #include "UnityCG.cginc"
23.                 #include "AutoLight.cginc"
24.                
25.                 struct v2f
26.                 {
27.                     float4  pos         : SV_POSITION;
28.                     float2  uv          : TEXCOORD0;
29.                     float3  viewDir     : TEXCOORD1;
30.                     float3  lightDir    : TEXCOORD2;
31.                     LIGHTING_COORDS(3,4)                            // Macro to send shadow  attenuation to the vertex shader.
32.                 };
33.  
34.                 v2f vert (appdata_tan v)
35.                 {
36.                     v2f o;
37.                    
38.                     o.pos = mul( UNITY_MATRIX_MVP, v.vertex);
39.                     o.uv = v.texcoord.xy;
40.                     TANGENT_SPACE_ROTATION;                         // Macro for unity to build the Object>Tangent rotation matrix "rotation".
41.                     o.viewDir = mul(rotation, ObjSpaceViewDir(v.vertex));
42.                     o.lightDir = mul(rotation, ObjSpaceLightDir(v.vertex));
43.  
44.                     TRANSFER_VERTEX_TO_FRAGMENT(o);                 // Macro to send shadow  attenuation to the fragment shader.
45.                     return o;
46.                 }
47.  
48.                 sampler2D _MainTex;
49.                 sampler2D _BumpMap;
50.                 fixed4 _Color;
51.                 half _Shininess;
52.  
53.                 fixed4 _SpecColor;
54.                 fixed4 _LightColor0; // Colour of the light used in this pass.
55.  
56.                 fixed4 frag(v2f i) : COLOR
57.                 {
58.                     i.viewDir = normalize(i.viewDir);
59.                     i.lightDir = normalize(i.lightDir);
60.                    
61.                     fixed atten = LIGHT_ATTENUATION(i); // Macro to get you the combined shadow  attenuation value.
62.  
63.                     fixed4 tex = tex2D(_MainTex, i.uv);
64.                     fixed gloss = tex.a;
65.                     tex *= _Color;
66.                     fixed3 normal = UnpackNormal(tex2D(_BumpMap, i.uv));
67.  
68.                     half3 h = normalize(i.lightDir + i.viewDir);
69.                    
70.                     fixed diff = saturate(dot(normal, i.lightDir));
71.                    
72.                     float nh = saturate(dot (normal, h));
73.                     float spec = pow(nh, _Shininess * 128.0) * gloss;
74.                    
75.                     fixed4 c;
76.                     c.rgb = UNITY_LIGHTMODEL_AMBIENT.rgb * 2 * tex.rgb;         // Ambient term. Only do this in Forward Base. It only needs calculating once.
77.                     c.rgb += (tex.rgb * _LightColor0.rgb * diff + _LightColor0.rgb * _SpecColor.rgb * spec) * (atten * 2); // Diffuse and specular.
78.                     c.a = tex.a + _LightColor0.a * _SpecColor.a * spec * atten;
79.                     return c;
80.                 }
81.             ENDCG
82.         }
83.  
84.         Pass {
85.             Tags {"LightMode" = "ForwardAdd"}                       // Again, this pass tag is important otherwise Unity may not give the correct light information.
86.             Blend One One                                           // Additively blend this pass with the previous one(s). This pass gets run once per pixel light.
87.             CGPROGRAM
88.                 #pragma vertex vert
89.                 #pragma fragment frag
90.                 #pragma multi_compile_fwdadd                        // This line tells Unity to compile this pass for forward add, giving attenuation information for the light.
91.                 //#pragma multi_compile_fwdadd_fullshadows                      // This line tells Unity to compile this pass for forward add and give shadow information as well as attenuation. Swap this line for the one above if you want forward add with shadows.
92.                 #pragma fragmentoption ARB_fog_exp2
93.                 #pragma fragmentoption ARB_precision_hint_fastest
94.                 #pragma target 3.0
95.                
96.                 #include "UnityCG.cginc"
97.                 #include "AutoLight.cginc"
98.                
99.                 struct v2f
100.                 {
101.                     float4  pos         : SV_POSITION;
102.                     float2  uv          : TEXCOORD0;
103.                     float3  viewDir     : TEXCOORD1;
104.                     float3  lightDir    : TEXCOORD2;
105.                     LIGHTING_COORDS(3,4)                            // Macro to send shadow  attenuation to the vertex shader.
106.                 };
107.  
108.                 v2f vert (appdata_tan v)
109.                 {
110.                     v2f o;
111.                    
112.                     o.pos = mul( UNITY_MATRIX_MVP, v.vertex);
113.                     o.uv = v.texcoord.xy;
114.                     TANGENT_SPACE_ROTATION;                         // Macro for unity to build the Object>Tangent rotation matrix "rotation".
115.                     o.viewDir = mul(rotation, ObjSpaceViewDir(v.vertex));
116.                     o.lightDir = mul(rotation, ObjSpaceLightDir(v.vertex));
117.  
118.                     TRANSFER_VERTEX_TO_FRAGMENT(o);                 // Macro to send shadow  attenuation to the fragment shader.
119.                     return o;
120.                 }
121.  
122.                 sampler2D _MainTex;
123.                 sampler2D _BumpMap;
124.                 fixed4 _Color;
125.                 half _Shininess;
126.  
127.                 fixed4 _SpecColor;
128.                 fixed4 _LightColor0; // Colour of the light used in this pass.
129.  
130.                 fixed4 frag(v2f i) : COLOR
131.                 {
132.                     i.viewDir = normalize(i.viewDir);
133.                     i.lightDir = normalize(i.lightDir);
134.                    
135.                     fixed atten = LIGHT_ATTENUATION(i); // Macro to get you the combined shadow  attenuation value.
136.  
137.                     fixed4 tex = tex2D(_MainTex, i.uv);
138.                     fixed gloss = tex.a;
139.                     tex *= _Color;
140.                     fixed3 normal = UnpackNormal(tex2D(_BumpMap, i.uv));
141.  
142.                     half3 h = normalize(i.lightDir + i.viewDir);
143.                    
144.                     fixed diff = saturate(dot(normal, i.lightDir));
145.                    
146.                     float nh = saturate(dot (normal, h));
147.                     float spec = pow(nh, _Shininess * 128.0) * gloss;
148.                    
149.                     fixed4 c;
150.                     c.rgb = (tex.rgb * _LightColor0.rgb * diff + _LightColor0.rgb * _SpecColor.rgb * spec) * (atten * 2); // Diffuse and specular.
151.                     c.a = tex.a + _LightColor0.a * _SpecColor.a * spec * atten;
152.                     return c;
153.                 }
154.             ENDCG
155.         }
156.     }
157.     FallBack "VertexLit"    // Use VertexLit's shadow caster/receiver passes.
158. }

 

The code is from UnityCG.cginc, but yes, they appear to be in view-space.

Surface shaders are nothing more than an abstraction over vert/frag shaders. I don't think there is anything you can do with surface shaders but is impossible with vert/frag shaders. #pragma debug is your friend. Try to add #pragma multi_compile_fwdadd_fullshadows to your ForwardAdd pass and then sample the maps in the same way Unity does.

 

Been using pragma debug, can't believe I missed that, though. Well caught

 

Thanks, Farfarer and Dolkar.

Using tangent-space light and view direction as interpolants a la Fanfarer's suggestion seems to be working well. I think that I will stick with this approach.

Using unity_LightPosition as in the ShadeVertexLights mentioned by Dolkar is not working. The values in unity_LightPosition don't change based on lights in the scene. Instead, each entry in the array appears to be an axis-aligned unit vector. Perhaps this stuff is legacy?

 

No, it's being used for vertex lighting... hmm... that might be it! Try to set your LightMode tag to "Vertex".

 

Setting it to Vertex works! Thanks!

I currently have two versions of the shader. One averages the lights in the vertex shader and then does lighting with this single, averaged light in the fragment shader. The other is the multipass ("ForwardBase" and "ForwardAdd") version. The single-pass shader works fine on both PC and Android Nexus 4. The multipass one works great on PC, but on the Android, the ForwardAdd passes aren't occurring. Is this a known limitation?

 

I suspect Android might be set to only have one pixel light, so it'll never have a use for the ForwardAdd pass.

You can change that in Quality settings.

 

Quality Settings are set to have 3 pixel lights. Quality settings are not per platform. The only per-platform settings that I can find are the Player Settings, which don't have any setting related to lights.

Thanks for being so helpful.

 

The quality settings are per-Quality Level, so you can either tell Android to use a higher quality setting (wherever the green tickbox is beneath the Android logo in the top grid) or you can create a new quality level based on Android's default, then you can up the Pixel Light Count and tell Android to use that quality level.

 

Thanks!