Determine how lit a vertex is in a Standard Shader?

I need to check if a vert is in shadow or not in a deferred modified standard shader. What's the most accurate way to do that? At the moment I take the value for outEmission and run Luminance() on it. The greyscale return value presumably indicates whether it is dark (0) or light (1). Is there a better way?

 

Short answer is you can't.

Longer answer is in deferred dynamic lighting isn't applied until long after you no longer know anything about the vertices, or really anything about the object itself. The whole point of deferred is it doesn't know anything about light or shadow when the object is initially rendered and the lighting doesn't have to know really anything about individual objects.

If you're using baked lightmaps you also don't really know if you're in shadow or not, but you can kind of guess. From the information stored in the lightmaps, but Unity doesn't store information about actual shadows in their lightmaps, only the combination of light, shadow, bounced lighting, and ambient lighting, so the best guess you can make is if the luminance of the value in the lightmap is less than a certain amount. If you're talking about outEmission that's no longer the vertex, that's the fragment shader, and that does indeed hold the decoded lightmap value which you can test with Luminance(), but again the same issue applies that you can guess, but it's highly dependant on the lighting of your scene.

If you're using the forward rendering path with a light set to mixed Unity actually does a bit of this "is this in shadow" against the lightmap data for real time shadows, but it uses information about the main directional light that I don't believe is available to deferred shaders by default, and is still just a guess.

 

To be honest I've always been a little dodgy on the various shader passes as far as what is interacting where. I guess I am indeed in the frag shader as I just wanted to hijack a little bit of a standard shader and blend the areas in the dark like this shader http://wiki.unity3d.com/index.php/DesaturatedDarks2 My code currently looks like this

Code (csharp):

1.  
2.     half3 fullColor = s.diffColor;
3. #if _VERTEXCOLOR
4.     fullColor*= i.color;
5. #endif
6.     half brightness = saturate(Luminance(color.xyz));
7.     half unlit = Luminance(lerp(fullColor, color, _Darkening));
8.     half3 final = lerp(half3(unlit, unlit, unlit), color.xyz, brightness);
9.     outEmission = half4(final, 1);
10.  

So that's as good as it is going to get?

 

Well, that's a vertex / fragment shader that'll be rendered using the forward rendering path regardless of what rendering path you've chosen, which is totally different than something that would have access to outEmissive.

 

Yeah but I'm trying to translate it to a surface shader. I already have a vertex colored standard shader and I was hoping to do something similar in my shader by blending desatured colors at the unlit areas.