Writing depth value in fragment program

[tomaszek]

Hi,

I've got problem with writing depth in fragment program. All fragment programs I saw for ShaderLab write only color, but it seems to be possible to output depth too (as I see in HLSL/Cg forums). I've been digging the forum for hours but I still got no idea how to do it. Was trying shader from attachment, but compiler throws such error:

Shader error in 'TestShader': D3D shader assembly failed with: (8): error X2022: scalar registers cannot be masked
Shader Assembly: ps_2_0
; 4 ALU
def c0, 0.00000000, 0, 0, 0
dcl t0
frc r1, t0
mov r0.z, c0.x

Any idea how to make it working ?

[dreamora]

The texture you write to in the fragment program is only a color buffer so writing depth will not work I guess.
Unity 3 renders the depth and normal buffer in a different pass (given you enable the camera to render a depth texture) than the color, but you can potentially access this one.

[tomaszek]

Thank you, fast and competent reply as usual by dreamora .

I've just analysed compiled version of simple surface shader. Indeed - as far as I see there is only color buffer written. I will have to sort things another way aquiring information from depth texture. The problem is - I want this shader (grass shader) to work on terrains which are usually queued as "geometry-100". So I'd need to write my material later as it will be alpha masked "transparent". Are there any known drawbacks when shifting terrain material later in render queue ? Dangerous side effects to be aware of ?

[dreamora]

likely totally non working depth sorting actually.
It already runs at its edge of usability as you might find out with some PostFX actually.

[duke]

Has this been addressed in 3.4?

[Noisecrime]

Just a note for those wanting to write to depth in D3D9 shader and getting the 'D3D shader assembly failed with: (36): error X2022: scalar registers cannot be masked' error.

Although this bug still remains, the problem is with the cg compiler, something that has been known since 2008 (according to this old cg thread on nvidia forums). So either its never been fixed or Unity is using a pretty old version of cg?

Thankfully it appears from initial testing this issue can be solved by changing the compiled shader code from

mov oDepth.z, r0.z

to

mov oDepth, r0.z

I.e. removing the .z mask, which since its on a scalar doesn't work and hence the error.

To change the compiled code simply create your shader normally, ignoring the error. Once finished select the shader and click 'open compiled shader' button in the inspector. Now copy the entire contents of the compiled shader and paste it into a new shader file. Look through the code and replace the lines causing the bug as above, then save the shader.

It does mean that your original shaderlab/cg shader will never compile or be usable in d3d9 environment so don't use it or include it in your builds! However your fixed 'compiled' version should work fine. At least it does for me in the editor and as a window build.

[tomaszek]

Good to know it's possible. Works in OpenGL/GLSL and D3D9 after NoiseCrime's fix, but still there are suspicious messages in compiled code:

1. for mobile profiles:

/* NOTE: GLSL optimization failed
0:32(15): error: `gl_FragDepth' undeclared
0:32(15): error: type mismatch
*/

so can't say if works for mobile

2. error about flash export incompatibility

Shader warning in 'TestShader': register type 9 not supported in AGAL (compiling for flash) at line 5

Anyway - it's big step further and could be useful, however I dropped the idea using this feature completelt for performance reasons (writing into depth buffer is told to be slow, but honestly I've never tested it since I couldn't do it well in Unity).

[Noisecrime]

Good point about Mobile ( where writing to depth buffer is not supported as far as I understand) and Flash.

As for performance, seems fine to me (Windows). I guess it could be slower than normal rendering since the gpu has no idea what depth you are going to set, so normal optimisations (e.g. early outs) can't be done. However its not likely that you'd use this frequently for many objects, more likely as in my case, you'd want to use it once per frame to populate the depth buffer with specific values.

[Aras]

Good to know it's possible. Works in OpenGL/GLSL and D3D9 after NoiseCrime's fix, but still there are suspicious messages in compiled code:

1. for mobile profiles:

/* NOTE: GLSL optimization failed
0:32(15): error: `gl_FragDepth' undeclared
0:32(15): error: type mismatch
*/

so can't say if works for mobile

We can't do much about mobile here. In OpenGL ES 2.0, writing to the fragment depth is not possible. There is just no way to do it.

2. error about flash export incompatibility

Shader warning in 'TestShader': register type 9 not supported in AGAL (compiling for flash) at line 5

Again, not much we can do. In Stage3D's AGAL shader language, writing to fragment depth is not possible.

To not produce any warnings, add "#pragma exclude_renderers gles flash" to the shader code.