world position of a pixel from depth texture

Well, i got dizzy looking for a solution on this one.
One way of doing it is explained in GPU GEMS 3 as follows:

Code (csharp):

1. float zOverW = tex2D(depthTexture, texCoord);
2. float4 H = float4(texCoord.x * 2 - 1, (1 - texCoord.y) * 2 - 1, 05.zOverW, 1);
3. float4 D = mul(H, g_ViewProjectionInverseMatrix);
4. float4 worldPos = D / D.w;

problem is there is no builtin ViewProjectionInverseMatrix in shaderlab, so i just pass it from c# script as follows:

Code (csharp):

1. (myCamera.cameraToWorldMatrix * myCamera.projectionMatrix).inverse;

but the 3d positions come out all messed up at this point with the passed inverse matrix, depth is reversed and view is mirrored.

Soo, any help is appreciated here. How can i get the 3d positions of 2d pixels in a shader using depthtexture as Z?

 

I've been trying to get this working for ages (as have other folk)... it's unclear whether the functionality is broken inside Unity or whether it's just been done wrong.

There's a _CameraToWorld built-in matrix in Unity, but as far as I can tell no one's gotten it to work properly.

Would be very interested to see if anyone has ideas on it.

 

_CameraToWorld Where is this matrix mentioned? i cant see it anywhere in the documents.

There also are afew more alternative ways of calculating this as i found in google but not 1 of them is supported by or possible with unity.

 

Eh, what i mentioned in the first post works, i just made a typo somewhere. Problem solved.

 

It's not documented, but it's there. It's mentioned as being "fixed" in the 3.3 update changelog. There's _CameraToWorld and _WorldToCamera matrices, you just have to define them and Unity will fill them in ( float4x4 _CameraToWorld; ).

What was the typo you made, btw? I'd be intrigued to figure this out.

 

I forgot that we multiply with matrix first (mul(g_ViewProjectionInverseMatrix, H))

About _CameraToWorld, what do the rows and columns represent anyways?

 

Not sure as I've never managed to get it to do anything useful :/

But I'd guess it's meant for you to multiply it with screen x/y/depth, it'd give you out the world coordinates of that point...

 

I will see what it does. Meanwhile Gpu Gems solution works flawless.

 

Sorry for reactivating this old thread, but I'm currently also struggling with this. What I'm trying to do is, reconstruct the world position from a single depth value.

I know there are several ways to accomplish this. The two most comon are:

1. Frustum corner method. Store a ray pointing to the four corners of the frustum near plane as vertex attributes of a fullscreen quad. In the fragment shader sample the depth buffer (linear view space depth) with the fragment coordinates and multiply the depth with the interpolated frustum corner ray. The result is the reconstructed position in view space. Multiplying with the inverse view matrix (_CameraToWorld) yields the world position. Check.

2. World position by unprojecting. Again sample depth based on the fragment coordinates. Remap uv's to [-1, 1] and set depth as z. This is the projected viewport position. Multiplying with the inverse viewProjection matrix yields the world position. Check.

I succesfully implemented the first version, however I just can't get the second method to work. My current approach is as follows:

Code (csharp):

1.  
2. Shader "Custom/RenderWorldPos"
3. {
4.   SubShader
5.   {
6.     Pass {
7.     CGPROGRAM
8.     #pragma vertex vert
9.     #pragma fragment frag
10.  
11.     #include "UnityCG.cginc"
12.  
13.     struct v2f {
14.       float4 pos : SV_POSITION;
15.       float4 uv : TEXCOORD0;
16.     };
17.  
18.     float4 _CameraDepthTexture_ST;
19.     sampler2D _CameraDepthTexture;
20.  
21.     float4x4 _ViewProjectInverse;
22.  
23.     v2f vert (appdata_img v)
24.     {
25.       v2f o;
26.  
27.         o.pos = mul (UNITY_MATRIX_MVP, v.vertex);
28.         o.uv.xy = ComputeScreenPos (o.pos);
29.  
30.       return o;
31.     }
32.  
33.     float4 frag (v2f i) : COLOR
34.     {
35.         float2 uv = i.uv.xy;
36.       uv.y = 1 - uv.y;
37.  
38.       float z = tex2D (_CameraDepthTexture, uv);
39.       if(z > 0.99) discard;
40.  
41.       float2 xy = uv * 2 - 1;
42.       float4 posProjected = float4(xy, z, 1);
43.       float4 posWS = mul(_ViewProjectInverse, posProjected);
44.  
45.       posWS = posWS/posWS.w;
46.  
47.       return float4(posWS.xyz, 1.0);
48.     }
49.     ENDCG
50.     }
51.   }
52.   Fallback off
53. }
54.  

The C# script for setting upo the camera and passing the matrix to the shader:

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections;
4.  
5. public class CamScript : MonoBehaviour
6. {
7.   private Material m_material;
8.   private Camera m_cam;
9.  
10.     // Use this for initialization
11.     void Start ()
12.   {
13.     m_material = new Material(Shader.Find("Custom/RenderWorldPos"));
14.     m_cam = gameObject.GetComponent<Camera>();
15.     m_cam.depthTextureMode = DepthTextureMode.DepthNormals;
16.     }
17.  
18.   void OnRenderImage(RenderTexture src, RenderTexture dst)
19.   {
20.     Matrix4x4 viewProjInverse = (m_cam.projectionMatrix * m_cam.worldToCameraMatrix).inverse;
21.     m_material.SetMatrix("_ViewProjectInverse", viewProjInverse);
22.     Graphics.Blit(src, dst, m_material);
23.   }
24. }
25.  

My result is close, but always a bit of. I'm using the reconstructed world position from the internal prepass as a reference.






I'd appreciate any hints on what I'm doing incorrectly.

Enclosed you find a unitypackage with a sample scene, that illustrates the issue. If you import this into an empty unity scene, make sure to delete the existing camera gameobject.

Have a nice evening.

 

MPanknin, have you been able to solve this problem?

I have very similar issue as well. My issue is getting view space position instead.

Code (csharp):

1. float4 ciipPos = float4(S*2.0-1.0, z, 1);
2. float4 viewPos = mul(_ProjectionInv,ciipPos);
3. viewPos = viewPos / viewPos.w;

and I am setting _ProjectionInv as :

Code (csharp):

1. camera.projectionMatrix.inverse;

My result is also a bit off as well.. looks kinda similar but no..

Something I have found out as well recently is that

camera.projectionMatrix != UNITY_MATRIX_P

I had to do this for my projection matrix to work

Code (csharp):

1. // Scale and bias from OpenGL -> D3D depth range
2.  
3. for (int i = 0; i < 4; i++) {
4.        P[2,i] = P[2,i]*0.5f + P[3,i]*0.5f;
5. }
6.  

but this was depth issue... I "think" inverse P has the same problems too! Any ideas?

Edit : I just tried going back and forth with correct view space position coming from vertex shader and my inverse projection matrix from script works (with modified P.inverse) So I think issue must be with how Z is passed to the equations.

I am getting my z straight like :

Code (csharp):

1. float d = tex2D(_CameraDepthTexture, uv.xy);

then

Code (csharp):

1. float depth = d;
2.  
3. // scale it to -1..1 (screen coordinates)
4. float2 projectedXY = float2( uv)*2-1;
5.  
6. // not sure why I need to invert on Y so I left it out.
7. //projectedXY.y = -projectedXY.y;
8.  
9. // create the position in screen space
10. float4 pos = float4(projectedXY,depth,1);
11.              
12. pos = mul(_ProjectionInv,pos);
13.              
14. // make it homogeneous
15. pos /= (pos.w);