How to get world-space normals in image-effects

So you can get the normals from the _CameraDepthNormalsTexture, but these are all in screen-space.
How do I convert them to world-space?

(I could swear I've done it before, but can't remember exactly and can't find anything)

I tried UnityObjectToWorldNormal() but that doesn't work.
(which is no surprise, I assume these are only for regular shaders, ... although, an image effect is just rendered on a screen-aligned quad, right? why doesn't it work then?)

I tried converting it myself by passing the camera direction to the shader, and rotating the vector accordingly, but the result isn't correct, and I can't find any problem in the logic.