full screen texture mask to see through a layer- how?

I'm trying to make an effect where I have a full screen circle gradient that would mask out a layer to be able to see through it.

ex: I have trees on one layer rendered by it's own camera. That layer also has a plane with a screen-space shader on it which I would like to use to mask out those trees to be able to see through them at the center. For whatever reason I can't get this to work. To give you a visual - imagine using a vignette as a mask for a layer's transparency.

I know a layer is not a one-pass thing, and I'm thinking that's my problem, but is there any way to have it act like that? Is this at all possible? Could it also have anything to do with the fact that I'm using a deferred render?

here's an example:

In this image the forest is being masked so that I can see the cow.

 

Your second camera which renders the cow or the forest should render to a rendertexture and your first camera should have a simple image effect which lerps from its own rendered scene (the opposite of second camera) to the first cameras render texture through a circular gradient texture.

 

Why can't you just combine 3 textures in one shader? one for trees, one for cow, and an alpha8 for the blend gradient. Then the shader is fairly simple.

 

hmmm.. I have no clue how to set that up. I'll have to take a look into it. I'm using strumpy for my shader editing, is this doable with that?

 

I'm not actually doing a 2D effect. that image is just to illustrate the effect. This all needs to be done in 3D. those trees represent unity trees placed in the world.

 

Ive no idea what strumpy can do and you shouldnt use it if you want to learn shaders.

 

I do appreciate the advice, but I'm not trying "learn" shaders. I'm not a coder, I'm a VFX artist. I use strumpy because it's a schematic editor, making my life much easier. I do understand that this might be out of the realm of possibilities for strumpy, but until that's confirmed, then I won't be learning to code shaders.

 

so this is what I'm going to try, let me know if you guys think this is a bad idea or not.

I have my screen-space "mask" working on it's own. But instead of applying the mask to a render layer and treating the render layers as a compositing software, I'm going to make a custom shade for the trees that will use my mask system multiplied into the alpha. I tried quickly applying a new material to a tree with my shader, but it seemed to do nothing at all, but I also had to run to work, so I'll look more into when I get home.