Unity Tegra Shadows?

One of the things introduced with Unity 4 was the ability to do shadow mapping on mobile platforms.
The sad bit was seeing that this does not include chipsets such as the Tegra 3, since they do not support depth textures.
However, someone mentioned that it would be possible to achieve a workaround by encoding depth values to RGBA channels and then decoding them before use.

I'm thinking, then, that it may be possible for me to create a custom shadow mapping system that packs depth values into a color render texture and reads them from a custom surface shader, and it should work just fine on mobile platforms that do not support depth textures (such as the new OUYA console).

If I were to approach such a project, would anybody be interested in an asset store release for a fee? Say, $50?

 

I am certainly interested.

 

Well, I so far have been able to successfully pack depth values into RGBA as well as unpack them. The next step is to output this into a render target for use by shadow receivers, which themselves will compute distance to light per pixel, compare this to the depth value in the render target, and depending on the depth comparison will determine whether the pixel should be in shadow or not.

 

Early tests have proven successful!



The above pic does not use the built-in Unity Shadow Mapping in any way, nor does it use depth textures (and thus would work on platforms that do not support depth textures!)
In fact, it does not rely on more than one render target either (it only needs the shadow map from the POV of the light), which means it also works even if you clear the main depth buffer (for example, if you have a separate camera rendering weapons on top of the main view, shadows will still work perfectly fine unlike the builtin Unity shadows)

 

Cool. Unity Pro, then.

 

It's simply not possible to implement shadow mapping without render targets

 

OK, so here's what I have so far.
The current system requires that you place a script on your main directional light (the system only supports a single directional light). You can configure the shadowmap size, range of coverage, and near/far planes for the shadowmap camera, as well as which layers will cast shadows. The system then automatically sets up the appropriate render target and camera. It also includes an option for "sparse updating", where it only re-renders the shadowmap when the main camera moves outside of a given threshold distance (better for performance, you can use this if you only have say environment casting shadows and player characters receiving them)
The shadow system also provides the shadowmap projection matrix and the shadowmap itself as shader globals, so that any shader can access everything needed for shadowing without any special setup. There ofc will be a demo surface shader included that uses a custom lighting model to demonstrate how to set up shadowmapping in custom shaders.

 

Can I ask unity why they have not coded this workaround for tegra 3 chipsets? I can understand it being too slow on earlier tegra, but this obviously is a case of feature parity.

 

I think they may have done some work at one of the Ninja camps... but who knows when it will be finalized/released? They haven't given any sort of ETA on it.
Even if it does get released though, I'll probably still release this but possibly for free (one nice feature I think is the "sparse update", which will be much less graphics intensive since it won't render the shadowmap every frame)

 

Update: Here's a demonstration of how one might actually use this in practice on a mobile device / OUYA

• Terrain / scenery is statically lightmapped (but is rendered into shadowmap)
• Character uses a custom lighting shader to compute shadows from shadowmap (but is not rendered into shadowmap)
• Character casts a projector shadow (yeah, I know this is generally a bad idea on Terrain... you could also use a simple quad or even no shadow)
• Shadowmap is rendered every 5 meters the character walks.
• Shadowmap is 512x512, spread over 10 meters

 

Just replaced an if-statement in my shadow receiver shader with a step call (avoiding a GPU branch)
It's not noticeably faster on PC, but should be more efficient on mobile platforms.
For reference:

Previous

Code (csharp):

1.  
2. half s = ( fragDepth < shadowmapDepth ) ? 1 : 0;
3.  

New

Code (csharp):

1.  
2. half s = 1.0 - step( shadowmapDepth, fragDepth );
3.  

 

OK, I think my next step is to attempt to create a shadow receiver shader with percentage-close-filtering to smooth out that aliasing.

EDIT: On second thought, I think I'll skip it. With too few samples, it still looks heavily aliased, but with more samples I start to encounter register limits (on PC. on mobile I would surely encounter them far sooner)

 

I have been gripped by madness!
Naw, JK. But seriously, I've decided I'm just going to release this for free.
Merry super-incredibly-early Christmas.
View attachment $TegraShadows.unitypackage

 

Mmm, download link does not work for me...

 

This happened to me with several attachments on the unity forum. This happens only with google chrome. I don't know if it has something to do with chrome or with the forum's code ...

With firefox yoy can download it.

And thank's for sharing this PhobicGunner !!!

 

I have been looking at doing the same thing over last few days, and I have archived almost identical result posted in here. I was looking for some ways to improve the shadowing issue after that and found this thread by accident. (I wish I could have bit earlier few days ago.. saving me some time to figure out few things.. But DIY things certainly helps you to understand how things works much better)

Anyway... after doing this shadow for myself, I was thinking why Unity doesn't make their existing shadow to work with this method of producing depth buffer using RGBA channel... I am sure if Unity wants to , they can implement this method internally.

I have been reading about ways to improve shadow mapping and boy.. there are indeed tons of out there, but that is just another time for me. The next thing I want to try is doing very simple perspective shadow mapping where it seems like you only need to multiply one more matrix to get the depth map to render little bit more perspective friendly way to cover more pixels near to the camera near plane.

One thing that I found my method was different from than method here is that I used Cull Back and bias instead of using Cull Front. I guess they both have their usage.

 

Yeah, Cull Back + bias is definitely the "ubiquitous" version that everybody else uses, I sort of independently figured out that if I culled the front faces instead I could avoid any sort of bias at all. I guess it doesn't matter that much, though.

I had read a little about perspective shadow maps, though never in great detail. If you could get that working it would be pretty sweet

BTW, there's also a little wierdness in my shadow system, and it's due to texture addressing: when reading from the shadowmap, objects outside the shadowmap coverage get the clamped edge pixels. I managed to somewhat solve this by clipping the shadow results by distance using a step function, but that feels kinda hacky to me. Would be great if I could somehow just wrap the entire shadowmap in some kind of 1-pixel border... oh well.

 

Yeah I have the same issue as well. It also happens not on the distance, but on the side as well, if camera rendering depth map doesn't cover the area enough. What I am going to try is that depends on the xy of the shadow map uv border , try to fade the shadow result to zero. First thing I can think of is some simple masking map and use uv that is used to pickup the depth map and just multiply so the results are faded off...

Or this could be archived using some simple maths instead...

Also, I was searching a bit more about shadow mapping, and been thinking reason why Unity didnt go for all those other shadow mapping techniques than CCSM.. I was thinking, there must be some good reason... and I think that Unity has found CCSM is best trade of all..

So I looked at doing CCSM myself...

In order to do CCSM, I have to basically , render n times of the shadow map depends on the z value of the depth map. But I can't think of doing it other way than setting up 4 cameras and manually render them into one depth map using manual static uv placement.

Now, I think this "can" be done. even in mobile.... but doing this will cause even more draw call increase ( which is true even for Unity CCSM ) so it may not be worth doing it for mobile at the moment. (maybe in few years..)

 

Here is my attempt to fade out shadow edge using additional masking map..

This will increase yet another texture fetch so if your shader is already texture fetching frenzy limit , then it may not be a good idea....

At first I tried doint it using tex1D so I do 2 tex1d fetch x and y direction but .... that will cause 1 more texture fetch.. so I went for tex2d. (causes more tex memory..)

 

Ok, here is my another attempt to do some quick soft shadowing.

Instead of trying to worry about shadow resolutions etc... I have tried brute force way of making soft shadow and see how that goes.

Basically, I have rendered my shadow into yet another render target and then applied blur post process.... then do multiply post process.

This must be one of the worst way of doing soft shadow because :

1. well.. obvious shadow inaccuracy and bleeding.
2. another render target = expensive
3. yet another post processing after that = expensive

But doing this way can reduce the shadow rendering render target texture so I have tried half size for rendering shadow. Then blur post process will 1/4 it again for blur process... then you just do multiply...

When you do this GPU texture filtering kicks in because of you are dealing with smaller texture sizes so you sort of get anti-alising effects.. (which helps a bit) so even if you don't do any blur, it is better.

On my S2 android phone , current scene runs at 60fps... but then again, there arn't much else going on the scene.

if I optimise post processing path a bit , then I may get little bit better performance ...

 

Yep, very nice!

 

Yeah , also doing shadow as post process also comes with nice side effect that the actual model you render does not have to contain any shadow codes. Because you have just separated that part to post processing. Also, this means you now have lot more rooms to play with your shader in terms of number of instructions and texture fetchs etc.. which I think it is important becuase mobile is limited to shader 2.0

 

Ah very cool!
I think at this point yours is way cooler than mine XD

 

Don't mind me asking but would this work on Unity 3.5?

 

+1?

Very nice progress.

 

Yes , I am using 3.5 at the moment.

It doesn't matter if it is 3.5 or 4. Even with Unity 4, mobile shadow doesn't work with Tegra2/3 devices. (and who knows what other...)

Today, I have tried to touch my post processing pipeline.

Before, I was just using Unity's built in image effects to get the blur to happen...

I did get some nice soft blur to my shadow, but I thought that doing this post processing on mobile is probably little too much than what you would usually do in mobile platform... so I decided to see if I can cut into the usual process and tweak them to work better on mobile.

Thing is that Unit's blur post processing effect will do couple of blits , down sizing on it's own and then running the image effect shader over and over... also adding allocating 2 temp render targets and then freeing them every frame!

So the goal was to do a single blit and single pass blur without allocating extra render targets. And + do multiply post processing at the same time, reducing yet another blit operation.

The above method was okish to produce except that on mobile it has even less room on the size of the vertex data I can pass over to the pixel shader so that was bit of challange... so instead of passing all 4 uv coordinates which I need to sample the shadow map from , I passed 2 and then for other 2 , I just calculated them in the pixel shader ( damn...)

Then again, I have tried yet another crazy idea....

what if I can modulate shadow intensity depends on the shading on the model... That is, darker the face (because of its natural lighting from lamert shading) , lighter than shadow gets... so sort of shadow "fade" or "blend".

The idea is that if polygon gets darker and eventually gets black lighting from light, there should be 100% in shadow right? so if I start modulating shadow intensity even on the lit side of the polygon, the shadow will get lighter until they are the same color as the polygon's color when it is turned away from the light.

It is hard to explain... but that is what I tried to do. So.. how to do that... well, when you do shading in your model, I saved the lambert term into the alpha channel. Then when it comes to the post processing, I used that alpha channel and modulated the scene and shadow intensity. Result is that model shading is done at post processing.. as well as shadow intensity modulation.

It is very non standard way of shading your model... and I guess it has its own problems if do transparency etc... but it was interesting nevertheless.

performance wise, I think it is bit faster than before... plus I am doing shadow intensity modulation now..

Still, the biggest performance killer is depth map generation.. if I increase depth map resolution from 512 to 1024 to 2048 then performance just drops significantly.

the scene is around 20k polygons with 32 drawcalls. running around 58fps using 512 map , shadow map at full res.

I think this kind of shadow mapping technique is ok for RTS top down games since AA is done at screen space... but if you use it on FPS or camera direction is close to parallel against ground then it won't look good...

 

Anyone knows the list of devices that supports Unity 4.0 shadow?

 

@Castor76: So the shadows you are working on aren't for Tegra? The shadows PhobicGunner is working on are for Tegra 3, even the package he posted is called 'TegraShadows'. I think you can understand the confusion.

 

This method , ( including PhobicGunner's) works regradless of what the device is...

It works on PC , should work on Mac and IOS and Android devices...

Because everything is done manually. Creathing depth map, storing it on rendertarget , shader to render etc... all custom made, so it works on Tegra , non Tegra devices..

The methos is not just for Tegra devices... it is just generalized method of doing shadow mapping if you have to do everything yourself.

 

Great!

So all the discussion about Tegra is irrelevant, that's what I wanted to know.

 

Sounds........FANTASTIC !

So, would you share it with us....




BTW, PhobicGunner - YOU ROCK BRO! THANKS !!

 

BTW, it seems Tatoforever will be releasing a mobile shadow too which seems to be using the same technique (generating depth texture from RGBA)

 

Yeah , that looks good.

I am not sure about doing it in single pass thou. Maybe they are talking about only rendering the shadow part. But to be able to render shadow or doing SSAA requires depth map in one form or another... which is separate pass of their own.

I will try to share this as soon as I have things bit more organized... at the moment, my project is in mess with other stuffs in there.

I also want to try to do another method of doing AA or soft shadowing.

 

Sounds good.
Am curious, how does yours look without any form of AA? Mine doesn't support it at all and in some cases ends up looking really jagged.

 

Ok, I have moved to Unity 4 and tested Unity 4 shadow on Android...

Pictures are direct capture from the phone.

Now I "think" that at high setting shadow map resolution is somewhere around 1024 and then 512 for medium...

FPS for high setting is too low to be used in the game.

Model uses just a standard lambert shader from Unity.

 

And here are the ones that I am trying...

Note that even the ones that are not bluring, it is actually doing it.. I just made offset to zero.. so FPS could be higher if I code to be hard edges to start with. (no post processing)

Now .... it seems like custom shadow is ... faster for some reason...

Maybe it is not doing some other stuffs that Unity shadow does... like CCSM. But I couldn't see CCSM happening on mobile using Unity shadow.. yet.. (if someone can confirm this works then let us know)

IF... that is IF... Unity is NOT doing anything special but just plan shadow map then.. well, I can only think of one reason why it could be slower.. Maybe.. creating depth map using hardware... is SLOWER than doing software custom depth buffer...!

The reason that I can think of is that depth buffer may be ( I am almost certain ) in float point format.. so mobile is having hard time accessing them.. and they are slow..

This is a good and a bad news... because you can potentially do custom shadow which may work faster and better for certain cases depends on what games you are making. But bad news is that it won't be integrated into Unity like how Unity shadow works, so difficulty on working with it. Also probably not a good idea of doing cross platform custom made shadow mapping. ( especially for PC )

Next , I am going to try using RG instead of RGBA to store depth map, I think it could be faster than using RGBA.. with some precision lost.. but Unity's depth buffer format is also usually at 16bit so I think it should be fine.

 

Sounds awesome. And looking good!!

Question, have you tried just saving the shadow map to smaller size texture and scale it up? In other engine I tried this they normally would automatically be blitted and yielding a nice blur without much cost.

 

Yes , this can happen in the post processing path.. which can make things even faster..

But I found that doing that alone is not enough in my taste , but yes, they are still better than nothing. (with bit of precision lost)

 

Ok, and Here is version with shadow modulated with diffuse lighting information. Notice with this version you don't get nasty shadow artifacts happening on the polygons facing other way from light. The artifacts are still produced... they are just been modulated to not show anymore using diffuse lighting term.

With this method, post processing shader becomes little bit more expensive so frame drops a couple.

(Thanks Hippo coder : What a dumb blind I was not looking at use 32bit display buffer.. sometimes it is right there and just can't see it)

I have also tried half the shadow post processing image size (not depth map) and it helped a bit.. hitting the same scene around 50 fps. Still looked ok to me.

 

I have been looking around to improve this shadow mapping method a bit further, like trying CCSM , Perspective shadow mapping or VSM etc..

But.. I think now I have hit the wall with this. I tried VSM and there are problems :

First , VSM requires high quality, high precision depth map. Paper suggest that even with 32bit precision, sometimes it may not be enough. And then, the VSM requires you to have depth map and another depth map containing depth * depth...

So I initially thought, well, use RG for depth and BA for depth^2... well, it did work... to some degree.. because using RG makes precision to become 16 bit, precision was too low for VSM... at least for the ones that I tried. This then ment, my near and far plane for depth map camera was very limited...
Also , you need to do a good blur to the depth map for this technique to be worth while, and that is actually was slower than doing view screen blur.

Now to the CCSM...

CCSM requires you to split rendering of depth map into different phases and then combine the textures together etc and do all other sort of things to figure out which part of the depth map you need to sample.. Well, easier said then done, it is just like almost everything in Unity. In Unity you can do things ( mostly easy things ) pretty well and quickly. But when you try to .. well, go beyond Unity, then you just hit the wall like I did... Just not enough inner knowledge of how Unity actually work inside and not enough exposed interface into the deep Unity makes you feel like going around circle and even if you do make things work, you feel like it can never be as efficient as how Unity would do it if it does it in native mode.

I did check out Unity 4 shadow a bit more, and it indeed does CCSM... so that is actually good thing using Unity4 shadow.

Perspective shadow... This also the same story. Not enough control over how things are rendered in Unity to be able to set up camera correctly and render them nicely.

It could be my lack of math knowledge , (probably is) but I think I have hit the wall. And I think I will leave this as it is for now.

I was thinking about releasing it here. But then I was going to do it if it is truly worth while using this instead of Unity's shadow. but alas,... it doesn't really seem that way. It is still nice to be able to use shadow using Unity 3.5 and Tegra for Unity 4. The code released here already satisfies that so there is not really any point... All the things I tried to do to improve the shadow quality was just simple post processing stuff anyway.

 

Awww, I was hoping to see if you did something differently.

 

if I ever succeed in anything different and meaningful then I will keep posting here.

 

I'm very interested in a shadow mapping solution for OUYA and tegra devices (and will put my money in that asset)
Any progress castor76?
it looks really cool right now. When will you have a public release?

 

Ouya.... humm that is something that I haven't thought about. So Ouya is Tegra based device... I see.. Well, then I guess custom shadow can help some games running on Ouya.

It is just that at the moment, there are too many restrictions of this custom shadow method to be used for a Unity like general purpose way.

Like I have mentioned, there are no automatic cascading of the shadow depth map, Limited screen space AA is only done through post processing and shadow depth diff artifacts are hidden only when you are using specific lighting method.

So , I don't think it is worth while to be released in the Asset Store. And the base of the method has already been released here.

However,......... what I am thinking is... I am wondering if there is way to hack into Unity's shadow method... From my limited understanding, the only thing that prevents Unity from doing hardware shadow on tegra like device is that they don't support hardware depth buffer. And I read somewhere that Unity could have come up with alternative depth buffer rendering method like the one that is shown here. So , "IF' we can hack into their process and only hijack the process or replace the part where it produces and uses depth map then maybe there can be something worth while.

 

Yes, Ouya is based on Tegra 3, and given the fact that, right now, we just can't make any realtime shadows on Ouya other than using projectors, if we could have some way to make them, even with limitations, it will be an improvement
I suppose this is something the Unity team will have to fix at some point, but until then we really need something (since it could take a bit of time for them to fix it)

 

how has your progress been going on this?

i've been doing testing with Tegra 3 on Surface RT and the performance is quite bad even on release builds..ie:

1 rigidbody, 1 plane, 1 directional light w/ hard shadows, no movement, no animation (completely still scene)... 30 fps at idle, remove shadows and i get 60 fps.

start throwing in shadows and multiple colliding rigidbodies and performance drops like a rock after 50 rigidbodies (20fps), and 200 rigidbodies (3 fps). quite disappointing.

 

The sunshine plugin in the asset store does make shadows on tegra3. The demo is very fast on ouya (apk provided to test) but I haven't been able to make it work as good with my own game and the fps falls dramatically. Don't know what am doing wrong

 

if anyone happens to check out Phobic's toolkit and runs into an error that doesn't allow the shader to compile (ShadowReceiver), you'll need to add

UNITY_INITIALIZE_OUTPUT(Input,o);

to void vert method before modifying the o variable...

don't ask how i figured this out cause, this stuff is magic to me.

 

Damn, homie. I love you.

So I guess Unity decided to not do the dynamic shadows for the Tegra, eh?

I found a load of great solutions out there and almost started writing one myself. You just saved me a lot of work.

 

Sorry to dig up this old thread, but...
Everyone says it works good on Tegra3 but when I try out your test scene, it just renders the outline of the shadow and nothing else.