Dynamic Volumetric Lighting replacement

You are correct but why would you have a shadow casting light if there is no object inside light's volume? The only problem I see here is that unity will turn the shadows off if the light is too far from the camera. But I can easily detect that and disable shadows for that particular volume. Anyway, here is what I do.

1. Shadow casting lights - I use LightEvent.AfterShadowMap to get light's shadow map and render light's volume. It takes some reverse-engineering to replicate all the built-in lighting parameters but it can be done.
2. Lights without shadows - I use CameraEvent.BeforeLighting to render volumes for all lights that don't cast shadows (including shadow casting lights too far from camera). I should cull them here based on visibility to save a few draw calls but I don't really care. I let the rasterizer clip them.

It is a pretty standard rendering setup otherwise (with all the drawbacks). Render to smaller light buffer, ray marching with dithering, blur, bilateral upscale. Animated 3D texture for volumetric noise and mie scattering. Everything else respects built-in unity lights.

I don't support directional lights. I did all of it just for fun. I have zero interest in using it in a game. And since I didn't need directional light for my demo I didn't bother. With that being said, I will need to trace cascade shadows for my next hobby project. I also love to hack and reverse-engineer stuff So I will look into it and tell you what I find.

Also, great job with your new lighting. Unity can be pretty slow sometimes. We actually have zero built-in shader policy at work for performance reasons.

 

You are correct, but you need to know when this happens. If you had a shadow casting light moving around and it was to go into the air then suddenly that LightEvent.AfterShadowMap would stop getting called. you wouldn't have anyway of knowing to render a none shadow casting volume there.

calculating the values for point and spot lights are do-able, its the directional that i see issues with. actually i was able to get the directional light render position, I should be able to calculate everything else from that. will have to look into it more.

 

That is a problem but I would say it is a pretty rare use case. You could add a tiny shadow casting 3d object to your light to hack your way around it. It is not very pretty I admit.

I definitely hope Unity will improve command buffers though. They are pretty limited and not all of it works to make it worse.

 

Yeah a custom material that doesn't write into the depth buffer but comes up in the shadow search to get around it would work. Kinda the same way "standard transparent shadows work" in unity.

yeah I'm really liking command buffers, I hope they keep expanding on it.

 

This is great stuff, on one hand it's cool to see how much you can extend Unity, on the other it's not nice to see all of the limitations and issues with the builtin Unity system (like using a cookie for spot lights, seriously?). Keep up the great work.

Also, you say this system will only work with the standard shader, if I'm using something like Uber in deferred rendering, would it work then as well?

 

I just hope graphics plumbers like @Aras notice little topics like this tucked away, particularly as he asked for command buffer feedback at one point but I don't think he got it. Here is some good feedback from @Michal_ and @Lexie

 

Ok, I have the directional light working. It turned out there is a simple solution. No reverse-engineering required. I'll explain tomorrow. I'm dead now...

 

Well done, would love to hear it, There is part of me that wants a custom directional light, and another part of me that just wants to use the inbuilt one. I know a few ways I could speed up the directional lighting and handle large worlds with massive shadow distance, Similarly to how farcray''s lighting works. But I don't think i need it for my game. soooo maybe the inbuilt lighting + volumetric will be enough.

The biggest issue with a directional light is I wanted to increase the shadow rendering part to offset the cost of volumetric lighting, Similar to how my stationary lights work. I was thinking about a tiled system that updates one tile per frame for stationary objects. then the dynamic light would fall back to using the tiled system for anything past the dynamic cascade. this would mean i could have a really short dynamic shadow distance (reduce shadow draw call) but still have a large shadow distance.

 

Pretty much the holy grail. Unity needs this built in or frankly, open world kind of sucks with any sort of time of day.

 

Sorry it took me so long. Crazy weekend.

There is one key difference between directional light and point (spot) light. As I said I use LightEvent.AfterShadowMap to retrieve light's shadow map and render light's volume. The problem is that all the shader parameters related to lighting and shadows aren't set at that moment by Unity. I'm talking about parameters like _World2Shadow matrices, _LightMatrix0, _LightPos etc. So you have to figure out what exactly they mean and how to compute them in the exact same way as Unity would. And that is the tricky part since there is no documentation and some of the parameters differ on different platforms to make it more interesting. I mean it is easy to guess that _LightPos.w is 1 / (range * range) but getting the matrices 1:1 is more difficult. Doing all that for more complex cascaded shadow maps would be even more difficult. Thankfully there is an easy way out.

For directional light I still use LightEvent.AfterShadowMap to store the shadow map to a global shader parameter but I don't render anything at this point. I wait for LightEvent.BeforeScreenspaceMask instead. It turned out that by the time BeforeScreenspaceMask event is triggered, all shader parameters related to cascaded shadows are already updated by Unity. All you have to do is look into builtin shader source code and copy the part that reads CSM. You don't even have to know what CSM is to use it. Here's the code snippet for reference (it doesn't handle shadow fading)

Code (CSharp):

1. UNITY_DECLARE_SHADOWMAP(_CascadeShadowMapTexture);
2.  
3.         inline fixed4 GetCascadeWeights_SplitSpheres(float3 wpos)
4.         {
5.             float3 fromCenter0 = wpos.xyz - unity_ShadowSplitSpheres[0].xyz;
6.             float3 fromCenter1 = wpos.xyz - unity_ShadowSplitSpheres[1].xyz;
7.             float3 fromCenter2 = wpos.xyz - unity_ShadowSplitSpheres[2].xyz;
8.             float3 fromCenter3 = wpos.xyz - unity_ShadowSplitSpheres[3].xyz;
9.             float4 distances2 = float4(dot(fromCenter0, fromCenter0), dot(fromCenter1, fromCenter1), dot(fromCenter2, fromCenter2), dot(fromCenter3, fromCenter3));
10.  
11.             fixed4 weights = float4(distances2 < unity_ShadowSplitSqRadii);
12.             weights.yzw = saturate(weights.yzw - weights.xyz);
13.  
14.             return weights;
15.         }
16.  
17.         inline float4 GetCascadeShadowCoord(float4 wpos, fixed4 cascadeWeights)
18.         {
19.             float3 sc0 = mul(unity_World2Shadow[0], wpos).xyz;
20.             float3 sc1 = mul(unity_World2Shadow[1], wpos).xyz;
21.             float3 sc2 = mul(unity_World2Shadow[2], wpos).xyz;
22.             float3 sc3 = mul(unity_World2Shadow[3], wpos).xyz;
23.             return float4(sc0 * cascadeWeights[0] + sc1 * cascadeWeights[1] + sc2 * cascadeWeights[2] + sc3 * cascadeWeights[3], 1);
24.         }
25.  
26.         inline float SampleCascadeShadowMap(float3 wpos)
27.         {
28.             float4 cascadeWeights = GetCascadeWeights_SplitSpheres(wpos);
29.             float4 samplePos = GetCascadeShadowCoord(float4(wpos, 1), cascadeWeights);
30.             return UNITY_SAMPLE_SHADOW(_CascadeShadowMapTexture, samplePos.xyz);
31.         }

 

That would be great. Personally, I'm putting all my hopes into hw instancing at the moment. We're rendering entire forests in Ylands and the draw call count can get pretty ridiculous even without shadows...

 

For some reason i never tested BeforeScreenspaceMask map, I only tested AfterScreenspaceMask shadow map hoping that these values would still be set, turns out the scene view overrides them in AfterScreenspaceMask.

 

I want this SO BAD.

 

Can't wait either. This is exactly what we need right now ^^
If only Unity would come up with something like this.

@Lexie: You got a private message from me, regarding a question ;-D

 

So I've added volumetric lighting to unity's inbuilt lighting now. It uses the command buffer to capture the shadow maps and renders the volumes into a temporary texture. It works with direction/point/spot lights. To get it working you simply drop a script on your camera and a script on any light you want to be volumetric. The rest is automatic.

The main reason for doing this is my custom lighting solution I've made has too many downside to really release. Its perfect for my game so I think it will stay as in house tech for now. By adding volumetric support to unity's inbuilt lighting I can be assured it will work for everyone with out them needing to majorly alter their rendering pipeline/artstyle.

@Michal_ is also working on volumetric support for unity's inbuilt lighting. they've said they will be open sourcing it soon in this thread for people that are interested, chances are it will beat me to release.

 

I will watch both threads from now on
Just Hope one gets released soon.

But its nice to see that there are now 2 solutions in the work.
Also I understand the reason to keep your lighting Solution unreleased

 

WHAT?
/SLAP
But your custom solution had better shadows and no light leakage!

 

I too was looking forward to the better shadows and faster static shadows. :\ Other than that everything sounds good.

 

It's also faster... you can modify unity's shaders to fix their awful shadow bias if you want to remove the light bleeding and have better shadows. I'm not doing anything special there. Just calculating bias correctly. You can also change the way softshadows work so that could also be added to unity's inbuilt

 

Yeah I think it would still be worth for you if you release your solution for point and directional lights. Even if it does not improve directional lights to much and has problems with fully dynamic lights, I am sure people will buy it.

I know I would.

 

Ill be releasing a volumetric plugin for unity's inbuilt lighting with-in a day or two. Just cleaning up all the editor stuff now.
I was also able to refine my upsampling to allow acceptable quality at eight resolution. I think I have finally come up with a solution to render a lot of volumetric lights at acceptable speeds to use in game. It supports dx9 and above, works in forward and deferred path.

I'm thinking about a price point of 10-15$. I know there are other volumetric lighting solutions on the store that cost way more then that but I wanted to make sure the price was reasonable for hobbyist.

If/when my custom replacement lighting is at an acceptable level, I'll release that as a beta branch to this asset, There are just too many downside to charge money for it so at the very least I can offer the volumetric support with inbuilt lighting.

 

Sounds great, a plug-and-play solution that doesn't go too deep will be very handy for those of us who already have a custom pipeline. I will definitely be picking this up when it comes out!

 

This really sounds great. Will definitly pick it up and check it out.
And the idea with your lighting solution sounds great as well. Keep it coming ;D

 

It looks very cool and i will be sure to buy this asset if it's priced fairly.

Keep up the great work.

 

I ran into some issues with DX9 support so it set me back a day, all fixed up now. As long as my closed beta goes well I should have it ready by the end of the week.

 

Any idea if this will render properly in VR? We've tried a couple of the solutions from the asset store, and they looked great in the editor, but once in VR the effect really fell apart.

 

I would also be very interested in the performance in VR.

 

I don't have a VR head set to test it out. Going to borrow one from a friend to see how hard it is to get it working with it.

My beta is complete, just doing up the documentation and marketing stuff now... No idea how long this will take.

 

That sounds fantastic. VR is of course still a very new thing, so you should be fine in releasing the asset as long as you initially state that VR is untested.

When do you think we will see a release on the asset store? Dying to mess around with it

 

I would like to see it work in VR also,

 

Any expected date to have this available to all? I'm really excited for this asset.

 

I'm borrowing the new oculus over the weekend, shouldn't be too hard to get it working. It already supports multi cameras.

I should have all the marketing and documentation ready by the end of tomorrow. so that + how ever long asset store approval takes would be my ETA.

 

NICE!! Can't wait.

 

So impressed by the demo!!
Looking forward the asset.

 

I should really update that, So what I'm releasing isn't that demo, That demo is my custom lighting solution that is a lot faster then unity's inbuilt lighting. It has a lot of downside so its hard to release it.

What I'll be releasing in a few days is volumetric lighting for unity's inbuilt lighting. I think Ill make another thread for this so its less confusing. The upside is it works out of the box. but it just volumetric lighting, it doesn't have any of the nice shadows like my custom lighting has, or the massive speed improvements.

I ran into some bugs in the closed beta. I think it will set me back a couple of days unfortunately. Nothing major, I just need to do some more testing on every platform to make sure it all works.

 

Thanks for the updates @Lexie , be sure to link the new thread also here so that we can follow the development

 

I am still looking forward to your custom lightning in the future.
Lighting is a major part of any game, so even if it has some drawbacks, I bet people can live with them.

Right now unitys lighting just looks S*** and performs bad.

 

Thanks man, it's really cool, can't wait to see this release. are you going to release this for free?

 

Could you post a few of the downsides that your custom lighting solution has to see if those would be major issues for other developers? I'm guessing some of us wouldn't mind the downsides if the upsides outweighed them. Thanks!

 

Deferred only, doesn't interface with light probes or enlighten, no lod support, Lights will not light transparency (might be able to fix this). Every shadow caster needs a script on it to cast shadows. 20-80 mb of VRAM is recommended to put aside for the system to work at full capacity. You can assign zero VRAM, but then its only slightly faster then unity's inbuilt lighting.

 

I got a little delayed on the volumetric plug-in for unity's inbuilt lighting. Doing one last beta test this weekend to make sure its running smoothly.

Note: This is volumetric lighting for unity's built in lighting. This is not the same as my custom lighting engine.
Here is a demo scene with it running. Its a pretty un-optimized scene, I should just buy a stock scene...

All the lights in the tree area are shadow casting lights, These are the most expensive to render.
If you look behind you there are 54 none shadow casting lights. They are really cheap to render in my system.
This was one of the main reasons I was pushing for quarter or half resolution with really good upsampling. I think I've achieved the best I can with out resorting to down sampling the camera's normal buffer as well.

There are a few settings for upsampling, its generally a fight between signal loss with sharp rays or blurred rays with less signal loss. The settings in this build are blurred rays as I prefer that over signal loss.

This build is using DX9. I still need to optimize my DX11 shaders (it all works, its just a little slower).

Right now all the lights simulate Mia scattering and light extinction, Everything is adjustable per camera or per light. I really need to implement Rayleigh scattering to get correct lighting of really distant objects and simulate a skybox.

Let me know if anyone has any issues running this demo.

 

Maybe you could use Unity's courtyard scene, but strip it down a bit so it isn't such a huge download for everyone.

 

Hi Lexie,

TBH, after playing around with it, I feel that Michal_'s volumetric lighting solution produces phenomenal results, is very performant, and is open source. I probably wouldn't personally be prepared to pay for your volumetric lighting solution, after having played with the downloadable demo. I'm not at all implying that what you've made isn't fantastic and valuable, just that in comparison to other options, it's not viable for me personally.

On the other hand, the shadowing you showed off in the beginning of the thread is unmatched by anything else I've found, and would be incredibly valuable for my game (it uses loads of dynamic lights, in a fully procedural world filled with destructible props, ruling out unity's light probes and enlighten entirely). If you were to release that asset, I wouldn't hesitate to purchase it, even in consideration of the downsides you've mentioned.

 

If I was working at unity I would seriously evaluate granting some source code access to @Michal_ and/or @Lexie !

Especially nowadays with epic giving source code access to ... everyone !

Oh btw, there is a market for a "procedural games lighting solution" asset. On demand baked GI, optimised dynamic but non-movable lights ...

 

Hi Lexie, any new on this project? really like what you're doing.

 

Yeah... Any news on how things are coming?

 

You must have a pretty good GPU or only rendering 1-2 lights cause I've tried Michal_ solution and its unusable on my 970m GPU when rendering more then a few lights. The half resolutions helped but the artifacts it produced were unacceptable for me. Just remember your users wont all have good GPU's. 10-15$ for 2x+ the performance isn't asking much.

Sorry I had family in town and then proceeded to get really sick. My last beta is finished, I'm sending it to end night games (the forest) to test it in a larger world setting. if everything comes back ok then ill release shortly after. Sorry for the wait.

 

Hey, do you think this would be compatible with sonicether's GI?
http://forum.unity3d.com/threads/fully-dynamic-diffuse-global-illumination.314748/
Your thing and his thing together would make me a very happy Cat in the Hat.

--IronDuke

 

My guess on how its implemented it wouldn't be too hard to add support for it. It would actually mean you could do most of the volumetric lighting in one raymarch pass by taking advantage of the voxelized light data that the GI uses.

So no, not out of the box, but support could be added pretty easily if its the method of GI i think it is.

 

Nice! You'll have to give that a shot at some point. I probably won't be needing it for a year or so, but I'm just researching what I could put in the game I'm planning.
Looking forward to seeing how that turns out. In the meantime, back to developing me lousy pinball game.

--IronDuke