[Next-Gen Soft-Shadows 2] Sophisticated dynamic penumbra Shadows for Unity

@Syberam or Shader Forge, just FYI

 

Shader Forge exists, but it's more expensive, generates less readable code, and I think no longer gets updates? I know the developer of a water asset I use that was written in SF is porting it to ASE because he's worried about compatibility between SF and new versions of Unity. I don't know if it's officially shutting down or if he's just being cautious.

ASE has some of the opposite problems, it's very young and maybe not quite as polished or full-featured in some places, but the developers are very active. There are some other node-based editors out there too. I don't mean to shill for Amplify, if you've got time to comparison shop you should. I ended up with ASE and I'm super happy with it.

It could be, but it isn't really possible without rewriting a ton of lighting stuff. For, say, a spot light, it first figures out if a point is in the cone of the light. If it is, it samples a shadow map, which is basically a depth texture from the point of view of the light, to see if the surface is the closest opaque surface to the light (and thus should be lit) or if it should be in shadow. If it's lit, it figures out how brightly lit and adds that to the existing light. If it's in shadow, it just doesn't add anything. It never takes light away.

What you're talking about is subtracting some light from surfaces that are in shadow, that is, surfaces that could be illuminated but aren't. It's mathematically possible, and I don't know enough about Unity's shadow system to tell you if there's anything to stop you trying a literal implementation. I'd expect it to do pretty crazy stuff most of the time at the very least. You probably only want it on directional lights, it'll draw attention to the edges of spot and point lights in a weird unpleasant way.

You'll also have to tackle how to get that "shadow darkness" information into the system. If you're modifying the default lighting, you can just add a little bit to the shadow strength probably, but NGSS uses that variable to control softness instead, so if you're modifying those shaders you'll have to hardcode the value or find some other way to sneak it in.

 

@Syberam
Darkening shadows artificially is not correct because shadows are just an absence of light, it's not a dark spot or texture that we put behind objects. So what remains if we remove direct light = ambient (also reflections). What remains if we remove ambient = pitch black (darkest shadows you can get = 0).

 

You can however artificially intercept Unity lighting functions (get ready to dig plenty of internal cgincludes) and clamp shadows attenuation to some value or substract it from the resulting light attenuation.
To do that, you have to override UnityStandardForward, UnityStandardForwardSimple, UnityStandardSetup, UnityStandardVariables, UnityStandardCore, UnityStandardCommon, AutoLight and UnityStandardShadow
Basically, gets the shadows attenuation function and clamp it to some desired values and then pass it to the PBS function. Is not hard, but you have to moddify a couple of files. I did this for a client not long ago (can't share his files though but that's how you darken shadows artificially).

 

@Syberam,
If you know how to read/code basic shading shot an email to support, I will explain how to do it and the exact files you need to modify.

 

@brisingre
Modulated (aka Colored) shadows with the current version of Unity renderer is tricky but definitely it's pankake with my custom renderer, will take note.

 

I usually do them with two lights...

 

Outch, that's gotta hurt. I hope you are not using current unoptimized forward renderer doing that. ^^

 

I mostly only use it with Deferred where the cost isn't too high, but for a few effects, unfortunately, yes.

 

@brisingre
Deferred wasn't a problem until 4K. Huge fat buffers at 4K+ resolutions are indeed a problem. ^^
That's why the urgent need of new custom renderers.

 

Yeah, and it's really not better than okay at lower resolutions. I don't like Deferred. Performance is alright, but it's super limited, so you end up having to use Forward for half your rendering anyway. Everything transparent and anything with a custom lighting function ends up using Forward, so you basically just have two renderers in your project. Then there's all the little inconveniences like not being able to clear a camera to another camera easily, and not being able to use MSAA. Most of the time being able to have loads of non-shadow-casting lights isn't all that useful, and shadow-casting lights aren't really any cheaper. It lets me do some tricks like faking colored cookies or colored shadows with multiple lights at not too much cost, and some cool effects like particle lights, and perhaps most important it means I don't have to worry about lights flickering between vertex and pixel lights, but I'm not particularly happy with it. I just can't live with the per-pixel light limits of the built-in Forward most of the time.

 

Lighting's never really been my biggest performance problem though. Thousands of procgen fish with gigs and gigs of individually-unique textures are my biggest performance problem.

Unrelated, another lighting feature that would be very valuable to me in a custom renderer is the ability to have much larger point lights. Even with NGSS, shadows on point lights start to look bad past like radius 25 or so because Unity won't render cubemap shadows larger than like 256 or something, and I run up against this limitation all the time when trying to light up interior spaces quickly. If I want reasonably even, soft interior lighting that isn't the same in every room, big point lights are the best way to do that quickly. If they don't cast shadows they bleed through walls, if they do cast shadows the shadows have resolution problems.

 

@brisingre,
That won't be a problem with shadowmap atlas in my custom renderer. You are sharing ~16K (or higher if hardware allows you) for all your interior visible shadows. It will allow you to use large parts of that atlas on point light shadows.

I've tried multiples shadowing techniques, none of them are perfect, Exponential and Variance shadowmapping have tons of light bleeding and leaking. The one that standout a bit are Moment shadowmaps which are kinda similar to Variance Shadowmaps, produces nice pre-filtered shadows but suffers from similar lights bleeding/leaking problems. The one that has less light leaking/bleeding but is the heaviest to compute is the old PCF.
All of them shows similar depth precision issues (acne, panning to avoid acne, etc).
So if you wonder why Unity didn't adopted any of those shadows type, there you have the answer.

Right now, I'm reworking the noise algorithm and optimizing depth fetching with various depth edge dilatation techniques to increase the tap number and have nicer/softer shadows for free.

 

Folks,
I'm experimenting with some pre-computed noise patterns. Precomputed noise has the advantage to store heavy calculations in a texture and provide stable patterns (specially upclose).
Demo video showing on the fly noise (default) vs new pre-computed noise:
https://s3.amazonaws.com/uploads.hi.../zDsLacgGvM6hJaw/PreComputedNoisePatterns.zip
Let me know what you think and if you want this as an option or by default (this will apply to PCSS and PCF).
Others noise algorithms underway (testing out blue noise right now).

 

Looks really good, waiting to see blue noise. Out of curiosity, how do you determine the onset and fade ratio of the shadow? Since it's a filtering technique, I assume you just take the original shadow and filter it; I'm not sure how this compares with reality, but from your earlier post, I assume this is the best strategy.

 

@buttmatrix,
It's done in two steps:
- First step, we average depth distances using a blocker search algorithm with a large kernel so second step won't fail (this provide us the distance between shadow caster and shadow receiver).
- Second step, we sample using a PCF algorithm and a kernel size that will change based on previous step distance between caster and receiver.
There's other optimizations such as skipping lit pixels or fully shadowed pixels but that's basically PCSS filtering in two easy steps.

 

Definitely looks good. I think it makes sense to have as default, but I wouldn't want it to be the only option. It looks like the tradeoff is between the high-frequency buzzing we get right now on the edges of shadows and some minor banding in the same sorts of places, is that correct?

I look forward to seeing blue noise. Blue noise usually looks good.

 

@brisingre,
Yes, with this filtering you get both, a bit of banding and noise. It also saves some trigonometric computation as those are already pre-computed in a texture. I'm experimenting with blue noise and will post something very soon.

 

I'm not seeing any difference in the shadows after following the setup procedure. I'm attaching a shot of one of the sample scenes. Would this be because my target platform in the project is set to mobile (iOS specifically)?

Thanks!

 

@alanmthomas
Unity has a bug that over-writes any custom replacement shadows on mobile. The bug has been reproduced but I'm not sure when it will land on Unity. I'll try to poke them out and see where they are.

 

Okay. So I won't see any change in appearance until that is fixed? Good to know.

 

@alanmthomas
You can switch platform (on example project) to Desktop and see shadows difference but on mobile it will show regular shadows.
Also, based on your image, you should enable cascade shadows, by default disabled on mobile (Graphics Menu). Of course if you can afford it, it has an extra bandwidth overhead.

 

So I saw Unity 2017 was released, and in the features it says it now has Percentage Closer Filtering (PCF). So does that mean other than contact shadows, NGSS isn't really needed anymore?

 

@ceebeee
Unity have been doing PCF since forever. In version 4 and 5 they added a bit more of PCF taps to make shadows a bit softer though.
NGSS does PCF at the final stage of PCSS filtering (or if PCSS is disabled entirely). Plus on top of that it also does hardware 2x2 linear filtering.

 

In addition to what was said above, NGSS offers you very granular control over the filtering that's applied to the shadow maps; Unity does not enable this out of the box besides hard vs. soft.

 

@buttmatrix
On top of that, Unity softshadows are hardcoded bilateral-blur shadows, you can't even tweak the values to make it softer or harder as it breaks shadows entirely.

 

Ok thanks, I really wasn't sure since Unity really didn't explain what was new or how to use it. but Now I see why. Thanks.

 

@TheSpecialDuckling
Yeah, thats the new directional Filtering (PCSS). I've already fixed that in v1.4.2 coming this week.
I'm finalizing some things up and will submit to the store.

 

@TheSpecialDuckling
Please, take a look at the visual improvements coming this week:

Slightly higher quality video in google drive:
https://drive.google.com/open?id=0B-BavlRTdAfjTkhmWk8ySllNNUk

 

@tatoforever aren't those banding artifacts due to the cascade blending bug in 2017.x?

 

@buttmatrix
No, those artifacts was me over-optimizing quite too much PCSS filtering but i got it fixed already.

Cascade Blending works fine, it's just that due to Unity culling aggressively shadows in v2017 that it cut out quite too much and shadows blending becomes white stretched at some point (enable it and see how it looks). I'm installing final 2017.1 version and see if it got fixed. Otherwise, I'll submit a case bug.

 

Hi

Bought NGSS some days ago because I was impressed with the variable penumbra feature...
Now, I haven't added the asset yet to my project, but then I am still on 5.5, most probably waiting for Unity 2017 to upgrade to a newer version.

I'd like to know what kind of performance improvements I could expect from using NGSS in my scene to replace the normal unity lights, especially in combination with the performance improvements promised for Unity 2017.

My current scene is rather heavy on tris, as its a room built out of finely tesselated rock. I have 8 realtime point lights in the room lighting the scene.... rather heavy, I know, but then I am also using volumetric lights, which of course only work with realtime lights. Most of my bad framerate at the moment (apart from the scene being completly unoptimized) come from the volumetric lights raymarching the scene, but besides that I noticed that the light shadows increase scene tris tenfold, going from 180k tris (which I would call reasonable for PC) to 2 million (which, while still running fine on a modern GPU, is rather high). Now I do not expect this scene to perform exceptionally well without a lot of optimization, nor am I trying to optimize it right now... I just don't want to work myself into a corner by early decisions like having to use realtime lights because of using volumetric light when the realtime lights perform so poorly in my case.

I already know that Unitys lights have a pretty expensive shadowing implementation for realtime lights (and a baked shadow format that obviously does not work for volumetric light implementations)... what kind of improvements does your system bring to the table? Can I expect your shadow system to increase tris count less than Unitys out of the box lights realtime shadows? Is this part of the performance improvements promised?


EDIT:

Forgot to ask one additional thing... Unreal Engine has that nice feature, distance field shadows, that should allow some basic dynamic shadowing outside of the realtime shadow cascades as far as I understand it, which of course is extremly important if you have an outside scene with fully dynamic celestial light (because of day/night cycle or the mentioned volumetric lights for example)...
Does your asset offer something similar to fill the view distance with some basic approximative shadows outside of the last shadow cascade? Is something like this planned?

 

@gian-reto-alig
Hi,
Current NGSS version do not bring any noticeable performance to the table, it won't make shadowmapping cheaper. It will make them look nicer and softer. However, reducing shadowmap resolution makes them cheaper to compute than default Unity soft-shadows (if their ress is higher) and still look better, specially for local point/spot lights (which is what you should do to improve performance a bit).
As far as promised performance improvements on actual NGSS product (coming soon) yes, edge dilatation depth will make them faster but I will use that free computation on making shadows a lot nicer so there won't be any noticeably performance improvement (unless you lower shadows quality of course). Future quality sets will be PCF 8*8, 9*9 and 12*12. Edge dilatation will reduce the number of shadows computation by a big large marge because it only focus on penumbra region, any pixel outside that area it's either fully lit or fully shadowed.

As for distance field shadows, this requires asset pre-computation and it doesn't play nice with rapid changes in lights/shadows, works better for static lights. Besides the results are incredibly awesome yeah. Although with some sort of dilated distance buffer, PCSS can become incredibly cheap and can even be computed at screen space.
We are working on Sample Distribution Shadowmaps for NGSS 2.0 that will allow you to display large distant shadows with little to no loss in shadows quality at all using logarithmical depth distribution. Other than that, capsule shadows and projected shadows (this one is ultra fast and ultra nice to compute).

 

Now noise patterns won't randomly jump when moving the camera, they will map to screen pixels.
Stabilizing noise patterns improvements:
https://s3.amazonaws.com/uploads.hi...jNF2Qhikgs8mjj/Stabilizing Noise Patterns.mov
I'm finalizing few things with optimizedPCF disks and we should be good for this week update which brings massive visual and speed improvements.

 

Best yet, I think.

 

Forgot to mention that I've added a banding/noise value that let you tweak the amount of banding and noise eg: 0.0 = banding and 1.0 = noisy.

 

Excuse me, I might have missed that (haven't read the full thread to be honest), but is NGSS only working with realtime, or also with baked shadows?
Given that Unity lightmapping AFAIK does not really take variable penumbra size into account yet (though my knowledge on that might be outdated, haven't really messed around with the lightmapper a lot since Unity dropped Beast), and with mixed lights back in the mix, that might be an important thing to have.

 

I would also be curious about the above topic in relationship to light modes

But, after going through the documentation and replacing the shader, deleting the shader cash, I can't get this to work with Steam VR. I can't enable the script on Camera (eye) after I drag in the Directional Light with the correct Directional Light Script attached. I can't check the box on the camera script.

Any ideas?

 

NGSS works with any mixed light mode and baked shadowmaks yeah.

 

@sbabb,
Can you please write to support email and include more detail about the way you are setting it up (video if possible)?
Regards,

 

I understand NGSS affecting shadows for dynamic objects in mixed mode, because they would receive precomputed GI, but will NGSS affect fully baked shadows? For example with baked shadows, it is possible to adjust the baked shadow angle. Will NGSS stack with the baked shadow angle, i.e. can you exaggerate the baked shadow angle more with the NGSS filter in the baked lightmap?

 

@buttmatrix
Yes, example showcasing NGSS point light shadows and shadowmasks:

This an old version of NGSS but shows the mixed light modes with NGSS shadows in the works.

 

Right, I think what I meant is that for fully baked lighting, there is no 'strength' option on a light component, it is replaced with 'baked shadow angle', therefore NGSS has no way of interfacing with the baked shadow angle; there is only 'intensity' and 'indirect multiplier'. For mixed lighting, this is not true, because dynamic objects still receive realtime shadows.

 

I don't think NGSS interferes at all with how the lightmappers bake shadows.

 

Right, I was saying NGSS cannot interface with baked lighting. For example, if a directional light has a 'strength' of 0 or a strength of 1 - more or less NGSS filtering - the resulting (fully) baked lightmap will not change, because baked shadows from a directional light are only affected by baked shadow angle; strength is not an option when the light source is set to baked.

 

@buttmatrix,
Yes, you can mix static baked lighting (indirect lighting and static shadows) with NGSS real-time shadows but you cannot bake the resulting NGSS shadows onto static lightmaps/shadowmaks. And that's unfortunate cause Enlighteen default baked shadows are garbage, as a comparison (left NGSS dynamic soft-shadows, right baked soft-shadows):



I mean, it's possible to have nice looking baked shadows but prepare to explode you scene with tons of high resolution lightmaps everywhere.

PS: And yeah, Shadows angle is just a property to define how soft baked-shadows will be (It's not exposed into real-time renderers though).

 

Exactly, I use NGSS in mixed mode a lot, and yeah, even when Enlighten works, it takes up a ton of memory :/
PLM is great though! Looking forward to mixed mode with PLM.

 

@buttmatrix
I think Unity is implementing an offline Monte Carlo renderer for PLM. This will works a lot better (and will look very similar to NGSS) because NGSS render shadows in a similar fashion, shooting rays randomly (using some noise patterns to alter ray direction). Another good example using noise patterns for rendering is Octane Renderer.
Anyway, I'm still working out some PCF filtering optimization and we should be good for pre-release tests (I will take a look at your VR thing too).

 

Yes, please do, the grass in our nature scene looks horrible without NGSS contact shadows

 

@tatoforever I just checked out that directional improvement video and was wondering; what extension or setting is it for the minimap like overview in visual studio?