[WIP] SE Screen Space Shadows

Hello, everyone. I'd like to share my work-in-progress implementation of screen-space raytraced shadows in Unity.

The name is pretty self-explanatory. The screen-space depth buffer is traced against in order to provide sharp and detailed contact shadows and shadows from objects not rendered to the shadow map. In addition to providing better contact and self-shadows, this can be very effective for fields of grass or small detail objects like rocks where rendering these objects to the shadow map would be very costly.

Here are some comparison images of the effect (I recommend opening them in a new tab and switching between them to see the difference more easily).

















This implementation works by using command buffers to render screen-space shadows to an intermediate buffer and then blend it with the screen-space shadow mask for the directional light. As a result, both diffuse light and specular highlights are occluded properly by the screen-space shadows. Since point and spot lights don't have an entry point for blending with a mask like this in Unity, unless I find another way, this will only work for directional lights.

And, of course, any information not directly visible will not contribute to screen-space shadows.

I still have a few things to finish, I will provide performance details when this implementation nears completion.

 

Looks very nice and makes a better more meaningful contribution than SSAO, so if it's cheaper than Unity's SSAO I don't see a reason why I wouldn't be interested!

 

That grass looks delicious enough to eat.

 

It's another one of those "why ain't this baseline in Unity FFS" threads. Wondering how it's done - does it result in drawing all those meshes again or just cleverly sampling depth buffer and calculating from light position?

 

Thanks, guys!

Performance as of now, in that grassy scene, is about the same as Unity's "Screen Space Ambient Occlusion" built-in image effect at its default settings. I think I can squeeze a bit more out of it with the things I have planned.

Yep, it simply uses the depth buffer, there's no additional rendering of objects. Basically, in view-space, a ray is traced towards the light source and tested for collisions with the depth buffer--a similar principle to screen-space reflections. Because of this, the basic technique is independent of scene complexity, since all pixels are performing the same calculations.

 

Looks like the UE contact shadows that were just released, i was wondering if there was something like it in unity

 

I'd love to be using @tomaszek Uber and this I think. Even if only directional it'll be lush.

The link to UE4's implementation is very impressive, particularly the point light shadowing. I figure Unity needs scriptable render loops?

 

Really nice looking shadows

 

Very nice indeed! Will this work fine with SEGI ?

 

looks good

 

Yep. I would need some sort of way to inject screen-space shadows into the point light shading before it's blended with the rest of the lighting buffer.

Yep! I'm actually using SEGI in those Viking Village screenshots.

 

That is awesome! I'm really impressed with your work. Looking forward to seeing it released on the Assets Store.

 

That looks absolutely fantastic @sonicether
And once again, sonic comes with an awesome new asset. That grass looks really good!
Looking forward to more info and getting this.

 

Screenshots look nice (although the grasses shadowed side is maybe a bit too dark without translucency) but screenshots do generally look nice for a screen-space effect

How are you dealing with the issue of pop-in of shadows from items not currently on the screen?

 

The screenshot of the grassy hills looks truly impressive. I'm interested in this asset, but I'm going to have to sincerely request that SESSAO owners be offered a discount for this release. Moreover, it would be nice to see similar screenshots without any additional post-processing added to compare against, e.g. only SE screen space shadows vs. no image effects.

 

Will this require unity 5.x?

 

Cody, how're doing? Sorry for your family member that's battling against cancer. I'll be praying for him/her.
Just wanted to ask if its possible to add in Unity a version of Nvidia's HFTS? I'd love to see that feature. ^^

 

Looks extremely nice! Finally there are solutions coming that will make Unity grass look comparable to UE.

 

Literally have no words for how great this looks

 

Absolutly stunning images! I will definitively buy it!

 

HULK WANT

--IronDuke

 

But it's out of the box in UE4

 

Well UE4 has a lot of stuff we would want to have out of the box... Epic knows what their developers want. ("We've also introduced a new automatic LOD generation feature for static meshes that does not require a third-party library.")

So I am really happy, that we have at least some amazing developers like sonicether in our community when the Unity Team can't get stuff done.

 

Unfortunately with screen-space effects there's really no way to get around this, though things can be tweaked to minimize the visual impact (like tracing z-thickness and a parameter that adjusts how smoothly or harshly samples fade out as they get to the screen edges). I've found that generally these issues aren't very noticeable unless you're looking for them. If you've ever played Skyrim with ENB Series, you'll have an idea of what screen-space shadows can do for a game (particularly for the grass) and how generally unnoticeable the pop-in artifacts are.

I'm looking into how to do this through the Asset Store. I think that's a great idea.

Yep.

I just took a closer look at HTFS, it looks like it's pretty hefty performance-wise (4-6 ms). I personally think that simple traced screen-space shadows are a better option since those milliseconds could be put to better use, like real-time GI


Oh, also, today I managed to speed up the tracing by about 2x! At high settings, it's about 70% the cost of Unity's Screen Space Ambient Occlusion at default settings, and at medium settings, it's about the cost of FXAA. It is noteworthy for me to point out that I haven't added a bilateral filter yet for smoothing out noise. Once that's in and enabled, it'll add to the rendering cost a little bit.

I do highly recommend using temporal anti-aliasing with this effect, since it can really clean up edges and flickering from movement. I've also added an option to leverage TAA by randomizing the jittering per-frame which smooths out noise and artifacts without having to add blur, which keeps the shadows extra sharp. Don't worry though, I will add bilateral blur for those who dislike TAA.

Here's without TAA


And here's with TAA and jitter frame randomization

 

Just figured I'd drop a few more screenshots here. It works surprisingly well with distant objects!







Also, here's a comparison with no image effects (no TAA either)

 

Faster the better.

 

As with all your work this looks stunning.
Assuming this will work ok for mac?

 

I can't personally test on Mac, but I can make sure it works for OpenGL.

Also, I just tried it out with UBER's z-write POM functionality, and the results are quite awesome.





So, any shader that writes to depth will have appropriate screen-space shadows!

 

what about transparent materials ? they have issues with depth and AO

 

I has suspected and hoped this would be the case and the results look very sweet.

 

Wow! Nice looking shadows. Say goodbye to "Peter Pan" effect I'm waiting for this asset!

 

I know this is not the most important thing at this stage, but could this effect be used to fake the way real life shadows work (the further from the shadow caster the shadow is, the smoother the shadow is) by sampling the distance or something similar? Even though, this fading is not really necessary since it does not quite work in off screen-space.

 

Add me to the "Wow!" list...looking wonderful, can't wait.
I know it is still WIP but wanted to ask about something.
I noticed translucency/transparency issue in one of your screenshots.
Looks like SS shadows effect translucency or transparency. It might be a problem with the grass shader but just wanted to point out.




Hope everything goes as you want!

 

Well, the screen-space shadows do a lot to harden contact shadows, so in that regard, they help with this effect. Combining screen-space shadows with PCSS would probably give some great results.

That issue actually has nothing to do with the screen-space shadows, that's actually an issue with SEGI (the global illumination I was using in these images). It's just more apparent when the grass has self-shadows on it. I'll have to look into that!

 

Really excited for this. Any idea on release date? Will you be selling this on the Asset Store?

 

This will be so perfect for tiny shadows in a spacecraft's cockpit, since the normal Unity shadows would be more used to let the ship cast a shadow on a moon 50,000km distant. When you have that big a distance, keeping good detail in the cockpit is a pain, even with shadow cascades and multiple cameras. Keep it up man! This is incredible! \o/

--IronDuke

 

Amazingly beautiful #_#

 

Yep!

 

Hey sonic ether, does your screen space shadow support WebGL? Currently unity shadow in WebGL looks pretty bad.

 

Looks fantastic. Great job as usual.

 

Hey everyone, I'm just wrapping things up and figured I'd give some comparison videos regarding noise, Temporal AA, and bilateral blur.

As I've said before, I highly recommend Temporal Anti-Aliasing to improve the look of the screen-space shadows. Anyway, here are the comparison videos.

No TAA or blur: http://www.sonicether.com/wp-content/uploads/2016/11/sesssnoblur.mp4
TAA: http://www.sonicether.com/wp-content/uploads/2016/11/sessstemporalaa.mp4
Bilateral Blur: http://www.sonicether.com/wp-content/uploads/2016/11/sesssbilateralblur.mp4

TAA really helps with shimmering from aliasing and helps to filter out noise. Of course, the bilateral blur is still an option for those who would rather use that. However, since the bilateral blur itself takes some render time, I'd personally rather put that into tracing more samples for a cleaner result, since blur would arguably defeat the purpose of sharp contact shadows in some situations. Ultimately you'll have the choice as the user!

Since Unity 5.5 was just released and in 5.5 the depth buffer is inverted, I'll have to make sure everything is working for both 5.5 and 5.4 and under, so that'll take some extra time, but I'm almost done. I'll keep you guys updated.

 

Hello Sonicether,

What Temporal Anti-Aliasing solution are you using?

 

How would we distinct between realtime shadows and contact ones - can they be mixed properly ? On your bottom most screenshot it's hard to say wheter we see contact screenspace shadows or self-shadows (as an option in POM when we _don't_ write into depth or realtime shadows generated by writing into depth on shadow caster). I'm wondering, because with UBER's POM z-write I've got shadow caster passes. To save GPU processing time (POM is expensive), would be fine to write into depth only for main buffer, use contact shadows for slef-shadowing, still having main object correctly casting shadows. As it casts realtime shadows on itself - what's the result then ?

Tom

P.S. Suprisingly I found that with inverted z-buffer U5.5 brings POM with depth write works out of the box in UBER (phew... was preparing myself for hard fight).

 

Does this only work with directional lights, or point and spot lights also?

 

I'm using a custom solution that I've been working on in my spare time, but Unity's new TAA should work fine.

So, I'm not quite sure what you're asking, but those screenshots show the z-write POM material with modified shadow caster depth. The first image of each pair shows the result without screen-space shadows, and the second shows the result with screen-space shadows. With the way I had things set up, the directional light shadow resolution was such that it couldn't properly provide self-shadowing on that material alone, which is where screen-space shadows come in to provide more detail.

If modifying the shadow caster depth was disabled, I think you would run into issues because the depth modification in the main pass would have some areas of the surface "behind" the shadow map depth which could cast incorrect self-shadows.

It only works with directional lights, since there is no entry point for me to combine a screen-space shadow result before adding a particular point or spot light to the light buffer.

 

Any ETA when screenspace shadows will be submitted to asset store?

Also, would you consider selling your custom TAA on the asset store?

 

I was rather wondering about realisation of screen-space shadows mixed together with realtime shadows. For UBER objects that has POM z-write shadows produced by Unity realtime shadow casting/receiving has lo-freq. So - your screenshot shown screenspace hi-freq shadows. You see no real chance to skip expensive POM calulations on shadow caster still could you tell something more on mixing together realtime shadows with screenspace - do you deal with collected shadows texture to add your shadows ? And - most interesting for me - performance. We've got pretty GPU intensive vegetation in our game. Skipping shadow casting on it would be huge benefit as far as your solution will be faster and good looking. I'm a bit worried about necessity for filtering. There are still no perfect TAA solution available, but I've got hope on incoming Livenda's solution.

Tom

 

Not that much on the topic but isn't Livenda's CTAA solution available at half price as "Launch Event" for a long time now only trough their web site... I was hoping to see CTAA on the store as well but seems it won't get available there...

P.S. Excited to see "SE Screen Space Shadows" in action !

 

Should be within the next few days.

Sure, I'd consider selling my TAA. It does use a novel technique for handling recently unoccluded pixels (no more streaks behind objects on detailed surfaces). There are a few areas that could improve still, though.

So, after unity's realtime shadows are gathered into a screen-space buffer, my screen-space shadows are simply multiplied with unity's shadow buffer. My screen space shadows have an adjustable "fade out" to ensure that they blend well with the traditional shadows.

 

@sonicether - Any idea if this could possibly be a solution for casting shadows onto alpha-blended water? I imagine the answer is no, but I figure I should ask.