Screen space raytracing in Unity 4 WIP

Hi,

Sharing some progress on a technique I've been working on in Unity 4. Goes under a number of labels, Realtime Local Reflections ect. Depending on the developer/engine. It's basically ray-traced reflections in screen space. 16 steps/samples per ray and 1 ray per pixel. I haven't done an optimisation pass, or added any complex reflection and surface properties. It's pretty much just the raw framework on which I will build on.

Cheers
Brn

 

Very cool. SSRTR should be a lot faster than the traditional multi-camera based reflections right? If it is and you could package this as a Water4 replacement and include flow maps, you might capture a good market.

 

So far the performance is working out well. The captured scene runs at around 550fps @ 1900 x 1200 on a Nvidia GTX 660ti. Im sure I can speed it up further.

Compared to Multi-Camera solutions it is hard to say which method will be faster. Both methods suffer from various visual errors. SSRTR has issues with foreground objects occluding surfaces behind them and cant handle reflections for objects not visible to the camera. From my point of view the biggest advantage to SSRTR is how friendly it is in terms of art pipelines. Not having to pre-bake reflection maps or to set up multiple cameras is a huge production time saver.

 

Cool, I also implemented this a while ago as part of a custom renderer for unity. The nice thing about it is that it has a constant performance impact, all surfaces can reflect, and of course there is no setup on the content creation side, it just works. Its also easy and cheap to make it use distance based fuzzy reflections based on a deferred glossiness value and cone tracing.

 

Wow. Looks neat. I did wonder about how you could possibly do reflections coming from surfaces that are behind other objects ... and then you said about how that's a limitation. Not sure how that `looks` or whether people would notice in a game? Either way I'm very impressed with the performance.... thinking back to how rendering a scene like that on my old little Amiga 500 would've taken a DAY, things have really come a long way. I was also wondering how long it would be until someone came up with a ray-tracing-based pipeline for Unity rendering .... if you could be the `first to market` on that on the asset store, with good performance and ease-of-use, you could really make a name for yourself.

 

Wonderful work again Bruno, really looking forward to playing with that when it is ready.

 

Looks really promising! Does it rely on DX11?

 

Thanks for the kind comments guys

Quick update. Testing the shader as an image effect. There are some real advantages to doing it this way. Any deferred shader with a specular component will reflect using this method. I'll make some further tweaks and separate the blur pass from the reflection pass.

@ Jcarpay. Right now it works on both dx9 and dx11 using almost exactly the same shader. DX11 is about 40% faster for the same scene.

 

Great job! Do you plan to sell it?

 

Very nice. So does this give you automatic environment-wide shadows and reflections everywhere? And what about lighting - can you do automatic ambient-occlusion/soft-shadows type of stuff?

 

I'll second that question - are the shadows and occlusion in your video processed through your raytracing shader as well?

Also: For specularity-based reflexions, can it be tweaked by using a specular texture on materials?

 

Screen-space, guys. Renders only what camera can see.

I was waiting for someone to ask, but ok...
@brn, are you doing this based on NVidia's paper? If so, how did you make rays return pixel info in Unity?
Also, it can't be exactly realtime raytracing and still running that fast, are each ray computing physically accurate reflections and refractions?
1024x768*3 = 2.359.296 rays to compute at a single frame, for simple raytracing...
If you render a frame with ray depth set to 8, as it is physically accurate, on a 1024 screen,
that is 1024x768*8*8 = 50.331.648 rays;
That is 192mb for a single frame, 11.5gb of rgba data per second if running at 60fps...

How are you doing this?? Is this really real time raytracing? You are faking a lot of stuff using Render-Textures, right?
I mean, not even NVidia's demos use raytraced real time scene with so many objects, and casting shadows... If this is really RtRT you are a genius and should tell Nvidia to put you in charge of this stuff.

 

There are probably a number of optimizations being made due to simplicity of materials or something? Also it's screen-space so it can't `really` do accurate reflection and stuff because it can't access the content of `space` that isn't visible.

Hardware shaders on mid-high-end graphics cards can easily push out 20-30 billion pixels of fillrate per second .. e.g. 100 full-screen fills at 1024x768... so his algorithm is within those limits.

 

I gotta admit I'm extremely interested in this project, amazing work!

 

Please, you know that is stipulation made by manufacturers when they are simply printing random pixels on screen; These numbers are not true when it comes to compute paths of rays;

This is accurate RtRT, research is by lead developers of Pixologic's Modo; You can see there is no technology to date to process all that data in real time, not even using a bunch of Kepler GPUs or Cell CPUs. Average fps on this research is at 14 to 16 and all objects in scene are static:

I don't understand how brn is making raytracing in so smooth fps even not using refractions;
Even for just reflections as shown on his videos it still would not run so smooth like that.

 

Hi guys, I better clear a few things up. The only data that is being "Ray Traced" is the reflections. Everything else is being done by an almost standard unity deferred piplelne. That includes shadows specular ect.

I'm only Ray tracing into the Gbuffers supplied by the deferred system which is why its only screen space, this is also why there are holes in the reflection and issue with off screen data. Im doing 16 samples per ray and one ray per pixel, so its roughly the equivalent on a SSAO shader in complexity.

I haven't seen Nvidia 's paper but i will look it up. If you want to compare what Ive done to similar approaches try this link http://raytracey.blogspot.com.au/2011/08/ray-tracing-in-cryengine-3.html Now to be absolutely technical I'm actually doing ray marching, I use the term Ray Tracing because its better understood by most people. I intend to aim the package at the serious games market where a single step solution to get their scene reflecting properly in unity would be very appealing. Most of the architectural people are very familiar with ray tracing so i figure its a good fit.

Cheers
Bruno

 

I have no idea about this, but it looks good so great job

 

Yes it does give automatic environment wide reflections. I use the specular power to determine the reflection. I know that this is counter intuitive to many who are use to the gloss value being the determining factor for specular intensity, but for a material with light conservation in mind , specular power is all that is needed to describe the reflection intensity and to a degree the Fresnel contribution as well. As for AO and global illumination this is defiantly possible but i am yet to investigate these options. Many of the guys from the demo scenes have managed to do it. I take much of my inspiration from their ground breaking work.

 

i would love to have this and i have no idea how to do it, im doing architectural stuff but im a beginner in unity how hard is this to achieve ?

 

Reviving this thread from the dead because its well deserved! Will you be selling is? I'm extremely interested in not relying on cubemaps and plane reflections

 

i second that ! pls détails on how the project is shaping up ..
Thks

 

When can we expect to see this on the asset store?

 

Impressive ! Any release date ?