Raytraced Diamonds in realtime in Unity with lightfields (WIP)

Hello!

I would like to share a demonstration of incorporating lightfields inside traditional 3D scenes in the Unity 3D engine. The diamond images are rendered using Blender's Cycles pathtracer into a lightfield and then new views are synthetized and blended with the scene goeometry within the Unity scene in realtime.

Essentially this allows you to raytrace (and later photograph) any object (or scene!) and throw it into Unity (or other engine) and have it behave just like another object within the scene in realtime, even if the raytracing of the lightfield takes a whole day on the GTX980 (like in this case)

More info coming soon!

 

Wow, astonishing work!
I am SO looking forward to lightfield tech becoming more available.
I hear OTOY will be making their renderer available for free in april (for a promotional duration).
A couple of questions:
Why are the contours of the diamonds inconsistent ?
Will it be possible to add transparency and refraction?
Also, I think some ao would have been nice.

 

Thanks. This work will be available for free use as well, including the engine integrations and lightfield processing tool chain. What I'm working on is open lightfield exchange format and associated processing software.

The contours are inconsistent because it uses earlier version of the rendering shader, which didn't have polished edge rejection yet (and has some other subtler things wrong). I have a newer version of the shader which renders much better edges (and fixes several other things).

Regarding transparency and refraction, yes, in fact that's one of the things I'm working on. Transparency is primitive, that's just adding another channel. Refraction is a little bit more complicated, but I'm working on refraction capture for lightfields, which will provide simplified refraction in realtime (translated into UV shifts).

Since the shader writes correct values into the depth buffer, you could use SSAO with this.

 

This is VERY impressive.

 

Sorry for the late reply, thanks very much for answering my questions.

Lightfields really are taking gfx to the next level.
So far I only see data size as a real obstacle, especially they would be animated.
Are there any other drawbacks that we should know about ?

 

WOW! Amazing!! when its out?

 

interesting stuff.

 

Seems awesome! Looking forward to it.

 

Amazing! This is going to be so cool for VR, would really like to start playing with it. I'm guessing it should be possible to create a 360 (real) camera rig that can autorotate arround a fixed center and take enough pictures following the surface of a sphere. Create the lightfield from that, load it up in Unity and get the holodeck feel using the Rift or Vive. (positional tracking)

Would love to start working on that demo.

Any idea on how you're going to distribute it? (Github repo?) Would you welcome help?

 

@Tom Goethals - Not sure if you noticed, but there was nothing more from the dev on that in a little over a year, and he hasn't been online with Unity forums in about half a year. You found an old thread.

 

I did notice after writing the post But i really don't understand why not more people are really excited over this tech. OTOY are doing the same kind of thing with their lightfields but also nothing is out yet. let's hope he re-surfaces, i think he's working on another really cool VR demo. ("Sightline the chair" was my favorite thing to show in early DK1 days)