Jove 2.0: DX11 Rendering System (Alpha Release)

Yeah Jove does its own cascaded shadow maps. It is basically cascaded exponential shadow maps.

Very little compared to what it would be without Jove. The cost of lighting is completely decoupled from scene geometry, meaning that doubling scene geometry does not increase lighting cost. LazyGunn gave me a scene with 2.2 million triangles that renders in ~80fps on my GTX 560ti (4 generation old midrange card), using the full range of Jove features.

The baked light is only indirect lighting, so dynamic objects can cast and receive shadows without any issue. The Lightmap in those screenshots is 512x512. I am in the process of implementing light probes for dynamic objects (this uses the default Unity light probes, except that I'm filling them with my own values. Their triangulation and interpolation is really solid so theres not much to improve there).
Cool idea about the logo, I'll think about it

 

Given that the time to compute the indirect lightning does not take that long, do you think it is possible to precompute lightning for a semi dynamic look?

I.e. bake it for every instance that your sun light could be in, for every time of day, and blending between the different instances of the indirect lightning bakes?

Rendering time should still be rather low, considering that it is only really needed for the SunLight which also only travels along a predefined path. Even if it is up to one hour on a medium sized map, that would be good enough, because such a semi-dynamic indirect lightning solution is still a great way to improve the looks of the game.

 

Wouldn't it be easier, since sunlight moves slowly, to just run the direct light and have the indirect lighting update behind it.

 

Baking is looking amazing, really looking forward to it (especially the speed, 10 sec on a 560ti :O, imagine what my 660 can do, or a 980 for that matter)

In addition with the timing of release I should have given a better timeframe, do you expect shadows for point lights within this month, thanks!

 

Yes, I think that could work too. You would just have to slowly blend between those 2 stages, so there is no huge sudden jump. Especially on newer hardware computing this should take 0 time...

It would be amazing if it was possible for any solution that makes indirect light dynamic, or semi-dynamic. Your Jove system would be far ahead anything else, even ahead of Unity5...

 

You mean just blending between a bunch of lightmaps? You could do that in standard Unity if you wanted to. Unless the bake times is whats stopping you, then this solution is gonna help. As for making updating the indirect light at runtime perhaps in a few generations. It isn't viable for now, at least using this approach.

The shadows are going to be baked in with all the new lighting features, and that is unlikely to be finished this month. Releasing those on its own without any of the other changes would be hard, there has been a lot of refactoring.

 

Actually, speaking of baked shadows - would it be possible to bake shadow maps from static light sources to relieve the system from updating them every frame? I am of course assuming the case where absolutely no shadow casting dynamic objects are required. Let's say the shadow map is calculated once per shadow casting light source (maybe on level load, maybe on last light update or creation, maybe on Jove Camera approach within a certain radius) and then stored with it, as some sort of per-light lightmap.

How feasible is that? I'm mostly thinking about it for background lighting of areas that can't be reached by the player, and for FPS games without the need for actor shadow casting.

 

Yes, exactly. Standard Unity is to slow, baking takes long, and your solution is nicer. It would be nice if we could blend between several light maps.

 

I'm not really talking about lightmap blending at all if you're about my post

 

Oh, no. I was replying to Aieth. Sorry.

 

That would be feasible. However, as I'm not the one working on it I'm gonna poke @Dolkar to give you a better answer

While that would work, hopefully the dynamic GI mode is going to be able to do all that in a much better fashion

 

Hey there, I'm the 'indie dev' helping Aieth with Jove from time to time... currently taking care of the point and spot light shadows whenever I can.

Performance of shadows is a huge concern for me. In many cases, this part can take the majority of rendering time. I was considering baking the shadowmaps. While that works for static scenes, it breaks down if you want to have dynamic objects casting shadows as well. I found a better way, though, in a presentation I can't find right now For static lights that don't move, it will bake all the static geometry that would affect the shadowmap into a single mesh. Using that mesh for rendering the shadowmap instead of the actual objects will massively reduce the draw calls, vertex count and overdraw. Another benefit is that you can render all the dynamic objects over it as well, so it should look exactly like a normal shadow mapping for lights that don't move, but hopefully a lot faster.

That's just one of the optimizations... The main philosophy is to not bother doing stuff that makes no difference, like updating the shadowmaps when they don't affect the scene or when they don't capture any dynamic objects, and of course decreasing the resolution and updating less often with distance. Treating point lights like 6 separate shadowmaps also helps.

As for when it's ready... I don't know. It should be usable quite soon, possibly in the next patch, but missing all the features I just listed.

 

Sounds brilliant. I agree that approach that makes no specific requirement to mark lights as static and drop dynamic shadowcasting support from them, while still optimizing cases where it's unused is the best - godspeed with that!

 

A nice picture using Jove I found in another thread, it's from the modular sci fi pack

 

The irregular fog patches look very cool. Does it work in VR though? If not, that would be a feature request.

Another feature request: support for TrueSky.

 

I heard someone in this thread had already ported true sky to Jove,its on one of the earlier pages of this thread.

 

That was me. While I did get it rendering it wasn't useable by any means. I was planning on fully integrating it but due to a few factors (running behind in current project, Jove being in heavy development and Aeith planning on doing his own thing with clouds down the track) I am currently not working on it. I definitely plan on finishing it off later on but I will wait until both plugins have matured a bit more.

 

https://www.unrealengine.com/blog/unreal-engine-45-released

Woah, shadows here look pretty interesting. Quote:

There is a new dynamic shadowing method which works by tracing a ray through mesh distance fields to the light. This allows for soft area shadows with sharp contacts!



Here is a directional light with ray traced soft shadows. Long shadows get softer and self-shadowing in the distance is still detailed.

Point and spot lights work too. ‘Source Radius’ is used to control how large the light source is and therefore how large the shadow penumbras are.

Ray traced shadows can be efficiently computed for longer view distances than traditional cascaded shadow maps (CSM). Distant shadows can be ray traced while nearby shadows are computed with CSM to get the best of both approaches.

Ray traced shadows are generally 20-30% faster on the GPU than cascaded shadow maps. For more information check out the documentation!

Add that to preexisting fancy distance field based AO they have, and it's a nice package. I wonder if any papers on that work are available in the open though, haven't seen any games or demos doing that sort of shadows (variable penumbra in CryENGINE is still an effect built on top of traditional cascaded shadow maps).

 

@bac9-flcl
That technique is super cool. I would love to play around with it and I can almost guarantee that I will at some point. They have not publicly shared how the technique works (you can of course read their source but that is easier said than done unless you are familiar with the engine) but I kind of understand how it works, I think. Unless they figured out some fancy optimization for it...


Since I'm posting anyway, somebody asked me before if emissive objects would cast indirect shadows. Here's a screenshot of the same room as before, but this time lit only by the sky and the very shiny pink... thing

 

@bac9-flcl
Distance Fields are pretty cool. They have been used mostly in the demo scene, where you need to describe complex shapes procedurally with very little code. Here is a image of a 13kb demo from iñigo quilez from 2008
He has been sharing a lot of the distance math on his site
http://www.iquilezles.org/www/articles/distfunctions/distfunctions.htm
even for penumbra shadows
http://www.iquilezles.org/www/articles/rmshadows/rmshadows.htm


But since you need to describe everything with mathematical shapes, they aren't that useful for dynamic objects and skinned Characters. Those still need their own shadow map. Transparent objects (as always) would need special treatment too.
UE4 renders the DistanceFieldShadows at half resolution. depth-aware upsample and temporal aliasing do the rest
Comes with it's own set of quirks and drawbacks, but still very pretty and interesting technique

 

A lot of cool stuff in this thread!.

That nice lighting of the room is kind of backed right?.

I'm building a racing game and would love to have a highway lighting like in this vid.

Hundred to a couple thousands of lights in an scenariocasting rt shadows at least to to the vehicles, ressult is pretty damm cool.

 

That's what she said!!

No but seriously, VERY cool.

 

From the unreal forums on how they implemented the distance field ao. I'm trying to find a detailed post on the shadow implementation

https://forums.unrealengine.com/sho...mbient-Occlusion-(Movable-Skylight-shadowing)

"Could you give an overview of how it works? I'm very curious about the implementation."

Sure thing. High level view is

Offline
Generate signed distance field for each UStaticMesh, store in a volume texture tightly bounding the mesh. The distance field stores the distance to the nearest surface, with sign indicating inside or outside.

Realtime
AO sample points are placed in world space using a Surface Cache, which places more samples in corners where AO is changing quickly. This is basically a GPU implementation of the standard Irradiance Caching algorithm used in offline raytracing. Irradiance caching isn't inherently parallelizable because each sample placement relies on the other samples placed before it. This is solved by doing multiple passes, with a sample grid aligned to the screen. The first pass only checks every 500 pixels of the screen for whether shading is needed, next pass every 250 pixels, etc. Shading is needed if no existing Surface Cache samples cover that position with a positive weight. Surface Cache samples from last frame are fed in, with some percent trimmed out to support dynamic scene changes (AO will converge over multiple frames).

The actual AO computation that happens at these sparse points from the Surface Cache is done by cone-stepping through the per-object distance fields. We trace 9 cones covering the hemisphere, accumulate the min visibility across all objects affecting the sample point. This is a super slow operation, which relies on the sparse sampling from the Surface Cache to make it realtime. Distance fields are nice to cone trace (compared to most other data structures) because you just do a series of sphere-occlusion tests along the cone axis. At each sample point on the cone axis you find the distance to the nearest surface from that object's distance field, and compute overall sphere occlusion using a heuristic. The cone visibility is the min of the sphere visibilities. We track the min distance to occluding surface for the shading point which is needed for the Surface Cache algorithm, it indicates how much area that sample is representative over (and therefore no other shading is necessary).

Then we interpolate all the generated AO Surface Cache samples onto the pixels of the screen by splatting the samples and normalizing with the final weight. This happens at half res. The tolerances of the Surface Cache interpolation are increased which effectively smooths the lighting in world space, in a way that preserves lighting details in corners.

The resulting AO is really unstable at this point, so we apply a temporal filter to stabilize it, a gap-filling pass for any pixels that still weren't covered with a valid AO sample, and finally a bilateral upsample. Then it is applied to the diffuse lighting of the Movable Skylight.

All of this is like 15 unique shaders, about half of which are compute. All of the lighting data structures exist only on the GPU, the CPU doesn't even know how much work there is to do, so Draw/DispatchIndirect is used a lot. There's still a fair amount of optimization potential, as the inner loop cone-stepping is pretty brute force.

The key benefits of this implementation are:
* Computes lighting on surfaces, not volumes. This allows much higher quality than something like LPV used for sky occlusion where you are limited by the volume texture resolution that lighting is computed in, and you don't know the surface normal anymore.
* Thin surfaces are represented well. The per-object distance field allows enough resolution to handle thin surfaces like walls which would disappear in a voxel based approach.
* Less work is done where less work is needed - static and flat surfaces cost less, while still supporting dynamic scene changes. Even in games that need fully dynamic lighting, not everything is moving all the time so you don't want to pay for that.

The weaknesses:
* Limited transfer distance, it doesn't scale up to global illumination
* Requires rigid objects for the distance field representation, only uniform scaling of mesh instances
* Cost can be much higher depending on scene content, a field of grass would be pretty much worst case (no interpolation).

 

Yeah I understand the AO part of it. What, so far, has me confused is how they extend it to lights as not every pixel shadowed by an object is now contained in its bounding box (AO only applies to close pixels, shadows can be from anything).

Thanks

@blueivy
Thanks for finding that. Turns out their implementation was way more complex than I thought it was lol.

 

Somebody asked epic on twitter was it the same technique as this http://www.iquilezles.org/www/articles/rmshadows/rmshadows.htm

And he said yes same technique, just a bunch of culling and optimizations are done to support a large world. Tweet is here:https://mobile.twitter.com/EpicShaders/status/513019076944592896

Also from the official documentation on them https://docs.unrealengine.com/lates...ws/RayTracedDistanceFieldShadowing/index.html

"To calculate shadowing, a ray is traced from the point being shaded through the scene's signed distance fields toward each light. The closest distance to an occluder is used to approximate a cone trace for about the same cost as the ray trace. This allows high quality area shadowing from spherical light source shapes."

A optimization they do:
"These shadows are computed at half resolution with a depth-aware upsample. Temporal AA does a great job of reducing the flickering from half res, but jaggies in the shadows still appear sometimes."

 

I played around with the technique in a raymarch shader in Unity today. Well, it's a start.
Look, Aieth. That's the broken boolean brother of your cannon

 

One question, does Jove work with Shuriken particles ? Or i would have to change the particle system in Shuriken based systems like my Particle Dynamic Magic pack, in order to apply the effects ?

Also how hard would something like this be (if that is the case).

Thanks

 

As far as I understand, only particle shader must be new (Jove provides all common types), otherwise existing particle systems are fully compatible.

 

That is great to know, thanks

 

Could you share that shader? Would be cool to play around with

 

Looks cool, are you doing that with boolean operations (CSG) or are you generating a distance field based off a model?

As bac9-flcl said, you would just have to change the particle shaders. It's not that regular Unity particle shaders don't work in Jove, they do, it's that smooth particles won't work with the default particle shaders since Jove doesn't use Unitys regular depth texture. I ported all of Unitys particle shaders (and added a few lit variants) with the only difference being that they use Joves depth texture instead of Unitys.

 

Thanks for clarifying further , very good news about the Shuriken compatibility .

 

Do the fog patch graphics work in VR?

 

Feature request:

Lens dirt (scratched window / rain drops) on a windscreen in world space.

 

I do not have access to a VR headset so I don't even know if standard Jove runs on it I intend to investigate it later on though.

You mean applying the lens shader to an object instead of doing it in screen space?

 

Hey Aieth, just a wonder'n how things are progressing... haven't heard any updates on the engine as of late so when you have a moment I'd love to hear wheat you have cooking...

 

I've been working on a whole bunch of small stuff. Lightmapping workflow, handling edge cases etc. Reworked everything that has to do with spherical harmonics, added lightmapping translucency and transparency. And a lot of groundwork for the light probe modes, nothing in that area is screenshot worthy so far though It's all debug stuff.
Basically, all the boring stuff that has to be done in between the major stuff.

Here's a screenshot of the same room as previously, but this time there's a yellow/orange translucent block covering all the windows.

 

Hey Aieth,

I don't know how I've missed Jove previously. What an amazing find!

A couple questions for you, if you don't mind. Hopefully, they aren't already talked about in this thread. (Sorry if I missed that.)

(Edit: Went back and re-read the entire thread. Forget my question about Shader Forge support--for now... )

- Have you been involved with the Unity 5 beta yet? If so, do you know how Jove 2.0 will work with Unity 5.0 and beyond?

Thanks!

 

Hummm... is this the beginnings of a SSS shader???

Very cool BTW...

 

Too much candy for my eyes!!

 

looks prety cool

 

Will Jove work with this asset?
http://forum.unity3d.com/threads/co...illumination-using-light-propagations.275808/
(Also Unity 5)

 

Extremely unlikely, I think it's using standard Unity lights and overlaps with some parts of Jove (like area lighting) in what it attempts to do, so compatibility is not there and porting makes little sense.

 

I have not been involved with the Unity 5 beta. One of my most talkative customers has though, and according to this person Jove 2.0 runs fine. I will naturally be supporting Unity 5 when it releases, the transition should be painless, if anything at all, given that almost no Unity features are used. The one thing that might or might not be tricky is Enlighten support, but that is a big priority.

I cannot tell without any information on how it works. In a general sense however, it is rarely so that Jove can be modified to support other plugins, other plugins need to be modified to support Jove.

 

For those of you asking for updates, I am going to have more exciting stuff to post the next week. My schedule is finally freeing up. In the mean time, here's the room with the yellow translucent blob covering the windows again

 

So beautiful! And thank you for the reassurance on Unity 5 support. That is one of my biggest fears for any third-party asset currently.

I have one other question, regarding shader logic/coding. I'm not super experienced with shader logic and CG, but I have a bit, and I can follow and modify examples fairly competently. How similar is writing shader logic in Jove vs how it's done in Unity currently?

The example shader in your dev diary looks similar to normal CG, but I'm not sure if I'm missing key differences. I do have some specific needs regarding color maps for "dye-ing" rendered clothing or armor, so I wanted to see if that would be difficult to accomplish here.

Thanks!

 

I have relatively little experience in shader writing too, but it's actually surprisingly easy to adapt to Jove. All that's changed is the format you output your final results in, nothing else. You don't have to rewrite your stuff, you don't have to include some monstrous equations into every shader, you just output colors in a neat container in the end of your shader, that's all.

 

Oh, that's awesome! Well I look forward to Jove 2.0's continued development and using it myself in the near future then.

 

Uh,,, this is hot.

 

As bac9-flcl said, it should be fairly easy to modify and add stuff. Explained in a very non technical way, the way deferred shading works is this;
1. All deferred shaders are rendered, passing data into a later calculation. You can do whatever you want with the pixel color at this stage, blending according to vertex colors or whatever you want to.
2. The lighting stage, using the passed data the lighting is calculated.
3. Forward objects are rendered in, you have a bunch of preincluded shaders but theres nothing stopping you from writing your own additions. Two examples are included in the current version (a simple lambert and a forward replication of the looks of the deferred lighting)
4. Post processing and tonemapping.

It sounds to me like you would want to write a shader for stage 1. This is simple to do currently but becoming even easier the next patch. This is how the main deferred shader is looking in my development version. I am considering possibly hiding the math related to normals by macros, although I'm not sure if that would be more or less confusing

Code (CSharp):

1. Shader "Jove/Deferred/Bumped"
2. {
3.     Properties
4.     {
5.         _Color   ("Color(RGB) Smoothness(A)", Color) = (1,1,1,1)
6.         _TranslucencyMult   ("Translucency multiplier", Range (0, 1)) = 1
7.         _MainTex ("Diffuse(RGB)", 2D) = "white" {}
8.         _AttributeTex ("Metallic(R) Occlusion(G) Translucency(B) Smoothness(A)", 2D) = "black" {}
9.         _BumpMap ("Normal(RGB)", 2D) = "bump" {}
10.     }
11.  
12.     Category
13.     {
14.         Fog { Mode Off }
15.  
16.         SubShader
17.         {
18.             Tags
19.             {  
20.                 "RenderType"="Opaque"        //This is for Unity, tells Unity this should be in the opaque render queue
21.                 "JoveRenderType"="Opaque"    //Is an opaque object
22.                 "JoveShadowCaster"="Opaque"    //Casts opaque shadows
23.                 "JoveLightMap"="Standard"    //Attribute used by the lightmapper. Tells it this is a standard object with no fancy features
24.             }
25.                  
26.             Pass
27.             {
28.                 //Tags the last stencil bit
29.                 //Only pixels with this bit will be shaded
30.                 Stencil
31.                 {
32.                     Ref 128
33.                     WriteMask 128
34.                     Comp Always
35.                     Pass Replace
36.                 }
37.                              
38.                 CGPROGRAM
39.                 #include "JoveCG.cginc"
40.                 #include "UnityCG.cginc"
41.                 #pragma vertex vert
42.                 #pragma fragment frag
43.                 #pragma target 5.0
44.              
45.                 #pragma multi_compile JOVE_AMBIENT_REFLECTIONPROBE JOVE_AMBIENT_LIGHTMAP JOVE_AMBIENT_RADIANCEPROBE JOVE_AMBIENT_DYNAMICRADIANCEPROBE  
46.            
47.                 uniform sampler2D _MainTex;
48.                 uniform sampler2D _BumpMap;
49.                 uniform sampler2D _AttributeTex;
50.              
51.                 uniform float _TranslucencyMult;
52.                 uniform fixed4 _Color;
53.                 uniform float4 _MainTex_ST;      
54.              
55.                 JOVE_AMBIENT_VARIABLES
56.              
57.                 v2fStandard vert(a2vStandard v)
58.                 {
59.                     v2fStandard o;
60.                  
61.                     o.tangentDir = normalize(mul((float3x3)_Object2World, v.tangent.xyz));
62.                     o.normalDir = normalize(mul(v.normal.xyz, (float3x3)_World2Object));
63.                     o.binormalDir = normalize(cross(o.normalDir, o.tangentDir) * v.tangent.w);
64.                          
65.                     o.tex = TRANSFORM_TEX(v.texcoord, _MainTex);
66.                     o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
67.                     o.depth = o.pos.w;
68.                  
69.                     JOVE_AMBIENT_VERTEX(v, o)
70.                  
71.                     return o;
72.                 }
73.              
74.                 GBufferProperties frag(v2fStandard i)
75.                 {      
76.                     float4 diffuseTex = tex2D(_MainTex, i.tex.xy);
77.                     float4 attributeTex = tex2D(_AttributeTex, i.tex.xy);
78.                     float3 normalTex = UnpackNormal(tex2D(_BumpMap, i.tex.xy));  
79.                      
80.                     GBufferInput bufferInput;
81.                     bufferInput.color = diffuseTex.xyz * _Color.xyz;
82.                     bufferInput.metallic = attributeTex.x;
83.                     bufferInput.occlusion = attributeTex.y;
84.                     bufferInput.translucency = attributeTex.z * _TranslucencyMult;  
85.                     bufferInput.smoothness = attributeTex.w * _Color.w;
86.                     bufferInput.packedNormal = EncodeTangentNormals(normalTex, i.tangentDir, i.binormalDir, i.normalDir);
87.                     bufferInput.depth = EncodeDepth(i.depth);  
88.                          
89.                     JOVE_AMBIENT_FRAGMENT_DEFERRED(i, bufferInput)
90.                  
91.                     return PackGBuffer(bufferInput);
92.                 }
93.              
94.                 ENDCG
95.             }
96.         }
97.     }
98. }
99.  

So by modifying the values passed into the PackGBuffer function you can implement any kind of blending, color changing or anything you can think of. If you need more control than this offers, you can always write a forward shader