Jove 2.0: DX11 Rendering System (Alpha Release)

Developer Diary 1
Volumetric Scattering and custom lighting shaders
​

I figured I would start writing a developer diary kind of thing. Mostly to make it easier for you guys to learn about the new features but also to gather information in a few select posts instead of spreading it randomly throughout a huge thread.
Jove 2 has been launched into alpha and I am hard at work at providing its first patch. I've decided to go with the naming scheme alpha 0.1, alpha 0.2 etc... as long as we are in alpha and then switch to beta 0.1, beta 0.2 etc... when we enter beta stage. Which makes the coming patch alpha 0.2.
This first patch is going to contain a bunch of minor stuff along with a few major features. The small stuff is mostly fixes to various things that have been reported in, now that my user base no longer consists out of only programmers
For example, you have a cap on the amount of light cookies you can use. You can tweak this cap under the "Cookies" tab on your JoveCamera. The maximum allowed point/spot cookies are 8 per type. However, using an identical cookie on two lights counts as two separate cookies. This is no longer the case in alpha 0.2.
There will also be a simple cut-out shader, since that has been request. I also intend on supplying a basic terrain shader. It is going to be a straight up port of the standard Unity shader, just with Jove lighting, so no fancy features there yet.
Now for the two major and perhaps most exciting features! They are volumetric scattering and better custom shader support.

Volumetric Scattering
​

There is already support for atmospheric scattering. This however only works based on distance and it only takes the atmosphere into account, as in sun and sky light. The volumetric scattering coming in alpha 0.2 is more local.
Volumetric scattering explained simply is that when light travels through the air, it collides with small particles in it. This causes some photons to leave their path and head somewhere else, some photons even get absorbed by the particles and the light is "lost" (transferred to heat). For example, you might have noticed that when it is foggy outside the fog tends to be bright white and not black. This is because the small water droplets don't actually absorb most of the light, they simply scatter it. And white light is just all other wavelengths combined, which is why fog tends to be white (or clouds for that matter). A heavier more water filled cloud turns dark instead, the light was absorbed and not scattered.
Now, as for the Volumetric Scattering implementation in Jove. It takes into account all possible light sources, the sky, the sun, all directional lights, all point and spot light and all reflection probes (localized IBL). It is animated by wind and you can specify the size and speed of the fog. You get direct control over multipliers to the different light types (if you want your point and spot lights to have very exaggerated scattering effects you can do so) and to the probes.


This picture is an example of a pretty high energy absorption. A lot of the incoming light is absorbed by particles in the air, causing it to become darker. What is not absorbed is scattered a lot, the entire image gets a grey/yellow tint from the scattering of the suns rays.


A more "normal" use case for the volumetric scattering, light shafts through a window/door opening.

Custom Lighting shaders​

Most of the shader code has gone through some pretty heavy refactoring. Towards the end, just before the release of the alpha, a few corners were cut to get it out in a timely manner This has been remedied and there is now very little duplication of code. I took this opportunity to setup a more user friendly interface into the Jove shading pipeline. If you don't know how to code shaders this might be of little interest to you, but for those who know you will now be able to create your own lighting shaders! The same principle can even be used to swap out the main deferred shader, if you want to use something else like a toon shader.
Now, less talk and more shader code! Below is an implementation of a very simple lambertian diffuse (also know as N dot L) and a uniform ambient.

Code (CSharp):

1. Shader "Jove/Forward/Simple Lambert(example)"
2. {
3.     Properties
4.     {
5.         _Color   ("Color(RGB) Transparency(A)", Color) = (1,1,1,1)
6.         _MainTex ("Diffuse(RGB) Transparency(A)", 2D) = "white" {}
7.     }
8.  
9.     Category
10.     {
11.         Fog { Mode Off }
12.  
13.         SubShader
14.         {            
15.             Tags
16.             {    
17.                 //Tells Unitys rendering queue that this is an opaque shader
18.                 "Queue" = "Geometry"            
19.                 //Tells Joves shadow rendering that this is an opaque shadow caster
20.                 "JoveShadowCaster"="Opaque"      
21.                 //Tells Joves AO that this contributes to AO. Transparent objects do not  
22.                 "JoveRenderType"="Opaque"        
23.             }
24.                    
25.             Pass
26.             {
27.                  cull Back
28.                  ZWrite On
29.                  ZTest LEqual
30.                  Blend Off
31.              
32.                 CGPROGRAM        
33.                 #define JOVE_LIGHT_FUNCTION MyLambertianLight
34.                 //DeferredCommon has the structs used in our lighting function
35.                 #include "DeferredCommon.cginc"
36.                
37.                 //Forward declare our lighting function so the compiler know what we are talking about
38.                 float3 MyLambertianLight(ShadingProperties shadingProps, LightProperties lightProps);
39.                
40.                 #include "JoveCG.cginc"
41.                 #include "UnityCG.cginc"
42.                
43.                 #pragma vertex vert
44.                 #pragma fragment frag
45.                 #pragma target 5.0
46.                
47.                 #pragma multi_compile JOVE_NO_CASCADES JOVE_ONE_CASCADE JOVE_TWO_CASCADES JOVE_THREE_CASCADES JOVE_FOUR_CASCADES JOVE_FIVE_CASCADES JOVE_SIX_CASCADES
48.                 #pragma multi_compile JOVE_SHADOWSAMPLE_POINT JOVE_SHADOWSAMPLE_POISSON_4 JOVE_SHADOWSAMPLE_POISSON_8 JOVE_SHADOWSAMPLE_POISSON_16 JOVE_SHADOWSAMPLE_POISSON_32
49.                 #pragma multi_compile JOVE_CASCADEVISUALIZATION_OFF JOVE_CASCADEVISUALIZATION_ON
50.                 #pragma multi_compile JOVE_SHADOW_CASTER_COOKIE_OFF JOVE_SHADOW_CASTER_COOKIE_ON
51.                
52.                 struct a2v
53.                 {
54.                     float4 vertex    : POSITION;
55.                     float3 normal    : NORMAL;  
56.                     float4 texcoord  : TEXCOORD0;
57.                 };
58.                
59.                 struct v2f
60.                 {
61.                     float4 pos             : SV_POSITION;
62.                     float2 tex             : TEXCOORD0;
63.                     float4 worldPos        : TEXCOORD1;
64.                     float3 worldNormal     : TEXCOORD2;
65.                 };
66.                      
67.                 uniform fixed4 _Color;
68.                 uniform sampler2D _MainTex;        
69.                 uniform float4 _MainTex_ST;
70.  
71.                 float3 MyLambertianLight(ShadingProperties shadingProps, LightProperties lightProps)
72.                 {  
73.                     float NdotL = saturate(dot(shadingProps.worldNormal, lightProps.lightDirection));
74.                     return NdotL * shadingProps.diffuseColor * lightProps.lightColor;
75.                 }  
76.                
77.                 v2f vert(a2v v)
78.                 {
79.                     v2f o;
80.                            
81.                     //Unity macro that scales texture coordinates
82.                     o.tex = TRANSFORM_TEX(v.texcoord, _MainTex);
83.                    
84.                     //Calculate clip space position        
85.                     o.pos = mul(UNITY_MATRIX_MVP, v.vertex);      
86.                    
87.                     //Calculate world position        
88.                     o.worldPos = mul(_Object2World, v.vertex);    
89.                    
90.                     //Transform vertex normal into world space        
91.                     o.worldNormal = mul(float4(v.normal.xyz, 0.0f), _World2Object).xyz;
92.                    
93.                     return o;  
94.                 }
95.                
96.                 float4 frag(v2f i, float4 pixelPosition : SV_Position) : COLOR
97.                 {        
98.                     //Sample main texture
99.                     float4 diffuseTex = tex2D(_MainTex, i.tex.xy);
100.                    
101.                     //Declare the ShadingProperties struct used in Jove light calculations
102.                     ShadingProperties shadingProps;
103.                    
104.                     //Screen position is always required. Transform SV_Position from [0, width] to [0, 1]
105.                     shadingProps.screenPosition         = pixelPosition.xy * ScreenSizeInv;    
106.                    
107.                     //Depth is always required            
108.                     shadingProps.depth                     = i.pos.w;
109.                    
110.                     //Normal is world space is always required. Normalize since interpolation can cause non unit length
111.                     shadingProps.worldNormal             = normalize(i.worldNormal);
112.                                    
113.                     //World position is always required
114.                     shadingProps.worldPosition            = i.worldPos.xyz;
115.                    
116.                     //View direction isn't required, but fragDistance is. And when calculating frag distance
117.                     //you *almost* get view direction for free so calculate that as well just becuz'
118.                     shadingProps.viewDirection            = CalculateViewDirectionAndDistance(i.worldPos.xyz, shadingProps.fragDistance);
119.                    
120.                     //Diffuse color, not required but we want it in our lighting calculation
121.                     shadingProps.diffuseColor            = diffuseTex.xyz * _Color.xyz;    
122.                    
123.                     //Ambient occlusion, not required but we want it for our ambient
124.                     shadingProps.occlusion                = SampleAO(shadingProps);
125.                    
126.                     //Calculates the indexes required for looking up lights and reflection probes
127.                     CalculateClusteredAndTiledIndexes(shadingProps, shadingProps.clusterParams);
128.                    
129.                     //Iterates over all directional lights (including sun)
130.                     float3 directionalLights = DirectionalLights(shadingProps);    
131.                    
132.                     //Iterates over all point and spot lights    
133.                     float3 pointAndSpotLights = PointAndSpotLights(shadingProps);    
134.                        
135.                     //Our own private flat ambient! Does not interact with any probes or sky, just a flat ambient
136.                     const float ambientStrength = 0.25f;
137.                     float3 ambientLighting = shadingProps.diffuseColor * ambientStrength * shadingProps.occlusion;
138.                    
139.                     //Summarize the final color and output!
140.                     float3 finalColor = directionalLights + pointAndSpotLights + ambientLighting;                        
141.                     return float4(finalColor, 1.0f);    
142.                 }                
143.                 ENDCG
144.             }
145.         }
146.     }
147. }
148.  
149.  

I apologize if the formatting is screwed up, I did my best but it seems hell bent on screwing up my tabs.
Here is the result of this shader

Not the fanciest shader, but it has full integration into the Jove pipeline. It receives and casts both AO and directional shadows.

Patch alpha 0.2 is hopefully coming later this week but some time next week is more likely.

 

The volumetrics sound stunning (some asset store products will become completely obsolete with that, haha), and great to hear about the light cookie fix!

As about the shader you have posted and shader changes you have described - as far as I understand, this is irrelevant if you want to write shaders that output data to GBuffer and it's about adding freedom to render things through custom lighting models, correct?

 

Exactly. A standard deferred shader that uses the existing lighting pipeline is unaffected. The change means that you can now implement your own lighting models for forward rendered objects OR/AND you can overwrite the main deferred shader if you would want to.

 

Hah now I want to get the test scene i'm doing done so I can get stuck into something more fitting of the scattering! That's going to look brilliant even on scenes just using Jovenated unity terrain (Although a mesh and getting stuck into shader writing isnt an absurd notion). And the cookies too, it does sound like an internet joke but i want them. Stuff to look forwards to, I feel bad for PM spam when you're obviously hard at work! Although I cant guarantee there won't be more but i'll keep it enthusing.

This is great stuff

 

Hi,
is there a maximum amount of lights allowed? I have a scene with a lot of Jove point lights and they seem to cull away when i turn in certain directions.. (sorry, couldn´t get a video uploaded so please take a look at the pictures below)
Anyone know what´s going on and if I can fix this somehow?
//TheHenk

 

Yes there is. I just realized I really should mention this in the documentation The maximum allow point lights is 512 and maximum allowed spot lights is 32. There is also another limit however, and that is how many "indexes" are allow to exist at one time. Usually you do not run into this limit but you can if you use large light sources. For example, 20 point lights with a range set to 1000 would hit it really fast.
You can fix this, if you want to take the extra performance hit (it shouldn't be that big anyway) by going into the JoveClustered.cs and upping the "MAX_LIGHT_INDEXES" variable. The default is 100 000, if you bump it to 200k that gives you twice the allowed indexes.

 

Does that also push the maximum allowed lights?
It seems I have about 2000 point lights and it renders the ones furthest away first. Otherwise I can maybe get some trigger spaces going and turn the lights on and off when entering rooms. Still there are a lot of lights in the main hall in the buildning...

 

Oh, you have that many point lights? In that case it won't help. If you need that many lights it would be a good idea to tie them to trigger zones. I mean, I will tell you how to change that as well, but keep in mind that just iterating over 2000 lights every frame and doing the culling... its gonna take its toll You are also feeding lighting data to the GPU for loads of lights that can't be seen by the camera.
But if you want to change the amount of point lights that are allowed to be active, go into JoveLights.cs and up "MAX_ACTIVE_POINT_LIGHTS" to your preferred value. If you notice lights go missing anyway, then it is the "MAX_LIGHT_INDEXES" I told you about before.

 

aha 2000 lights, thats quite something

 

Thanks a lot @Aieth! I have gone through and arranged the lights per level now. But maybe I can get a little 'trial and horror' action going and see how much our rigs can take without coding the trigger zones.. (It´s running on pretty decent hardware in our new Visualization Centre..) @lazygunn A lot of lights... but it will look stellar!

 

Let me know how it goes I have not tested it properly myself.

 

Would that SECTR Vis asset help for something like this?

 

Small feature request - is it possible to add support for ambient lights? Essentially, point lights that do not contribute to reflections and do not remove ambient occlusion contribution (I guess that can be also described as a light of an infinite radius but limited range). Similar effect can be achieved through carefully configured environment probes, but ambient lights would offer artists great degree of control over scene lighting, especially with proper GI being nowhere close to release.

Use cases: Interior scenes, with those lights placed in the windows to outdoors and other openings. The option is especially attractive considering how well Jove handles large quantities of light sources.

 

@bac9-flcl I think that's what Crytek does as well in Ryse. +1

 

@bac9-flcl
That is a good idea. Ambient lights can be a great addition to make the scenes look even better. Ambient lights + Jove should give us enough power to make really great looking scenes.

 

It could be argued that a bunch of Crytek's decisions regarding Ryse were questionable but i'm fairly sure the same idea was used in Crysis 3 which is an unquestionably high graphical achievement. It's a good idea, i'd like to light areas without affecting the AO for sure

 

I've been planning to do that. I was gonna go "Surprise! Cool feature!" but now you've spoiled it I need to make some changes to lighting pipeline first though, but it should be feasible within the not too distant future.

 

Great to hear!

On another subject, how is that cookie patch going?

 

It is going The fog is still being worked on, and after that I need to create a terrain shader.

 

The volumetric scattering is starting to come together as a whole now. It supports all existing light sources, including reflection probes and things like light cookies.
Previously, effects like these would have been done by hand using a transparent mesh. While that can surely look great as well, it requires a lot of tweaking and handling of special case scenarios (what if your mesh goes through a wall?). Using this system all you do is crank up a slider and you have volumetric fog

Two point lights and a white spot light


The obligatory cinematic spot light effect​

 

Looks great. Keep it up!
I like how you already replaced more than 6 assets that I had to use in combination to achieve effects similar to what you have done. Just that it mostly looked worse, and performed worse.

This all in one solution is amazing. It is going to save me a lot of time and money, aaaaand it looks better!

 

Agreed. The consolidation idea is great for efficiency and ... sanity.

 

Interesting keep up the good work.

 

This just keeps getting more and more ridiculously awesome! Those volumetric effects are killer.

 

Seeing this stuff about fog and volumetric lighting has got me thinking.

Just for some background, I'm kind of a noob that's been very slowly working on a 3D space game. Currently I'm using Skyshop shaders, mostly just for some decent ambient lighting via IBL (I'm not going for a massively realistic look). I've been following this thread closely, as it seems that the efficient rendering and better shadows of Jove 2.0 could be quite helpful. Also, the PBR stuff I kind of want to learn more about anyway.

I hadn't considered attempting stuff like foggy/dusty environments, but these recent Jove screenshots have captured my imagination. With Jove 2.0, would something like this be possible, at least sometime in the future?
http://indus3.org/blog/atmospheric-effects-in-games/

Or even the same type of thing, but with "god rays"? I don't have a feel for how expensive that would be on frame rate, if it's even possible to do. If it is possible, I suppose the performance hit would depend greatly on the number of objects affecting the rays.

 

as i saw in jove 2.0 there is some kind of fog. im not sure you can do such stuff but at least i can try on boxes and spheres.

 

I think that would be entirely possible with the next patch. And properly implemented god rays are not a separate additive effect but naturally arising phenomena from volumetric scattering, so you will get the rays from the sun for free.

 

Wow, that would be cool. If that's possible with the next patch, my interest in the whole thing just went up a tick or two.

 

Really cool, can recreate the bf4 shanghai tower falling

 

Is localized fog in this as well? It would be cool for like vents and stuff

Anyways looks really cool

 

Waah, I need that volumetric light stuff now! ..at least before wednesday.. It would make my visualization project super awesome! Btw, all those lights don´t seem to cause much trouble... I have about 800 lights in scene at the moment and it runs smooth enough..

 

There's a very nice compute-powered asset that's good for this kind of thing called Fluidity that maybe could do with an update but if that kind of thing would be great to have appear at some point, in fact there's quite a few things i'd love to see but all in good time I guess, it seems it can all just keep going and going

 

That is a lot of comments. I really appreciate the feedback

Yeah that would be possible, even pretty easy. The only part which might require some special configuration is the falloff. I'm having a vertical falloff setting, to simulate fog gathering close to the surface, while that would require a vertical falloff that works both up and down. As for god rays, yeah you get that "for free". God rays are basically the result of fog being in shadow. If the fog is not hit by light, it cannot scatter light towards the viewer.

The way I'm setting up this system is that you get to specify how far it is volumetric and an "earth scale". The ray marching is performed along this volumetric distance, this is the distance where local lights (point, spot), local reflection probes and shadows will affect the scattered light. If a rendered pixel is further away than the maximum volumetric distance it takes the maximum volumetric value (aka all lights within, say 100 meters) and then it computes the rest of the scattering analytically. In plain English, you get fancy local effects up until the volumetric distance, and then only directional lights and sky lighting.

You also get several settings affecting quality, like resolution settings, temporal reprojection strength (the higher the better quality, but it "blends" into previous frames causing trails if lights move fast), volumetric spread bias (the higher the value the more concentrated the samples are towards the camera).

Well that depends on how you define localized fog Technically, this could be called localized fog. If you mean fog as in "place a fog volume here and let there be fog!" then no, that is not included in this patch. It is very likely that I will do it in the future though, as it would require little new code. If you mean crank of the fog slider and see a spot light with a fan cookie rotating cause cool volumetric light effects, then yes that is in.

I would love it to be finished now Unfortunately it is not that simple. The foundation is there, but a lot of work remains to make it user friendly/optimize it. Also, Unity does not support #pragma multi_compile for compute shaders, so I have to do my own variation... I need 42 permutations of my fog compute shader.

 

Quick question, what do you think of the task to move TC Particles (a dx11 gpu particle system) into Jove under my own steam? Would it need much more than a shader switch? Regarding the project i discussed, this would be quite spectacular for various aspects of snowfall, and it mainly occured to me as you said the volumetric scattering might be modulated by wind.. syncing that kind of thing would be very neat (along with Lars' tree script to sway with proper respect)

 

A shader switch *should* be all that is needed, I don't know if it does something else as well though. You'll have to test it

 

How suitable the volumetric effects will be for unorthodox environments like vast vertical interiors? For example, if there is some sort of height-dependent density or direction based scattering variation, I'd like to have an option to disable that to have fog that workis consitently no matter the direction you're looking in.

 

It uses noise to simulate varying fog density. Are you asking for an option to turn off all the fog attenuation factors all together? So uniform fog everywhere you look? I can do that, feedback duly noted

 

No-no, noise-based density variations are great, especially as you have mentioned that noise can be animated. I'm just asking if there are any features tailored for classic outdoor environments, for example, fog density being different above then in the direction of the horizon. If there are, I might have to turn them off in some environments so that, for example, some vertical tunnel won't look strange, but omnidirectional features like noise animation are useful everywhere.

 

Yeah there is a vertical attenuation. You can of course turn that off

 

Not bad for a few slider adjustments and scaled primitives

 

Let me at the bugger, or i'll write an angry letter haha. Such exciting stuff to work with

 

Wow, it sounds perfect. Looking forward to trying it out!

 

If all jove features are in place working then i now where the unity upgrade cash is going
this got almost all what we are asking from unity since v3 render wise, yet unity 4 (and i am 100% sure 5) renderer are from the past and you are already way ahead of them with your beta.
you have my salute.

 

Alloy is using the same channels as Jove, right? So when I use Alloys tool to mix up the textures for me, and create 1 mask texture it should work?
Because it does weird stuff for me. My metal is absorbing all the light, becoming completely black, and the rest of the entire object is reflecting stronger than the metal. I am 100% Certain that all the input maps for Alloy are right...

I don't get it. I am probably too stupid.

http://abload.de/img/23413qj4.jpg

 

Maybe the effects white/black have on the specular/roughness are different in jove? Maybe try inverting some of your maps.

 

@Olafson, you need to show us your input maps, not the result.

 

That looks strange. I do not know how Alloy works, what color is the main texture? I should probably rename that from diffuse to just color. A dark metal has a color value of 0.5, ~127 in RGB. If you want me to take a look at it you can PM me the textures or something.

 

One potential issue is the blue channel of your attribute texture - in Jove it's not specularity like in Alloy (there are no specular maps at all) but translucency, and it should be black for non-translucent materials like your metal. I also see the translucency multiplier cranked up, that will make misapplied translucency even more noticeable.

Speaking of which, @Aeith, can you explain the system behind different widespread PBR texture setups? I mean, I understand what exactly roughness, occlusion, translucency and metalness maps in Jove are doing, that system makes perfect sense to me. But there are others! For instance, I don't understand why certain PBR implementations (like Alloy or Unity 5 PBR, judging from it's presentation on Unite) would use specular. How do different input schemes can be simultaneously considered physically correct?

It's pretty obvious to me how to make a surface more reflective: you simply make it less rough so that less light is scattered omnidirectionally (to diffuse) and more light is conserved for proper opposite angle bouncing (specular reflection). It's pretty obvious to me how to make a base surface color different, be it hue or base brightness - you change it's albedo texture, and that texture is basically used to show what wavelength the color reflected from the surface is (for dielectrics).

Now, how the hell do you add a specular map there, assuming that specular map still means reflectivity map like it used to? Using a map that's directly influencing reflection force when there exists a separate dedicated roughness map makes zero sense to me. How is Unity 5 explaining the presence of that map simultaneously with roughness? Alloy docs explain it's use as "for setting the specular f0 for non-metals. Black to White in this channel maps to 0.02 to 0.08 f0", but I'm afraid I'm not familiar with those terms. That does not sound like the old definition of specular, I guess.

 

Oh, screw me. Thank you man! It was indeed the translucency thing. I, for some reason thought (dont ask me why) That it was specularity, just like in Alloy.

Fixed it now! Thanks for your help, everyone. Sorry for that, hopefully wont happen again.

 

Haven't looked at how exactly Jove does it, but roughness doesn't define specular reflectivity, it defines (in "traditional shader terminology") something closer to gloss - in PBR, it influences both diffuse and specular, but it isn't specular reflectivity (or diffuse).

Rather, in a metalness workflow, metalness along with albedo defines specular reflectivity. Metals have black diffuse - so the albedo map is repurposed to define the specular reflectivity for those (both its color and intensity). For dielectrics, the base reflectivity (F0*) is set to a constant low "grayscale" value (somewhere between the Alloy-mentioned 0.02 and 0.08 - or, according to Disney, 0.00 and 0.08) - and the albedo map is used for diffuse like you'd expect.

So, what the specular map does is allow you to change that constant base reflectivity value. With only metalness and no specularity, you can't change reflectivity for dielectrics - but it doesn't tend to vary much anyway (again, F0 between 0.00 and 0.08 - notably this range doesn't include the specular for gems and diamond). The Unity 5 PBR appears to use the purely specular workflow, while Alloy does use the metalness workflow, but allows you to change the reflectivity within that 0.02-0.08 range.

Well, the old definition (as much as there was a standard definition) tended to include a vague idea of roughness. But other than that, it's just that metals tend to have very high specular (mostly 0.50 to 1.00), while dielectrics have extremely low specular - and since albedo is used for the specular of metals, Alloy only allows the "realistic" range of 0.02 to 0.08 for the rest. As mentioned, Disney specified 0.00-0.08 in their PBR paper (although I can't think of any cases where it should be 0.00 - or even below 0.02).

*) F0 really = specular reflectance at normal incidence