Jove 2.0 - Replacing the rendering pipeline

​

For the past seven months I've been working on support for deferred rendering in Jove. Early on in the process I came to the conclusion that simply extending Jove to support Unitys deferred rendering would not suffice. It required too many workarounds and was generally a sub-optimal way of doing what I wanted to do. And since I've been wanting to do my own shadow maps, SSAO and a few other things anyway... I figured why not write my own deferred renderer. I also have to give credit to Dolkar for his inspiring work.

Why?
Because Unity needs an overhaul in the graphical department. This is most evident as shown by the increasing amount of assets directed at providing better graphics for Unity. There is, either currently on the market or in development, a wide range of assets ranging from IBL implementations to reflection systems and shaders. Jove 1.0 is one of those, being a shader replacement asset. The issue with Jove 1.0, as well as the idea of graphical assets in general, is the difficulty of combining several in one project.
Using one or two is generally not an issue, but if you end up wanting to combine eight of them you're in compatibility hell and chances are you are going to end up with a sub-par FPS. Why? Because no asset creator is going to say "oh well in the big picture my creation does not really deserve more than 2.0ms, so I'm going to scale it down for the good of the end user". When it comes to real time graphics, you really can not implement eight different systems without weighing the costs against the benefits. The best AA in the world might look incredible, but what use is it if it takes 25ms rendering time? Especially with systems like Global Illumination or AA you can throw an infinite amount of GPU power at them and they can still use more. That, plus the fact that a bunch of effects can often be semi combined to save performance. There are a lot of uses for downsampling the final image to half its original size, but there is no point in doing that three times per frame for three different image effects.

Unity does deferred rendering with a technique that is known as a light pre-pass renderer. I'm not going to go into technical stuff here but what it does put plainly is:
Render the data necessary for lighting into a screen space texture ->
Calculate the light from all light sources in screen space ->
Render all objects again, this time reading the lighting from the screen space textures.
The main disadvantage of this technique is that it has to render all objects twice. Which means drawcalls = 2 * meshes. Jove 2.0 is instead using full deferred shading. Deferred shading is not as fast for simple rendering (previous generation or this generation mobile phones), but is faster for this generation (Xbox One, PS4 and desktops). Another advantage of deferred shading is that it does not place arbitrary constraints on the user like light pre pass does, like limiting the user to a single color for overlapping lights.

Jove 2.0 renders all renders all data necessary for complete shading into screen space textures and then does everything in screen space. This eliminates the need for a second draw pass, freeing up CPU time as well.
Another reason for rewriting the pipeline, is that while Unity is good at providing access it's still just access at certain parts. For example, ambient occlusion should really only be used to occlude the ambient. If you slap on the default SSAO shader it occludes direct light as well as ambient light. If you think about, why would ambient occlusion occlude anything but ambient light?

Next generation graphics for Unity
DISCLAIMER: I do not own the model. It was made by d1ver over on the Polycount forums
http://www.polycount.com/forum/showthread.php?t=130641​

Now, here's the interesting part. How it works for the user and what it does.

To start with, lets get a few basics out of the way. These are the features of Unitys rendering pipeline which Jove 2.0 still uses:

• Static Batching
  [*]Dynamic Batching
  [*]Frustum Culling
  [*]The actual render calls

Everything else, from Unitys shadow maps to ambient to shading and screen-space effects, is replaced or removed. Removed as in, there is no support for it, it will not work.

[Modular] PBR SciFi Packby Jakob R. (http://forum.unity3d.com/threads/modular-pbr-scifi-pack.257239/)​

Area Lights
This needs it's own paragraph. Unity "Light" class include directional, point and spot for real time usage. This class serves no purpose in Jove 2.0. The entire light system is replaced, and infinitely small lights are no longer supported. This means every light source, point, spot or directional, now has a radius as well as a range (except for directional, which only has radius).
A 0 radius light leads to infinitely small highlights on smooth objects, which in practice is usually compensated for by simply not making materials that are that smooth. With lights using a radius objects can now be completely smooth without specular highlights vanishing.


Here is a list of features that are guaranteed to make their way into Jove 2.0. The color coding indicates if they are
Finished at a beta level (e.g could still use optimization)
Half done
Not started
Please note that every feature on this list is subject to change, nothing is final

• Physically Based Shading
  [*]Exponential Shadow Maps
  [*]Parallel Split Shadow Maps
  [*]Cubemap System
  [*]HDR Rendering
  [*]Bloom
  [*]Tonemapping
  [*]Screen space occlusion
  [*]Atmospheric Scattering
  [*]Dynamic Sky
  [*]Area Lights
  [*]Clustered Shading
  [*]Parallax Cubemaps
• [*]Screen Space Reflections
• [*]Depth Of Field
• [*]Water Shading
  [*]Motion Blur

Clustered shading allows hundreds of dynamic point lights at a time


Glass shaders in Jove 2.0. For an explanation, see this post http://forum.unity3d.com/threads/jo...rendering-pipeline.230141/page-3#post-1706460​

Demos
Atmospheric Scattering: https://dl.dropboxusercontent.com/u/233290703/ScatteringDemo.html
Clustered Shading: https://dl.dropboxusercontent.com/u/233290703/ClusteredDemo.html
Both demos require a DX11 capable computer

FAQ

• Does it require Unity Pro?
  Yes
• Does it require DX11?
  Yes. In the future there will be a very downscaled DX9 version, aimed more at actual compatibility than fancy features
• Does it work on a mac?
  Eventually. When it works on DX9 it is going to work on mac
• Does it work on mobiles?
  No and it likely won't until Moores law allows it to
• When will it be released?
  When it is done. Jove 2.0 is heading for a public alpha release early August
• Will Jove 2.0 be a separate asset from Jove 1.0?
  No. The price will rise however
• How is the performance?
  A lot better than what you might think My GTX 560ti, which is not a high end card, does 500 point lights and post process at over 100fps

As progress continues this thread will be continuously updated. All kinds of critique and suggestions are welcome.
Thanks for reading!

 

All that progress! Great job so far!

It's probably worth noting that by saying "point lights are no longer supported", you most likely don't mean that one can't place an omnidirectional light source with Jove, but that every light source needs to have a physical width, or radius. That in turn, I assume, also makes specular highlights wider based on the distance to the light source, in the case of point and spot lights.

HBAO looks better than I expected... though a bit too blurry. Is the performance that bad for higher resolutions?

Advanced tonemapping? I don't think I've read a post about it. Whats the deal?

I still can't help but feel that clustered shading is a huge overkill. According to the paper, it outperforms tiled shading only with 1000+ lights... and you're not planning on filling the scene with virtual lights for the GI afaik...

Order Independent Transparency: Are you going for an exact solution or an approximate one? .... or both?

Also.. when you post fps measurements... you should also mention at which resolution it was running, especially with resolution dependent deferred shading.

 

Yes, that is exactly what I mean. I edited the main post for clarity. Thanks for pointing that out.

No it's actually running at full resolution. I just over blurred it I need to figure a good way to increase the maximum depth difference (bilateral filtering) when distance increases. Without the blur far away areas (25-30+ away) start flickering on camera movement.

That's my own term. At the core it's filmic tonemapping, but with (optional) automatic exposure and automatic key value based on the luminance of the scene. It's pretty flexible.

I agree with you, and it's possible that I'm going with tiled shading instead for the lights. We'll have to see how it performs. But even if it's not used for lights, I'm still implementing it, as I've got another major use planned for it.

Exact solution. OIT is definitely not a guaranteed feature however, it's all down to how it actually performs.

I'm being vague on purpose, I don't want to tout the performance of something unoptimized It's mostly meant to ensure that it's actually running at game speed and not just interactive slideshow speed. The resolution is 1680x1050.

 

This looks great overall. It's ability to handle virtually limitless numbers of lights looks interesting. It's only when you start getting into 1000s of lights that you can start realistically doing lighting from particle systems or mesh surfaces accurately. Is it still possible to use multiple cameras in a scene and combine the images so that you can use existing unity shaders if necessary though ? I would like to be able to use refractive glass shaders as well as flow map shaders in my project. I'm assuming that existing unity shaders will not show up if jove is being used to do the rendering is that correct ?

If thats the case it would be great if you made a script that could recompile custom shaders to use Jove's lighting a bit like the Go Linear plugin does. Also rather than selecting which lighting method to use for jove based on performance it might be better to make a performance manager for it that could could change the lighting quality settings based on the performance of the device being used. If you implemented multiple lighting algorithms so that jove could run on more platforms. You could create a low quality mode so that people could run games made with Jove even if they didn't necessarily look very good and also make this mode compatible with opengles3 mobile devices. Jove could also switch between these quality options on the fly to ensure a consistent framerate on lower end PC's as well.

 

It is possible to specify a layer that will be rendered using standard forward rendering, allowing custom shaders to be used at will. The only downside is that the standard forward rendering has to be done after the main deferred pass, meaning that each object will cause overdraw. Crysis for example does this for its weapons and heads, everything else is done in the main deferred pass. Except for that, it won't be possible to use custom shaders. You could of course alter the main pass to implement whatever shading model you would like, something which would be trivial for anyone used to writing shaders.

Recompiling custom shaders would be... hard to say the least. Correct me if I'm wrong, but Go Linear just changes the ways textures are sampled? That's trivial compared to changing every input data.

I fully intend on keeping it scalable, but at least to start with everything is going to go the physically based route. Different lighting models is something that would be considered later, I need to get the core running first

 

I have to admit I'm very impressed with this, I tried something similar but I wanted to leverage some of the features of GL4 and improve DR from another engines example classes.. I was looking at how Unity would compile the shaders / then get DR working with the whole thing VIA a plugin without breaking anything else. You can't really do anything easy like extend the sealed matrix class mainly because it ideally needs to be with the C++ via another class in the .DLL. (Amongst the obvious reason it's sealed) Then I looked at how to call GL lighting correctly in regards to 3D picking whilst still trying to keep the UI intact and it became messy very very quickly.

I gave up and started building my own Engine, because it actually seemed easier to do . It seemed to me like Unity would need a re-write and a completely new rendering pipeline .. I suppose it's not too bad with DX as you have an up to date version to work with.

But here is the thing, as you already know Jove will be closed into windows only. If it was do-able GL4 would be the way to go, but without source code it seems nigh on impossible.

Sorry to go full geek on the situation, but I'll be looking forward to see how this turns out..

 

In some ways it's easier to just give up and create your own thing I actually got that feeling when starting on Jove 2.0, as I felt limited by conforming to the lighting system of standard forward mode. At least this way the only way I'm limited is frustum culling and batching (I can live with that tbh...).

Being limited to windows is a downside, but on the other hand most high end games are windows only anyway. I suppose Unity is going to support both Xbox One and PS4 at some point, and when that happens I will definitely work in support in Jove as well.

Is there a specific reason why you absolutely need GL4? For mac?

 

Well mac and port's and things like consoles, from what I've heard sony have dumped PSGL and gone directly OpenGL.. There were tons of benefits to this, our lighting system heavily impacted the amount of draw calls on a scene and from previous experience of working with consoles and frostbite there were tons of optimisations that could be done to eradicate a lot of these issues. I was also speaking with certain people about the OpenGL Mantle derivative which will be coming sooner or later and in the end the whole thing will of been far more optimised than going with DX11. I'm sure there are some DX guru's who could show me up on the matter, but I had my reasons. GL and DX in terms of tech has little between them and it's down to experience, I'm used to Vis studio / glew and GL.

Also the plan was to use OpenCL for raytracing cut-scenes with offline rendering.. With Optix I had a small demo running in real time, but the performance was around 20 FPS on a 780TI, so not much use for anything else.

I'm looking forward to this Aieth, let me know if you need anything ..

 

I see. Fortunately for me, I'm hiding behind Unity When Unity supports Xbox One/PS4, so do I.

Thanks for the offer of help, I've been getting quite a few of those It is great to know that this is something the community wants. If there's anything I can use it's test scenes and/or models. I'm definitely not the best artist around, in fact I'm more towards the opposite end of the spectrum, and I could use some for testing purposes. In many cases I'm tweaking low level stuff to get it to look good and it helps if I'm looking at actual models and not spheres with programmer textures while doing it

 

Looks really cool, I will keep an eye on this, subscribed

 

OpenGL is a factor because of linux, and valve's steamos initiative may not take hold for a few years, but it's worth bearing in mind that OpenGL, and Mantle (i hope) will be much more a priority at some point

 

All this in just 3 Months of Partime Work In between University ...

Dude Im Extremely impressed. Congratulations for the Achievment.
Congrats To you and Dolkar for all constant Support and bumpup ...
I Was not wrong about that Awesome open Sourçe guy ...

HO And Dont forget-me as beta tester .... I been the first and will be the last ...

After all im basicaly one of the fathers of this idea ^^

Its very Cool to see My dreams and Visions of joinventures And needed features Cometrue http://goo.gl/VWZzlV

And Turned from Mere Visions into Hard WOrked reality .

The Best of Luck With this August !


PS: Dont Forget of After Have This JOVE 2 FRAMEWORK Selling as Crazy ...
To Whraph around Your Work in JOVE 1 : And Release it Latter JO1 as Free Asset ! *Giggles*


BEST REGARDS


SOULS

 

Thanks!

Yes of course, as soon as there is a good reason for supporting OpenGL, I will. It is too bad you can't run compute shaders on mac.

Thanks for your support.

 

Ive wrapped up work on the first beta patch, Jove 2 enters beta 0.1!

I finalized work on the area lights, ported Unity's Depth of Field shader over to Jove and restructured a lot of code.
Please note that the ambient systems are placeholders. This causes especially the metal spheres to look worse than they should.
This scene, with full settings on shadows, HBAO, bloom and DoF runs at 3ms It's 150 000 tris.
Right click and open image for high resolution.

 

I wasn't even aware it was in beta! Is it open to owners of Jove the first? I've hardly even looked at Jove as i bought it because i keep getting distracted but this is very interesting and however practical it turns out to be in terms of wether I used it in something very time consuming, it would definitely be interesting to try out

 

This first 0.1 version only goes out to a select few, it's not ready for an open beta yet. At some point, I can't promise when though, I will have an open beta version for all customers

 

I'm planning on buying Jove, But i do want to know some points before i'm completely convinced in buying your package.

How hard will it be to convert shaders from other packages or generated shaders from e.g. ShaderForge.
And how hard will it be to convert image effects e.g. HDR Bloom from Unity or Color3 from amplify
Is it possible that in your PRTV implementation to recalculate the lighting data at runtime.

*Bonus question (Don't have to be answered): 2 camera shadow sharing for e.g. first person weapons. This is something i never got working in unity where a camera in 2 different depths rendering different objects but sharing the shadow mapping. Would be nice if this was possible without a lot of work.

If you would like to answer these questions that will be great.

 

Depends on what you mean by converting. Due to the difference in how a forward and deferred renderer get access to data such as normals, light vectors etc just changing a few variables isn't enough. If you however can write shaders replacing the main lighting function would be trivial. Jove 2.0 sends all variables into a main lighting function which then does the PBL calculations. If you know how to write shaders, that's easy.

Also, keep in mind that Jove 2.0 isn't available to the public so don't go rushing in just yet

Trivial. Instead of using Unitys "_CameraDepthTexture" and its functions to unpack depth, you instead use "_JoveDepthTexture" and joves functions to unpack instead.

Yes, but if you want to do it without a noticable drop in frame rates you can't update more than a few probes per second. If you want to do dynamically generate a level that's completely possible.

I'm not sure I follow your bonus question. You want to cameras to share the same shadow map? In theory that's possible, but the thing is when rendering shadows you want to make the fit as tight as possible. There's no point in rendering shadows that are not seen right? So you'd have to redo the calculations so that the shadow fits both cameras. If this is what you're asking, the technique is totally possible but I won't implement it as it isn't really general. Specialized solutions are great and all, but if only 1% want them I can't really motivate working on them, sorry.

 

Thank you for replying, Now i am 100% convinced in getting your package.

And about my bonus question. It is indeed something not used normally but for some games including mine it is a requested thing that shadows can be shared over multiple cameras. In unity self its impossible without hacking much around. But if you say its possible. That's already a great start. Thanks

 

I got another question, Will you be supporting particle lit shaders, Like shown in most next gen engines.
And particle shadow receivers and casters, basicly transparancy shadows.

Even if you are not planning on making that, Would Jove2 support it in like making all the variables open so it can be made by myself or somebody else.

 

I have ideas and plans for particles as well, but I can't really say at the moment as it is going to be a later concern. The particle system won't survive untouched though.

I have yet to decide if I am going to include full source code or not, so I can't say at this point.

 

Hi everyone! I've been quiet as of late but development has continued on. Much has changed on the scene (UE4/U5 anyone?) but Jove 2.0 is still on the way. I'm also no longer working alone but have entered into a partnership kind of deal with a company and an indie group. A lot of time has gone into polishing the core (it's amazing how that can take as much time as actually implementing it) and the new cubemap system is finally in place. It delivers far better results than the previous system (it required "interesting" environment to produce "interesting" results whereas the new system much more closely mimics how light actually behaves) and in addition it's cheaper to evaluate at run time.

Anyway, small update just to let you all know I and Jove are still alive. Here's a picture and a webplayer
Ignore the sky for now as it has gotten 0 attention so far. It's just one of the included Skyboxes without any tweaks or fancy stuff to make it "fit" with the rest of the image.

The bottom row is gold going from smoothness 1 to smoothness 0
The top row is black plastic from smoothness 1 to smoothness 0
The plastic cube is smoothness 1
DISCLAIMER: The gun model, called Cerberus, is not mine but is made by "d1ver" over on the Polycount forums. Here is a link http://www.polycount.com/forum/showthread.php?t=130641




And here is the webplayer demo. Requires DX11 capable hardware/OS
https://dl.dropboxusercontent.com/u/233290703/Showcase.html

 

Looks good , glad to hear you are making progress. At some point you should make a demo with changing lighting conditions to show how your well it handles changing lighting conditions. A sunset and sunrise for example. Also are you creating Jove in such a way that when Unity 5 adds support for opengles 3.1 compute shaders your shaders should also compile for that. A bit like how most existing Unity 4 shaders could be compiled for opengles 3 when it was added for Unity 4 ?

Opengles 3.1 has already been accepted as the standard for webgl so if Jove is going to be compatible with the web gl platform in Unity 5 I'm assuming it will have to. The reason it is a big deal is that opengles 3.1 runs the kernel for the graphics chip on the gpu itself rather than using a graphics driver running on the cpu. In theory that should give a performance boost to computer shaders as well. With both microsoft and sony signing up to support web gl it means they will have to support it's shaders even if they are converting them on the fly to another standard like directx 12 in microsofts case.

I think Jove should position itself as a high end lighting platform for Unity 5 designed to run on PC and next gen platforms. If it uses clustered rendering then it will be far more suited for scenes with lots of dynamic lights than Unity 5's solution. Unity 5's built in solution will probably be faster and therefore be able to run on mobile chips but has limitations such as dynamic lights not contributing to the GI for example.

 

I fully plan on demoing the capabilities of a dynamic sky, as soon as that's implemented In the picture above the sky is just a skybox, but I'm currently working on a physically based atmospheric scattering system, which really is the last thing to add before you can make complete outdoor scenes in Jove 2.0 (not indoor though).

Unfortunately I can't use most of Unitys functions, almost everything relating to rendering is custom made from the ground up. This means it won't flawlessly compile for Opengles 3.1. However, I do fully intend on supporting it when and if Unity supports it. When it does, give me a couple of weeks The issue lies with the difference between how different systems handle matrices, and especially when it comes to shadow mapping there are a *lot* of matrix multiplications back and forth between different spaces.

I fully intend on positioning Jove as a high end platform for next generation graphics. Unity is limited by its desire to support everything from mobiles to high end and it ends in a jack of all trades master of none position.

Does Geomerics Enlighten not support GI from dynamic lights? I'm not familiar with the system, how it works and what it can do. All I really know is that it works by simplifying meshes? If you happen to know or have a link (I couldn't find anything except for sales pitches on google) I would appreciate it

 

You might want to go this link to learn about how enlighten works.

http://advancedgraphics.marries.nl/lectures/07a_approximate_global_illumination_part2.pdf


I'm only guessing but I think Unity is taking advantage of the fact that lightmaps with enlighten are very small and going to create some system of pre backing light animations to have animated GI lightmaps. I think that reactor animation running on the ipad in the Unity 5 promo is an example of it. I suppose enlighten might have updated their algorithm for Unity but otherwise I don't see how it will be able to do GI with truly dynamic lights at best it will only be able to bake animations..

 

Thanks for the link!
Now I'm no longer that impressed by the software I guess it's great that it works on low end devices, but I don't have much interest in those What I like about GI is that there is no clear winner among approaches, you only trade compromises. Out of all the techniques I've seen the one developed for Far Cry 3 speaks most to me. Anyway, I've decided to push my GI system further back, relying on either cubemaps projected onto spherical harmonics or just diffuse lightmaps for now. There are a lot of stuff I can do, but the necessities need to be given priority. SSR and parallax reflections, clustered shading and atmospheric effects need to go in first.

Also, I want to add that I am hopefully going to be able to start a closed beta program for Jove 2.0 in a few months. If any customer of Jove reads this and is interested in being put on the candidate list you can drop me a PM or just write here in the thread.

 

Yes Enlighten supports dynamic lights here's a demo of it:https://www.youtube.com/watch?v=Wrt5aLHI8ME

 

Did you see Lionheads new take on Light Propagation Volumes that is now going to be the standard lighting system in unreal engine 4 in the future?:http://www.lionhead.com/blog/2014/april/17/dynamic-global-illumination-in-fable-legends/ Do you think they've solved the problems that this technique previously had like light leaking?

 

At least for me the main issue of the fully dynamic techniques such as LPV has never been light leaking. It is an issue of performance and the fact that you need to inject light via reflective shadow maps (meaning every light you want to contribute to GI increases performance demands). It works great for outdoor scenes where you have only the sun, but it quickly breaks down on indoor scenes.

That said, going to a 2nd order SH should help a lot with the light leaking and lessen demands on the geometry part of the algorithm. I would love to experiment with LPV but I have a lot of stuff to do before I can start experimenting with special features

 

I did some more digging to see how enlighten is doing the GI in Unity 5 and found this

https://www.youtube.com/watch?v=H4wi6loLWbY

It turns out that it also has a mode where it is in effect baking cube indirect specular cube maps at runtime on the cpu to produce the realtime GI. The maps are very low res though only 64 by 64 but I have to admit the results are pretty convincing. I was almost right about unity taking advantage of the fact that enlighten can get good results from low res lightmaps in enlighten then, it's just that it is able to bake them at runtime so that they are dynamic. I'd love to know how much cpu usage the enlighten runtime uses though.

I would suggest for Jove that if you aren't planning to implement your GI solution yet that you make it so that Jove can use the low res GI cube maps generated by Enlighten and focus your attention on making the most convincing direct PBR lighting , shadows and reflections as well as efficient rendering of a lot of dynamic lights.

It would also be great if when you find out the format of PBR shaders that Unity 5 is using and make an equivalent shader for Jove so that a user could potentially use Unity 5's lighting for a mobile device and then switch the shader and use Jove for more advanced direct lighting on consoles and PC.

Also it would be great if you could create a glass shader that would be suitable for Jove which supported reflection and refraction. You also need to be able to make transparent and emissive objects that work well with your lighting

Also for compositing PBR materials together it would be great if you could link colours on a texture map to particular PBR materials and have Jove use them as layer masks to add multiple PBR materials to a mesh efficiently . I would love to be able to for example use Zbrush or even a painting tool in Unity to just paint areas of a mesh red and blue and then tell Jove to make all areas of the mesh that are red look like they are made of silver and and all areas that are painted blue look like plastic. You could even make it so that you could layer multiple texture maps on top of each other and then be able to blend multiple materials together. I'm guessing this would also result in far more compressible texture maps as a result so that levels would load faster.


I would be happy to participate in your beta for Jove 2. Out of interest though why don't you add the beta of Jove 2 as an optional package in the Jove package on the asset store ?

 

If it's possible, I'm going to support geomerics enlighten for Jove 2.0 (unless the system is super unflexible). I'll play with my own GI system(s) later on when/if I have time.

As for transparency we have a pretty robust system planned for 2.0 which won't have any of the traditional drawbacks of transparency (limited light count etc). Emissive materials are naturally possible as well.

Compositing materials is coming as well, we're hoping to do a custom material system but nothing is final yet. It's mostly an issue of whether or not we can work around Unity as Unity, especially when it comes to textures, likes to do things a certain way (reimporting on moving a texture etc).

I don't think there's going to be an open beta. I prefer to release it to a select few and hold off the update until everything is ready - the last thing I want is to end up spending more time answering questions than adding the last parts of the system

 

Hey, i'm not sure wether i'm in the position to say what looks good and what doesn't, however it being targeted solely at the high end is very much up my alley on some things, in quite a while mind, which suits things fine as this is obviously a major project. I own Jove and i'm not sure i ever ended up importing it as things went a bit quiet then 2.0 happened. From then i saw Jove as an investment, rather than something to get stuck into just now. Certainly put me down for the beta

 

Everyone is in the position to say what looks good I welcome all kinds of feedback. Based on that image though keep in mind that there's no image effects running bare tonemapping and that the sky is a placeholder.

It is a pretty major project indeed. I was working on a GI system for Jove 1.1 but I got fed up spending so much time working around Unity (I had serious issues with how to pass localized data to the shaders at the right time) so I gave up on it and implemented a new pipeline instead. Gives me a lot of freedom I'll gladly keep you in mind for the beta. I'm hoping to reach a usable state by this summer.

 

If I called myself a technical artist i'd be much more heavily inclined towards artist than technical so what i've seen thus far shows it works, thus far, but how it'll relate to how i use it remains to be seen. It is a very interesting project though and i look forward to the results

 

In crysis 2, In this document http://crytek.com/assets/Crysis-2-Key-Rendering-Features.pdf page 17 they mention batching image effect like motion blur and depth of field into 1 drawcall. It only costs them 1ms on consoles. Will you do something similar for Jove 2? Batching effects into 1 or several draw calls.

 

In unity 5 they have said they will support true deferred rendering,based on this does your system have any advantages over unity 5's built in deferred rendering?

 

There is no motion blur as of yet. But yes, I am batching together what can be batched for the sake of performance. Unitys approach of layering effects on top of each other is very flexible but creates unnecessary overhead. For example, the bloom is sampled in the same pass as tonemapping is performed, avoiding an unnecessary full screen pass.

There is a lot more to it than that. The main reason for writing my own rendering pipeline had little to do with the chosen deferred approach, although full deferred is faster, it had more to do with the fact that it gives me complete control over everything. I can use my own lighting system and a bunch of other stuff I could not do with Unitys system. The actual implementation of full deferred shading was done in a few days after starting this, so it's not really the main selling point
The main selling point is a complete rewrite of all the lighting code, including, but not limited to, new shadow mapping, area lights, physically based lighting, atmospheric scattering, volumetric fog and a bunch of other buzzwords The foundation has been built so stuff is going to be happening at a faster pace now.

 

Atmospheric scattering and a dynamic sky are finally ticked off. Now I just need to implement a night mode
While you wait, here is a sunrise over Jove land!

​

 

That's looking very promising! To be honest i'd probably be using a different system to Jove's for the sky were i to use Jove, would that require any heavy rework? It's using volumetric clouds and a good few custom shaders so there's that. Additionally and this has become an issue for me with current PBR, i'm assuming you'll use IBL for PBR, and this will involve the use of cubemaps. As you are aiming at the high end anyways, will you be supplying very fast ways of producing environment maps for PBR for at least almost-dynamic runtime generation? Will you be using an entirely different paradigm? Atm i'm looking at doing this myself for the PBR shaders I use with the SH using OpenCL (There appears to be an implementation for Unity and i'm trying to avoid directcompute completely).

 

That depends on how that system is implemented. The sky is currently rendered as a full screen quad into pixels tagged by a stencil mask from the rendering phase. If the system you use can be modified so that it renders only tagged pixels, then yes it should be pretty easy. You'd just have to comment out JoveSkyPass() and add in the rendering commands for whatever system you choose to use. The advantage of source code

About the IBL, yes I am using it. The issue with dynamically rendered cubemaps is that most IBL implementations require cubemaps that are prefiltered. Jove is no different. If it is feasible depends on your exact need, do you mean rendering a cubemap every frame at the players position or do you mean rendering a cubemap once in a while, like when moving between rooms or loading levels? If it is the second case, it is very feasible. Jove 1.0 used a CPU filtering algorithm, Jove 2.0 uses RenderTextures so it does the filtering on the GPU. It does this not with a brute force approach but by using importance sampling. This means that creating a cubemap is very fast, I do not have any exact measurements but it is less than 100ms.

I'm going to work around this limitation by parallax correcting cubemaps (think box projection but more accurate) and screen space reflections. It is obviously not the same as rendering a cubemap every frame but it is far more performant and is going to look great when it is done.

Why are you looking at using spherical harmonics for IBL? Not to be that guy but SH is not exactly very high frequency (unless you go for higher orders, in which case the data required increases exponentially) and is not very suitable for anything but diffuse light.

 

SH is low frequency yes but it can be used quickly for convolution of cubemaps for diffuse and specular irradiance especially when parallelised, at low orders at least. Theres a good example of the basic idea, serial, here http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter10.html. I've found a few examples of parallelised CUDA implementations and i'm guessing it could be done multiplatform in a compute language, preferably OpenCL developed to compile on nvidia, amd and intels own hardware without fuss

If importance sampling is a better way to do it I can definitely look into that (I know there's something about them in gpu gems 3)

I am definitely thinking of generating them over a few seconds (or a second, or every few tens of frames) rather than per frame as it's expensive, and was also wondering if it was worth grabbing dual paraboloid maps instead of cubemaps as it takes two grabs instead of six

I'm not amazingly experienced with this stuff and mainly regurgitating recent research, as i was happily beavering away making pretty things then realised the lack of almost-dynamic environment maps to go with my time of day + weather with PBR is going to get extremely annoying. I could, I suppose, just grab a map and filter it offline for various combinations of weather and lighting and blend according i just find that a bit inelegant and bodgy and could look completely wrong in some situations hence my interest in doing it a little more dynamically

 

Would I be able to use specialized shaders like the Pre-integrated Skin Shader ?

Thanks...

 

You have to go very high order to get useful radiance information from spherical harmonics, high frequencies get lost very quickly at low orders. I do believe importance sampling is probably a better approach for specular reflections, the math is kind of hairy but you don't need to understand the proof why it works in order to be able to use it The Nvidia article has code for how to importance sample for a blinn-phong distribution, if you want a GGX instead you can look at Epics presentation on shading in Unreal Engine 4.
I agree that it's hard to get IBL to work good with dynamic time of day and weather. The approach I'm looking at is trying to bake a cubemap that's as general as possible (meaning no specularity from any lights but static ones) and then modulate it with intensity and color based on time of day (e.g at night you would multiply the sampled cubemap by a dark blue color). It's not perfect but combined with screenspace reflections it is convincing, and cheap, enough for this generation.

The bad news is that very few shaders are going to work without modifcation. The good news is that it is going to be very easy to convert shaders to run in Joves forward pass. All you would have to do is change the RenderType tag and instead of accessing lights with Unitys function (what are they now again? Something like _LightColor0, I have not used them in quite a while) you would access lights using Jove functions. Although when Jove is finished I doubt you are even going to want to use many other shaders, unless they are for special effects.




Demonstrating the capabilities of this system, here's a foggy day.

 

I put together a small demo of a sunrise, with sliders to control the sky intensity and scattering intensity. It's not a complex scene but it shows off the capabilities. Here's the link https://dl.dropboxusercontent.com/u/233290703/ScatteringDemo.html
As usual, requires DX11

 

I am getting weird artifacts on your demo, My graphics card is an HD7850

 

That looks weird indeed. It looks like it works except for the artifacts. Have you run the demo I posted a page or two back? Did that work without any artifacts? Also, have you updated your drivers?

 

Aieth I'm also getting issues with that demo. I don't think you are clearing the frame buffer or depth buffer correctly or something like that. I have a gtx680 and it constantly flickers after a couple of seconds. I can stop it flickering if I minimize the tab in the browser and reopen it. It looks like the graphics driver is force clearing the frame buffer to get round the issue after it fills after a couple of seconds. The previous demo you posted works fine.

 

The previous demo works perfectly fine

 

Very strange, it works fine for me on a GTX 560 ti. I added a bunch of unnecessary clears just to make sure it's fine, test this version https://dl.dropboxusercontent.com/u/233290703/ScatteringDemo.html
Have you guys updated your drivers recently? I know there was an issue with 3d rendertextures in Unity with older drivers, it caused strange behaviours when more than 2 were active at a time (which there are in my system during the precalculation phase). My issues disappeard on a driver update. I also reduced size of the demo from 50mb to 1mb